- Troubleshooting >
- Automation
Automation¶
On this page
Cloud Manager Automation allows you to deploy, configure, and manage MongoDB deployments with the Cloud Manager UI. Cloud Manager Automation relies on an MongoDB Agent, which must be installed on every server in the deployment. The MongoDB Agents periodically poll the Cloud Manager service to determine the current goal, and continually report their status to Cloud Manager.
Automation Runs Only on 64-bit Architectures¶
Cloud Manager provides only 64-bit downloads of the MongoDB Agent.
Using Own Hardware¶
If you deploy Automation manually, ensure that you have one MongoDB Agent on every server.
If you deploy the agent manually, you must create MongoDB’s
dbpath
and the directory for the MongoDB binaries and ensure that the user running the agent owns these directories.If you install using the
rpm
package, the agent runs as themongod
user; if using thedeb
package, the agent runs as themongodb
user. If you install using thetar.gz
archive file, you can run the agent as any user.
Networking¶
Connectivity to MongoDB Ports¶
All hosts must be able to allow communication between MongoDB
ports. The default is 27017
, but you can configure alternate port
ranges in the Cloud Manager interface.
The MongoDB Agent must be able to connect to
cloud.mongodb.com
on port 443
(HTTPS).
For more information on access to ports and IP addresses, see
Security Overview.
Intra-cluster Connectivity Issue¶
When performing a rolling update, the MongoDB Agent tries to avoid
downtime. It needs to collect state from other members of the cluster.
A connectivity issue (between mongod
s and mongos
es), such as
hostname resolution or misconfigured firewall, may prevent the
MongoDB Agent from determining the remote processes state and
completing the change.
To ensure all members of the cluster can communicate with each other:
- For a non-sharded cluster:
- For a sharded cluster:
Frequent Automation Connections¶
MongoDB Agent gathers the state from each member of the cluster every 10 seconds to ensure that the environment is in the expected state. As part of this assessment, MongoDB Agent creates a connection, checks certain files to determine state, and then closes the connection. These frequent, short-lived connections are part of MongoDB Agent’s routine activity and should not impact performance.
Automation Configuration¶
After completing the automation configuration, ensure that the deployment plan satisfies the needs of your deployment. Check hostnames and ports before confirming the deployment.
Sizing¶
- Ensure that you provision hosts with enough space to run MongoDB and support the requirements of your data set.
- Ensure that you provision a sufficient number of hosts to run your
deployment. Each
mongod
should run on its own host.
Frequent Connection Timeouts¶
The MongoDB Agent may frequently time out of connections for one or more of the following reasons:
- Connection timeouts
- High network latency
- High server load
- Large SSL keys
- Insufficient CPU speed
By default, connections time out after 40 seconds. MongoDB recommends
gradually increasing the value of the dialTimeoutSeconds
MongoDB Agent configuration setting to prevent frequent premature
connection timeouts. However, increasing this value also increases the
time required to deploy future configuration changes. Experiment with
small, incremental increases until you determine the optimum value for
your deployment.
To learn more, see dialTimeoutSeconds
in
Connection Settings at
MongoDB Agent Configuration.
Deployments¶
A banner that displays We have detected a potential problem while deploying… appears when certain conditions apply. These are some examples.
Deployment Change Unable to Complete / Not Proceeding¶
If you have added or restarted a deployment and the deployment remains
in the In Progress
state for several minutes, the banner displays.
At this point, you have four options:
Click View Diagnostics.
The View Diagnostics modal displays any errors that may have happened during deployment.
Click View Status.
The Automation Status modal displays the deployment processes, when they last reported their deployment status, what task they are trying to complete, and the deployment status.
To learn more about the status of any of the individual processes, you can click the ellipsis icon and select either View Details or View Agent Logs.
- View Details shows what the deployment plan for the process is and which stage of that plan the MongoDB Agent currently is.
- View Agent Logs opens a new browser window with the Deployment > Agents > Agent Logs screen shown and the contents of the MongoDB Agent log displayed by default. Click the View menu to select a different agent log.
Click View Agent Logs.
A new browser window opens with the Deployment > Agents > Agent Logs screen shown and the contents of the MongoDB Agent log displayed by default. Click the View menu to select a different agent log.
Click Allow Override & Edit Configuration.
If you diagnose an error and need to correct the deployment configuration, follow the procedure to edit the deployment.
If you shut down the deployment and still cannot find a solution, remove the deployment from Cloud Manager.
MongoDB Agents Are Down or Not Communicating¶
If Automation Module of MongoDB Agent can’t communicate with the
Cloud Manager API endpoint
or the MongoDB
Server processes, Cloud Manager displays a warning banner in the Project. You
can resolve this in one of two ways depending upon whether or not you
expect the MongoDB Agents to be communicating:
MongoDB Agents need to communicate¶
If the MongoDB Agent(s) should be communicating with the Cloud Manager host or MongoDB instances, confirm the following for each MongoDB Agent:
- The Agent is up and running on each host.
- The Agent and the Cloud Manager
API endpoint
can communicate.
MongoDB Agents Do Not Need to Communicate¶
If Automation Module of MongoDB Agent(s) should be communicating with
the Cloud Manager API endpoint
or MongoDB
Server processes, confirm the following for each automated MongoDB
Server deployment:
- Click the Allow Editing & Override Current Configuration link in the warning banner.
- Remove all processes
(
mongod
andmongos
) running on the hosts that serve the unneeded MongoDB Agents.
MongoDB Agent Permissions¶
A permissions issue may prevent automation from completing a change.
If View Status or View Diagnostics report an
permissions-related error (such as open /data/db/mongod.lock:
permission denied
), ensure that the MongoDB Agent user owns and
has read and write permissions to the dbpath
and logpath
files.
Remove Problematic MongoDB Hosts¶
You can use the console or the API to remove stale, broken, or problematic hosts from automation. This may include the circumstance when the MongoDB Agent can’t be reached.
To remove a problematic host using the console:
Navigate to your project.
Click Servers.
Find your problematic host.
Click Remove Server.
, thenCloud Manager displays the Are you sure you want to remove this server> modal.
Enter the provided hostname into the Enter the host name field an click Remove if you want to remove this server.
Warning
Cloud Manager removes all monitoring data for this host when you click Remove. Cloud Manager provides no confirmation or cancellation for this action.
If you don’t want to remove the server, click Cancel.
Click Review & Deploy to review your changes.
Cloud Manager displays your proposed changes.
If you are satisfied, click Confirm & Deploy.
Cloud Manager removes all processes and agents at this time.
If you want to make further configuration changes, click Cancel.
To remove a problematic host using the API:
- Get the current automation config.
- Edit the automation configuration JSON file.
- Remove the stale node from processes and replica sets.
- Update the automation config file.
- Wait for a few minutes.
- Check the Agents view.
- Confirm the host no longer appears on the list of Agents.