Navigation

Health Check Solutions

This page lists issues that can be raised by a Cloud Manager health check and provides their solutions.

Agent version outdated

Agents get upgraded for better performance and user experience. You may choose to update the agents for your Cloud Manager group:

If you use Automation:
Update the agents through the Cloud Manager Deployment tab.
If you do not use Automation

Follow the update procedures for the operating system of the servers on which your agents currently run:

Host has decreasing available disk space

Cloud Manager considers any disk on any host as needing more disk capacity if it estimates that the disk will be full in two weeks or less.

To remedy this problem, move your database to disk(s) with greater capacity.

Host has excessive disk utilization

Cloud Manager considers any disk on any host as having excessive disk utilization if it is actively storing or retrieving data for a prolonged period of time.

To remedy this problem, move your database to disk(s) with greater throughput.

Host has startup warnings

Limits Startup Warning

Process and user limits with low default values can cause a number of issues in the course of normal MongoDB operation. Please see UNIX ulimit Settings in the MongoDB Manual for further information and recommendations.

NUMA Enabled Startup Warning

Running MongoDB on a system with NUMA can cause a number of operational problems, including slow performance for periods of time and high system process usage. Please see MongoDB and NUMA Hardware in the MongoDB Manual for further information and recommendations.

Readahead

Please see the readahead information in this section of the MongoDB Manual for information and recommendations about the Readahead startup warning.

Transparent Huge Pages + Defrag

Please see Disable Transparent Huge Pages (THP) for information and recommendations about the Transparent Huge Pages and Defrag startup warning.

Host is unreachable

The Monitoring Agent connects to each MongoDB process in your deployment to collect diagnostic data.

If your Monitoring Agent cannot connect to a process, consider the following possible resolutions:

Reason Resolution
Host no longer exists. Remove host from Cloud Manager.
Monitoring Agent cannot reach host. See Remedies for a Host Down Alert for possible resolutions.

MongoDB version outdated

For MongoDB deployments managed by Cloud Manager, Cloud Manager supports safe automatic upgrade and downgrade operations between releases of MongoDB while maximizing the availability of your deployment. Cloud Manager supports upgrade and downgrade operations for sharded clusters, replica sets, and standalone MongoDB instances.

Configure Available MongoDB Versions describes how to choose which versions of MongoDB are available to Cloud Manager.

If your deployment is not managed by Cloud Manager, you will need to manually change the version of MongoDB. The MongoDB Manual provides upgrade tutorials with each release. For example, see Upgrade MongoDB to 3.2 for upgrading to MongoDB 3.2 from an earlier version.

For managed deployments:

  1. Click Deployment, then the Processes tab.
  2. Click the Topology view.
  3. On the line listing the cluster, replica set, or process, click the Modify button.
  4. In the Version field, select the version. Then click Apply.
  5. Click Review & Deploy.
  6. Click Confirm & Deploy.

For more information and precautions, see Change the Version of MongoDB.

Replica set has an even number of votes

An even number of voting members in a replica set can lead to election issues in the event of a primary node failure. You should consider adding an additional voting node to your replica sets to ensure an odd number of votes.

You can add an arbiter to your replica set to allow an uneven number of members without the overhead of a member that replicates data.

If your deployment is not managed by Cloud Manager, follow the MongoDB Manual’s instructions to manually add an arbiter to your replica set.

For managed deployments:

  1. Click Deployment, then the Processes tab.
  2. Click the Topology view.
  3. On the line listing the replica set, click the Modify button.
  4. Add and configure the new member: Under Member Options, click Add and select Arbiter.
  5. Click Apply.
  6. Click Review & Deploy. Cloud Manager displays your proposed changes.
  7. Click Confirm & Deploy.

Replica set has less than three data-bearing nodes

We recommend that your replica set includes at least three data-bearing nodes to ensure high availability. For factors that affect high availability, see the MongoDB Manual’s pages on high availability, elections, and failover.

If your deployment is not managed by Cloud Manager, follow the MongoDB Manual’s instructions to manually add a node to your replica set.

For managed deployments:

  1. Click Deployment, then the Processes tab.
  2. Click the Topology view.
  3. On the line listing the replica set, click the Modify button.
  4. Add and configure the new member: Add the member by increasing the number of members in the MongoDs Per Replica Set field.
  5. Click Apply.
  6. Click Review & Deploy. Cloud Manager displays your proposed changes.
  7. Click Confirm & Deploy.

Replica set has mixed version nodes

Because of potential incompatibilities, it is recommended you upgrade outdated versions of MongoDB instances to the most recent in your cluster.

If your deployment is not managed by Cloud Manager, you will need to manually change the version of MongoDB. The MongoDB Manual provides upgrade tutorials with each release. For example, see Upgrade MongoDB to 3.2 for upgrading to MongoDB 3.2 from an earlier version.

For managed deployments:

  1. Click Deployment, then the Processes tab.
  2. Click the Topology view.
  3. On the line listing the replica set, click the Modify button.
  4. In the Version field, select the version. Then click Apply.
  5. Click Review & Deploy.
  6. Click Confirm & Deploy.

For more information and precautions, see Change the Version of MongoDB.

Replica set has more than one arbiter

An arbiter is added to a replica set with an even number of members to add a vote in elections for primary. Arbiters always have exactly one vote, and thus allow replica sets to have an uneven number of members, without the overhead of a member that replicates data. Only one arbiter is required to break election ties.

If your deployment is not managed by Cloud Manager, follow the MongoDB Manual’s instructions to manually remove a member from your replica set.

For managed deployments:

  1. Click Deployment, then the Processes tab.
  2. Click the Topology view.
  3. For the arbiter to be removed, click the ellipsis icon and select Remove from Replica Set.
  4. Click Remove to confirm.
  5. Click Review & Deploy. Cloud Manager displays your proposed changes.
  6. Click Confirm & Deploy.

For more information on deployment architectures, see Replica Set Deployment Architectures in the MongoDB Manual.

Shared cluster has mixed version nodes

The components of the sharded cluster run different versions of MongoDB.

To avoid compatibility issues, use the same version of MongoDB for all the mongos and mongod processes that make up your sharded cluster. This includes all the mongod processes used for the cluster’s config servers and shards.

To change the version of a mongod or mongos process, see Change the Version of MongoDB.

Too many queued operations

Queued operations are operations that are waiting to be processed. This may occur when you have reached your hardware capacity or if you have poorly performing queries.

If you have access to Cloud Manager Premium, you can track long running operations using the Cloud Manager Profiler. To enable the profiler tool in Cloud Manager:

  1. Click Deployment, then the Processes tab.
  2. Click the Topology view.
  3. On the line listing the process, click the Metrics button.
  4. Click the Profiler tab and follow instructions to enable the profiler.

If you don’t have access to Cloud Manager Premium, you can still get access to profiling data for statistics about performance and database operations. To read more about profiling databases, see Profile Databases.

Too much replication lag

Replication lag is a delay between an operation on the primary and the application of that operation from the oplog to the secondary. Replication lag can be a significant issue and can seriously affect MongoDB replica set deployments. Excessive replication lag makes “lagged” members ineligible to quickly become primary and increases the possibility that distributed read operations will be inconsistent.

For information on how to troubleshoot replication lag, please see Check the Replication Lag in the MongoDB Manual.