Navigation

View and Retrieve Logs

Overview

Cloud Manager collects log information for both MongoDB processes and its agents. For MongoDB processes, you can access both real-time logs and on-disk logs.

  • The MongoDB logs provide the diagnostic logging information for your mongod and mongos processes.
  • The Agent logs provide insight into the behavior of your Cloud Manager agents.

MongoDB Real-Time Logs

The Monitoring Agent issues the getLog command with every monitoring ping. This command collects log entries from RAM cache of each MongoDB process.

Cloud Manager enables real-time log collection by default. You can disable log collection for either all MongoDB processes in a Cloud Manager project or for individual MongoDB processes. If you disable log collection, Cloud Manager continues to display previously collected log entries.

View MongoDB Real-Time Logs

1

Click Deployment, then the Processes tab, then the Topology view.

2

(Optional) For sharded clusters, filter which process type is listed.

The four buttons are listed in the following order, left to right: Shards, Configs, Mongos, and BIs.

Process Displays
Shards mongod processes that host your data.
Configs mongod processes that run as config servers to store a sharded cluster’s metadata.
Mongos mongos processes that route data in a sharded cluster.
BIs BI processes that access data in a sharded cluster.
3

On the line listing the process, click Metrics.

4

Click the Logs tab.

The tab displays log information. If the tab is not displayed, see Enable or Disable Log Collection for a Deployment to enable log collection.

5

Refresh the browser window to view updated entries.

Enable or Disable Log Collection for a Deployment

1

Click Deployment, then the Processes tab, then the Topology view.

2

On the line for any process, click the ellipsis [] icon then click Monitoring Settings.

3

Toggle Collect Logs For Host as desired.

  1. Click the Logs tab.
  2. Toggle the Collect Logs For Host to Off or On, as desired.
4

Click X to close the Monitoring Settings box.

If you turn off log collection, existing log entries remain in the Logs tab, but Cloud Manager does not collect new entries.

Enable or Disable Log Collection for the Project

1

Click Settings, then Project Settings.

2

Toggle the Collect Logs For All Hosts option to Yes or No, as desired.

MongoDB On-Disk Logs

Cloud Manager collects on-disk logs even if the MongoDB instance is not running. The Automation Agent collects the logs from the location you specified in the MongoDB systemLog.path configuration option. The MongoDB on-disk logs are a subset of the real-time logs and therefore less verbose.

You can configure log rotation for the on-disk logs. Cloud Manager rotates logs by default.

View MongoDB On-Disk Logs

1

Click the deployment whose logs you want to download, then click Request Logs

Click the ellipsis [] icon on the line for any process, replica set, or sharded cluster in the project, then click Request Logs.

2

Select which types of log Cloud Manager per process to collect and the maximum cumulative size of these logs.

To choose logs to download, perform the following actions:

Action Purpose
Click MongoDB Logs Gather logs from deployed MongoDB processes.
Click FTDC Data Gather the diagnostic data files from the (FTDC) collection mechanism, such as server statistics and status messages.
Click Automation Agent Logs Gather logs from the deployed Automation Agents.
Click Backup Agent Logs

Gather logs from all deployed Backup Agents.

Note

This differs from other logs. The logs collected are not limited to the selected hosts, but include all Backup Agent logs in the deployment.

Click Monitoring Agent Logs

Gather logs from all deployed Monitoring Agent.

Note

This differs from other logs. The logs collected are not limited to the selected hosts, but include all Monitoring Agent logs in the deployment.

Set Size per Log Type in MB

Enter the maximum cumulative uncompressed size in megabytes of all selected log files.

  • This limit is cumulative. If you set this value to 50 MB, Cloud Manager gathers a total of 50 MB of uncompressed log file from all logs you chose to download. Those log files are then archived and compressed.
  • If the current log file is less than the specified size, Cloud Manager gathers the most recent rotated file as well.
  • If the total size of the log files reaches the specified size part way through a log file, this last log file is truncated to the most recent line that falls within the specified size.
  • The maximum amount of log files that can be collected is 20 GB. This maximum includes all collected log files that have not expired. If you request additional logs and collecting those logs results in more than 20 GB of logs collected, Cloud Manager generates an error. The total amount of logs collected compared to the limit is displayed as Estimated Total Size.

Example

You choose to collect 20 MB of logs from all processes in a replica set. This replica set has three mongod processes on two hosts:

  • host1:27017
  • host2:27017
  • host2:27018

Your deployment runs the following agents:

Automation Agent host1, host2, host4, host5
Backup Agent host4
Monitoring Agent host4, host5

When you choose all the log types for this replica set, and limit to 20 MB per process, Cloud Manager shows that the Estimated Total Size is 220 MB (11 processes * 20 MB) out of 20 GB.

Once the log collection begins, Cloud Manager scans the log directories for the mongod processes and their associated FTDC from the most current log entry until 20 MB of log files or the end of the last log is collected. All the monitoring and backup agents in the deployment are also scanned.

  • The Backup Agent has 60 MB of logs.
  • Each MongoDB process (3) has 7 MB of logs plus 15 MB of FTDC data per process.
  • Each Monitoring Agent (2) has 30 MB of logs.
  • Each Automation Agent (2) has 12 MB of logs.

The total size of logs collected is 150 MB:

(20 + (3 * (7 + 15)) + (2 * 20) + (2 * 12)) = 150

  • The maximum of 20 MB of logs from the Backup Agent is collected.
  • All of the logs for each MongoDB process are collected: 7 MB of MongoDB + 15 MB of FTDC data.
  • The maximum of 20 MB of logs from each Monitoring Agent is collected.
  • All of the Automation Agent logs from host1 and host2 are collected. host4 and host5 do not host any processes in the replica set.

The resulting archive structure within the downloaded archive is:

host1/27017/mongodb
host1/27017/ftdc
host1/automation_agent
host2/27017/mongodb
host2/27017/ftdc
host2/27018/mongodb
host2/27018/ftdc
host2/automation_agent
host4/backup_agent
host4/monitoring_agent
host5/monitoring_agent
3

View the progress on the Log Request History page.

Entry Status shows Collecting Logs… and automatically updates its status as log collection continues.

  • If Cloud Manager fails to retrieve log files, click Retry to retrieve the failed log files again.
  • If a failure has occurred, you can still download the archive. Some of the requested log files will be missing.
4

Download the collected logs archive.

Click the Download icon.

The size listed for the archive on the Log Request History page is the uncompressed size. The archive consumes that amount of disk space on the target host once it is extracted.

This download can only be restarted and not resumed. If the download fails, you must download the logs again.

The archive is named mongodb-logfiles_<instance_or_process>_<ISO8601_Format_Date>.tar.gz.

The extracted files use the following directory structure:

<host>
  automation_agent
    automation-agent-verbose.log
    automation-agent-verbose.log.<ISO8601_Format_Date>
  backup_agent
    backup-agent-verbose.log
    backup-agent-verbose.log.<ISO8601_Format_Date>
  monitoring_agent
    monitoring-agent-verbose.log
    monitoring-agent-verbose.log.<ISO8601_Format_Date>
  <replica_set> // Sharded Cluster Only
    <port>
      ftdc
        metrics.<ISO8601_Format_Date>
        metrics.interim
      mongodb
        mongodb.log
        mongodb.log.<ISO8601_Format_Date>
  <port> // Replica Set or Standalone
    ftdc
      metrics.<ISO8601_Format_Date>
      metrics.interim
    mongodb
      mongodb.log
      mongodb.log.<ISO8601_Format_Date>

Note

When you extract the tar archive on a Microsoft Windows host, use an archive extraction utility that supports PAX extended headers. Some Windows archive utilities have issues with PAX extended headers for tar.

Collected logs expire and are removed after 7 days. To extend the lifetime of a particular log file, click the extend link for that archive on the Log Request History page.

Configure Log Rotation

1

Open the MongoDB Log Settings modal.

  1. Click Deployment.
  2. In the More drop-down list, click MongoDB Log Settings.
2

Configure the log rotation settings.

Setting Description
Rotate Logs Toggle to On or Off.
Size Threshold (MB) The size where the logs rotate automatically. The default value is 1000.
Time Threshold (Hours) The duration of time when the logs rotate automatically. The default value is 24.
Max Uncompressed Files Optional. The greatest number of log files that should stay uncompressed. The suggested value is 5.
Max Percent of Disk Optional. The greatest amount of disk space on your MongoDB hosts that the logs should consume. The suggested value is 2%.

When you are done, click Save.

3

Click Review & Deploy to review your changes.

4

Click Confirm & Deploy to deploy your changes.

Otherwise, click Cancel and you can make additional changes.

Agent Logs

Cloud Manager collects logs for all your Automation Agents, Monitoring Agents, and Backup Agents.

View Agent Logs

1

Click Deployment, then the Agents tab, then Agent Logs.

The page displays logs for the type of agent selected in the View drop-down list. The page filters logs according to any filters selected in through the gear icon.

2

Filter the log entries.

To display logs for a different type of agent, use the View drop-down list.

To display logs for a specific hosts or MongoDB processes, click the gear icon and make your selections.

To clear filters, click the gear icon and click Remove Filters.

To download the selected logs, click the gear icon and click Download as CSV File.

Note

To view logs for a specific agent, you can alternatively click the Agents tab’s All Agents list and then click view logs for the agent.

Configure Agent Log Rotation

If you are using Automation to manage your cluster, use the following procedure to configure rotation of the agent log files.

Note

If you are not using Automation, see the following documentation for information about how to manually configure logging settings in the agent configuration files:

1

Click Deployment, then the Agents tab.

2

Click Downloads & Settings.

3

Scroll down to the Agent Log Settings section.

4

Edit the log settings.

Click the pencil icon to edit the Monitoring Agent or Backup Agent log settings:

Name Type Description
Linux Log Path string

Conditional: Logs on a Linux host. The path to which the agent writes its logs on a Linux host.

The suggested value is:

/var/log/mongodb-mms-automation/monitoring-agent.log
Windows Log Path string

Conditional: Logs on a Windows host. The path to which the agent writes its logs on a Windows host.

The suggested value is:

%SystemDrive%\MMSAutomation\log\mongodb-mms-automation\monitoring-agent.log
Rotate Logs Toggle A toggle to select if the logs should be rotated.
Size Threshold (MB) integer The size where the logs rotate automatically. The default value is 1000.
Time Threshold (Hours) integer The duration of time when the logs rotate automatically. The default value is 24.
Max Uncompressed Files integer Optional. The greatest number of log files, including the current log file, that should stay uncompressed. The suggested value is 5.
Max Percent of Disk integer Optional. The greatest percentage of disk space on your MongoDB hosts that the logs should consume. The suggested value is 2%.

When you are done, click Confirm.

5

Click Review & Deploy to review your changes.

6

Click Confirm & Deploy to deploy your changes.

Otherwise, click Cancel and you can make additional changes.