Navigation

Restore a Replica Set from a Backup

When you restore a replica set from backup, Cloud Manager provides you with a restore file for the selected restore point. For an overview of the restore process, please see /core/restore-overview.

Considerations

BinData

The BSON specification changed the default subtype for the BSON binary datatype (BinData) from 2 to 0. Some binary data stored in a snapshot may be BinData subtype 2. The Backup Agent automatically detects and converts snapshot data in BinData subtype 2 to BinData subtype 0. If your application code expects BinData subtype 2, you must update your application code to work with BinData subtype 0.

See also

The notes on the BSON specification explain the particular specifics of this change.

restoreInfo.txt

The backup restore file includes a metadata file, restoreInfo.txt. This file captures the options the database used when the snapshot was taken. The database must be run with the listed options after it has been restored.

This file contains:

  • Group name
  • Replica Set name
  • Cluster Id (if applicable)
  • Snapshot timestamp (as Timestamp at UTC)
  • Last Oplog applied (as a BSON Timestamp at UTC)
  • MongoDB version
  • Storage engine type
  • mongod startup options used on the database when the snapshot was taken

Prerequisites

Oplog Size, Start and End Timestamps

To seed each replica set member you need to know the current size, start timestamp, and end timestamp of your oplog.

To find the size of your oplog for your replica set member:

Connect to the member in a mongo shell and run the rs.printReplicationInfo() method. This method returns the following data:

configured oplog size:   10.10546875MB
log length start to end: 94400 (26.22hrs)
oplog first event time:  Mon Mar 19 2012 13:50:38 GMT-0400 (EDT)
oplog last event time:   Wed Oct 03 2012 14:59:10 GMT-0400 (EDT)
now:                     Wed Oct 03 2012 15:00:21 GMT-0400 (EDT)
To make a point-in-time or oplog timestamp restore:

Run the mongodb-backup-restore-util with the following options:

  • --seedReplSetMember
  • --oplogSizeMB
  • --seedTargetPort
  • --opStart
  • --opEnd

The MBRU needs the oplog size as an integer in MB. Set this value using the --oplogSizeMB option.

The Cloud Manager application provides the start timestamp and end timestamp.

To make a snapshot restore

Run the seedSecondary.sh or seedSecondary.bat script included in the backup restore file.

The seedSecondary script needs the replica set’s oplog size as an integer in GB.

To learn how to seed a secondary using this script, see Seed a New Secondary from Snapshot.

See also

For learn how the oplog affects replication, see Check the Size of the Oplog section of the Troubleshoot Replica Sets.

Client Requests During Restoration

You must ensure that the MongoDB deployment does not receive client requests during restoration. You must either:

  • Restore to new systems with new hostnames and reconfigure your application code once the new deployment is running, or
  • Ensure that the MongoDB deployment will not receive client requests while you restore data.

Secure Copy (SCP) Delivery

Important

You need to generate a key pair before using SCP. SCP transfers files faster than HTTP.

Note

Microsoft Windows does not include SCP. Installing SCP is outside the scope of this manual.

SCP is performed in parallel and, by default, Secure Shell Daemon (sshd) installations use a small number of concurrent connections. Changing this setting in sshd_config allows SCP to support sufficient connections to complete the restore.

Automatic Restore

To have Cloud Manager restore the snapshots, perform the select the snapshot procedure. Choose Choose Cluster to Restore to at Step 2 of the restore wizard.

Manual Restore

To restore the backup manually, perform the following procedures:

  1. Select and Prepare the Snapshot.
  2. Retrieve the Snapshot using HTTPS or Send the Snapshot using SCP.
  3. Prepare and Distribute Snapshot
  4. Unmanage the Replica Set.
  5. Restore the Replica Set Manually.
  6. Reimport the Replica Set.

Select and Prepare the Snapshot

1

Click Backup, then the Overview tab.

2

Click the deployment then click Restore or Download.

3

Select the restore point.

Choose the point from which you want to restore your backup.

Restore Type Description Action
Snapshot Allows you to choose one stored snapshot. Select an existing snapshot to restore.
Point In Time

Creates a custom snapshot that includes all operations up to but not including the selected time. By default, the oplog stores 24 hours of data.

Example

If you select 12:00, the last operation in the restore is 11:59:59 or earlier.

Important

You cannot perform a PIT restore that covers any time prior to the latest backup resync. For the conditions that cause a resync, see Resync a Backup.

Select a Date and Time.
Oplog Timestamp

Creates a custom snapshot based on the timestamp of an oplog entry (its ts field). Cloud Manager includes all operations up to and including the time of the timestamp.

The oplog entry’s ts field is a BSON timestamp and has two components: the timestamp and the increment.

Type the following:

Timestamp
The value in seconds since the Unix epoch.
Increment
An incrementing ordinal for operations within a given second.

Click Next.

4

Choose how to restore the files.

Choose to restore the snapshot to an existing MongoDB process or download a copy of the data.

To restore to an existing instance, click Choose Cluster to Restore to.
  1. Complete the following fields:

    Field Action
    Project Select a project to which you want to restore the snapshot.
    Cluster to Restore to

    Select a cluster to which you want to restore the snapshot.

    Warning

    Automation removes all existing data from the cluster. All backup data and snapshots for the existing cluster are preserved.

  2. Click Restore.

    Cloud Manager notes how much space the restore requires.

    Important

    You can skip the remainder of this page.

To download the data, click Download.

You can choose to download a copy using HTTPS or have Cloud Manager send you a copy using SCP.

Retrieve the Snapshot using HTTPS

1

Configure the snapshot download.

  1. Configure the following download options:

    Pull Restore Usage Limit Select how many times the link can be used. If you select No Limit, the link is re-usable until it expires.
    Restore Link Expiration (in hours) Select the number of hours until the link expires. The default value is 1.
  2. Click Finalize Request.

  3. If you use 2FA, Cloud Manager prompts you for your 2FA code. Enter your 2FA code, then click Finalize Request.

2

Retrieve the snapshot.

The restore panel shows the restore point type and time, the delivery method, and the download link once Cloud Manager has generated it.

By default, this link is available for an hour and can be used just once. To download the snapshot:

  1. If you closed the restore panel, click Backup, then Restore History.
  2. When the restore job completes, click (get link) for the restore.
  3. Click:
    • The copy button to the right of the link to copy the link to use it later, or
    • Download to download the snapshot immediately.

Extra step for point-in-time restores

For point-in-time and oplog timestamp restores, additional instructions are shown. The final step shows the full command you must run using the MBRU. It includes all of the necessary options to ensure a full restore.

Select and copy the mongodb-backup-restore-util command provided under Run Binary with PIT Options.

Important

Skip to Prepare and Distribute Snapshot.

Send Snapshot using SCP

Direct Cloud Manager to copy the restore files to your server via SCP.

1

Configure how to secure copy the data.

  1. Select from the following options:

    Format

    Select the format in which you want to receive the restore files:

    Individual DB Files
    Transfers individual MongoDB data files that Cloud Manager produces directly to the target directory.
    Archive

    Transfers MongoDB data files in a single archive (tar or tar.gz) that you must extract before restoring the data files to a working directory.

    This option displays only if the archive size can be calculated.

    With Archive delivery, you need sufficient storage space on the destination host for both the archive and the extracted files.

    SCP Host Type the hostname of the host to receive the files.
    SCP Port Type the port of the host to receive the files.
    SCP User Type the username used to access to the host.
    Auth Method Select whether to use a username and password or an SSH certificate to authenticate to the host.
    Password Type the user password used to access to the host.
    Passphrase Type the SSH passphrase used to access to the host.
    Target Directory Type the absolute path to the directory on the host to which to copy the restore files.
  2. Click Finalize Request.

  3. If you use 2FA, Cloud Manager prompts you for your 2FA code. Enter your 2FA code, then click Finalize Request.

2

Retrieve the snapshot.

The restore panel shows the restore point type and time, the delivery method, and the download link once Cloud Manager has generated it.

The files are copied to the host directory you specified. To verify that the files are complete, see how to validate a secure copy restore.

Extra step for point-in-time restores

For point-in-time and oplog timestamp restores, additional instructions are shown. The final step shows the full command you must run using the MBRU. It includes all of the necessary options to ensure a full restore.

Select and copy the mongodb-backup-restore-util command provided under Run Binary with PIT Options.

Prepare and Distribute Snapshot

1

Restore the snapshot data files to the destination host.

Extract the snapshot archive to a temporary location where a temporary mongod instance can access the archive contents. Use a different data directory than any other database running on the host.

Example

tar -xvf {backupRestoreName}.tar.gz
mv {backupRestoreName} {temp-database-path}
2

Run the MongoDB Backup Restore Utility (Point-in-Time Restore Only).

  1. Download the MongoDB Backup Restore Utility to your host.

    Note

    If you closed the restore panel, click Backup, then More and then Download MongoDB Backup Restore Utility.

  2. Run the MongoDB Backup Restore Utility on your destination host.

    Pre-configured MBRU command

    Cloud Manager provides the mongodb-backup-restore-util with the appropriate options for your restore on the restore panel under Run Binary with PIT Options.

    Select and copy the mongodb-backup-restore-util command provided.

    ./mongodb-backup-restore-util --https --host <targetHost> \
      --port <targetPort> \
      --opStart <opLogStartTimeStamp> \
      --opEnd <opLogEndTimeStamp> \
      --logFile <logPath> \
      --oplogSourceAddr <oplogSourceAddr> \
      --apiKey <apiKey> \
      --groupId <groupId> \
      --rsId <rsId> \
      --whitelist <database1.collection1, database2, etc.> \
      --blacklist <database1.collection1, database2, etc.> \
      --seedReplSetMember \
      --oplogSizeMB <size> \
      --seedTargetPort <port> \
      --ssl \
      --sslCAFile <path> \
      --sslPEMKeyFile <path>
    

    The mongodb-backup-restore-util command uses the following options:

    Option Required Description
    --https Optional Use if you need TLS/ SSL to connect to the --oplogSourceAddr.
    --host Required Provide the hostname or IP address for the host that serves the mongod to which the oplog should be applied.
    --port Required Provide the port for the host that serves the mongod to which the oplog should be applied.
    --opStart Required Provide the BSON timestamp for the first oplog entry you want to include in the restore.
    --opEnd Required Provide the BSON timestamp for the last oplog entry you want to include in the restore.
    --logFile Optional Provide a path, including file name, where the MBRU log is written.
    --oplogSourceAddr Required Provide the URL for the oplog.
    --apiKey Required Provide your Cloud Manager Agent API Key.
    --groupId Required Provide the group ID.
    --rsId Required Provide the replica set ID.
    --whitelist Optional Provide a list of databases and/or collections to which you want to limit the restore.
    --blacklist Optional Provide a list of databases and/or collections to which you want to exclude from the restore.
    --seedReplSetMember Optional

    Use if you need a replica set member to re-create the oplog collection and seed it with the correct timestamp.

    Requires --oplogSizeMB and --seedTargetPort.

    --oplogSizeMB Conditional

    Provide the oplog size in MB.

    Required if --seedReplSetMember is set.

    --seedTargetPort Conditional

    Provide the port for the replica set’s primary. This may be different from the ephemeral port used.

    Required if --seedReplSetMember is set.

    --ssl Optional Use if you need TLS / SSL to apply oplogs to the mongod. Requires --sslCAFile and --sslPEMKeyFile.
    --sslCAFile Conditional

    Provide the path to the CA file.

    Required if --ssl is set.

    --sslPEMKeyFile Conditional

    Provide the path to the PEM certificate file.

    Required if --ssl is set.

3

Copy the snapshot to each replica set member to restore.

Unmanage the Replica Set

Before attempting to restore the data manually, remove the replica set from Automation.

Restore the Replica Set Manually

Follow the tutorial from the MongoDB Manual to restore the replica set

Reimport the Replica Set

To manage the replica set with automation again, import the replica set back into Cloud Manager.