Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 25 additions & 17 deletions docs/usage/oplog-replay.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
# Replay oplog from arbitrary start time

You can replay the [oplog](../reference/glossary.md#oplog) for a specific period on top of any backup: logical, physical, storage level snapshot (like [EBS-snapshot](../reference/glossary.md#ebs-snapshot)). You can save oplog slices without the mandatory base backup snapshot. This behavior is controlled by the [`pitr.oplogOnly`](../reference/pitr-options.md) configuration parameter:
You can have PBM save oplog slices without the mandatory base backup snapshot. This behavior is controlled by the [`pitr.oplogOnly`](../reference/pitr-options.md) configuration parameter:

```yaml
pitr:
oplogOnly: true
```

By replaying these oplog slices on top of the backup snapshot with the [`pbm oplog-replay`](../reference/pbm-commands.md#pbm-oplog-replay) command, you can manually restore sharded clusters and non-sharded replica sets to a specific point in time from a backup made by any tool and not only by Percona Backup for MongoDB. Plus, you reduce time, storage space, and administration efforts on making the redundant base backup snapshot.
You can then replay the [oplog](../reference/glossary.md#oplog) for a specific period on top of any backup: logical, physical, storage-level snapshot (like [EBS-snapshot](../reference/glossary.md#ebs-snapshot)).

By replaying these oplog slices on top of a backup snapshot with the [`pbm oplog-replay`](../reference/pbm-commands.md#pbm-oplog-replay) command, you can manually restore sharded clusters and non-sharded replica sets to a specific point in time from a backup made by any tool and not only by Percona Backup for MongoDB. Plus, you reduce time, storage space, and administration efforts on making the redundant base backup snapshot.

Check warning on line 12 in docs/usage/oplog-replay.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/usage/oplog-replay.md#L12

[Vale.Spelling] Did you really mean 'sharded'?
Raw output
{"message": "[Vale.Spelling] Did you really mean 'sharded'?", "location": {"path": "docs/usage/oplog-replay.md", "range": {"start": {"line": 12, "column": 172}}}, "severity": "WARNING"}

!!! warning

Use the oplog replay functionality with caution, only when you are sure about the starting time from which to replay oplog. The oplog replay does not guarantee data consistency when restoring from any backup. However, it is less error-prone for backups made with Percona Backup for MongoDB.
Use the oplog replay functionality with caution, only when you are sure about the starting time from which to replay oplog.

## Ways to specify time for oplog replay

Expand All @@ -29,11 +31,11 @@

!!! note ""

Starting with version 2.2.0, oplog replay on top of a physical backups made with Percona Backup for MongoDB is done automatically as part of [point-in-time recovery](pitr-physical.md).
Starting with version 2.2.0, oplog replay on top of physical backups made with Percona Backup for MongoDB is done automatically as part of [point-in-time recovery](pitr-physical.md).

This section describes how to **manually** replay oplog on top of physical backups made with Percona Backup for MongoDB version 2.1.0 and earlier.
This section describes how to **manually** replay oplog on top of physical backups. This is useful for backups made with Percona Backup for MongoDB version 2.1.0 and earlier, where oplog apply was not automatically done by PBM.

After you [restored a physical backup](restore.md), do the following:
After you [restore a physical backup](restore-physical.md), do the following:

1. Stop point-in-time recovery, if enabled, to release the lock.

Expand All @@ -57,24 +59,30 @@

4. After the oplog replay, make a fresh backup and enable the point-in-time recovery oplog slicing.

## Oplog replay for storage level snapshots
## Oplog replay for storage-level snapshots

This section describes how to **manually** replay oplog on top of snapshot backups.

When making a backup, Percona Backup for MongoDB stops the point-in-time recovery. This is done to maintain data consistency after the restore.
After you [restore a snapshot backup](restore-external.md), do the following:

Storage-level snapshots are saved with point-in-time recovery enabled. Thus, after the database restore from such a backup, point-in-time recovery is automatically enabled and starts oplog slicing. These new oplog slices might conflict with the existing oplogs saved during the backup. To replay the oplog in such a case, do the following after the restore:
1. Start all `mongod` nodes

2. Start all `pbm-agents`

1. Disable point-in-time recovery.
2. Delete the oplog slices that might have been created.
3. Re-sync the data from the storage.
4. Run the `pbm oplog-replay` command and specify the `--start` and `--end` flags with the timestamps.
3. Run the following command to resync the backup list with the storage:

```bash
pbm oplog-replay --start="2022-01-02T15:00:00" --end="2022-01-03T15:00:00"
```
pbm config --force-resync
```

5. After the oplog replay, make a fresh backup and enable the point-in-time recovery oplog slicing.
4. Run the `pbm oplog-replay` command and specify the `--start` and `--end` flags. Use the `last_write_time` from the output of `pbm describe-backup` as the start. See [how you can specify the time](#ways-to-specify-time-for-oplog-replay).

[Known limitations](../features/known-limitations.md#oplog-replay-from-arbitrary-start-time){.md-button}
```bash
pbm oplog-replay --start="2026-01-02T15:00:00" --end="2026-01-03T15:00:00"
```

5. Start the balancer and start `mongos` nodes.

6. Make a fresh backup to serve as the new base for future restores.

[Known limitations](../features/known-limitations.md#oplog-replay-from-arbitrary-start-time){.md-button}
14 changes: 10 additions & 4 deletions docs/usage/restore-external.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
pbm restore-finish <restore_name> -c </path/to/pbm-config.yaml>
```

At this stage, Percona Backup for MongoDB reads the metadata from the backup, prepares the data for the cluster / replica set start and ensures its consistency. The database is restored to the timestamp specified in the `restore_to_time` of the metadata.
At this stage, Percona Backup for MongoDB reads the metadata from the backup, prepares the data for the cluster / replica set start and ensures its consistency. The database is restored to the timestamp specified in the `last_write_time` of the metadata.

!!! note

Expand All @@ -67,9 +67,15 @@
pbm config --force-resync
```

4. Start the balancer and start `mongos` nodes.
4. Optional. Perform point in time recovery by running [`pbm oplog-replay`](./oplog-replay.md#oplog-replay-for-storage-level-snapshots) using the `last_write_time` as the start, for example:

5. Make a fresh backup to serve as the new base for future restores.
```bash
pbm oplog-replay --start="2026-01-02T15:00:00" --end="2026-01-03T15:00:00"
```

5. Start the balancer and start `mongos` nodes.

6. Make a fresh backup to serve as the new base for future restores.

### Restore from a backup made outside PBM

Expand All @@ -83,7 +89,7 @@

To restore an external backup made without PBM, you need to specify the following for the `pbm restore` command:

* a path to the configuration file of the `mongod` node on the source cluster from where the backup was made. This is the configuration file that PBM will use during the restore. It should contain the [storage options :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/configuration-options/#storage-options ) per replica set name, for example:
* a path to the configuration file of the `mongod` node on the source cluster from where the backup was made. This is the configuration file that PBM will use during the restore. It should contain the [storage options :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/configuration-options/#storage-options) per replica set name, for example:

Check warning on line 92 in docs/usage/restore-external.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/usage/restore-external.md#L92

[Google.Will] Avoid using 'will'.
Raw output
{"message": "[Google.Will] Avoid using 'will'.", "location": {"path": "docs/usage/restore-external.md", "range": {"start": {"line": 92, "column": 151}}}, "severity": "WARNING"}

```yaml
rs1:
Expand Down
Loading