Skip to content

Add explanation for larger indexes after migration #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
igroene wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add explanation for larger indexes after migration #53

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/faq.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,25 @@
## What database read and write concerns are used?
By default, Percona ClusterSync for MongoDB uses the `"majority"` read concern level for reads on the source cluster. For writes to the destination cluster, the tool uses a write concern level of `"majority"` with `j: true`.

## Why are indexes larger after migrating data with Percona ClusterSync?

In some cases, indexes on the destination cluster may become significantly larger after migration. This is usually caused by differences in physical storage layout rather than logical differences in the data or indexes.

Possible causes include:

- Different WiredTiger compression settings between source and destination (for example `zstd` vs `snappy`)

Check notice on line 63 in docs/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/faq.md#L63 

[Google.Parens] Use parentheses judiciously.
Raw output 
{"message": "[Google.Parens] Use parentheses judiciously.", "location": {"path": "docs/faq.md", "range": {"start": {"line": 63, "column": 1}}}, "severity": "INFO"}
- Percona ClusterSync performs inserts, updates, and deletes during synchronization, which can fragment collections and indexes
- Percona ClusterSync copies documents in `_id` order; if secondary indexes are not correlated with `_id`, index pages may become inefficiently organized, causing additional B-tree page splits and larger indexes

Check notice on line 65 in docs/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/faq.md#L65 

[Google.Contractions] Use 'aren't' instead of 'are not'.
Raw output 
{"message": "[Google.Contractions] Use 'aren't' instead of 'are not'.", "location": {"path": "docs/faq.md", "range": {"start": {"line": 65, "column": 77}}}, "severity": "INFO"}
- Different MongoDB or WiredTiger versions
- Different prefix compression settings

Potential mitigation strategies include:

- Rebuilding indexes after migration (`reIndex` or drop/recreate indexes)

Check notice on line 71 in docs/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/faq.md#L71 

[Google.Parens] Use parentheses judiciously.
Raw output 
{"message": "[Google.Parens] Use parentheses judiciously.", "location": {"path": "docs/faq.md", "range": {"start": {"line": 71, "column": 1}}}, "severity": "INFO"}
- Running `compact`
- Performing a rolling initial sync after the data migration is complete

Larger index files do not necessarily indicate data inconsistency or corruption, and query performance may remain unchanged.

Check notice on line 75 in docs/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/faq.md#L75 

[Google.Contractions] Use 'don't' instead of 'do not'.
Raw output 
{"message": "[Google.Contractions] Use 'don't' instead of 'do not'.", "location": {"path": "docs/faq.md", "range": {"start": {"line": 75, "column": 20}}}, "severity": "INFO"}

## What features are planned for future releases?

Expand Down
Loading