Multiple passes for creating an archive

[drws]

When Borg is used to backup a live/changing dataset, and it takes some time to finish, it is a good practice to do another run. The second one finishes much quicker, only filling in a few recent changes, and the resulting backed-up data is much more coherent. This produces two similar archives.

Borg could be able to do two passes in a single create run, producing only a single archive. One possibility is to write a .checkpoint archive at the end of the first pass and then finish up in the second one.

[infectormp]

From docs...

Backup speed is increased by not reprocessing files that are already part of existing archives and weren’t modified.

That's why second run finishes much quicker.

Each repository can hold multiple archives, which represent individual backups that contain a full archive of the files specified when the backup was performed.

So, this is not two similar archives.

[ThomasWaldmann]

@drws Valid observation.

But by doing N passes, one is just trying to catch up with a changing filesystem. How successful that is depends on the rate of changes also.

So the real solution is not trying to catch up, but to back up a fs snapshot (which is supported by modern filesystems).