Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/backup/configuration.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "Backup configuration options"
sidebar_label: "Configuration options"
description: "Server-wide configuration keys that control backup behavior: paths, allowed destinations, concurrency, low-memory delays, and the responsible-node grace period."
sidebar_position: 85
---

Expand Down Expand Up @@ -29,6 +30,7 @@ Learn how to apply these keys in the [Configuration Overview](../server/configur
* [Backup.MaxNumberOfConcurrentBackups](../backup/configuration#backupmaxnumberofconcurrentbackups)
* [Backup.ConcurrentBackupsDelayInSec](../backup/configuration#backupconcurrentbackupsdelayinsec)
* [Backup.LowMemoryBackupDelayInMin](../backup/configuration#backuplowmemorybackupdelayinmin)
* [Backup.MoveToNewResponsibleNodeGracePeriodInMin](../backup/configuration#backupmovetonewresponsiblenodegraceperiodinmin)
* [Server.CpuCredits.ExhaustionBackupDelayInMin](../backup/configuration#servercpucreditsexhaustionbackupdelayinmin)

</Admonition>
Expand Down Expand Up @@ -158,6 +160,26 @@ Number of minutes to delay the backup if the server enters a low-memory state.

<ContentFrame>

### Backup.MoveToNewResponsibleNodeGracePeriodInMin

Number of minutes the [cluster observer](../server/clustering/distribution/cluster-observer.mdx) waits
before reassigning a backup task to a new responsible node,
when the current responsible node has entered the
[Rehab state](../server/clustering/distribution/distributed-database.mdx#database-topology).

During this grace period, the task remains assigned to its current responsible node.
This gives the node a chance to recover, and prevents a duplicate backup
from starting elsewhere in the cluster.

- **Type**: `TimeSetting`
- **TimeUnit**: `TimeUnit.Minutes`
- **Default**: `30`
- **Scope**: Server-wide only

</ContentFrame>

<ContentFrame>

### Server.CpuCredits.ExhaustionBackupDelayInMin

EXPERT: When CPU credits are exhausted, backup tasks are cancelled.
Expand Down
23 changes: 22 additions & 1 deletion docs/backup/faq.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "FAQ"
sidebar_label: "FAQ"
description: "Frequently asked questions about backup and restore in RavenDB: one-time backups, server-wide backups, retention, failure handling, and behavior in a cluster."
sidebar_position: 90
---

Expand Down Expand Up @@ -152,7 +153,27 @@ For example, you can set a backup task to keep backups locally as well as send t

### What happens if the node responsible for a backup task is down?

When the responsible node is down during the scheduled time, another node from the database group will assume ownership of the task, so there will be no gaps in the backup schedule.
When the responsible node enters the
[Rehab state](../server/clustering/distribution/distributed-database.mdx#database-topology),
the [cluster observer](../server/clustering/distribution/cluster-observer.mdx)
does not move the backup task immediately.
Instead, it waits a grace period (30 minutes by default) before reassigning the task
to another database-group member.
The grace period gives the original node a chance to recover,
and prevents the same backup from running on two nodes at once.

When you create a backup task, you can pick the responsible node yourself,
or leave it for the cluster to decide.
The choice (yours or the cluster's) is persisted cluster-wide,
so all nodes agree on a single responsible node for the task.

If the current responsible node is in Rehab due to resource constraints on the host machine,
the cluster observer will leave the backup task on the same node rather than reassign it.
The backup will run on that node once it recovers,
or be reassigned to another node if this one is eventually removed from the database group.

The grace period is configurable via
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../backup/configuration#backupmovetonewresponsiblenodegraceperiodinmin).

</ContentFrame>

Expand Down
20 changes: 20 additions & 0 deletions docs/server/clustering/distribution/highly-available-tasks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,26 @@ Learn more [here](../../../server/clustering/distribution/distributed-database.m
about database nodes' relations and states.
</Admonition>

<Admonition type="note" title="">

**Backup tasks are an exception** to the local-decision model described above.

For backup tasks, the responsible node is either picked when the task is created
(via the **Set responsible node** option in Studio, or `MentorNode` via the API),
or chosen by the [cluster observer](../../../server/clustering/distribution/cluster-observer.mdx)
when no explicit choice was made.
Either way, the choice is persisted cluster-wide,
so all nodes agree on a single responsible node.

When the current responsible node enters the `Rehab` state,
the cluster observer waits a configurable grace period (30 minutes by default)
before moving the task to another node.
This prevents the same backup from running on two nodes at once.
The grace period is set by
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../backup/configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).

</Admonition>



## Tasks Relocation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import LanguageContent from "@site/src/components/LanguageContent";
* [Does RavenDB automatically delete old backups?](../../../../client-api/operations/maintenance/backup/faq.mdx#does-ravendb-automatically-delete-old-backups)
* [Are there any locations that backup files should NOT be stored at?](../../../../client-api/operations/maintenance/backup/faq.mdx#are-there-any-locations-that-backup-files-should-not-be-stored-at)
* [What happens when a backup process fails before it is completed?](../../../../client-api/operations/maintenance/backup/faq.mdx#what-happens-when-a-backup-process-fails-before-completion)
* [What happens if the node responsible for a backup task is down?](../../../../client-api/operations/maintenance/backup/faq.mdx#what-happens-if-the-node-responsible-for-a-backup-task-is-down)

</Admonition>
## FAQ
Expand Down Expand Up @@ -108,5 +109,29 @@ While in progress, the backup content is written to an **.in-progress* file on d
If the failed process was an incremental-backup task, any future incremental backups will
continue from the correct place before the file was created so that the backup is consistent with the source.

### What happens if the node responsible for a backup task is down?

When the responsible node enters the
[Rehab state](../../../../server/clustering/distribution/distributed-database.mdx#database-topology),
the [cluster observer](../../../../server/clustering/distribution/cluster-observer.mdx)
does not move the backup task immediately.
Instead, it waits a grace period (30 minutes by default) before reassigning the task
to another database-group member.
The grace period gives the original node a chance to recover,
and prevents the same backup from running on two nodes at once.

When you create a backup task, you can pick the responsible node yourself,
or leave it for the cluster to decide.
The choice (yours or the cluster's) is persisted cluster-wide,
so all nodes agree on a single responsible node for the task.

If the current responsible node is in Rehab due to resource constraints on the host machine,
the cluster observer will leave the backup task on the same node rather than reassign it.
The backup will run on that node once it recovers,
or be reassigned to another node if this one is eventually removed from the database group.

The grace period is configurable via
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).



Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,26 @@ Learn more [here](../../../server/clustering/distribution/distributed-database.m
about database nodes' relations and states.
</Admonition>

<Admonition type="note" title="">

**Backup tasks are an exception** to the local-decision model described above.

For backup tasks, the responsible node is either picked when the task is created
(via the **Set responsible node** option in Studio, or `MentorNode` via the API),
or chosen by the [cluster observer](../../../server/clustering/distribution/cluster-observer.mdx)
when no explicit choice was made.
Either way, the choice is persisted cluster-wide,
so all nodes agree on a single responsible node.

When the current responsible node enters the `Rehab` state,
the cluster observer waits a configurable grace period (30 minutes by default)
before moving the task to another node.
This prevents the same backup from running on two nodes at once.
The grace period is set by
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).

</Admonition>



## Tasks Relocation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import LanguageContent from "@site/src/components/LanguageContent";
* [Backup.MaxNumberOfConcurrentBackups](../../server/configuration/backup-configuration.mdx#backupmaxnumberofconcurrentbackups)
* [Backup.ConcurrentBackupsDelayInSec](../../server/configuration/backup-configuration.mdx#backupconcurrentbackupsdelayinsec)
* [Backup.LowMemoryBackupDelayInMin](../../server/configuration/backup-configuration.mdx#backuplowmemorybackupdelayinmin)
* [Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin)

</Admonition>
## Backup.TempPath
Expand Down Expand Up @@ -112,3 +113,21 @@ Number of minutes to delay the backup if the server enters a low memory state.



## Backup.MoveToNewResponsibleNodeGracePeriodInMin

Number of minutes the [cluster observer](../../server/clustering/distribution/cluster-observer.mdx) waits
before reassigning a backup task to a new responsible node,
when the current responsible node has entered the
[Rehab state](../../server/clustering/distribution/distributed-database.mdx#database-topology).

During this grace period, the task remains assigned to its current responsible node.
This gives the node a chance to recover, and prevents a duplicate backup
from starting elsewhere in the cluster.

- **Type**: `TimeSetting`
- **TimeUnit**: `TimeUnit.Minutes`
- **Default**: `30`
- **Scope**: Server-wide only



Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import LanguageContent from "@site/src/components/LanguageContent";
* [Does RavenDB automatically delete old backups?](../../../../client-api/operations/maintenance/backup/faq.mdx#does-ravendb-automatically-delete-old-backups)
* [Are there any locations that backup files should NOT be stored at?](../../../../client-api/operations/maintenance/backup/faq.mdx#are-there-any-locations-that-backup-files-should-not-be-stored-at)
* [What happens when a backup process fails before it is completed?](../../../../client-api/operations/maintenance/backup/faq.mdx#what-happens-when-a-backup-process-fails-before-completion)
* [What happens if the node responsible for a backup task is down?](../../../../client-api/operations/maintenance/backup/faq.mdx#what-happens-if-the-node-responsible-for-a-backup-task-is-down)

</Admonition>
## FAQ
Expand Down Expand Up @@ -108,5 +109,29 @@ While in progress, the backup content is written to an **.in-progress* file on d
If the failed process was an incremental-backup task, any future incremental backups will
continue from the correct place before the file was created so that the backup is consistent with the source.

### What happens if the node responsible for a backup task is down?

When the responsible node enters the
[Rehab state](../../../../server/clustering/distribution/distributed-database.mdx#database-topology),
the [cluster observer](../../../../server/clustering/distribution/cluster-observer.mdx)
does not move the backup task immediately.
Instead, it waits a grace period (30 minutes by default) before reassigning the task
to another database-group member.
The grace period gives the original node a chance to recover,
and prevents the same backup from running on two nodes at once.

When you create a backup task, you can pick the responsible node yourself,
or leave it for the cluster to decide.
The choice (yours or the cluster's) is persisted cluster-wide,
so all nodes agree on a single responsible node for the task.

If the current responsible node is in Rehab due to resource constraints on the host machine,
the cluster observer will leave the backup task on the same node rather than reassign it.
The backup will run on that node once it recovers,
or be reassigned to another node if this one is eventually removed from the database group.

The grace period is configurable via
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).



Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,26 @@ Learn more [here](../../../server/clustering/distribution/distributed-database.m
about database nodes' relations and states.
</Admonition>

<Admonition type="note" title="">

**Backup tasks are an exception** to the local-decision model described above.

For backup tasks, the responsible node is either picked when the task is created
(via the **Set responsible node** option in Studio, or `MentorNode` via the API),
or chosen by the [cluster observer](../../../server/clustering/distribution/cluster-observer.mdx)
when no explicit choice was made.
Either way, the choice is persisted cluster-wide,
so all nodes agree on a single responsible node.

When the current responsible node enters the `Rehab` state,
the cluster observer waits a configurable grace period (30 minutes by default)
before moving the task to another node.
This prevents the same backup from running on two nodes at once.
The grace period is set by
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).

</Admonition>



## Tasks Relocation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import LanguageContent from "@site/src/components/LanguageContent";
* [Backup.MaxNumberOfConcurrentBackups](../../server/configuration/backup-configuration.mdx#backupmaxnumberofconcurrentbackups)
* [Backup.ConcurrentBackupsDelayInSec](../../server/configuration/backup-configuration.mdx#backupconcurrentbackupsdelayinsec)
* [Backup.LowMemoryBackupDelayInMin](../../server/configuration/backup-configuration.mdx#backuplowmemorybackupdelayinmin)
* [Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin)

</Admonition>
## Backup.TempPath
Expand Down Expand Up @@ -112,3 +113,21 @@ Number of minutes to delay the backup if the server enters a low memory state.



## Backup.MoveToNewResponsibleNodeGracePeriodInMin

Number of minutes the [cluster observer](../../server/clustering/distribution/cluster-observer.mdx) waits
before reassigning a backup task to a new responsible node,
when the current responsible node has entered the
[Rehab state](../../server/clustering/distribution/distributed-database.mdx#database-topology).

During this grace period, the task remains assigned to its current responsible node.
This gives the node a chance to recover, and prevents a duplicate backup
from starting elsewhere in the cluster.

- **Type**: `TimeSetting`
- **TimeUnit**: `TimeUnit.Minutes`
- **Default**: `30`
- **Scope**: Server-wide only



Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import LanguageContent from "@site/src/components/LanguageContent";
* [Does RavenDB automatically delete old backups?](../../../../client-api/operations/maintenance/backup/faq.mdx#does-ravendb-automatically-delete-old-backups)
* [Are there any locations that backup files should NOT be stored at?](../../../../client-api/operations/maintenance/backup/faq.mdx#are-there-any-locations-that-backup-files-should-not-be-stored-at)
* [What happens when a backup process fails before it is completed?](../../../../client-api/operations/maintenance/backup/faq.mdx#what-happens-when-a-backup-process-fails-before-completion)
* [What happens if the node responsible for a backup task is down?](../../../../client-api/operations/maintenance/backup/faq.mdx#what-happens-if-the-node-responsible-for-a-backup-task-is-down)

</Admonition>
## FAQ
Expand Down Expand Up @@ -108,5 +109,29 @@ While in progress, the backup content is written to an **.in-progress* file on d
If the failed process was an incremental-backup task, any future incremental backups will
continue from the correct place before the file was created so that the backup is consistent with the source.

### What happens if the node responsible for a backup task is down?

When the responsible node enters the
[Rehab state](../../../../server/clustering/distribution/distributed-database.mdx#database-topology),
the [cluster observer](../../../../server/clustering/distribution/cluster-observer.mdx)
does not move the backup task immediately.
Instead, it waits a grace period (30 minutes by default) before reassigning the task
to another database-group member.
The grace period gives the original node a chance to recover,
and prevents the same backup from running on two nodes at once.

When you create a backup task, you can pick the responsible node yourself,
or leave it for the cluster to decide.
The choice (yours or the cluster's) is persisted cluster-wide,
so all nodes agree on a single responsible node for the task.

If the current responsible node is in Rehab due to resource constraints on the host machine,
the cluster observer will leave the backup task on the same node rather than reassign it.
The backup will run on that node once it recovers,
or be reassigned to another node if this one is eventually removed from the database group.

The grace period is configurable via
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).



Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,26 @@ Learn more [here](../../../server/clustering/distribution/distributed-database.m
about database nodes' relations and states.
</Admonition>

<Admonition type="note" title="">

**Backup tasks are an exception** to the local-decision model described above.

For backup tasks, the responsible node is either picked when the task is created
(via the **Set responsible node** option in Studio, or `MentorNode` via the API),
or chosen by the [cluster observer](../../../server/clustering/distribution/cluster-observer.mdx)
when no explicit choice was made.
Either way, the choice is persisted cluster-wide,
so all nodes agree on a single responsible node.

When the current responsible node enters the `Rehab` state,
the cluster observer waits a configurable grace period (30 minutes by default)
before moving the task to another node.
This prevents the same backup from running on two nodes at once.
The grace period is set by
[Backup.MoveToNewResponsibleNodeGracePeriodInMin](../../../server/configuration/backup-configuration.mdx#backupmovetonewresponsiblenodegraceperiodinmin).

</Admonition>



## Tasks Relocation
Expand Down
Loading
Loading