Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions src/current/_includes/v23.2/backward-incompatible/alpha.1.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
- CockroachDB no longer performs environment variable expansion in the parameter `--certs-dir`. Uses like `--certs-dir='$HOME/path'` (expansion by CockroachDB) can be replaced by `--certs-dir="$HOME/path"` (expansion by the Unix shell). [#81298][#81298]
- In the Cockroach CLI, [`BOOL` values](../v23.1/bool.html) are now formatted as `t` or `f` instead of `True` or `False`. [#81943][#81943]
- Removed the `cockroach quit` command. It has been deprecated since v20.1. To [shut down a node](../v23.1/node-shutdown.html) gracefully, send a `SIGTERM` signal to it. [#82988][#82988]
- Added a cluster version to allow the [Pebble storage engine](../v23.1/architecture/storage-layer.html#pebble) to recombine certain SSTables (specifically, user keys that are split across multiple files in a level of the [log-structured merge-tree](../v23.1/architecture/storage-layer.html#log-structured-merge-trees)). Recombining the split user keys is required for supporting the range keys feature. The migration to recombine the SSTables is expected to be short (split user keys are rare in practice), but will block subsequent migrations until all tables have been recombined. The `storage.marked-for-compaction-files` time series metric can show the progress of the migration. [#84887][#84887]
- In the Cockroach CLI, [`BOOL` values]({% link {{ page.version.version }}/bool.md %}) are now formatted as `t` or `f` instead of `True` or `False`. [#81943][#81943]
- Removed the `cockroach quit` command. It has been deprecated since v20.1. To [shut down a node]({% link {{ page.version.version }}/node-shutdown.md %}) gracefully, send a `SIGTERM` signal to it. [#82988][#82988]
- Added a cluster version to allow the [Pebble storage engine]({% link {{ page.version.version }}/architecture/storage-layer.md %}#pebble) to recombine certain SSTables (specifically, user keys that are split across multiple files in a level of the [log-structured merge-tree]({% link {{ page.version.version }}/architecture/storage-layer.md %}#log-structured-merge-trees)). Recombining the split user keys is required for supporting the range keys feature. The migration to recombine the SSTables is expected to be short (split user keys are rare in practice), but will block subsequent migrations until all tables have been recombined. The `storage.marked-for-compaction-files` time series metric can show the progress of the migration. [#84887][#84887]
- Using a single TCP port listener for both RPC (node-node) and SQL client connections is now deprecated. This capability **will be removed** in the next version of CockroachDB. Instead, make one of the following configuration changes to your CockroachDB deployment:
- Preferred: keep port `26257` for SQL, and allocate a new port, e.g., `26357`, for node-node RPC connections. For example, you might configure a node with the flags `--listen-addr=:26357 --sql-addr=:26257`, where subsequent nodes seeking to join would then use the flag `--join=othernode:26357,othernode:26257`. This will become the default configuration in the next version of CockroachDB. When using this mode of operation, care should be taken to use a `--join` flag that includes both the previous and new port numbers for other nodes, so that no network partition occurs during the upgrade.
- Optional: keep port `26257` for RPC, and allocate a new port, e.g., `26357`, for SQL connections. For example, you might configure a node with the flags `--listen-addr=:26257 --sql-addr=:26357`. When using this mode of operation, the `--join` flags do not need to be modified. However, SQL client apps or the SQL load balancer configuration (when in use) must be updated to use the new SQL port number. [#85671][#85671]
- If no `nullif` option is specified while using [`IMPORT CSV`](../v23.1/import.html), then a zero-length string in the input is now treated as `NULL`. The quoted empty string in the input is treated as an empty string. Similarly, if `nullif` is specified, then an unquoted value is treated as `NULL`, and a quoted value is treated as that string. These changes were made to make `IMPORT CSV` behave more similarly to `COPY CSV`. If the previous behavior (i.e., treating either quoted or unquoted values that match the `nullif` setting as `NULL`) is desired, you can use the new `allow_quoted_null` option in the `IMPORT` statement. [#84487][#84487]
- [`COPY FROM`](../v23.1/copy.html) operations are now atomic by default instead of being segmented into 100 row transactions. Set the `copy_from_atomic_enabled` session setting to `false` for the previous behavior. [#85986][#85986]
- If no `nullif` option is specified while using [`IMPORT CSV`]({% link {{ page.version.version }}/import.md %}), then a zero-length string in the input is now treated as `NULL`. The quoted empty string in the input is treated as an empty string. Similarly, if `nullif` is specified, then an unquoted value is treated as `NULL`, and a quoted value is treated as that string. These changes were made to make `IMPORT CSV` behave more similarly to `COPY CSV`. If the previous behavior (i.e., treating either quoted or unquoted values that match the `nullif` setting as `NULL`) is desired, you can use the new `allow_quoted_null` option in the `IMPORT` statement. [#84487][#84487]
- [`COPY FROM`]({% link {{ page.version.version }}/copy.md %}) operations are now atomic by default instead of being segmented into 100 row transactions. Set the `copy_from_atomic_enabled` session setting to `false` for the previous behavior. [#85986][#85986]
- The `GRANT` privilege has been removed and replaced by the more granular [`WITH GRANT OPTION`]({% link v23.2/grant.md %}#grant-privileges-with-the-option-to-grant-to-others), which provides control over which privileges are allowed to be granted. [#81310][#81310]
- Removed the ability to cast `int`, `int2`, and `int8` to a `0` length `BIT` or `VARBIT`. [#81266][#81266]
- Removed the deprecated `GRANT` privilege. [#81310][#81310]
Expand Down
8 changes: 4 additions & 4 deletions src/current/v23.2/architecture/reads-and-writes-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ First, imagine a simple read scenario where:
- Ranges are replicated 3 times (the default).
- A query is executed against node 2 to read from table 3.

<img src="{{ 'images/v23.2/perf_tuning_concepts1.png' | relative_url }}" alt="Perf tuning concepts" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/perf_tuning_concepts1.png" alt="Perf tuning concepts" style="max-width:100%" />

In this case:

Expand All @@ -41,13 +41,13 @@ In this case:

If the query is received by the node that has the leaseholder for the relevant range, there are fewer network hops:

<img src="{{ 'images/v23.2/perf_tuning_concepts2.png' | relative_url }}" alt="Perf tuning concepts" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/perf_tuning_concepts2.png" alt="Perf tuning concepts" style="max-width:100%" />

## Write scenario

Now imagine a simple write scenario where a query is executed against node 3 to write to table 1:

<img src="{{ 'images/v23.2/perf_tuning_concepts3.png' | relative_url }}" alt="Perf tuning concepts" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/perf_tuning_concepts3.png" alt="Perf tuning concepts" style="max-width:100%" />

In this case:

Expand All @@ -60,7 +60,7 @@ In this case:

Just as in the read scenario, if the write request is received by the node that has the leaseholder and Raft leader for the relevant range, there are fewer network hops:

<img src="{{ 'images/v23.2/perf_tuning_concepts4.png' | relative_url }}" alt="Perf tuning concepts" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/perf_tuning_concepts4.png" alt="Perf tuning concepts" style="max-width:100%" />

## Network and I/O bottlenecks

Expand Down
6 changes: 3 additions & 3 deletions src/current/v23.2/architecture/storage-layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Pebble uses a Log-structured Merge-tree (hereafter _LSM tree_ or _LSM_) to manag

SSTs are an on-disk representation of sorted lists of key-value pairs. Conceptually, they look something like this (intentionally simplified) diagram:

<img src="{{ 'images/v23.2/sst.png' | relative_url }}" alt="Structure of an SST file" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/sst.png" alt="Structure of an SST file" style="max-width:100%" />

SST files are immutable; they are never modified, even during the [compaction process](#compaction).

Expand All @@ -78,7 +78,7 @@ The SSTs within each level are guaranteed to be non-overlapping: for example, if
- To allow LSM-based storage engines like Pebble to support ingesting large amounts of data, such as when using the [`IMPORT`]({% link {{ page.version.version }}/import.md %}) statement.
- To allow for easier and more efficient flushes of [memtables](#memtable-and-write-ahead-log).

<img src="{{ 'images/v23.2/lsm-with-ssts.png' | relative_url }}" alt="LSM tree with SST files" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/lsm-with-ssts.png" alt="LSM tree with SST files" style="max-width:100%" />

##### Compaction

Expand Down Expand Up @@ -128,7 +128,7 @@ Another file on disk called the write-ahead log (hereafter _WAL_) is associated

The relationship between the memtable, the WAL, and the SST files is shown in the diagram below. New values are written to the WAL at the same time as they are written to the memtable. From the memtable they are eventually written to SST files on disk for longer-term storage.

<img src="{{ 'images/v23.2/memtable-wal-sst.png' | relative_url }}" alt="Relationship between memtable, WAL, and SSTs" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/memtable-wal-sst.png" alt="Relationship between memtable, WAL, and SSTs" style="max-width:100%" />

##### LSM design tradeoffs

Expand Down
22 changes: 11 additions & 11 deletions src/current/v23.2/aws-dms.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ As of publishing, AWS DMS supports migrations from these relational databases (f
1. In the AWS Console, open **AWS DMS**.
1. Open **Endpoints** in the sidebar. A list of endpoints will display, if any exist.
1. In the top-right portion of the window, select **Create endpoint**.
<img src="{{ 'images/v23.2/aws-dms-create-endpoint.png' | relative_url }}" alt="AWS-DMS-Create-Endpoint" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-create-endpoint.png" alt="AWS-DMS-Create-Endpoint" style="max-width:100%" />

A configuration page will open.
1. In the **Endpoint type** section, select **Target endpoint**.
Expand All @@ -107,10 +107,10 @@ As of publishing, AWS DMS supports migrations from these relational databases (f
{{site.data.alerts.callout_info}}
To connect to a CockroachDB {{ site.data.products.standard }} or {{ site.data.products.basic }} cluster, set the **Database name** to `{host}.{database}`. For details on how to find these parameters, see [Connect to your cluster]({% link cockroachcloud/connect-to-your-cluster.md %}?filters=connection-parameters#connect-to-your-cluster). Also set **Secure Socket Layer (SSL) mode** to **require**.
{{site.data.alerts.end}}
<img src="{{ 'images/v23.2/aws-dms-endpoint-configuration.png' | relative_url }}" alt="AWS-DMS-Endpoint-Configuration" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-endpoint-configuration.png" alt="AWS-DMS-Endpoint-Configuration" style="max-width:100%" />
1. If needed, you can test the connection under **Test endpoint connection (optional)**.
1. To create the endpoint, select **Create endpoint**.
<img src="{{ 'images/v23.2/aws-dms-test-endpoint.png' | relative_url }}" alt="AWS-DMS-Test-Endpoint" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-test-endpoint.png" alt="AWS-DMS-Test-Endpoint" style="max-width:100%" />

## Step 2. Create a database migration task

Expand All @@ -124,7 +124,7 @@ To conserve CPU, consider migrating tables in multiple replication tasks, rather

1. While in **AWS DMS**, select **Database migration tasks** in the sidebar. A list of database migration tasks will display, if any exist.
1. In the top-right portion of the window, select **Create task**.
<img src="{{ 'images/v23.2/aws-dms-create-db-migration-task.png' | relative_url }}" alt="AWS-DMS-Create-DB-Migration-Task" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-create-db-migration-task.png" alt="AWS-DMS-Create-DB-Migration-Task" style="max-width:100%" />

A configuration page will open.
1. Supply a **Task identifier** to identify the replication task.
Expand All @@ -135,17 +135,17 @@ To conserve CPU, consider migrating tables in multiple replication tasks, rather
{{site.data.alerts.callout_danger}}
If you choose **Migrate existing data and replicate ongoing changes** or **Replicate data changes only**, you must first [disable revision history for backups](#setup).
{{site.data.alerts.end}}
<img src="{{ 'images/v23.2/aws-dms-task-configuration.png' | relative_url }}" alt="AWS-DMS-Task-Configuration" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-task-configuration.png" alt="AWS-DMS-Task-Configuration" style="max-width:100%" />

### Step 2.2. Task settings

1. For the **Editing mode** radio button, keep **Wizard** selected.
1. To preserve the schema you manually created, select **Truncate** or **Do nothing** for the **Target table preparation mode**.
<img src="{{ 'images/v23.2/aws-dms-task-settings.png' | relative_url }}" alt="AWS-DMS-Task-Settings" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-task-settings.png" alt="AWS-DMS-Task-Settings" style="max-width:100%" />
1. Optionally check **Enable validation** to compare the data in the source and target rows, and verify that the migration succeeded. You can view the results in the [**Table statistics**](#step-3-verify-the-migration) for your migration task. For more information about data validation, see the [AWS documentation](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html).
1. Check the **Enable CloudWatch logs** option. We highly recommend this for troubleshooting potential migration issues.
1. For the **Target Load**, select **Detailed debug**.
<img src="{{ 'images/v23.2/aws-dms-cloudwatch-logs.png' | relative_url }}" alt="AWS-DMS-CloudWatch-Logs" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-cloudwatch-logs.png" alt="AWS-DMS-CloudWatch-Logs" style="max-width:100%" />

### Step 2.3. Table mappings

Expand All @@ -161,7 +161,7 @@ When specifying a range of tables to migrate, the following aspects of the sourc
1. Select **Add new selection rule**.
1. In the **Schema** dropdown, select **Enter a schema**.
1. Supply the appropriate **Source name** (schema name), **Table name**, and **Action**.
<img src="{{ 'images/v23.2/aws-dms-table-mappings.png' | relative_url }}" alt="AWS-DMS-Table-Mappings" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-table-mappings.png" alt="AWS-DMS-Table-Mappings" style="max-width:100%" />

{{site.data.alerts.callout_info}}
Use `%` as an example of a wildcard for all schemas in a PostgreSQL database. However, in MySQL, using `%` as a schema name imports all the databases, including the metadata/system ones, as MySQL treats schemas and databases as the same.
Expand Down Expand Up @@ -205,7 +205,7 @@ If your migration succeeded, you should now:

If your migration failed for some reason, you can check the checkbox next to the table(s) you wish to re-migrate and select **Reload table data**.

<img src="{{ 'images/v23.2/aws-dms-reload-table-data.png' | relative_url }}" alt="AWS-DMS-Reload-Table-Data" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-reload-table-data.png" alt="AWS-DMS-Reload-Table-Data" style="max-width:100%" />

## Optional configurations

Expand All @@ -221,7 +221,7 @@ The `BatchApplyEnabled` setting can improve replication performance and is recom
1. Choose your task, and then choose **Modify**.
1. From the **Task settings** section, switch the **Editing mode** from **Wizard** to **JSON editor**. Locate the `BatchApplyEnabled` setting and change its value to `true`. Information about the `BatchApplyEnabled` setting can be found [here](https://aws.amazon.com/premiumsupport/knowledge-center/dms-batch-apply-cdc-replication/).

<img src="{{ 'images/v23.2/aws-dms-batchapplyenabled.png' | relative_url }}" alt="AWS-DMS-BatchApplyEnabled" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-batchapplyenabled.png" alt="AWS-DMS-BatchApplyEnabled" style="max-width:100%" />

{{site.data.alerts.callout_info}}
`BatchApplyEnabled` does not work when using **Drop tables on target** as a target table preparation mode. Thus, all schema-related changes must be manually copied over if using `BatchApplyEnabled`.
Expand All @@ -244,7 +244,7 @@ The `BatchApplyEnabled` setting can improve replication performance and is recom

To prevent this error, set the [`expect_and_ignore_not_visible_columns_in_copy` session variable]({% link {{ page.version.version }}/session-variables.md %}#expect-and-ignore-not-visible-columns-in-copy) in the DMS [target endpoint configuration](#step-1-create-a-target-endpoint-pointing-to-cockroachdb). Under **Endpoint settings**, add an **AfterConnectScript** setting with the value `SET expect_and_ignore_not_visible_columns_in_copy=on`.

<img src="{{ 'images/v23.2/aws-dms-endpoint-settings.png' | relative_url }}" alt="AWS-DMS-Endpoint-Settings" style="max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/aws-dms-endpoint-settings.png" alt="AWS-DMS-Endpoint-Settings" style="max-width:100%" />

## Troubleshooting common issues

Expand Down
6 changes: 3 additions & 3 deletions src/current/v23.2/backup-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ At a high level, CockroachDB performs the following tasks when running a backup

The following diagram illustrates the flow from `BACKUP` statement through to a complete backup in cloud storage:

<img src="{{ 'images/v23.2/backup-overview.png' | relative_url }}" alt="A flow diagram representing the process of a backup job from statement through to backup data stored." style="border:0px solid #eee;max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/backup-overview.png" alt="A flow diagram representing the process of a backup job from statement through to backup data stored." style="border:0px solid #eee;max-width:100%" />

## Job creation phase

Expand Down Expand Up @@ -85,7 +85,7 @@ In the following diagram, nodes that contain replicas of the relevant key spans

While processing, the nodes emit progress data that tracks their backup work to the coordinator. In the diagram, **Node 3** and **Node 1** will send progress data to **Node 2**. The coordinator node will then aggregate the progress data into checkpoint files in the storage bucket. The checkpoint files provide a marker for the backup to resume after a retryable state, such as when it has been [paused]({% link {{ page.version.version }}/pause-job.md %}).

<img src="{{ 'images/v23.2/backup-processing.png' | relative_url }}" alt="Three-node cluster exporting backup data from the leaseholders" style="border:0px solid #eee;max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/backup-processing.png" alt="Three-node cluster exporting backup data from the leaseholders" style="border:0px solid #eee;max-width:100%" />

## Metadata writing phase

Expand Down Expand Up @@ -114,7 +114,7 @@ For example, in the following diagram there is a three-node cluster split across

During a [restore]({% link {{ page.version.version }}/restore.md %}) job, the job creation statement will need access to each of the storage locations to read the metadata files in order to complete a successful restore.

<img src="{{ 'images/v23.2/locality-aware-backups.png' | relative_url }}" alt="How a locality-aware backup writes to storage buckets in each region" style="border:0px solid #eee;max-width:100%" />
<img src="/docs/images/{{ page.version.version }}/locality-aware-backups.png" alt="How a locality-aware backup writes to storage buckets in each region" style="border:0px solid #eee;max-width:100%" />

#### Job coordination on Standard and Basic clusters

Expand Down
Loading
Loading