Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions _site/redirects.json
Original file line number Diff line number Diff line change
Expand Up @@ -2678,5 +2678,17 @@
{
"source": "/sql-reference/formats",
"destination": "/core/reference/formats"
},
{
"source": "/integrations/pg_clickhouse",
"destination": "/cloud/managed-postgres/extensions/pg_clickhouse"
},
{
"source": "/integrations/pg_clickhouse/tutorial",
"destination": "/cloud/managed-postgres/extensions/pg_clickhouse/tutorial"
},
{
"source": "/integrations/pg_clickhouse/reference",
"destination": "/cloud/managed-postgres/extensions/pg_clickhouse/reference"
}
]
11 changes: 1 addition & 10 deletions integrations/connectors/navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -214,16 +214,7 @@
"integrations/connectors/data-integrations/integrations/retool",
"integrations/connectors/data-integrations/integrations/easypanel",
"integrations/connectors/data-integrations/integrations/middleware",
"integrations/connectors/data-integrations/integrations/splunk",
{
"group": "pg_clickhouse",
"expanded": false,
"pages": [
"integrations/connectors/tools/pg_clickhouse/introduction",
"integrations/connectors/tools/pg_clickhouse/reference",
"integrations/connectors/tools/pg_clickhouse/tutorial"
]
}
"integrations/connectors/data-integrations/integrations/splunk"
]
},
{
Expand Down
10 changes: 5 additions & 5 deletions integrations/connectors/tools/pg_clickhouse/introduction.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebarTitle: 'Introduction'
description: 'Run analytics queries on ClickHouse right from PostgreSQL without rewriting any SQL'
slug: '/integrations/pg_clickhouse'
slug: '/cloud/managed-postgres/extensions/pg_clickhouse'
title: 'pg_clickhouse reference documentation'
doc_type: 'landing-page'
keywords: ['PostgreSQL', 'Postgres', 'FDW', 'foreign data wrapper', 'pg_clickhouse', 'extension']
Expand All @@ -19,7 +19,7 @@ PostgreSQL schema. Then run your existing PostgreSQL queries to against those
tables, preserving your existing code base while pushing execution down to
ClickHouse.

## Getting Started {#getting-started}
## Getting started {#getting-started}

The simplest way to try pg_clickhouse is the [Docker image], which contains
the standard PostgreSQL Docker image with the pg_clickhouse and [re2]
Expand All @@ -34,7 +34,7 @@ docker exec -it pg_clickhouse psql -U postgres -c 'CREATE EXTENSION pg_clickhous
See the [tutorial] to get started importing ClickHouse tables and pushing down
queries.

## Test Case: TPC-H {#test-case-tpc-h}
## Test case: TPC-H {#test-case-tpc-h}

This table compares [TPC-H] query performance between regular PostgreSQL
tables and pg_clickhouse connected to ClickHouse, both loaded at scaling
Expand Down Expand Up @@ -67,7 +67,7 @@ memory.
| [Query 21] | 1349 ms | 4434 ms | |
| [Query 22] | 258 ms | 1415 ms | |

### Compile From Source {#compile-from-source}
### Compile from source {#compile-from-source}

#### General Unix {#general-unix}

Expand Down Expand Up @@ -106,7 +106,7 @@ sudo yum install \

See [PostgreSQL Yum] for details on pulling from the PostgreSQL Yum repository.

#### Install From PGXN {#install-from-pgxn}
#### Install from PGXN {#install-from-pgxn}

With the above dependencies satisfied use the [PGXN client] (available as
[Homebrew], [Apt] and Yum packages named `pgxnclient`) to download, compile,
Expand Down
16 changes: 8 additions & 8 deletions integrations/connectors/tools/pg_clickhouse/reference.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebarTitle: 'Reference'
description: 'Complete reference documentation for pg_clickhouse'
slug: '/integrations/pg_clickhouse/reference'
slug: '/cloud/managed-postgres/extensions/pg_clickhouse/reference'
title: 'pg_clickhouse reference documentation'
doc_type: 'reference'
keywords: ['PostgreSQL', 'Postgres', 'FDW', 'foreign data wrapper', 'pg_clickhouse', 'extension']
Expand All @@ -13,7 +13,7 @@ pg_clickhouse is a PostgreSQL extension that enables remote query execution
on ClickHouse databases, including a [foreign data wrapper]. It supports
PostgreSQL 13 and higher and ClickHouse 23 and higher.

## Getting Started {#getting-started}
## Getting started {#getting-started}

The simplest way to try pg_clickhouse is the [Docker image], which contains
the standard PostgreSQL Docker image with the pg_clickhouse and [re2][re2
Expand All @@ -40,7 +40,7 @@ CREATE SCHEMA taxi;
IMPORT FOREIGN SCHEMA taxi FROM SERVER taxi_srv INTO taxi;
```

## Versioning Policy {#versioning-policy}
## Versioning policy {#versioning-policy}

pg_clickhouse adheres to [Semantic Versioning] for its public releases.

Expand Down Expand Up @@ -68,7 +68,7 @@ be accompanied by SQL upgrade scripts, and all existing database that contain
the extension must run `ALTER EXTENSION pg_clickhouse UPDATE` to benefit from
the upgrade.

## DDL SQL Reference {#ddl-sql-reference}
## DDL SQL reference {#ddl-sql-reference}

The following SQL [DDL] expressions use pg_clickhouse.

Expand Down Expand Up @@ -372,7 +372,7 @@ Use the `CASCADE` clause to drop them, too:
DROP FOREIGN TABLE acts CASCADE;
```

## DML SQL Reference {#dml-sql-reference}
## DML SQL reference {#dml-sql-reference}

The SQL [DML] expressions below may use pg_clickhouse. Examples depend on
these ClickHouse tables:
Expand Down Expand Up @@ -681,7 +681,7 @@ LOAD
```

It's not normally necessary to use [LOAD], as Postgres will automatically load
pg_clickhouse the first time any of of its features (functions, foreign
pg_clickhouse the first time any of its features (functions, foreign
tables, etc.) are used.

The one time it may be useful to [LOAD] pg_clickhouse is to [SET](#set)
Expand Down Expand Up @@ -823,7 +823,7 @@ shared_preload_libraries = pg_clickhouse
Useful to save memory and load overhead for every session, but requires the
cluster to be restart when the library is updated.

## Data Types {#data-types}
## Data types {#data-types}

pg_clickhouse maps the following ClickHouse data types to PostgreSQL data
types. [IMPORT FOREIGN SCHEMA](#import-foreign-schema) uses the first type in
Expand Down Expand Up @@ -974,7 +974,7 @@ As a rule, only use [TEXT] columns for encoded strings and use [BYTEA] columns
only for binary data, and never switch between them.
</Tip>

## Function and Operator Reference {#function-and-operator-reference}
## Function and operator reference {#function-and-operator-reference}

### Functions {#functions}

Expand Down
10 changes: 5 additions & 5 deletions integrations/connectors/tools/pg_clickhouse/tutorial.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebarTitle: 'Tutorial'
description: 'Learn how to connect pg_clickhouse to ClickHouse and query a New York City taxi example dataset.'
slug: '/integrations/pg_clickhouse/tutorial'
slug: '/cloud/managed-postgres/extensions/pg_clickhouse/tutorial'
title: 'pg_clickhouse tutorial'
doc_type: 'guide'
keywords: ['PostgreSQL', 'Postgres', 'FDW', 'foreign data wrapper', 'pg_clickhouse', 'extension', 'tutorial', 'taxi']
Expand All @@ -22,7 +22,7 @@ docker run -d --network host --name clickhouse -p 8123:8123 -p9000:9000 --ulimit
docker exec -it clickhouse clickhouse-client
```

## Create a Table {#create-a-table}
## Create a table {#create-a-table}

Let's borrow from the [ClickHouse tutorial] to create a simple database with The New York
City taxi dataset:
Expand Down Expand Up @@ -87,7 +87,7 @@ PARTITION BY toYYYYMM(pickup_date)
ORDER BY pickup_datetime;
```

## Add the Data Set {#add-the-data-set}
## Add the data set {#add-the-data-set}

And then import the data:

Expand Down Expand Up @@ -461,7 +461,7 @@ your own SQL query.
Time: 17.450 ms
```

## Create a Dictionary {#create-a-dictionary}
## Create a dictionary {#create-a-dictionary}

Create a dictionary associated with a table in your ClickHouse service. The
table and dictionary are based on a CSV file that contains a row for each
Expand Down Expand Up @@ -620,7 +620,7 @@ table.
Time: 2.012 ms
```

2. This query returns rows for the the 1000 trips with the highest tip
2. This query returns rows for the 1000 trips with the highest tip
amount, then performs an inner join of each row with the dictionary:

```sql
Expand Down
68 changes: 68 additions & 0 deletions products/managed-postgres/migrations/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
slug: /cloud/managed-postgres/migrations/overview
sidebarTitle: 'Overview'
title: 'Managed Postgres data migration'
description: 'Compare the four migration paths into ClickHouse Managed Postgres and pick the one that fits your source database and downtime requirements.'
keywords: ['managed postgres', 'migration', 'postgres migration', 'clickpipes', 'peerdb', 'pg_dump', 'pg_restore', 'logical replication']
doc_type: 'guide'
---

import PrivatePreviewBadge from "/snippets/components/PrivatePreviewBadge/PrivatePreviewBadge.jsx";

# Managed Postgres data migration

<PrivatePreviewBadge link="https://clickhouse.com/cloud/postgres" galaxyTrack={true} slug="migration-overview" />

You can migrate to Managed Postgres through four different paths. Which one
fits depends on whether you need ongoing replication, what source you're
migrating from, and how much downtime your application can tolerate during
cutover.

| Method | Ongoing replication (CDC) | Where it runs | Best for |
| --------------------------------------------------------------------------------------------------- | ------------------------- | -------------------------- | ------------------------------------------------------------------------- |
| [ClickPipes](/products/managed-postgres/migrations/clickhouse-cloud) | Yes | ClickHouse Cloud console | Most migrations — guided wizard with initial load and CDC out of the box |
| [PeerDB](/products/managed-postgres/migrations/peerdb) | Yes | Self-hosted (Docker) | Sources or workflows not covered by the ClickPipes UI |
| [pg_dump and pg_restore](/products/managed-postgres/migrations/pg_dump-pg_restore) | No | Your local machine | One-time moves of small or static datasets where downtime is acceptable |
| [Logical replication](/products/managed-postgres/migrations/logical-replication) | Yes | Source and target Postgres | Direct control over native Postgres replication, no third-party tooling |

## ClickPipes {#clickpipes}

[ClickPipes](/products/managed-postgres/migrations/clickhouse-cloud) is the recommended
path for most migrations. It runs entirely inside the ClickHouse Cloud console
and walks you through connecting to the source, exporting and importing the
schema, and starting an initial load with or without CDC. Pre-built source
connectors cover Amazon RDS, Aurora, Supabase, Google Cloud SQL, Azure
Flexible Server, Neon, Crunchy Bridge, TimescaleDB, and any generic Postgres
instance.

## PeerDB {#peerdb}

[PeerDB](/products/managed-postgres/migrations/peerdb) is a self-hosted migration
tool you run via Docker. Use it when your source or workflow isn't a fit for
the ClickPipes wizard — for example, when you need to script peer creation
across many databases or run the migration entirely inside your own network.
PeerDB doesn't migrate indexes, constraints, or triggers automatically; you
recreate those on the target after the data lands.

## pg_dump and pg_restore {#pg-dump-pg-restore}

[pg_dump and pg_restore](/products/managed-postgres/migrations/pg_dump-pg_restore)
take a snapshot of the source and replay it on the target. There's no ongoing
replication, so writes must stop on the source for the duration of the dump
and restore. This is the right choice for small or static datasets, or
non-production environments where a maintenance window is acceptable.

## Logical replication {#logical-replication}

[Logical replication](/products/managed-postgres/migrations/logical-replication)
uses native Postgres publications and subscriptions to stream changes from
the source to the target. You configure `wal_level`, replication slots, and
the `REPLICATION` privilege yourself — no third-party tooling sits in the
middle. Pick this path when you want full control over the replication
mechanics or your environment rules out external migration tools.

## After the migration {#after-migration}

Once data is moving, use [data validation](/products/managed-postgres/migrations/data-validation)
to confirm row counts and content match between source and target before
cutting over application traffic.
89 changes: 51 additions & 38 deletions products/managed-postgres/navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,59 +6,72 @@
"group": "Get started",
"pages": [
"products/managed-postgres/overview",
"products/managed-postgres/quickstart"
"products/managed-postgres/quickstart",
"products/managed-postgres/connection",
{
"group": "Data migration",
"expanded": false,
"pages": [
"products/managed-postgres/migrations/overview",
"products/managed-postgres/migrations/clickhouse-cloud",
"products/managed-postgres/migrations/peerdb",
"products/managed-postgres/migrations/pg_dump-pg_restore",
"products/managed-postgres/migrations/logical-replication",
"products/managed-postgres/migrations/data-validation"
]
}
]
},
{
"group": "Features",
"group": "Use it",
"pages": [
"products/managed-postgres/connection",
"products/managed-postgres/settings",
"products/managed-postgres/scaling",
"products/managed-postgres/security",
"products/managed-postgres/clickhouse-integration",
"products/managed-postgres/high-availability",
"products/managed-postgres/read-replicas",
"products/managed-postgres/scaling",
"products/managed-postgres/backup-and-restore",
"products/managed-postgres/extensions",
"products/managed-postgres/upgrades",
"products/managed-postgres/openapi"
]
},
{
"group": "Benchmarks",
"pages": [
"products/managed-postgres/benchmarks"
"products/managed-postgres/security"
]
},
{
"group": "ClickHouse Integration",
"group": "Configure",
"pages": [
"products/managed-postgres/clickhouse-integration"
]
},
{
"group": "Migrations",
"pages": [
"products/managed-postgres/migrations/pg_dump-pg_restore",
"products/managed-postgres/migrations/logical-replication",
"products/managed-postgres/migrations/peerdb",
"products/managed-postgres/migrations/data-validation",
"products/managed-postgres/migrations/clickhouse-cloud"
]
},
{
"group": "Monitoring",
"pages": [
"products/managed-postgres/monitoring/overview",
"products/managed-postgres/monitoring/dashboard",
"products/managed-postgres/monitoring/metrics",
"products/managed-postgres/monitoring/prometheus",
"products/managed-postgres/monitoring/query-insights"
"products/managed-postgres/settings",
{
"group": "Extensions",
"expanded": true,
"pages": [
"products/managed-postgres/extensions",
{
"group": "pg_clickhouse",
"expanded": false,
"pages": [
"integrations/connectors/tools/pg_clickhouse/introduction",
"integrations/connectors/tools/pg_clickhouse/tutorial",
"integrations/connectors/tools/pg_clickhouse/reference"
]
}
]
},
"products/managed-postgres/upgrades"
]
},
{
"group": "FAQ",
"group": "Operate & reference",
"pages": [
{
"group": "Monitoring",
"expanded": false,
"pages": [
"products/managed-postgres/monitoring/overview",
"products/managed-postgres/monitoring/dashboard",
"products/managed-postgres/monitoring/query-insights",
"products/managed-postgres/monitoring/prometheus",
"products/managed-postgres/monitoring/metrics"
]
},
"products/managed-postgres/benchmarks",
"products/managed-postgres/openapi",
"products/managed-postgres/faq"
]
}
Expand Down