Skip to content

docs: add Pub/Sub to ClickHouse Dataflow template docs #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mintlify wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: add Pub/Sub to ClickHouse Dataflow template docs #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added images/integrations/data-ingestion/google-dataflow/pubsub-inqueue-job.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion integrations/connectors/data-ingestion/GCP/google-dataflow/dataflow.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ However, this option requires knowledge of Java programming and familiarity with
- Requires coding and understanding of the Beam API.

## Predefined templates {#2-predefined-templates}
ClickHouse offers [predefined templates](/integrations/connectors/data-ingestion/GCP/google-dataflow/templates) designed for specific use cases, such as importing data from BigQuery into ClickHouse. These templates are ready-to-use and simplify the integration process, making them an excellent choice if you prefer a no-code solution.
ClickHouse offers [predefined templates](/integrations/connectors/data-ingestion/GCP/google-dataflow/templates) designed for specific use cases, such as batch imports from BigQuery or streaming ingestion from Pub/Sub into ClickHouse. These templates are ready-to-use and simplify the integration process, making them an excellent choice if you prefer a no-code solution.

### Key features {#key-features-1}
- No Beam coding required.
Expand Down
2 changes: 1 addition & 1 deletion integrations/connectors/data-ingestion/GCP/google-dataflow/templates.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,5 @@ For detailed step-by-step instructions, refer to the [Google Dataflow Run Pipeli

## List of ClickHouse Templates {#list-of-clickhouse-templates}
* [BigQuery To ClickHouse](/integrations/connectors/data-ingestion/GCP/google-dataflow/templates/bigquery-to-clickhouse)
* [Pub/Sub To ClickHouse](/integrations/connectors/data-ingestion/GCP/google-dataflow/templates/pubsub-to-clickhouse)
* [GCS To ClickHouse](https://github.com/ClickHouse/DataflowTemplates/issues/3) (coming soon!)
* [Pub Sub To ClickHouse](https://github.com/ClickHouse/DataflowTemplates/issues/4) (coming soon!)
5 changes: 4 additions & 1 deletion ...nectors/data-ingestion/GCP/google-dataflow/templates/bigquery-to-clickhouse.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,4 +177,7 @@ This error occurs when ClickHouse runs out of memory while processing large batc

## Template source code {#template-source-code}

The template's source code is available in ClickHouse's [DataflowTemplates](https://github.com/ClickHouse/DataflowTemplates) fork.
The template's source code is available in:

- [`GoogleCloudPlatform/DataflowTemplates`](https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/main/v2/googlecloud-to-clickhouse) — the upstream Google Cloud Platform repository.
- [`ClickHouse/DataflowTemplates`](https://github.com/ClickHouse/DataflowTemplates) — ClickHouse's fork.
Loading