Skip to content

Clarify Parquet continuous export performance for wide tables #2900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
silasmendes-ms wants to merge 1 commit into
base: main
Choose a base branch
from
+1 −0
Open

Clarify Parquet continuous export performance for wide tables #2900

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions data-explorer/kusto/management/data-export/continuous-data-export.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ To define continuous export to a delta table, do the following steps:
* The following formats are allowed on target tables: `CSV`, `TSV`, `JSON`, and `Parquet`.
* Continuous export isn't designed to work over [materialized views](../materialized-views/materialized-view-overview.md), since a materialized view might be updated, while data exported to storage is always appended and never updated.
* Continuous export can't be created on [follower databases](/azure/data-explorer/follower) since follower databases are read-only and continuous export requires write operations.
* Continuous export to Parquet can experience reduced performance for tables with a large number of columns due to per-column encoding overhead.
* Records in source table must be ingested to the table directly, using an [update policy](../update-policy.md), or [ingest from query commands](../data-ingestion/ingest-from-query.md). If records are moved into the table using [.move extents](../move-extents.md) or using [.rename table](../rename-table-command.md), continuous export might not process these records. See the limitations described in the [Database Cursors](../database-cursor.md#restrictions) page.
* If the artifacts used by continuous export are intended to trigger Event Grid notifications, see the [known issues section in the Event Grid documentation](/azure/data-explorer/ingest-data-event-grid-overview#known-event-grid-issues).

Expand Down