Clarify Parquet continuous export performance for wide tables#2900
Clarify Parquet continuous export performance for wide tables#2900silasmendes-ms wants to merge 1 commit into
Conversation
|
@silasmendes-ms : Thanks for your contribution! The author(s) and reviewer(s) have been notified to review your proposed change. |
|
Learn Build status updates of commit 7f8d684: ✅ Validation status: passed
For more details, please refer to the build report. |
|
Hi @silasmendes-ms - This pull request was opened in the public repo. PMs should work in the private repo, per the Microsoft Docs contributor guide. We can keep this PR open for review and merge, but would you make future content updates in the private repo? Thank you! Can you review the proposed changes? Important: When the changes are ready for publication, adding a #label:"aq-pr-triaged" |
prmerger-automator
Bot
added
the
aq-pr-triaged
There was a problem hiding this comment.
Pull request overview
Documents a known limitation in Azure Data Explorer continuous export behavior by noting that exporting to Parquet may see reduced performance on very wide tables, helping set expectations for customers designing schemas and export pipelines.
Changes:
- Added a limitation note about Parquet continuous export performance for tables with many columns.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * The following formats are allowed on target tables: `CSV`, `TSV`, `JSON`, and `Parquet`. | ||
| * Continuous export isn't designed to work over [materialized views](../materialized-views/materialized-view-overview.md), since a materialized view might be updated, while data exported to storage is always appended and never updated. | ||
| * Continuous export can't be created on [follower databases](/azure/data-explorer/follower) since follower databases are read-only and continuous export requires write operations. | ||
| * Continuous export to Parquet can experience reduced performance for tables with a large number of columns due to per-column encoding overhead. |
Adds a limitation note documenting that continuous export to Parquet can experience reduced performance for tables with a large number of columns due to per-column encoding overhead.
This clarification helps set customer expectations around export performance characteristics for wide tables.