docs: Update compression support for AWS plugins

[ShelbyZ]

Summary

• out_s3 - Update docs for support of snappy compression
• out_firehose - Update docs for support of snappy compression
• out_kinesis_streams - Add compression parameter

Related PR - fluent/fluent-bit#11400

Summary by CodeRabbit

• Documentation
  • Firehose docs: added per-record compression formats (gzip, zstd, snappy), new Compression section, and clarified buffering/encoding behavior.
  • Firehose config: expanded public parameters (log_key, profile, region, role_arn, simple_aggregation, sts_endpoint, time_key, time_key_format, workers) and added worker usage examples and guidance.
  • Kinesis docs: added compression parameter and Compression section (gzip, zstd, snappy).
  • S3 docs: added zstd and snappy alongside gzip and clarified Parquet/Arrow availability.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Documentation updates for Firehose, Kinesis, and S3 outputs: per-record compression options expanded to gzip, zstd, and snappy (with Parquet/Arrow conditional on compile-time Arrow). Firehose docs add several configuration parameters and a Workers section with examples; Kinesis and S3 docs add/clarify compression behavior.

Changes

Cohort / File(s)	Summary
Firehose — compression & config additions pipeline/outputs/firehose.md	Added compression values: gzip, zstd, snappy; clarified Arrow availability is compile-time optional. Added parameters: log_key, profile, region, role_arn, simple_aggregation, sts_endpoint, time_key, time_key_format, workers. New "Compression" and "Worker" sections with examples and guidance.
Kinesis — compression parameter pipeline/outputs/kinesis.md	Introduced compression parameter (values: gzip, zstd, snappy) and new "Compression" section describing per-record compression and consumer decompression requirements.
S3 — compression clarification pipeline/outputs/s3.md	Updated compression options to gzip, zstd, snappy; clarified that Parquet/Arrow require Apache Arrow at compile time and adjusted Content-Encoding/format notes.

Sequence Diagram(s)

(omitted — changes are documentation-only and do not introduce new runtime control flow)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• out_firehose: out_kinesis: add simple_aggregation config parameter and cleanup #2317: Edits to Firehose and Kinesis docs adding/adjusting compression and configuration parameters (strong overlap).
• out_s3: Add a description for pure C parquet #2111: Edits to S3 compression/Parquet/Arrow documentation clarifying build-time Arrow requirements.

Suggested labels

4.2.1

Poem

🐇 I hopped through docs with joy and cheer,
gzip, zstd, snappy now appear,
Firehose, Kinesis, S3 aligned,
Arrow waits till builds are signed,
Workers hum — the logs hop near. 🎉

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main changes: updating AWS plugin documentation to reflect new compression support (snappy, zstd) across S3, Firehose, and Kinesis outputs.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@pipeline/outputs/firehose.md`:
- Line 20: The `compression` description is misleading: update the `compression`
field documentation to replace "Each log record is individually compressed and
sent to Firehose" with a clear explanation that while Fluent Bit can apply
record-level compression before sending, AWS Firehose buffers multiple records
and compresses the entire buffered output file when delivering to destinations
(e.g., S3), so consumers receive and must decompress complete compressed files
(not individually compressed records); keep the supported values (`gzip`,
`zstd`, `snappy`, `arrow`) and recommendation about Arrow vs Firehose Parquet
conversion and the default of no compression.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@pipeline/outputs/firehose.md`:
- Around line 94-101: Update the "Compression" section to clarify that
file-level compression is applied only when the destination's compression
setting is enabled (it's destination-configured, not automatically applied by
Firehose); mention that Fluent Bit compresses each record using the
`compression` parameter before upload, Firehose buffers records into files, and
if the destination has file-level compression enabled the delivered files will
be compressed — consumers must first decompress the file (if applicable) and
then decompress each compressed record inside it. Reference the "Compression"
heading, the `compression` parameter, and mentions of "Fluent Bit" and
"Firehose" when making the wording change.