Skip to content

docs: add RFC 0004 for Kafka to Iceberg export#4

Open
yatharthranjan wants to merge 1 commit intorfc-0003-radar-data-mapperfrom
rfc-0004-iceberg-s3
Open

docs: add RFC 0004 for Kafka to Iceberg export#4
yatharthranjan wants to merge 1 commit intorfc-0003-radar-data-mapperfrom
rfc-0004-iceberg-s3

Conversation

@yatharthranjan
Copy link
Copy Markdown
Member

Summary

  • Add RFC 0004 in rfcs/platform/0004-streaming-radar-kafka-to-iceberg-s3.md for production-ready Kafka-to-Iceberg export architecture on RADAR-Kubernetes.
  • Specify concrete implementation guidance including the community Iceberg Kafka Connect sink (org.apache.iceberg.connect.IcebergSinkConnector), published-first deployment artefact policy, and a detailed rollout sequence.
  • Add architecture diagram and update RFC index in README.md.

Test plan

  • Verify RFC file renders correctly in Markdown, including Mermaid diagram.
  • Confirm RFC numbering/title/metadata and index entry are consistent.
  • Validate links to Helm charts and connector references are present.
  • Review content with maintainers for final operational SLO values and image pinning decisions.

Document a production-ready RADAR platform architecture for streaming all Kafka topics to Iceberg on S3 via dedicated Kafka Connect, Nessie catalog, and Trino consumption.
@github-actions github-actions bot added rfc RFC document and related discussion area: platform Platform-wide concerns, infra, deployment labels Mar 30, 2026
@yatharthranjan yatharthranjan added the status: draft RFC is in draft and under review label Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: platform Platform-wide concerns, infra, deployment rfc RFC document and related discussion status: draft RFC is in draft and under review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant