Skip to content

crypto: switch rdkafka from vendored OpenSSL to AWS-LC#35861

Draft
jasonhernandez wants to merge 1 commit intojason/sec-200-fivetran-destination-crypto-tlsfrom
jason/sec-237-rdkafka-awslc
Draft

crypto: switch rdkafka from vendored OpenSSL to AWS-LC#35861
jasonhernandez wants to merge 1 commit intojason/sec-200-fivetran-destination-crypto-tlsfrom
jason/sec-237-rdkafka-awslc

Conversation

@jasonhernandez
Copy link
Copy Markdown
Contributor

Summary

Switch librdkafka's crypto backend from vendored OpenSSL to AWS-LC for FIPS 140-3 compliance. This affects every Kafka source and sink.

Changes

Fork (MaterializeInc/rust-rdkafka, branch jasonhernandez/ssl-awslc):

  • Added aws-lc-sys optional dependency to rdkafka-sys
  • Added ssl-awslc feature that links librdkafka against AWS-LC instead of OpenSSL
  • Build script dynamically discovers aws-lc-sys root directory (versioned env vars)
  • No changes to librdkafka C code — v2.12.1 already has BoringSSL/no-ENGINE patches

This repo:

  • Switched all 7 crates from ssl-vendored to ssl-awslc
  • Updated workspace [patch] to point at the fork branch

Why this works

  • librdkafka v2.12.1 has all necessary compatibility patches (PR Make mz_kafka_sinks a local input #3535: guards ENGINE behind OPENSSL_NO_ENGINE)
  • AWS-LC defines OPENSSL_NO_ENGINE, reports as OpenSSL 1.1.1g — correct code paths
  • All ~30 OpenSSL APIs librdkafka calls (TLS, SCRAM, OAUTHBEARER, X509, Base64) confirmed present in AWS-LC

Note

The fork branch needs to be merged to MaterializeInc/rust-rdkafka main before this PR can land. Currently using branch = "jasonhernandez/ssl-awslc".

Part of SEC-237.

Test plan

  • cargo check -p mz-kafka-util passes locally
  • CI: Kafka source/sink integration tests
  • CI: SASL/SCRAM authentication tests

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone.

PR title guidelines

  • Use imperative mood: "Fix X" not "Fixed X" or "Fixes X"
  • Be specific: "Fix panic in catalog sync when controller restarts" not "Fix bug" or "Update catalog code"
  • Prefix with area if helpful: compute: , storage: , adapter: , sql:

Pre-merge checklist

  • The PR title is descriptive and will make sense in the git log.
  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).

@jasonhernandez jasonhernandez force-pushed the jason/sec-237-rdkafka-awslc branch 10 times, most recently from 6bac7ec to 1fd6fde Compare April 3, 2026 17:05
@jasonhernandez jasonhernandez changed the base branch from main to jason/sec-220-tier3-4-leaf-crate-migrations April 3, 2026 17:05
@jasonhernandez jasonhernandez force-pushed the jason/sec-237-rdkafka-awslc branch 3 times, most recently from b8a7586 to 474920d Compare April 3, 2026 18:15
Replace librdkafka's vendored OpenSSL with a pre-built AWS-LC static
library for FIPS 140-3 compliance on all Kafka sources and sinks.

Changes:
- Switch all 7 crates from ssl-vendored to ssl-awslc feature
- Point rdkafka/rdkafka-sys at fork branch with ssl-awslc support
- Add AWS-LC static library build to CI builder Dockerfile (with ninja)
- Add AWS-LC build to macOS clippy CI step (with Go installer)
- rdkafka-sys build.rs reads AWS_LC_DIR (default /opt/aws-lc) and passes
  it to librdkafka's CMake as OPENSSL_ROOT_DIR

Part of SEC-237.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jasonhernandez jasonhernandez force-pushed the jason/sec-237-rdkafka-awslc branch from 474920d to 6913ede Compare April 3, 2026 18:25
@jasonhernandez jasonhernandez changed the base branch from jason/sec-220-tier3-4-leaf-crate-migrations to jason/sec-200-fivetran-destination-crypto-tls April 3, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant