Skip to content

fix: respect TlsSpec for segment push client#18530

Open
wolfkill wants to merge 1 commit into
apache:masterfrom
wolfkill:fix/segment-push-tls-client
Open

fix: respect TlsSpec for segment push client#18530
wolfkill wants to merge 1 commit into
apache:masterfrom
wolfkill:fix/segment-push-tls-client

Conversation

@wolfkill
Copy link
Copy Markdown

Summary

  • create a TLS-aware FileUploadDownloadClient from SegmentGenerationJobSpec.getTlsSpec() for segment push paths
  • keep the shared default upload client for jobs without TlsSpec
  • extract TlsUtils.createSslContext(...) so callers can build an SSLContext without mutating JVM-wide HTTPS defaults
  • add regression coverage for default vs TLS-aware upload client selection

Root Cause

SegmentPushUtils always used a static FileUploadDownloadClient, so push jobs that supplied TlsSpec did not build a client from that job-level TLS configuration. Direct/core/Hadoop segment push flows could therefore ignore the intended key/trust store settings.

Fixes #17702

Tests

  • Red check before implementation: JAVA_HOME=/opt/homebrew/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home PATH=/opt/homebrew/opt/openjdk@21/bin:$PATH ./mvnw -pl pinot-segment-local -am -Dtest=org.apache.pinot.segment.local.utils.SegmentPushUtilsTest#testGetFileUploadDownloadClientHonorsTlsSpec -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false test failed because SegmentPushUtils.getFileUploadDownloadClient(...) did not exist yet.
  • JAVA_HOME=/opt/homebrew/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home PATH=/opt/homebrew/opt/openjdk@21/bin:$PATH ./mvnw -pl pinot-common,pinot-segment-local -am -Dtest=org.apache.pinot.common.utils.tls.TlsUtilsTest,org.apache.pinot.segment.local.utils.SegmentPushUtilsTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false test
  • git diff --check upstream/master...HEAD

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 22, 2026

Codecov Report

❌ Patch coverage is 5.51724% with 137 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.75%. Comparing base (d338a9b) to head (30a8421).
⚠️ Report is 47 commits behind head on master.

Files with missing lines Patch % Lines
...he/pinot/segment/local/utils/SegmentPushUtils.java 0.00% 132 Missing ⚠️
...va/org/apache/pinot/common/utils/tls/TlsUtils.java 61.53% 5 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (d338a9b) and HEAD (30a8421). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (d338a9b) HEAD (30a8421)
unittests 2 1
unittests2 1 0
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18530      +/-   ##
============================================
- Coverage     63.73%   56.75%   -6.98%     
+ Complexity     1932        7    -1925     
============================================
  Files          3292     2567     -725     
  Lines        201471   148983   -52488     
  Branches      31317    24100    -7217     
============================================
- Hits         128398    84558   -43840     
+ Misses        62787    57241    -5546     
+ Partials      10286     7184    -3102     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (?)
java-21 56.75% <5.51%> (-6.98%) ⬇️
temurin 56.75% <5.51%> (-6.98%) ⬇️
unittests 56.75% <5.51%> (-6.98%) ⬇️
unittests1 56.75% <5.51%> (+0.97%) ⬆️
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

assertSame(SegmentPushUtils.getFileUploadDownloadClient(defaultJobSpec), defaultClient);

SegmentGenerationJobSpec tlsJobSpec = new SegmentGenerationJobSpec();
tlsJobSpec.setTlsSpec(new TlsSpec());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test doesn't test anything ;)
You need to add a test with a custom truststore/keystore, ideally against a local HTTPS endpoint that fails with the default client and succeeds through SegmentPushUtils with TlsSpec.

Copy link
Copy Markdown
Contributor

@xiangfu0 xiangfu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm otherwise

@xiangfu0 xiangfu0 force-pushed the fix/segment-push-tls-client branch from 0b92350 to 30a8421 Compare May 27, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SegmentPushUtils should respect TlsSpec from SegmentGenerationJobSpec

3 participants