Skip to content

Comments

[CELEBORN-2267][CIP-14] Add Cpp-Write Java-Read integration tests for…#3606

Open
afterincomparableyum wants to merge 1 commit intoapache:mainfrom
afterincomparableyum:cpp-client/celeborn-2267
Open

[CELEBORN-2267][CIP-14] Add Cpp-Write Java-Read integration tests for…#3606
afterincomparableyum wants to merge 1 commit intoapache:mainfrom
afterincomparableyum:cpp-client/celeborn-2267

Conversation

@afterincomparableyum
Copy link

This is a follow up to #3575

  • Add compression codec argument to C++ DataSumWithWriterClient and set it in CelebornConf so the writer uses LZ4/ZSTD when enabled
  • Pass codec from runCppWriteJavaRead to the C++ writer command
  • Add CppWriteJavaReadTestWithLZ4 and CppWriteJavaReadTestWithZSTD (mirroring CppWriteJavaReadTestWithNONE)

How was this patch tested?

I compiled and ran tests locally, all passed.

… LZ4 and ZSTD

- Add compression codec argument to C++ DataSumWithWriterClient and set
  it in CelebornConf so the writer uses LZ4/ZSTD when enabled
- Pass codec from runCppWriteJavaRead to the C++ writer command
- Add CppWriteJavaReadTestWithLZ4 and CppWriteJavaReadTestWithZSTD
  (mirroring CppWriteJavaReadTestWithNONE)
@afterincomparableyum
Copy link
Author

Ping @SteNicholas @HolyLow @RexXiong for review. Thank you for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant