Refactor benchmark jobs and remove hardcoded paths (Fixes #418)#718
Refactor benchmark jobs and remove hardcoded paths (Fixes #418)#718makarandhinge wants to merge 6 commits intoapache:mainfrom
Conversation
- Replace hardcoded input path with configurable argument - Replace hardcoded output path 'lala.out' with configurable argument - Add default output path as <input-file>.out if not specified - Add usage message and argument validation - Addresses issue apache#418
|
Thanks @makarandhinge. It would be better if we keep the JavaNativeAPI examples and add the respective ones with the planbuilder instead of replacing them. |
|
Thank you for the suggestion @zkaoudi. I will restore the original JavaNativeAPI examples and add separate implementations using JavaPlanBuilder instead of replacing them. This way both approaches will remain available in the benchmark module. I will update the PR accordingly. |
fdc7ca9 to
bf7a708
Compare
|
Hi @zkaoudi, I have updated the PR accordingly. The original JavaNativeAPI examples are now preserved, and separate implementations using Please let me know if any further adjustments are needed. Thanks! |
zkaoudi
left a comment
There was a problem hiding this comment.
Thank you @makarandhinge
| } | ||
|
|
||
| WayangContext wayangContext = new WayangContext(); | ||
| WayangContext wayangContext = new WayangContext(new Configuration()); |
There was a problem hiding this comment.
What's the purpose of adding an empty configuration in the context?
| } | ||
|
|
||
| WayangContext wayangContext = new WayangContext(); | ||
| WayangContext wayangContext = new WayangContext(new Configuration()); |
| @@ -0,0 +1,116 @@ | |||
| /* | |||
There was a problem hiding this comment.
There is already a Wordcount with the planbuilder so we do not need a second benchmark.
…tWithPlanBuilder.java
|
Thanks, @zkaoudi |
Description
This PR revisits and cleans up benchmark jobs in the
wayang-benchmarkmodule as requested in issue #418.Based on maintainer feedback, the original JavaNativeAPI examples were preserved and equivalent implementations using
JavaPlanBuilderwere added instead of replacing them. This keeps both APIs available for benchmarking and comparison.Changes Made
Benchmark Improvements
Grep.java.Added JavaPlanBuilder Implementations
TPCHQ1WithPlanBuilder.javaas a JavaPlanBuilder version ofTPCHQ1WithJavaNative.java.WordCountWithPlanBuilder.javaas a JavaPlanBuilder version ofWordCountWithJavaNativeAPI.java.Code Cleanup
Main.javaby removing unused imports and addingConfiguration.WordCountParquet.javaby removing unused imports and addingConfiguration.WordCount.javaby removing commented code.Current Benchmark Structure
Testing
Related Issue
Fixes #418