perf: optimize identity function composition#826
Draft
He-Pin wants to merge 1 commit intodatabricks:masterfrom
Draft
perf: optimize identity function composition#826He-Pin wants to merge 1 commit intodatabricks:masterfrom
He-Pin wants to merge 1 commit intodatabricks:masterfrom
Conversation
e3e480f to
19fb7ae
Compare
19fb7ae to
26bedd2
Compare
26bedd2 to
53556cf
Compare
Contributor
Author
|
Safety update pushed in
Validation:
|
24dc899 to
8390d28
Compare
Motivation: bench.07 builds a deep chain of function(x) f(f(x)) over identity functions. Scala Native overflows the stack on this case with --max-stack 100000, and the JVM path creates tens of thousands of lazy values and function calls. Modification: Add an apply1 fast path for unary identity functions and recognize the exact non-tailstrict function(x) f(f(x)) shape. The wrapper preserves laziness, keeps explicit tailstrict eager semantics, and checks identity-composition chains iteratively instead of recursively. Result: bench.07 now passes on Scala Native, reduces the JVM debug counters from lazy_created=32786/function_calls=65550 to lazy_created=19/function_calls=16, and reports 0.036 ms/op in the single-case JMH run.
8390d28 to
515691b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
bench/resources/cpp_suite/bench.07.jsonnetbuilds a lazy array of composed functions:On upstream master, Scala Native overflows the stack for this case even with
--max-stack 100000. On JVM, the same identity-equivalent chain still materializes a deep nested function/lazy evaluation path.Modification
Val.Func.apply1.tailstrictfunction(x) f(f(x))shape and return a lightweight identity-composition wrapper.BaseIdentityProbeInProgressmarks active probes, whileBaseIdentityKnownIdentity/BaseIdentityKnownNonIdentitycache the final predicate result. Recursive composition cycles are treated as non-identity for this optimization and fall back to normal application/max-stack handling.tailstrictsemantics: constructingf2(error "...")stays lazy, calling it still forces the original error, andf2(error "...") tailstrictstill forces eagerly.Result
Compared revisions:
upstream/master:8b67cb1515691b35b43fa8JMH,
bench/resources/cpp_suite/bench.07.jsonnet:Scala Native CLI, same input:
StackOverflowError5b43fa8The native CLI case is tiny and includes process startup, so the stronger signal is the JMH delta plus the master Native failure/pass change.
Verification
Local verification after the semantic state-name update:
All passed.
Boundary Checks
function(x) f(f(x)).tailstrictcalls are not folded into the lazy identity-composition path.local g = f2(error "..."); g(1).f2(1)(1).f2(function(x) x + 1)(1) == 3and a multi-layer non-identity chain.