feat: support serializing closures in CompactedRegion.save#13891
Open
Kha wants to merge 2 commits into
Open
Conversation
Kha
added
the
release-ci
Member
Author
|
!bench |
|
Benchmark results for 91f59d5 against 6de1d62 are in. There are significant results. @Kha
Large changes (2🟥)
Small changes (1🟥)
|
This PR adds opt-in support for serializing closures (functions with captured values) to `.olean` files via `CompactedRegion.save (allowClosures := true)`, so a saved function can be loaded back and called, including from a separate process. Regular module data is unaffected and continues to reject closures. With `allowClosures := true` the writer emits a new `v3` `.olean` layout, `[header][data_size][data][closure offsets][lib relocation table]`, whose sections are all length-prefixed so the reader parses forward. The relocation table records `(base_addr, id)` for only the loaded libraries that contain a compacted closure's function pointer; on load each is matched by `id` against the currently-loaded libraries to compute a per-library address delta that relocates the closures' `m_fun` pointers under ASLR. A library that is no longer loaded is a hard error rather than a silent mis-relocation. `v3` files are mapped copy-on-write so function pointers are patched in place without modifying the file. The closure-offset list points the loader directly at each `m_fun` field, avoiding a scan of the compacted region, and a zero closure count lets closure-less `v3` files skip the relocation table and `get_loaded_libs()` entirely. The full pointer-fixup walk still runs only when a dependency region fails to load at its saved base address. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
github-actions
Bot
added
the
toolchain-available
|
Mathlib CI status (docs):
|
Collaborator
|
Reference manual CI status:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds opt-in support for serializing closures (functions with captured values) to
.oleanfiles viaCompactedRegion.save (allowClosures := true), so a saved function can be loaded back and called, including from a separate process. Regular module data is unaffected and continues to reject closures.With
allowClosures := truethe writer emits a newv3.oleanlayout,[header][data_size][data][closure offsets][lib relocation table], whose sections are all length-prefixed so the reader parses forward. The relocation table records(base_addr, id)for only the loaded libraries that contain a compacted closure's function pointer; on load each is matched byidagainst the currently-loaded libraries to compute a per-library address delta that relocates the closures'm_funpointers under ASLR. A library that is no longer loaded is a hard error rather than a silent mis-relocation.v3files are mapped copy-on-write so function pointers are patched in place without modifying the file. The closure-offset list points the loader directly at eachm_funfield, avoiding a scan of the compacted region, and a zero closure count lets closure-lessv3files skip the relocation table andget_loaded_libs()entirely. The full pointer-fixup walk still runs only when a dependency region fails to load at its saved base address.