-Zstaticlib-hide-internal-symbols and Zstaticlib-rename-internal-symbols: hide/rename internal symbols in staticlibs#155338
-Zstaticlib-hide-internal-symbols and Zstaticlib-rename-internal-symbols: hide/rename internal symbols in staticlibs#155338cezarbbb wants to merge 10 commits into
-Zstaticlib-hide-internal-symbols and Zstaticlib-rename-internal-symbols: hide/rename internal symbols in staticlibs#155338Conversation
|
rustbot has assigned @petrochenkov. Use Why was this reviewer chosen?The reviewer was selected based on:
|
This comment has been minimized.
This comment has been minimized.
|
This would also need to rename symbols to avoid conflicts between two rust staticlibs ending up getting linked together, right? |
Why exactly is that the case? |
ff707ad to
7ac49d1
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
My primary goal right now is to reduce binary size, so I don't have immediate plans to implement symbol renaming. This means that linking multiple Rust staticlibs together can still result in multiple definition errors. Would you like me to address that in this PR as well? It seems feasible to implement — for example, by rehashing symbols and updating their references accordingly. |
I previously assumed this symbol needed to remain externally visible to support scenarios requiring cross-language exception propagation. Do you think we should also set rust_eh_personality as hidden? |
|
If it isn't too hard it would be nice to do symbol renaming too. I think doing in-place modification isn't going to work for that though. Adding a unique suffix would require growing the size of the string table. |
|
Got it. I will first fix the |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
5e1c3a1 to
c7d4e98
Compare
|
@bors delegate=try |
|
✌️ @cezarbbb, you can now perform try builds on this pull request! You can now post |
|
@bors try |
This comment has been minimized.
This comment has been minimized.
`-Zstaticlib-hide-internal-symbols`: Hide non-exported internal symbols from staticlibs
|
@bors try jobs=x86_64-* |
This comment has been minimized.
This comment has been minimized.
|
@cezarbbb Awesome, I think to test it out I just checkout this PR on my computer and build |
|
Correct. Fetch my PR and build |
|
I'll check it out in a sec and report it goes! |
|
Hey @cezarbbb, so I've been trying to get this to work for both Android and iOS, and I'll let you know what happened for each Android (
|
|
Oh my bad, looks like I forgot to set |
|
Ok a new short summary is I tried building with this command for both iOS and Android: And for both targets I got this error: However if I build with just: Then build succeeds. In other words it seems like |
|
It seems that I made a stupid mistake — the flag was using a The fix is to just emit a warning and continue. I'll fix it shortly. |
|
You can try again. |
|
Already in-progress, should report back in a bit :) But one thing I noticed is that now builds emit a warning every dependency crate which is a bit spammy / likely unintended. |
|
Preliminarily speaking, I think it is renaming user-defined symbols, causing undefined symbol errors when the output goes into the linking stage: (I am using cxx.rs btw) Simply toggling the rename flag off fixes this build error. Still investigating. |
|
I think the rename flag indeed modifies public symbols:
When linking against It also seems like the flag breaks mangling somehow as such it could demangle |
|
I see. The |
|
I think I can filter by filename and only process |
|
I have no idea, but let me test that already on my setup |
|
I think I'm like 99% sure renaming works flawlessly now. So my setup was 2 Rust libraries being ported to C++ via cxx.rs and linked independently into a mobile app. There were
For one of the Rust libraries I built it with the That was for Android though, for iOS I'll get that tested very soon too, maybe within the next hour. |
|
Took me long time to test for iOS cause I'm having other unrelated issues with the iOS build, but nonetheless judging by how far I am I think the patch just works on iOS too ✅ |
|
@cezarbbb asked me to take over the review since @bjorn3 didn't respond for some time. r? @petrochenkov |
What do the numbers in the tables mean?
|
|
|
| # `staticlib-hide-internal-symbols` | ||
|
|
||
| When building a `staticlib`, this option hides all non-exported Rust-internal | ||
| symbols by setting their ELF visibility to `STV_HIDDEN`. |
There was a problem hiding this comment.
What is the difference with -Z default-visibility=hidden here?
As far as I can see, that option will also set visibility to hidden for all SymbolExportLevel::Rust symbols.
There was a problem hiding this comment.
-Z default-visibility=hidden only affects the current crate's codegen. A staticlib also bundles .o files from upstream crates (std, core, etc.) that were compiled without it. -Zstaticlib-hide-internal-symbols post-processes the entire archive including those pre-compiled upstream objects.
| multiple Rust static libraries are linked into the same final binary. | ||
|
|
||
| the Rust compiler already sets `STV_HIDDEN` visibility on non-exported | ||
| symbols by default in the generated `.o` files, so renamed internal symbols |
There was a problem hiding this comment.
Hm? It does not?
If it does then why is -Zstaticlib-hide-internal-symbols needed.
There was a problem hiding this comment.
You're right, that doc statement is wrong. The default visibility is Interposable (STV_DEFAULT), not hidden. Will fix.
| @@ -0,0 +1,40 @@ | |||
| #![crate_type = "staticlib"] | |||
There was a problem hiding this comment.
Some test files, e.g. these ones
tests/run-make/staticlib-hide-internal-symbols-macho/lib.rs
tests/run-make/staticlib-hide-internal-symbols/lib.rs
tests/run-make/staticlib-rename-internal-symbols-macho/lib.rs
tests/run-make/staticlib-rename-internal-symbols/lib.rs
are nontrivial but their contents are exactly the same.
Perhaps it's possible to deduplicate the tests to avoid copypasting source files several times?
There was a problem hiding this comment.
Agreed, will deduplicate.
| .metadata_symbol | ||
| .rsplit_once('_') | ||
| .map(|(_, hash)| hash.to_string()) | ||
| .unwrap_or_else(|| format!("{:08x}", crate_info.local_crate_name.as_u32())); |
There was a problem hiding this comment.
Can two short hashes be the same if two different crates have the same name?
I guess not due to the stable_crate_id component of the metadata_symbol?
There was a problem hiding this comment.
Yes, exactly. StableCrateId incorporates -Cmetadata which Cargo always sets uniquely per crate, so same-name crates get different suffixes. The only way to get a collision is compiling two same-name crates manually without -Cmetadata, but then their mangled symbols are already identical anyway (v0 uses the same StableCrateId as disambiguator).
I'll clean this up to read StableCrateId directly from CrateInfo instead of parsing it out of the metadata symbol string. Is that OK?
| items without `#[no_mangle]`) are renamed. | ||
|
|
||
| This option can only be used with `--crate-type staticlib`. Using it with | ||
| other crate types will result in a compilation error. |
There was a problem hiding this comment.
| other crate types will result in a compilation error. | |
| other crate types will result in a compilation warning. |
(Same for the hiding flag.)
|
|
||
| Tests for the `-Zstaticlib-hide-internal-symbols` flag, which hides non-exported symbols in ELF static libraries. | ||
|
|
||
| ## `tests/ui/staticlib-rename-internal-symbols/`: `-Zstaticlib-rename-internal-symbols` command line flag |
There was a problem hiding this comment.
I don't think there's need in separate subdirectories, can move this together with some other linking tests.
There was a problem hiding this comment.
Got it, will merge into an existing linking test directory.
|
(I started reviewing, will continue on Monday.) |
Sorry for the confusing column names. Here's what the table shows: The test links the staticlib into a shared library using
The column names are bad — I'll rewrite the table with clearer labels when I update the PR description. |
Makes sense, I'll split it. |
| if let Some(name) = tab.read_name(data, i) { | ||
| if hide_set.contains(&name) { | ||
| let buf = result.get_or_insert_with(|| data.to_vec()); | ||
| buf[off + tab.layout.st_other_offset] = elf::STV_HIDDEN; |
There was a problem hiding this comment.
Is it safe to clobber the upper bits of st_other here (and in the other place this is done in this PR)?

View all comments
According to issue #104707, when building a staticlib, all Rust internal symbols — mangled symbols,
#[rustc_std_internal_symbol]items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only#[no_mangle]symbols via a linker version script.Two flags are provided, respectively solved the problems of staticlibs exporting many unnecessary Rust internal symbols and multiple staticlibs causing duplicate symbol conflicts:
-Zstaticlib-hide-internal-symbolsdirectly post-processes ELF object files in the archive: parsing theSHT_SYMTABsections and settingSTV_HIDDENvisibility on anyGLOBAL/WEAKdefined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead.-Zstaticlib-rename-internal-symbolstakes a two-pass global approach: first, it collects all definedGLOBAL/WEAKsymbol names that are not in the exported symbol set across all .o files; then it renames those symbols in each .o file by appending a suffix (e.g.__rust_internal_), handling both definitions and undefined references so that cross-object-file references remain consistent. The implementation uses a "move strtab to end" strategy: it builds a newstrtabwith the renamed names, places it at the end of the file, and patches thestrtabsection header and the ELFe_shoff. When combined with-Zstaticlib-hide-internal-symbols, the renamed symbols also receiveSTV_HIDDENvisibility.The two flags for symbol hiding and symbol renaming need to be decoupled, because hiding incurs virtually no overhead, whereas renaming comes with unavoidable costs. Reducing binary size and resolving duplicate symbol conflicts between multiple staticlibs are two distinct requirements. For this reason, I want to let developers choose and trade off between them based on their own needs.
Both flags only affect ELF targets; a warning is emitted for non-ELF targets
The test code are as follows:
1.a std rust staticlib:
1.b downstream c program:
The test results with different compiler flags(which might cause binary size reduction) are as follows:
1.c result with
-Zstaticlib-hide-internal-symbols1.d result with
-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols2.a no_std rust staticlib
2.b downstream c program
The test results with different compiler flags(which might cause binary size reduction) are as follows:
2.c result with
-Zstaticlib-hide-internal-symbols2.d result with
-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbolsTest results show that this compiler option is beneficial for scenarios where LTO cannot be enabled.
r? @bjorn3 @petrochenkov