-Zstaticlib-hide-internal-symbols and Zstaticlib-rename-internal-symbols: hide/rename internal symbols in staticlibs

[cezarbbb]

View all comments

According to issue #104707, when building a staticlib, all Rust internal symbols — mangled symbols, #[rustc_std_internal_symbol] items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only #[no_mangle] symbols via a linker version script.

Two flags are provided, respectively solved the problems of staticlibs exporting many unnecessary Rust internal symbols and multiple staticlibs causing duplicate symbol conflicts:

-Zstaticlib-hide-internal-symbols directly post-processes ELF object files in the archive: parsing the SHT_SYMTAB sections and setting STV_HIDDEN visibility on any GLOBAL/WEAK defined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead.

-Zstaticlib-rename-internal-symbols takes a two-pass global approach: first, it collects all defined GLOBAL/WEAK symbol names that are not in the exported symbol set across all .o files; then it renames those symbols in each .o file by appending a suffix (e.g. __rust_internal_), handling both definitions and undefined references so that cross-object-file references remain consistent. The implementation uses a "move strtab to end" strategy: it builds a new strtab with the renamed names, places it at the end of the file, and patches the strtab section header and the ELF e_shoff. When combined with -Zstaticlib-hide-internal-symbols, the renamed symbols also receive STV_HIDDEN visibility.

The two flags for symbol hiding and symbol renaming need to be decoupled, because hiding incurs virtually no overhead, whereas renaming comes with unavoidable costs. Reducing binary size and resolving duplicate symbol conflicts between multiple staticlibs are two distinct requirements. For this reason, I want to let developers choose and trade off between them based on their own needs.

Both flags only affect ELF targets; a warning is emitted for non-ELF targets

The test code are as follows:

1.a std rust staticlib:

use std::collections::HashMap;
use std::panic::{catch_unwind, AssertUnwindSafe};

#[no_mangle]
pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b }

#[no_mangle]
pub extern "C" fn my_hash_lookup(key: u64) -> u64 {
    let mut map = HashMap::new();
    for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); }
    *map.get(&key).unwrap_or(&0)
}

pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() }

#[no_mangle]
pub extern "C" fn my_format_number(n: i32) -> i32 {
    let s = format!("number: {}", n); s.len() as i32
}

#[no_mangle]
pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 {
    match catch_unwind(AssertUnwindSafe(|| {
        if b == 0 { panic!("division by zero!"); }
        a / b
    })) {
        Ok(result) => result,
        Err(_) => -1,
    }
}

#[no_mangle]
pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); }

1.b downstream c program:

extern int my_add(int a, int b);
extern unsigned long my_hash_lookup(unsigned long key);
extern int my_format_number(int n);
extern int my_safe_div(int a, int b);
extern void my_uncaught_panic(void);

int main() {
    int failures = 0;
    if (my_add(10, 20) != 30) failures++;
    if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++;
    if (my_format_number(42) != 10) failures++;
    if (my_safe_div(100, 5) != 20) failures++;
    if (my_safe_div(100, 0) != -1) failures++;
    pid_t pid = fork();
    if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); }
    else { waitpid(pid, &status, 0); }
    return failures;
}

The test results with different compiler flags(which might cause binary size reduction) are as follows:
1.c result with -Zstaticlib-hide-internal-symbols

  settings                   OFF        ON  -Zsave     ALL    OFF.dynsym ON.dynsym 
  ------------------------------------------------------------------------
  default                 1.7M      1.5M  204K (12%)    1735       5    1730
  lto_thin                616K      584K  33K (5%)     246       5     241
  lto_fat                 525K      525K    0 (0%)       6       5       1
  opt_s                   1.7M      1.5M  204K (12%)    1735       5    1730
  opt_z                   1.7M      1.5M  204K (12%)    1735       5    1730
  lto_thin_z              602K      570K  32K (5%)     246       5     241
  lto_fat_z               514K      514K    0 (0%)       6       5       1
  full                    514K      514K    0 (0%)       6       5       1

1.d result with -Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols

  settings                   OFF        ON  -Zsave     ALL    OFF.dynsym ON.dynsym 
  ------------------------------------------------------------------------
  default                 1.7M      1.5M  162K (9%)    1735       5    1730
  lto_thin                616K      599K  18K (2%)     246       5     241
  lto_fat                 525K      535K  -1% (-1%)       6       5       1
  opt_s                   1.7M      1.5M  162K (9%)    1735       5    1730
  opt_z                   1.7M      1.5M  162K (9%)    1735       5    1730
  lto_thin_z              602K      585K  18K (2%)     246       5     241
  lto_fat_z               514K      524K  -1% (-1%)       6       5       1
  full                    514K      523K  -1% (-1%)       6       5       1

2.a no_std rust staticlib

#![no_std]
#![feature(core_intrinsics)]

use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! { loop {} }

#[no_mangle]
pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) }

#[no_mangle]
pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 {
    if data.is_null() { return 0; }
    let slice = unsafe { core::slice::from_raw_parts(data, len) };
    let mut sum: u8 = 0;
    for &byte in slice { sum = sum.wrapping_add(byte); }
    sum
}

fn internal_helper() -> i32 { 42 }
#[no_mangle]
pub extern "C" fn call_internal() -> i32 { internal_helper() }

#[no_mangle]
pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); }

2.b downstream c program

extern int embedded_add(int a, int b);
extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len);
extern int call_internal(void);
extern void embedded_trigger_abort(void);

int main() {
    int failures = 0;
    if (embedded_add(10, 20) != 30) failures++;
    unsigned char data[] = {1, 2, 3};
    if (embedded_checksum(data, 3) != 6) failures++;
    if (call_internal() != 42) failures++;
    pid_t pid = fork();
    if (pid == 0) { embedded_trigger_abort(); _exit(0); }
    else { waitpid(pid, &status, 0); }
    return failures;
}

The test results with different compiler flags(which might cause binary size reduction) are as follows:
2.c result with -Zstaticlib-hide-internal-symbols

  settings                   OFF        ON  -Zsave     ALL    OFF.dynsym ON.dynsym 
  ------------------------------------------------------------------------
  default                 485K      429K  56K (11%)     490       4     486
  lto_thin                180K      180K    0 (0%)       4       4       0
  lto_fat                 179K      179K    0 (0%)       4       4       0
  opt_s                   485K      429K  56K (11%)     490       4     486
  opt_z                   485K      429K  56K (11%)     490       4     486
  lto_thin_z              180K      180K    0 (0%)       4       4       0
  lto_fat_z               179K      179K    0 (0%)       4       4       0
  full                    179K      179K    0 (0%)       4       4       0

2.d result with -Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols

  settings                   OFF        ON  -Zsave     ALL    OFF.dynsym ON.dynsym 
  ------------------------------------------------------------------------
  default                 485K      447K  39K (7%)     490       4     486
  lto_thin                180K      189K  -5% (-5%)       4       4       0
  lto_fat                 179K      189K  -5% (-5%)       4       4       0
  opt_s                   485K      448K  38K (7%)     490       4     486
  opt_z                   485K      448K  38K (7%)     490       4     486
  lto_thin_z              180K      189K  -5% (-5%)       4       4       0
  lto_fat_z               179K      189K  -5% (-5%)       4       4       0
  full                    179K      189K  -5% (-5%)       4       4       0

Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled.

r? @bjorn3 @petrochenkov

[rustbot]

r? @petrochenkov

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: codegen, compiler
• codegen, compiler expanded to 69 candidates
• Random selection from 16 candidates

[bjorn3]

This would also need to rename symbols to avoid conflicts between two rust staticlibs ending up getting linked together, right?

[bjorn3]

The rust_eh_personality symbol is always kept visible to ensure .eh_frame unwinding works correctly for C consumers.

Why exactly is that the case? rust_eh_personality is actually the symbol that is most likely to cause conflicts as it is the only one whose name doesn't get mangled depending on the rustc version.

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[cezarbbb]

This would also need to rename symbols to avoid conflicts between two rust staticlibs ending up getting linked together, right?

My primary goal right now is to reduce binary size, so I don't have immediate plans to implement symbol renaming. This means that linking multiple Rust staticlibs together can still result in multiple definition errors. Would you like me to address that in this PR as well? It seems feasible to implement — for example, by rehashing symbols and updating their references accordingly.

[cezarbbb]

The rust_eh_personality symbol is always kept visible to ensure .eh_frame unwinding works correctly for C consumers.

Why exactly is that the case? rust_eh_personality is actually the symbol that is most likely to cause conflicts as it is the only one whose name doesn't get mangled depending on the rustc version.

I previously assumed this symbol needed to remain externally visible to support scenarios requiring cross-language exception propagation. Do you think we should also set rust_eh_personality as hidden?

[bjorn3]

If it isn't too hard it would be nice to do symbol renaming too. I think doing in-place modification isn't going to work for that though. Adding a unique suffix would require growing the size of the string table.

[bjorn3]

I previously assumed this symbol needed to remain externally visible to support scenarios requiring cross-language exception propagation. Do you think we should also set rust_eh_personality as hidden?

rust_eh_personality is only meant to be referenced by the .eh_frame section of rust object files. The only reason it's name isn't mangled is because LLVM hard codes the name to determine the exception table format to emit.

[cezarbbb]

If it isn't too hard it would be nice to do symbol renaming too. I think doing in-place modification isn't going to work for that though. Adding a unique suffix would require growing the size of the string table.

Got it. I will first fix the rust_eh_personality issue, and then try to implement symbol renaming.

[SparrowLii]

@bors delegate=try

[rust-bors]

✌️ @cezarbbb, you can now perform try builds on this pull request!

You can now post @bors try to start a try build.

[cezarbbb]

@bors try

[rust-bors]

☀️ Try build successful (CI)
Build commit: a9431d3 (a9431d37da1d0346038257cec9d94f2783997621, parent: e8e4541ff19649d95afab52fdde2c2eaa6829965)

[cezarbbb]

@bors try jobs=x86_64-*

[rust-bors]

💔 Test for 12ba282 failed: CI. Failed job:

• Calculate job matrix (web logs, enhanced plaintext logs)

[cezarbbb]

@bors try jobs=aarch64-*

[rust-bors]

☀️ Try build successful (CI)
Build commit: 3e76129 (3e7612913ab931b859e2e0633efea5ee09e9e265, parent: e8e4541ff19649d95afab52fdde2c2eaa6829965)

[cezarbbb]

@bjorn3 Both symbol rename and symbol hide have been implemented. For now, support is limited to ELF targets; I’m not sure whether the current symbol rename approach works for PE or Mach-O. I’ve updated the documentation with full details and test results for reference.

[cezarbbb]

@rustbot ready