Add symbol prefixing feature for BoringSSL

[BarbossHack]

Goal

Add a prefix to all symbols in libcrypto and libssl to prevent conflicts with other OpenSSL or BoringSSL versions that might be linked in the same process.

Why?

When statically linking both OpenSSL and BoringSSL in the same project, we encountered duplicate symbols, for example:

rust-lld: error: duplicate symbol: AES_encrypt
>>> defined at aes-x86_64.s:328 (crypto/aes/aes-x86_64.s:328)
>>>            libcrypto-lib-aes-x86_64.o:(AES_encrypt) in archive /home/user/debug/target/debug/deps/libopenssl_sys-aa966f6339bd703d.rlib
>>> defined at aes.c:62 (/home/user/debug/target/debug/build/boring-sys-daeba9923fc21b33/out/boringssl/src/crypto/fipsmodule/aes/aes.c:62)
 >>>            bcm.c.o:(.text.AES_encrypt+0x0) in archive /home/user/debug/target/debug/deps/libboring_sys-b3f993188e8b17ea.rlib

How?

The first approach was to use the BoringSSL-provided Go script make_prefix_headers.go. It mostly worked, but:

• It required two compilation steps: first to build the static libraries and extract symbols with read_symbols.go, then to generate boringssl_prefix_symbols.h with make_prefix_headers.go, and finally rebuild again the static libraries with this header. (We could have prebuilt it like gRPC does, but I preferred a fully automated workflow.)
• Some ASM/Perl-generated symbols were missed (even if present in boringssl_prefix_symbols.h), which I ended up fixing with the Binutils objcopy tool, not very straightforward.
• It required Go to run.

Therefore, I decided to use only the Binutils tools nm and objcopy to fully automate the symbol prefixing.

Steps (build.rs):

1. List static libraries to prefix symbols in (libssl.a and libcrypto.a).
2. Use nm to list all exported symbols in these static libraries.
3. Generate a mapping file and use objcopy to prefix all symbols.
4. Update the generated bindings (bindings.rs) with #[link_name] where necessary.

Cross-compilation

Currently, this workflow is tested for Linux and Android only.

For Android, the target-specific nm and objcopy from the NDK are used.

macOS/iOS and Windows toolchains are not tested. I currently don’t have access to a macOS environment and have limited experience with Windows toolchains.

Usage

I've added an opt-in feature prefix-symbols, as it is not tested on macOS/iOS/Windows yet.

cargo add boring --features prefix-symbols

[kaukabrizvi]

This PR would help resolve #197. Would it be possible for a maintainer to take a look or assign a reviewer? Thank you!

[BarbossHack]

I don’t think Cloudflare will review it because this patch doesn’t support Windows or macOS yet. But I’m working on it (or at least trying to).

[kornelski]

I don't expect to use this internally at Cloudflare, since we have pretty strict requirements about which libraries we use, and even if symbols don't collide, we can't end up with random copies of openssl.

I don't mind if this is unix-only.

However, I am concerned about reliability. What are the failure modes here? If it fails to rename some symbols, does the result fail to link, or is there a risk of mixing symbols between libraries?

[BarbossHack]

I don't mind if this is unix-only.

Perfect, then I will focus on linux and android for this PR.

I’ve added a check (panic!) to disable this feature on macOS, iOS, and Windows for now, since it doesn’t work on those targets yet. If the PR is merged, I’ll open a follow-up PR for those platforms later.

However, I am concerned about reliability. What are the failure modes here? If it fails to rename some symbols, does the result fail to link, or is there a risk of mixing symbols between libraries?

I’ve added a GitHub Action to verify that the build and run work on Linux (and Android), when linking with both openssl and boringssl. It first checks that there are indeed “duplicate symbols” errors without using this feature, and then verifies that enabling this feature fixes them. See the successful workflow here: https://github.com/BarbossHack/boring/actions/runs/20194450649

On macOS, the linker (ld64) allows duplicate symbols in static libraries by default, simply picking one of the definitions to include. So the link will succeed. However, this can lead to a runtime failure with a “segmentation fault”, which is what happened in my experiments.

Linux and Android linkers are strict about multiple definitions in the same executable. If two objects define the same symbol and are linked statically, the link will fail with duplicate symbol errors. So, if my symbol prefixing misses some symbols (not detected by nm from binutils), it will fail to link with a “duplicate symbols” error. That’s what happened during my tests: first with the read_symbols.go script, and later with nm when I was filtering out too many symbols.