Skip to content

compat/openssl: Add ASAN check for OpenSSL compat layer - #48958

Closed
tedjpoole wants to merge 4 commits into
envoyproxy:mainfrom
tedjpoole:add-openssl-asan-check
Closed

tedjpoole wants to merge 4 commits into
envoyproxy:mainfrom
tedjpoole:add-openssl-asan-check

Conversation

@tedjpoole

Copy link
Copy Markdown
Contributor

Add an openssl.asan presubmit CI job that runs the compat layer tests under AddressSanitizer.

Also removed the use of the RTLD_DEEPBIND flag when loading the OpenSSL shared libraries, because ASAN's dlopen() interceptor does not work with it. Instead, the compatibility layer is now compiled with the -fvisibility=hidden flag, which excludes its symbols from .dynsym entirely, eliminating the need to use RTLD_DEEPBIND and so gives ASAN full coverage of the OpenSSL shared libraries.

Also added two new tests: a symbol visibility check confirming that compat layer symbols are not dynamically visible (preventing the subtle misresolution bugs that RTLD_DEEPBIND was originally introduced to solve), and a LeakSanitizer canary that verifies leak detection is active.

Add an `openssl.asan` presubmit CI job that runs the compat layer tests
under AddressSanitizer.

Unfortunately, ASAN's dlopen() interceptor does not work with the
RTLD_DEEPBIND flag that was being used to prevent OpenSSL's internal
symbols from resolving to the compat layer's identically-named wrappers.
To get around this, the compat layer is now compiled with
-fvisibility=hidden. This excludes its symbols from .dynsym entirely,
eliminating the need to use RTLD_DEEPBIND and so gives ASAN full
coverage of the dlopened OpenSSL libraries.

Also added two new tests: a symbol visibility check confirming that
compat layer symbols are not dynamically visible (preventing the subtle
misresolution bugs that RTLD_DEEPBIND was originally introduced to
solve), and a LeakSanitizer canary that verifies leak detection is
active.

Signed-off-by: Ted Poole <tpoole@redhat.com>
Signed-off-by: Ted Poole <tpoole@redhat.com>

@jwendell jwendell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can split this into two pr's? One that makes the changes to compat/openssl files, the other would be adding the ci job? Also, I think it makes sense if this job would be post-submit only?

cc @phlax thoughts?

@phlax

phlax commented Nov 3, 2026

Copy link
Copy Markdown
Member

would be good to see the ci running somewhere to get an idea of what is being added

Signed-off-by: Ted Poole <tpoole@redhat.com>
Comment thread ci/do_ci.sh Outdated
build_openssl_presubmit
;;

openssl.asan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this as well. Let's make this pr only touching the openssl bits

@jwendell

jwendell commented Nov 3, 2026

Copy link
Copy Markdown
Member

@tedjpoole since you're re-using this pr, could you please also change title/description to reflect last changes?

Signed-off-by: Ted Poole <tpoole@redhat.com>
@tedjpoole tedjpoole closed this Nov 3, 2026
@tedjpoole

Copy link
Copy Markdown
Contributor Author

Replaced by #48972

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants