Skip to content

Fully support lazy and immediate binding #335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jserv merged 8 commits into from
Oct 11, 2026
Merged

Fully support lazy and immediate binding #335

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
86d8178
Remove the DYN_BIND_NOW flag from the codebase
DrXiao Oct 11, 2026
0577bae
Implement a PLT entry calling DL resolver for the AArch64 backend
DrXiao Oct 11, 2026
8a5c718
Implement immediate binding and the corresponding compiler option
DrXiao Oct 11, 2026
ef8287d
Introduce a Makefile variable to determine the binding strategy
DrXiao Sep 9, 2026
7e58283
Improve test suites to validate immediate binding mode
DrXiao Sep 15, 2026
72808a4
Validate immediate binding mode in CI
DrXiao Sep 20, 2026
d5be4bc
Document the usage of the binding option
DrXiao Oct 11, 2026
9fbbe1b
Improve the dynamic linking documentation
DrXiao Oct 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 88 additions & 20 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
# Bootstrap each target and run the consolidated behavioral suite against
# the stage 0 and stage 2 compilers.
host-x86:
name: ${{ matrix.architecture }}/${{ matrix.link_mode }} (${{ matrix.compiler }})
name: ${{ matrix.architecture }}/${{ matrix.link_mode }}/${{ matrix.binding_mode }} (${{ matrix.compiler }})
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
Expand All @@ -31,11 +31,47 @@ jobs:
matrix:
compiler: [gcc, clang]
architecture: [arm, arm64, riscv, x64]
link_mode: [static, dynamic]
link_mode: [dynamic]
binding_mode: [lazy, immediate]
include:
Comment thread
DrXiao marked this conversation as resolved.
- compiler: gcc
architecture: arm
link_mode: static
binding_mode: none
- compiler: clang
architecture: arm
link_mode: static
binding_mode: none
- compiler: gcc
architecture: riscv
link_mode: static
binding_mode: none
- compiler: clang
architecture: riscv
link_mode: static
binding_mode: none
- compiler: gcc
architecture: arm64
link_mode: static
binding_mode: none
- compiler: clang
architecture: arm64
link_mode: static
binding_mode: none
- compiler: gcc
architecture: x64
link_mode: static
binding_mode: none
- compiler: clang
architecture: x64
link_mode: static
binding_mode: none
env:
CC: ${{ matrix.compiler }}
ARCH: ${{ matrix.architecture }}
DYNLINK: ${{ matrix.link_mode == 'dynamic' && '1' || '0' }}
# This variable is ignored when verifying static linking mode
BINDING: ${{ matrix.binding_mode == 'immediate' && 'now' || 'lazy' }}
steps:
- name: Checkout code
uses: actions/checkout@v7
Expand All @@ -45,18 +81,18 @@ jobs:
architecture: ${{ matrix.architecture }}
link-mode: ${{ matrix.link_mode }}
github-token: ${{ github.token }}
# ARCH and DYNLINK are named on the command line rather than left to the
# environment: the build session the tree records assigns ARCH outright,
# ARCH, DYNLINK and BINDING are named on the command line rather than left
# to the environment: the build session the tree records assigns ARCH outright,
# and a makefile assignment wins over the environment.
- name: Build artifacts
run: make ARCH="$ARCH" DYNLINK="$DYNLINK"
run: make ARCH="$ARCH" DYNLINK="$DYNLINK" BINDING="$BINDING"
- name: Unit tests
run: make check ARCH="$ARCH" DYNLINK="$DYNLINK"
run: make check ARCH="$ARCH" DYNLINK="$DYNLINK" BINDING="$BINDING"
- name: Upload the test logs
if: failure()
uses: actions/upload-artifact@v7
with:
name: logs-${{ matrix.compiler }}-${{ matrix.architecture }}-${{ matrix.link_mode }}
name: logs-${{ matrix.compiler }}-${{ matrix.architecture }}-${{ matrix.link_mode }}-${{ matrix.binding_mode }}
path: |
out/*.log
if-no-files-found: ignore
Expand All @@ -68,7 +104,7 @@ jobs:
# sanitizer step of host-x86 used to do. Building "sanitizer" first, in a
# job that builds nothing else, is what puts the instrumentation in.
sanitizer:
name: Sanitizers ${{ matrix.architecture }}/${{ matrix.link_mode }}
name: Sanitizers ${{ matrix.architecture }}/${{ matrix.link_mode }}/${{ matrix.binding_mode }}
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
Expand All @@ -81,10 +117,26 @@ jobs:
# roughly fifteen hours. Restoring the dimension means finding out
# why first.
architecture: [arm, arm64, riscv, x64]
link_mode: [static, dynamic]
link_mode: [dynamic]
binding_mode: [lazy, immediate]
include:
- architecture: arm
link_mode: static
binding_mode: none
- architecture: riscv
link_mode: static
binding_mode: none
- architecture: arm64
link_mode: static
binding_mode: none
- architecture: x64
link_mode: static
binding_mode: none
env:
ARCH: ${{ matrix.architecture }}
DYNLINK: ${{ matrix.link_mode == 'dynamic' && '1' || '0' }}
# This variable is ignored when verifying static linking mode
BINDING: ${{ matrix.binding_mode == 'immediate' && 'now' || 'lazy' }}
steps:
- name: Checkout code
uses: actions/checkout@v7
Expand All @@ -95,9 +147,9 @@ jobs:
link-mode: ${{ matrix.link_mode }}
github-token: ${{ github.token }}
- name: Build the stage 0 compiler with sanitizers
run: make sanitizer ARCH="$ARCH" DYNLINK="$DYNLINK"
run: make sanitizer ARCH="$ARCH" DYNLINK="$DYNLINK" BINDING="$BINDING"
- name: Sanitizer-enabled stage 0 tests
run: make check-sanitizer ARCH="$ARCH" DYNLINK="$DYNLINK"
run: make check-sanitizer ARCH="$ARCH" DYNLINK="$DYNLINK" BINDING="$BINDING"

# Preprocess shecc with itself, then compile the result: the stage 1 source
# is the largest input the preprocessor gets, and the only one that exercises
Expand Down Expand Up @@ -135,17 +187,27 @@ jobs:
# AArch64 that is the only place the real loader is exercised: QEMU-user maps
# the image itself, and on an x86-64 host it always presents 4 KiB pages.
host-arm:
name: ${{ matrix.architecture }}/${{ matrix.link_mode }} on Arm64
name: ${{ matrix.architecture }}/${{ matrix.link_mode }}/${{ matrix.binding_mode }} on Arm64
runs-on: ubuntu-24.04-arm
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
architecture: [arm, arm64]
link_mode: [static, dynamic]
link_mode: [dynamic]
binding_mode: [lazy, immediate]
include:
- architecture: arm
link_mode: static
binding_mode: none
- architecture: arm64
link_mode: static
binding_mode: none
env:
ARCH: ${{ matrix.architecture }}
DYNLINK: ${{ matrix.link_mode == 'dynamic' && '1' || '0' }}
# This variable is ignored when verifying static linking mode
BINDING: ${{ matrix.binding_mode == 'immediate' && 'now' || 'lazy' }}
steps:
- name: Checkout code
uses: actions/checkout@v7
Expand All @@ -156,14 +218,14 @@ jobs:
link-mode: ${{ matrix.link_mode }}
github-token: ${{ github.token }}
- name: Build artifacts
run: make ARCH="$ARCH" DYNLINK="$DYNLINK"
run: make ARCH="$ARCH" DYNLINK="$DYNLINK" BINDING="$BINDING"
- name: Unit tests
run: make check ARCH="$ARCH" DYNLINK="$DYNLINK"
run: make check ARCH="$ARCH" DYNLINK="$DYNLINK" BINDING="$BINDING"
- name: Upload the test logs
if: failure()
uses: actions/upload-artifact@v7
with:
name: logs-arm64-host-${{ matrix.architecture }}-${{ matrix.link_mode }}
name: logs-arm64-host-${{ matrix.architecture }}-${{ matrix.link_mode }}-${{ matrix.binding_mode }}
path: |
out/*.log
out/tests/*.log
Expand All @@ -174,15 +236,21 @@ jobs:
# build that produces the object files, and that job has already made them
# without instrumentation.
host-arm-sanitizer:
name: Sanitizers arm/${{ matrix.link_mode }} on Arm64
name: Sanitizers arm/${{ matrix.link_mode }}/${{ matrix.binding_mode }} on Arm64
runs-on: ubuntu-24.04-arm
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
link_mode: [static, dynamic]
link_mode: [dynamic]
binding_mode: [lazy, immediate]
include:
- link_mode: static
binding_mode: none
env:
DYNLINK: ${{ matrix.link_mode == 'dynamic' && '1' || '0' }}
# This variable is ignored when verifying static linking mode
BINDING: ${{ matrix.binding_mode == 'immediate' && 'now' || 'lazy' }}
steps:
- name: Checkout code
uses: actions/checkout@v7
Expand All @@ -193,9 +261,9 @@ jobs:
link-mode: ${{ matrix.link_mode }}
github-token: ${{ github.token }}
- name: Build the stage 0 compiler with sanitizers
run: make sanitizer ARCH=arm DYNLINK="$DYNLINK"
run: make sanitizer ARCH=arm DYNLINK="$DYNLINK" BINDING="$BINDING"
- name: Sanitizer-enabled stage 0 tests
run: make check-sanitizer ARCH=arm DYNLINK="$DYNLINK"
run: make check-sanitizer ARCH=arm DYNLINK="$DYNLINK" BINDING="$BINDING"

coding-style:
name: Coding style
Expand Down
15 changes: 8 additions & 7 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,12 +84,13 @@ TRANSLATION_DEFS = "\#define SHECC_TRANSLATION_DATE \"$(TRANSLATION_DATE)\"\n\#d
STAGE0_FLAGS ?= --dump-ir
STAGE1_FLAGS ?=
DYNLINK ?= 0
BINDING ?= lazy

COMMENTFLOW ?= commentflow
SHFMT ?= shfmt
ifeq ($(DYNLINK),1)
STAGE0_FLAGS += --dynlink
STAGE1_FLAGS += --dynlink
STAGE0_FLAGS += --dynlink -z $(BINDING)

@jserv jserv Oct 11, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing records BINDING, so make DYNLINK=1 followed by make DYNLINK=1 BINDING=now keeps the lazy stage 1 and stage 2 binaries, and the bootstrap comparison still passes because both stages were built the same way. DYNLINK already has this gap. A follow-up could write both values into a stamp file that stage 1 depends on, so switching either one rebuilds the bootstrap.

@DrXiao DrXiao Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has not yet been resolved. I will think of a proper approach to address it later.

@DrXiao DrXiao Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this pull request has been merged, I will submit another one later to address the concern mentioned above.

STAGE1_FLAGS += --dynlink -z $(BINDING)
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
endif

SRCS := $(wildcard $(patsubst %,%/main.c, $(SRCDIR)))
Expand Down Expand Up @@ -236,23 +237,23 @@ uninstall-hooks:

check-stage0: $(OUT)/$(STAGE0) tests/driver.sh
$(VECHO) " TEST STAGE 0\n"
tests/driver.sh 0 $(DYNLINK)
tests/driver.sh 0 $(DYNLINK) $(BINDING)

check-stage2: $(OUT)/$(STAGE2) tests/driver.sh
$(VECHO) " TEST STAGE 2\n"
tests/driver.sh 2 $(DYNLINK)
tests/driver.sh 2 $(DYNLINK) $(BINDING)
Comment thread
DrXiao marked this conversation as resolved.

check-sanitizer: $(OUT)/$(STAGE0)-sanitizer tests/driver.sh
$(VECHO) " TEST STAGE 0 (with sanitizers)\n"
$(Q)cp $(OUT)/$(STAGE0)-sanitizer $(OUT)/shecc
tests/driver.sh 0 $(DYNLINK)
tests/driver.sh 0 $(DYNLINK) $(BINDING)
$(Q)rm $(OUT)/shecc

check-abi-stage0: $(OUT)/$(STAGE0)
tests/$(ARCH)-abi.sh 0 $(DYNLINK);
tests/$(ARCH)-abi.sh 0 $(DYNLINK) $(BINDING);

check-abi-stage2: $(OUT)/$(STAGE2)
tests/$(ARCH)-abi.sh 2 $(DYNLINK);
tests/$(ARCH)-abi.sh 2 $(DYNLINK) $(BINDING);

# Both prerequisites are order-only, and both exist because "make -j" would
# otherwise let a compile start beside the thing it reads. Selecting a target
Expand Down
19 changes: 18 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,20 @@ out/shecc-stage2.elf: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dy
out/shecc-stage2.elf: ELF 64-bit LSB executable, ARM aarch64, dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, no section header
```

When using `DYNLINK=1`, adding `BINDING=now` can also enable the use of **immediate binding**.
```shell
# Dynamic linking with lazy binding
$ make DYNLINK=1
$ make DYNLINK=1 BINDING=lazy

# Dynamic linking with immediate binding
$ make DYNLINK=1 BINDING=now

# BINDING is ignored by the build system if the static linking mode is enabled.
$ make BINDING=now
$ make DYNLINK=0 BINDING=lazy
```

For development builds with memory safety checks:
```shell
$ make sanitizer
Expand All @@ -172,7 +186,7 @@ $ make check-sanitizer

File `out/shecc` is the first stage compiler. Its usage:
```shell
$ shecc [-o output] [+m] [--dot] [--no-libc] [--dump-ir] [--dynlink] [-E] <infile.c>
$ shecc [-o output] [+m] [--dot] [--no-libc] [--dump-ir] [--dynlink] [-z <lazy | now>] [-E] <infile.c>
```

Compiler options:
Expand All @@ -182,6 +196,9 @@ Compiler options:
- `--no-libc` : Exclude embedded C library (default: embedded)
- `--dump-ir` : Dump intermediate representation (IR)
- `--dynlink` : Use dynamic linking (default: disabled)
- `-z`: Use lazy binding or immediate binding (default: lazy)
- This option is valid only when `--dynlink` is also enabled. Otherwise, the compiler
reports an error message if this option is used with static linking.
- `-E` : Preprocess only; write the expanded token stream and stop

Example 1: static linking mode
Expand Down
9 changes: 8 additions & 1 deletion docs/dynamic-linking.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,13 @@
Build the dynamically linked version of shecc:

```shell
# The 'DYNLINK' variable can control the use of dynamic linking mode
$ make ARCH=arm DYNLINK=1
$ make ARCH=riscv DYNLINK=1

# Enable immediate binding using the 'BINDING' variable (default: lazy)
$ make ARCH=arm DYNLINK=1 BINDING=now
$ make ARCH=riscv DYNLINK=1 BINDING=now
```

Next, you can use shecc to build dynamically linked programs by adding the `--dynlink` flag:
Expand All @@ -17,6 +22,9 @@ $ out/shecc --dynlink -o <output> <input.c>
# Use the stage 1 or stage 2 compiler
$ qemu-arm -L <LD_PREFIX> out/shecc-stage2.elf --dynlink -o <output> <input.c>
$ qemu-riscv32 -L <LD_PREFIX> out/shecc-stage2.elf --dynlink -o <output> <input.c>
# Enable immediate binding by the '-z now' option
$ qemu-arm -L <LD_PREFIX> out/shecc-stage2.elf --dynlink -z now -o <output> <input.c>
$ qemu-riscv32 -L <LD_PREFIX> out/shecc-stage2.elf --dynlink -z now -o <output> <input.c>

# Execute the compiled program
$ qemu-arm -L <LD_PREFIX> <output>
Expand Down Expand Up @@ -331,7 +339,6 @@ For the current implementation of dynamic linking, note the following:

* GOT is located in a writable segment (`.data` segment).
* The `PT_GNU_RELRO` program header has not yet been implemented.
* `DT_BIND_NOW` (force immediate binding) is not set.

This implies that:

Expand Down
3 changes: 1 addition & 2 deletions mk/arm64.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,10 @@ ARCH_DEFS = \
\#define DYN_LINKER \"/lib/ld-linux-aarch64.so.1\"\n$\
\#define LIBC_SO \"libc.so.6\"\n$\
\#define LIBDL_SO \"libdl.so.2\"\n$\
\#define PLT_FIXUP_SIZE 0\n$\
\#define PLT_FIXUP_SIZE 32\n$\
\#define PLT_ENT_SIZE 16\n$\
\#define RESERVED_GOT_NUM 3\n$\
\#define R_ARCH_JUMP_SLOT 1026 /* R_AARCH64_JUMP_SLOT */\n$\
\#define DYN_BIND_NOW 1\n$\
"

# An Arm64 Linux host runs this target's output itself, so nothing has to stand
Expand Down
1 change: 0 additions & 1 deletion mk/x64.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ ARCH_DEFS = \
\#define RESERVED_GOT_NUM 3\n$\
\#define R_ARCH_JUMP_SLOT 7 /* R_X86_64_JUMP_SLOT */\n$\
\#define REG_CNT 11 /* rdi rsi rdx rcx r8 r9 rax rbx r14 r12 r13 */\n$\
\#define DYN_BIND_NOW 1 /* this PLT has no lazy-resolution path */\n$\
\#define HAVE_COND_MOVE 1 /* CMOVcc */\n$\
\#define CALLEE_SAVED_REGS 4 /* the file ends rbx r14 r12 r13 */\n$\
"
Loading