Repository navigation
Fixes open-metadata/openmetadata-collate#7380: Keep JAVA_TOOL_OPTIONS secrets out of the startup logs - #35402
Conversation
… secrets out of the startup logs
The JVM prints JAVA_TOOL_OPTIONS to stderr at startup ("Picked up
JAVA_TOOL_OPTIONS: ..."), so a truststore password passed that way ended up
in the server and migration logs. bin/openmetadata-server-start.sh,
bootstrap/openmetadata-ops.sh and bin/openmetadata.sh now pass the options to
java as arguments, ahead of their own so precedence is unchanged, and unset
the variable. A value with quotes is left to the JVM, which strips quotes
that shell word splitting would keep.
The docker smoke test starts the image with JAVA_TOOL_OPTIONS set and checks
that the value stays out of the log while the options still reach the JVM.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
Passing the options as plain arguments traded the startup log for ps: /proc/<pid>/cmdline is readable by every local user, while /proc/<pid>/environ is readable only by the process owner. The launchers now hand the options to the java launcher as an @argfile on fd 3, fed by a heredoc, so the JVM's command line only shows @/dev/fd/3 and no file is left behind. musl JDKs re-exec the launcher with the options expanded, so on Alpine-based images ps still shows them. A value containing '#' now also stays in the environment, because argfile syntax reads '#' as the start of a comment. openmetadata-ops.sh now builds its classpath before taking the variable, so the dev-mode mvn call still sees JAVA_TOOL_OPTIONS. The docker smoke test also checks that the value is not on PID 1's command line.
The argfile could not carry values containing '#' or quotes, so those stayed in the environment and the JVM still printed them, with no warning. Generated passwords often contain '#'. The launchers now write the options to a -XX:VMOptionsFile instead, which the JVM parses exactly like JAVA_TOOL_OPTIONS, quotes and '#' included, without echoing it. The file is created in /dev/shm (tmpfs, writable even with a read-only root filesystem) or $TMPDIR, opened on fd 3 and unlinked at once, so nothing stays on disk and ps only shows -XX:VMOptionsFile=/dev/fd/3. The JVM reads the file itself after the musl launcher re-exec, so this also holds on Alpine-based images. If neither directory is writable, the launchers print a one-line warning without the value and leave the variable to the JVM. The docker smoke test now puts a '#' in its value and checks that no options file is left behind.
Code Review ✅ Approved 1 closed / 1 findings🟡 Medium risk · Changing JAVA_TOOL_OPTIONS delivery across server and migration launchers could alter JVM option handling or prevent those processes from starting. Moves ✅ 1 closed✅ Security: Secret moves from env to world-readable argv instead of out of logs
VerificationRules No rules evaluated Functional validation Not enabledAuto-approval Not enabledFor more verification details, comment OptionsDisplay: compact → Showing compact verification results. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
Describe your changes:
Fixes open-metadata/openmetadata-collate#7380
When
JAVA_TOOL_OPTIONSis set, for example to pass a truststore password from a Kubernetes secret, its value showed up in the server and migration logs:Why it happened: our scripts never echo it. The JVM prints
JAVA_TOOL_OPTIONSto stderr at startup whenever the variable is set, and no flag turns that off. So every launcher that starts a JVM with the variable in its environment leaked it: the server container, the Helmrun-db-migrationsinit container and composeexecute-migrate-all(both viaopenmetadata-ops.sh), and bare metal (logs/catalog.err).What changed:
bin/openmetadata-server-start.sh,bootstrap/openmetadata-ops.shandbin/openmetadata.shnow writeJAVA_TOOL_OPTIONSto a-XX:VMOptionsFileand unset the variable.JAVA_TOOL_OPTIONS, quotes and#included, and doesn't echo it./dev/shm(tmpfs, still writable with a read-only root filesystem), falling back to$TMPDIR. It is opened on fd 3 and unlinked at once, so nothing stays on disk.psor/proc/<pid>/cmdline, only shows-XX:VMOptionsFile=/dev/fd/3. That also holds on musl JDKs (Alpine), because the JVM reads the file itself after the launcher re-execs.openmetadata-ops.shbuilds its classpath first, so in dev mode themvncall still seesJAVA_TOOL_OPTIONS.sh, so it runs under bash, dash and busyboxshalike.scripts/docker_smoke_test.shgets a new assertion. It starts the image withJAVA_TOOL_OPTIONS="-XX:+PrintCommandLineFlags -Dsmoke.secret=...#1", then checks four things:/dev/shmor/tmp;ps. The second used an@argfile, which can't carry#or quotes, so those values stayed in the environment and still printed. This version has neither problem.The
2.0backport is #35518. The Collate server launchers get the same change in open-metadata/openmetadata-collate#7411. The Collate image takesbootstrap/from this repo, so its migration init container needs this PR.Type of change:
High-level design:
N/A — small change.
Tests:
Use cases covered
JAVA_TOOL_OPTIONSset, its options still reach the JVM. The value no longer appears in the server log, the migration log,logs/catalog.err, or the JVM's command line.#or quotes arrive exactly asJAVA_TOOL_OPTIONSwould deliver them.JAVA_TOOL_OPTIONSones, as before.JAVA_TOOL_OPTIONS, nothing changes. With no writable temp dir, the launcher prints one warning line and falls back to the old behavior.Unit tests
Not applicable: shell scripts only. The new docker smoke test assertion is the regression test.
Backend integration tests
make java_affectedselects the full IT suite, but only because no impact-map area ownsbin/**. The ITs start the server in-process and never run these scripts, so I left them to the merge queue.Ingestion integration tests
Playwright (UI) tests
Manual testing performed
Reproduced the leak on
docker.getcollate.io/openmetadata/server:2.0.5, withJAVA_TOOL_OPTIONS="-Dsecret=hunter2". The server log and./bootstrap/openmetadata-ops.sh --helpboth printPicked up JAVA_TOOL_OPTIONS: -Dsecret=hunter2.Confirmed the review's premise in a container: a second uid can read a process's
/proc/<pid>/cmdline, but gets "permission denied" on its/proc/<pid>/environ.Ran this branch's
bin/openmetadata-server-start.shandbootstrap/openmetadata-ops.shwith stub main classes. Each stub reports whether the property arrived, whetherJAVA_TOOL_OPTIONSis still in its environment, whether its own command line contains the value, and its max heap.ashon the 2.0.5 Alpine image (musl).-Dsmoke.p=changeit -Xmx256m,-Dsmoke.p=pa#ss, and-Dsmoke.p='pa#s"s' -Dsmoke.q="a b".Results, in every shell:
changeit,pa#ss,pa#s"s,a b), the same as the env var gives;JAVA_TOOL_OPTIONSis gone from the environment and from the command line, including on musl;-Xmx1Gstill wins, and 256 MB for the ops tool;An unmatched quote still makes the JVM refuse to start, as it does with the env var. Its error names
/dev/fd/3, not the value.On
gcr.io/distroless/cc-debian13:debug-nonroot, as uid 65532 with a Temurin 21 JRE:/dev/shm;/dev/shmread-only as well, it prints the one-line warning and falls back.bash -n, plussh -nunder dash and busybox, pass for every changed script. Shellcheck reports the same warnings asorigin/mainand nothing new.On this PR, the "Smoke test the server image" step runs the new assertion against the OSS image, since
bin/**triggers that step.UI screen recording:
Not applicable — launcher scripts only, no UI.
Checklist:
I have read the CONTRIBUTING document.
My PR title is
Fixes <issue-number>: <short explanation>My PR is linked to a GitHub issue via
Fixes #<issue-number>above.I have commented on my code, particularly in hard-to-understand areas.
I have added tests (unit / integration / Playwright as applicable) and listed them above.
Decision check: no record needed. The rule is stated in each launcher, and the smoke assertion enforces it.
I have added a test that covers the exact scenario we are fixing.
🤖 Generated with Claude Code