fix(challenges): boot when sk/session placeholders are empty - #226
Conversation
Compose always sets BASE_CHALLENGE_SK_FILE and BOUNTY_SESSION_SECRET_FILE. remote-deploy materializes empty files so Docker does not create directories at those paths; both bins treated that as a hard boot error (exit 1), which crash-looped staging master and 502/503'd /challenge/*/health. Missing/invalid sk now warns and skips signing. Empty session secret falls back to ephemeral bytes. InferenceOffer missing stays warn + can_score false (unchanged). remote-deploy fills a 32-byte session secret from urandom when the file is missing or 0-length. Co-authored-by: Mathis <echobt@users.noreply.github.com>
Greptile SummaryThe bounty and proof challenge services now remain available when deployment-created signing-key placeholders are empty or unavailable. Bounty falls back to fresh in-memory session material when its session-secret file is unreadable or empty, while remote deployment creates persistent random session material when possible. T-Rex validation blockedThe completed health-check evidence could not be uploaded or have its artifact references verified because the validation environment disabled further access after reaching its step limit. The saved execution records show both updated health endpoints returned HTTP 200 with empty and unavailable placeholders. Confidence Score: 5/5Safe to merge: no unresolved product defects were identified, and the recorded executable checks show the updated services remain reachable with the deployment placeholder states addressed by this change. No final findings remain. The recorded checks exercised the changed boot paths and health endpoints, but their artifact upload status could not be verified after the validation environment disabled further access. Files Needing Attention: No files require follow-up.
What T-Rex did
Reviews (1): Last reviewed commit: "fix(challenges): boot when sk/session pl..." | Re-trigger Greptile |
Summary
Staging master has been crash-looping
bounty-challengeandproof-challenge(exit 1 →Restarting) since well before #223. GHA 33952922508 is the same pattern as #224 / #222 / #210: gateway/healthzis fine,/challenge/{bounty,proof}/healthstays 502/503 because the processes never listen.Root cause is not InferenceOffer. #223 already treats a missing/closed offer as warn +
can_score=false/ submit 503.ProofPin::validateaccepts the committed pin with empty[inference].model/base_url. ComposePROOF_INFERENCE_OFFER_FILE=/run/base/proof/inference_offer.jsonis a file inside the existingdeploy/secrets/proofdirectory mount — a missing file is ENOENT, not a Docker bind-mount poison.The hard boot errors are the compose-always-set secret files:
BASE_CHALLENGE_SK_FILEdeploy/secrets/bounty_sk,proof_sk: >empty file (so Docker does not create a directory at a file mount)load_challenge_secret→Err→ exit 1BOUNTY_SESSION_SECRET_FILEdeploy/secrets/bounty/session_secretBoth services therefore die before
/health. The 502/503 oscillation is the gateway proxy hitting a restarting backend, not a failed backend registry (reseed returned 201).This PR:
BASE_CHALLENGE_SK_FILE→ warn, skip emitter (same as unset)BOUNTY_SESSION_SECRET_FILE→ warn, ephemeral 32 bytesBASE_CHALLENGE_SK_FILE→ warn, keep serving/healthremote-deploy.sh: fillsession_secretfrom urandom when missing or 0-length (matcheslocal-e2e.sh)Not invented: no InferenceOffer JSON, no judge provider URL/key. Missing offer stays fail-closed on submit.
Host files that must still exist for real weight (not for boot):
deploy/secrets/bounty_skanddeploy/secrets/proof_sk— 32 raw bytes or 64 hex, pubs matching the staging trust root (#215ceremony). Empty placeholders keep/healthup but cannot sign leaves (bounty seal will 409 D24 until a real mini-secret is installed).deploy/secrets/bounty/session_secret— non-empty for pairing that survives restart (remote-deploy now generates one).deploy/secrets/proof/inference_offer.jsonandinference_api_key— optional. Missing → warn,can_score=false, submit 503. Do not invent a provider.Greptile
Every PR is reviewed by Greptile before merge. Config:
.greptile/.@greptileai reviewTest plan
cargo test -p bounty-challenge-bin -p proof-challenge-bincargo fmt --all -- --checkcargo clippy -p bounty-challenge-bin -p proof-challenge-bin --all-targets -- -D warningscargo run -p xtask -- loc-cap(bins still well under cap)Risk
No emission / scoring / consensus change. Staging
/healthshould come up without a live judge offer. Operators still need realbounty_sk/proof_skbefore a seal can include signed leaves.Naming
I did not rename
BASE_*environment variables, deployed host paths(
/opt/base,/run/base, …), GHCRbaseintelligence/basepackage names, orbase-*-v1cryptographic domain tags, unless this PR’s purpose is a coordinatedcutover documented in
docs/NAMING.md.