codex-516-hook — warn the user when gpt-5.5 truncates in Codex

gpt-5.5 serving Codex sometimes stops its hidden reasoning at exactly 516 output tokens and answers anyway, in its normal confident register (openai/codex#30364, HN discussion) — under both ChatGPT-subscription and API-key auth. In our measurements, truncated turns were wrong on many nontrivial reasoning tasks, while every untruncated run of the same prompts was correct.

This Stop hook reads the per-turn reasoning_output_tokens that Codex already writes to the local session transcript. When a turn hits exactly 516, a [c516] warning renders in the Codex TUI so you know the answer above it is suspect before acting on it. The warning is user-facing only: the hook does not inject anything into the model's context or tell the next agent turn what to do.

If you want the truncation automatically repaired rather than flagged, see CodexCont and codexcomp: they run a local proxy that catches the truncation in-flight and auto-continues the reasoning. This hook is the lighter-touch complement — it leaves Codex's connection to the API untouched and simply flags truncations to you after the fact.

Install

Requirements: Python 3.8+ on PATH (stdlib only). We've tested on codex-cli 0.142.3 on Windows only.

1. Clone this repo anywhere; on macOS/Linux chmod +x c516-hook.sh. The shims locate hook.py relative to themselves, so keep the files together.
2. Add to ~/.codex/config.toml, using the absolute path of your clone. command is what macOS/Linux runs; command_windows overrides it on Windows — set whichever applies (or both):

[[hooks.Stop]]
[[hooks.Stop.hooks]]
type = "command"
command = '/path/to/clone/c516-hook.sh stop'
command_windows = 'C:\path\to\clone\c516-hook.cmd stop'

3. Start Codex interactively and approve the one-time trust prompt for the Stop-hook registration (/hooks shows its status). Until trusted, Codex silently skips the hooks; sessions already open when you edited the config don't load them either — use a fresh session.

Upgrading from the earlier two-hook version

Remove only the [[hooks.UserPromptSubmit]] block whose command invokes c516-hook; leave unrelated submit hooks alone. Keep the [[hooks.Stop]] registration shown above, including its stop argument, then start a fresh Codex session. The current script also ignores stale submit-side invocations so an already-open session cannot inject the retired warning into model context.

Testing that it works

The puzzles/ directory contains two prompts with machine-verified unique answers, tuned against this bug.

1. puzzles/clamp-trigger.txt — makes the warning fire (correct answer: 24)

An easy-looking drawing puzzle that triggered the truncation in ~90% of our runs (gpt-5.5, reasoning effort xhigh). It is deliberately easy enough that the model usually still answers 24 correctly even when truncated — its job is to fire the hook reliably, not to produce wrong answers.

Expected behavior in a fresh interactive session:

1. Paste the puzzle. The turn ends fast (~20 s; a healthy run takes longer and visibly thinks). If it truncated, the [c516] Stop warning appears in the TUI right after the answer: "This turn's reasoning was truncated at exactly 516 tokens…".
2. No warning? Check the log (below). If the previous turn's count isn't 516, the run escaped the bug (~10% on this puzzle) — retry in a fresh session.

2. puzzles/wrong-answer-zebra.txt — demonstrates the damage (correct answer: house 2)

A zebra-style logic puzzle (unique solution, verified by exhaustive search; the fish is never mentioned in the clues, so the answer requires solving the whole grid). It truncates in only ~15% of runs, but truncated runs answered wrong in 5 of our 6 observed truncations (scattered guesses: 1, 4, 5) — with a confident-sounding justification. Healthy runs think for a minute or more and consistently answer house 2.

Re-roll it in fresh sessions until one answers suspiciously fast: that run should combine a wrong answer with the [c516] TUI warning — the full failure mode, caught in the act.

The log

Every Stop-hook invocation is logged (default ~/.local/state/c516-hook.log, override with C516_HOOK_LOG), one JSON line per invocation, including the just-ended turn's full reasoning-count list — useful both for verifying the hook and for passively monitoring how often the bug hits your real sessions:

{"now": "...", "mode": "stop", "counts": [516], "truncated": [516], "emitted": true, ...}

To smoke-test the plumbing without the bug: start Codex with C516_FORCE_EMIT=1 to force the warning on the next Stop-hook invocation.

Expected behavior summary

Situation	Stop-hook result
Turn had a response at exactly 516 reasoning tokens	User-facing TUI warning at turn end
Healthy turn (any other counts, incl. 1034/2070/2588/3106)	silent
First turn with no model response	silent
Transcript missing, unreadable, or format changed	silent (fail-open, logged)

Counts at 1034/1552/2070/2588/3106… (516+518k) are common in healthy runs — in our data every such run answered correctly (it appears to be count quantization, not harmful truncation), so the hook deliberately ignores them. If the serving behavior changes, adjust TRUNCATION_VALUES in hook.py.

What triggers it: measurements (2026-07-05)

The truncation follows Codex's request shape, not the account type. Same trigger prompt (puzzles/clamp-trigger.txt), gpt-5.5, reasoning effort xhigh, codex-cli 0.142.3:

Request	Runs	exactly 516 reasoning tokens
codex exec, ChatGPT auth	20	18
codex exec, API-key auth (same context)	20	18
bare /v1/responses, API key, same prompt, no tools	20	0

Exactly what in Codex's request shape triggers it is unknown; a quick check hinted that tool declarations matter and input size alone doesn't, but we didn't chase it further.

Reasoning counts on every path land disproportionately on the 516+518k ladder (1034, 1552, 2070, 2588, ...) — the quantization comb shows even in healthy runs; the harmful event is specifically termination at the first rung.

Design note: mid-turn vs final-response truncation

A turn can contain many model responses (each tool-call round is one), and the hook fires when any of them hit exactly 516. In our own session history the hits split roughly 55% mid-turn-only / 40% final-response / 5% single-response turns. Those differ in severity: mid-turn truncation clips intermediate steps (what to read, what to explore next) — real but diffuse damage — while final-response truncation clips the response that writes the actual answer, which is where the wrong-answer correlation was measured.

The warning text is calibrated to the final-response case, so when it fires treat it as an upper bound on alarm: the log line's counts list shows where the 516s sat (last element = the answer-writing response). A natural refinement would be to soften the displayed warning when the hits were mid-turn only; we haven't shipped that because we haven't yet tested it in live use.

Caveats

• Not an account-type issue: every Codex user is exposed regardless of how they pay (see the measurements section above).
• A turn whose reasoning naturally lands on exactly 516 would false-positive — and from where this hook sits it would be indistinguishable from the bug, so we can't count how often that happened to us. Two indirect reasons to think it's rare: healthy runs landed near 516 without hitting it (503, 533, 554, 595), and the issue's aggregate data puts non-gpt-5.5 models' exact-516 rate at ~1.3%. But on long terms the probability increases - we haven't done enough testing here yet.
• Maturity: this works on the author's setup (Windows 11, codex-cli 0.142.3, ChatGPT Pro, Python 3.12), verified end-to-end there, but it has not been tested much beyond that — treat it as a working example rather than hardened tooling. The rollout format is platform-independent, but the .sh shim in particular is untested. Reports welcome.
• The hook reads only the local session transcript; nothing is sent anywhere.

License

MIT