Skip to content

Document exact model CLI review workflow #255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jmanico merged 1 commit into from
Oct 20, 2026
Merged

Document exact model CLI review workflow #255

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 186 additions & 11 deletions MODELS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,14 @@ merging, releases, or changes to those controls.

## Lineup

Use the lineup below, verifying current availability through the installed tools and
official provider documentation. Record the actual model ID and tool version;
report unavailable slots and substitutions explicitly. Do not claim a review
ran merely because a CLI accepted a model name.
Use these exact model IDs and CLI routes. This is the repository's selected
lineup, not a request to discover the latest models. Start with the local
commands below; do not repeat broad web searches to map model names to CLIs.
Check installed CLI versions and actual authenticated execution. Consult the
linked official documentation only when a command, access failure, deprecation,
or requested lineup change needs investigation. Record requested IDs, identities
reported by the tools, and tool versions separately. CLI acceptance of a name
does not establish that a review ran.

| Step | Model | Java Encoder responsibility |
| --- | --- | --- |
Expand All @@ -24,19 +28,173 @@ ran merely because a CLI accepted a model name.
| 3. Independent review | Sonnet 5.5 (`claude-sonnet-5-5`, through Claude Code) | Checks Javadocs, examples and migration notes for accurate context selection, complete surrounding syntax, concision and plain US English. |
| 4. Resolve and deliver | Astra (`gpt-6-astra`, through Codex) | Resolves findings with evidence, validates the final revision, and reports the verdict, remaining blockers and results to the operator. |

### Start here: scope and orchestration

The coordinating Codex agent owns the task and accepted edits. This workflow
explicitly permits independent model reviews in parallel. Use the installed
`codex` executable for OpenAI models and `claude` for Anthropic models; a Codex
subagent cannot select a Claude model. Do not recursively launch this lineup
from each reviewer.

For implementation, run Astra's build and Opus's second build sequentially,
then freeze the revision for independent reviews. Use an isolated branch/checkout
when the user's working tree has unrelated changes. Build agents may edit only
their assigned checkout under the session's existing permissions and repository
controls. Never bypass approval or sandbox controls for either CLI. The commands
below are for reviews, not for granting build agents write access.

For an issue or proposed behavior review, skip the implementation steps and
give the reviewers the same current SHA, proposal, and relevant source. State
that no implementation diff exists. Opus can review feasibility and test gaps
without making a second build. Do not create a fix, publish a comment, or promise
a release merely because a review was requested.

1. Read this file and the repository guidance; record `git status --short` and
`git rev-parse HEAD`. Preserve the user's uncommitted work.
2. Prepare one immutable review packet outside the working tree. Include the
reviewed SHA, task intent, complete diff (or explicit proposal), full relevant
files with line numbers, callers/tests, repository guidance, and relevant
primary-specification excerpts with retrieval dates and source links. Retain
the packet and its SHA-256. If reviewing uncommitted changes, identify the SHA
as the baseline and hash the captured diff too; do not call it a committed
revision. Do not include credentials or unrelated private files.
3. Record `codex --version` and `claude --version`. Check Codex's authenticated
catalog once per review session using the `model/list` procedure below.
Reuse that response for all Codex slots. Perform the Blue preflight once before
its actual review. For Claude, a successful review result is also the access
check; a separate greeting probe is unnecessary.
4. Run independent reviews against that same packet. Keep each slot's role
instruction separate from the shared evidence, then combine them into a
retained per-slot input file. Require findings before other reviewers'
conclusions are shared. Tell reviewers not to edit, publish, delegate, or run
other models. The coordinator runs additional tests or retrieves specifications
requested by any reviewer, then supplies that evidence for follow-up.
5. Retain stdout, stderr, exit status, reported identity, and verdict per slot.
Resolve findings against code, specifications, and tests. Send material fixes
back to affected reviewers at the new SHA.
6. Deliver the verdict and evidence with any unfilled slots and unrun checks.
A review after a release is supplemental, never a retroactive release gate.

### CLI recipes

These routes were checked with **Codex CLI 0.163.0** and **Claude Code 2.1.304**
on **2026-10-20**. Recheck local `--help` when the installed version changes or
a flag fails; do not silently change models. The commands assume absolute
`review_dir`, `review_packet`, and `review_role_file` paths. The packet contains
shared evidence; the role file contains the slot's instructions. Use a fresh
directory and one output basename per run, including follow-up reviews.

```sh
codex --version
claude --version
mkdir -p "$review_dir/empty"
```

After selecting `review_model` and its role file, prepare the per-slot input:

```sh
review_model=gpt-6.1-sol
review_input="$review_dir/$review_model.input.txt"
cat "$review_role_file" "$review_packet" > "$review_input"
shasum -a 256 "$review_packet" "$review_input"
```

Use `gpt-6.1-sol` for independent review or `gpt-6-astra` for proposal analysis
and resolution. Feed the input through stdin rather than embedding it in a shell
argument. The final report and JSONL event log are separate artifacts:

```sh
codex exec --ignore-user-config --ephemeral --skip-git-repo-check \
-C "$review_dir/empty" -s read-only \
-m "$review_model" -c 'model_reasoning_effort="high"' \
--json -o "$review_dir/$review_model.txt" \
< "$review_input" > "$review_dir/$review_model.jsonl" \
2> "$review_dir/$review_model.stderr"
review_status=$?
printf '%s\n' "$review_status" > "$review_dir/$review_model.exit"
```

For Claude, use exactly `claude-opus-5-5`, `claude-mythos-5-1`, or
`claude-sonnet-5-5`. This packet-only recipe disables tools and MCP servers;
provide source/specification/test evidence in the packet. Select the Claude ID
and prepare its per-slot input as above before running:

```sh
(
cd "$review_dir/empty" || exit 1
claude -p --model "$review_model" --effort high \
--output-format json --tools "" \
--strict-mcp-config --mcp-config '{"mcpServers":{}}' \
--disable-slash-commands --no-session-persistence \
< "$review_input" > "$review_dir/$review_model.json" \
2> "$review_dir/$review_model.stderr"
review_status=$?
printf '%s\n' "$review_status" > "$review_dir/$review_model.exit"
exit "$review_status"
)
```

Tool disabling is not full process isolation: installed settings, hooks and
managed policies can still apply. Preserve required controls and record any
behavior that adds context or affects the review. Do not use `--bare` as an isolation
shortcut: in the verified version it disables OAuth/keychain authentication.
Do not add `--fallback-model` or permission-bypass flags. `Opus 5.5 Cyber` is
not a separate model ID in this lineup; use `claude-opus-5-5` and report any
provider access program only if the tool actually attests it.

For every command, capture the exit code immediately. For Codex require a
completed turn and a substantive final verdict; an empty output file or a
`turn.failed` event is not a review. For Claude inspect `is_error`, `result`,
and the `modelUsage` keys in the JSON response. Record the identity actually
reported, including any unexpected model. A model's own claim about its name
is not identity evidence. Codex CLI 0.163.0's observed JSONL turns did not expose
an underlying backend model identity: retain the catalog match and requested ID,
and record the backend as unreported. Do not invent an identity field. If a tool
reports a different primary reviewer, mark the requested slot unfilled and
describe the actual run; record auxiliary model usage separately. Do not claim
tests ran based on an analysis-only response. No automatic model substitution
is allowed.

Official references for troubleshooting:
[Codex CLI](https://developers.openai.com/codex/cli/reference),
[Codex app-server](https://developers.openai.com/codex/app-server),
[Astra](https://developers.openai.com/api/docs/models/gpt-6-astra),
[Sol 6.1](https://developers.openai.com/api/docs/models/gpt-6.1-sol),
[Claude models](https://platform.claude.com/docs/en/models/overview),
[Mythos 5.1](https://platform.claude.com/docs/en/models/mythos-5-1/overview),
and [Claude CLI](https://code.claude.com/docs/en/cli-reference).

### Daybreak Blue preflight

This slot means the **Daybreak Blue entry in the Codex model picker**. Do not
silently replace it with GPT-6 Sol plus Blue access: those are different model
selections. The [API model catalog](https://developers.openai.com/api/docs/models/gpt-daybreak-blue-latest)
marks the alias deprecated, but an operator may explicitly request the installed
Codex picker entry. Check its availability for the current account instead of
marks the alias deprecated, but this workflow selects the installed Codex picker
entry with Blue access. When the user invokes this workflow, attempt this slot
and report any failure; do not treat it as an optional substitute for Sol.
Check its availability for the current account instead of
inferring access from environment variables, configuration keys or API deprecation.

Record `codex --version` and inspect the authenticated Codex app-server
`model/list` result: match the display name, model ID and
`availableAccessPrograms.cyber`. Then use an empty temporary directory for this
harmless probe (replace `/path/to/empty-directory`):
`availableAccessPrograms.cyber`. Use `codex app-server --stdio` with newline
delimited JSON-RPC. Send `initialize`, wait for its response, send `initialized`,
then request `model/list`:

```json
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"clientInfo":{"name":"encoder-review","version":"1.0"},"capabilities":{"experimentalApi":true}}}
{"jsonrpc":"2.0","method":"initialized"}
{"jsonrpc":"2.0","id":2,"method":"model/list","params":{"includeHidden":true}}
```

Retain the catalog response and follow `nextCursor` if it is non-null. Match
`displayName: "Daybreak Blue"`, `model: "gpt-daybreak-blue-latest"` and cyber
access `daybreakBlue`. The CLI spelling is `daybreak_blue`; the app-server
spelling is `daybreakBlue`. Close the catalog process after collecting results.
Do not dump auth files or infer access from environment variables.

Then use an empty temporary directory for this harmless probe
(replace `/path/to/empty-directory`):

```sh
codex exec --ignore-user-config --ephemeral --skip-git-repo-check \
Expand All @@ -47,7 +205,22 @@ codex exec --ignore-user-config --ephemeral --skip-git-repo-check \

Require a completed turn with `READY`, not just CLI acceptance of the name.
For the actual review, use the same explicit model and program with the immutable
review packet required below; a preflight is not a code review. Keep repository
review packet and a Blue role instruction; a preflight is not a code review.
Prepare `review_input` for this slot as above:

```sh
codex exec --ignore-user-config --ephemeral --skip-git-repo-check \
-C "$review_dir/empty" -s read-only \
-m gpt-daybreak-blue-latest --cyber-access-program daybreak_blue \
-c 'model_reasoning_effort="high"' \
--json -o "$review_dir/daybreak-blue.txt" \
< "$review_input" > "$review_dir/daybreak-blue.jsonl" \
2> "$review_dir/daybreak-blue.stderr"
review_status=$?
printf '%s\n' "$review_status" > "$review_dir/daybreak-blue.exit"
```

Keep repository
review and approval controls in force. Do not enable model/provider fallback.
If using app-server directly, set `allowProviderModelFallback=false` on
`thread/start` and `cyberAccessProgram=daybreakBlue` on `turn/start`.
Expand All @@ -65,7 +238,8 @@ Do not change account entitlements or silently substitute another model. Report
an unfilled slot as a coverage gap, not a passing review.

On 2026-10-18, Codex 0.163.0 advertised the exact picker entry with Blue access,
and its explicit probe completed successfully. The earlier #243 review had
and its explicit probe completed successfully. The catalog and explicit probe
were verified again on 2026-10-20 with the same CLI version. The earlier #243 review had
not attempted that probe; its unavailable label did not establish an access
failure. This dated result does not guarantee future access. Any review added
after publication must be labeled supplemental, not retroactively counted as a
Expand Down Expand Up @@ -134,7 +308,8 @@ later edits.
Require a **MERGE**, **REQUEST_CHANGES**, or **CLOSE** recommendation with severity,
exact file/line locations, the failing context or contract, evidence, and a
concrete fix. Separate introduced regressions from inherited issues and clearly
label hypotheses. A MERGE recommendation is advisory, not permission to merge.
label hypotheses. All model recommendations, including Astra's final synthesis,
are advisory; none authorize merging, closing, or publication.

For security claims, read the primary specification or implementation and verify
that it supports the claim. Identify the parser and version used by a test and
Expand Down
Loading