Skip to content

fix(llm): let the task schema disambiguate JSON candidates in model prose - #610

Draft
zzwong wants to merge 1 commit into
mainfrom
zzwong/issue-608/schema-disambiguated-extraction
Draft

zzwong wants to merge 1 commit into
mainfrom
zzwong/issue-608/schema-disambiguated-extraction

Conversation

@zzwong

@zzwong zzwong commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Closes #608

Problem

Structured-output decoding already recovers a JSON object from surrounding prose, via extractSingleJSONObject. It requires exactly one syntactically valid top-level object, deliberately, so ambiguous output is never guessed at.

A reasoning preamble routinely breaks that. Running the resident extractor against representative inputs:

preamble before the answer recovered today
no braces here yes
draft: {"decision":"skip"} no — two candidates
I must return JSON, not {}. no — {} is a candidate
answer itself truncated no — correctly unrecoverable

The budget is two validation attempts and an exhausted task is blocking, so a model that drafts before answering loses half its retry budget systematically rather than occasionally.

Change

Extraction returns every balanced valid candidate, and the task's own decoder picks among them: exactly one passing the schema is accepted, zero or several keep the existing error. A lone failing candidate still reports its own schema error, since that describes the real violation.

This cannot loosen validation. A candidate failing DisallowUnknownFields, or carrying the wrong thread_id or schema_version, was never a possible answer — only the bytes reaching the decoder change, never what the decoder accepts. Trailing-data rejection is unaffected: each candidate is a single balanced object, and the raw-text check still runs first.

Evidence

Every behavior change mutation-checked by reverting it alone:

Mutation Fails
Restore the single-candidate rule recovers_the_schema-valid_object_when_the_preamble_drafts_another
Accept when passed >= 1 rejects_two_schema-valid_objects
Skip the i = end advance TestExtractJSONObjects/nested_objects_and_arrays
Ignore json.Valid prose_braces_alongside_one_valid_object, valid_object_nested_in_invalid_prose_braces

go build ./..., go vet ./internal/... and go test -count=1 ./... are clean.

Note on scope

The originally filed issue claimed no extraction existed; that was wrong and the issue has been corrected. A truncated response remains unrecoverable by extraction and is the retry path's job, not this one's.

…rose

Structured output already recovers a JSON object wrapped in prose, but only
when exactly one syntactically valid top-level object is present. A model that
drafts its answer inside a reasoning preamble emits a second valid object, and
even the sentence "return JSON, not {}" is enough, so recovery gave up and a
validation attempt was spent on text that could never parse.

Extraction now returns every balanced candidate and the task's own decoder
chooses among them: exactly one passing the schema is accepted, zero or several
keep the existing error. A candidate that fails DisallowUnknownFields, or
carries the wrong thread or schema version, was never a possible answer, so
this changes which bytes reach the decoder without changing what the decoder
accepts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Structured-output extraction gives up when a reasoning preamble contains its own JSON object

1 participant