Skip to content

fix(cli, api): keep the overall extraction instruction in ixp packages - #455

Merged
anachili merged 1 commit into
masterfrom
chi.heng/package-overall-extraction-instruction
Oct 8, 2026
Merged

anachili merged 1 commit into
masterfrom
chi.heng/package-overall-extraction-instruction

Conversation

@anachili

@anachili anachili commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

An IXP project's overall extraction instruction goes into the model's prompt, but re package download / re package upload dropped it. A project uploaded from a package ran a different prompt from the one it was downloaded from, and scored differently in Measure, with nothing to show anything was missing.

A customer promoting projects from a Test to a Production tenant hit this. The annotated data transferred exactly — same documents, same annotations, field for field — but the project score fell by twelve points, concentrated on the identifier and name fields that most depend on the instruction to tell them apart.

It was missed because the instruction lives on the default label group, not on the dataset, so the label defs the group is created with do not carry it. Field group, field and field type instructions already round tripped — this was the only level that did not. It is the last open item of RE-7680, after model config in #392 and field types in #394.

What changed

  • LabelGroup gains instructions, so re package download writes it into the package's dataset.json.
  • UpdateDataset gains default_label_group_instructions, sent as _default_label_group_instructions.
  • re package upload sends it for IXP projects, in the update_dataset call that already restores the model config. Every dataset has a default group from creation, so it does not need a separate request after the group's label defs.

Only sent when the package has an instruction, so an older package leaves the new project's own default alone instead of blanking it.

Breaking (reinfer-client API): LabelGroup and UpdateDataset each gain a field, so struct literals of them need an extra initializer.

Test plan

  • Two unit tests: the instruction parses off a dataset response, and UpdateDataset serializes it under the right key and omits it when unset.
  • cargo test -p reinfer-client — 41 passed. cargo clippy --workspace --all-targets and cargo fmt --all --check clean.
  • Updated the label_groups literal in the test_create_update_dataset_custom integration test.
  • Verified against the platform source rather than a live tenant: the dataset response always carries label_groups[].instructions (LabelGroup.instructions is a required field), and PUT accepts _default_label_group_instructions and applies it to the default group (_update_custom_label_groups_with_default_group_instructions, in place since before March 2026, so every ring has it). Both ends cap the instruction at 4096 characters, so a packaged instruction can never be too long for the update.
  • Not covered: the score itself. That needs an annotated project, a retrain and Measure, so it is worth confirming once on a real promotion that a package now reproduces the source's score.

Not in scope

  • No CLI flag sets the instruction on re update dataset — it only comes from a package.
  • Packages still drop field level business rules (rule_set on MoonFormFieldDef).
  • Per choice instructions cannot reach the model at all: the platform hardcodes instructions="" when serializing field choices, so nothing the CLI sends would take effect.

RE-7680

An IXP project's overall extraction instruction is part of the model's
prompt, but packages dropped it. A project uploaded from a package ran a
different prompt from the one it was downloaded from, and scored
differently in Measure, with nothing to show anything was missing.

The instruction lives on the `default` label group, not on the dataset,
and neither `LabelGroup` nor `UpdateDataset` had a field for it. Add one
to each, and send it from `re package upload`.

Only send it when the package has one, so that an older package leaves
the new project's own default alone instead of blanking it.
@anachili
anachili force-pushed the chi.heng/package-overall-extraction-instruction branch from f70a83a to 2279f3c Compare October 8, 2026 13:03
@anachili anachili changed the title fix(cli, api): round trip the overall extraction instruction in ixp packages fix(cli, api): keep the overall extraction instruction in ixp packages Oct 8, 2026
@anachili
anachili merged commit 3c6b1af into master Oct 8, 2026
3 checks passed
@anachili anachili mentioned this pull request Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant