Skip to content

fix(petab): read edition-2 parameter records on export, so a new-era free parameter no longer vanishes from the problem (#733) - #735

Merged
wshlavacek merged 1 commit into
mainfrom
petab-export-parameter-records-733
Sep 17, 2026
Merged

wshlavacek merged 1 commit into
mainfrom
petab-export-parameter-records-733

Conversation

@wshlavacek

Copy link
Copy Markdown
Collaborator

Closes #733. Found while fixing #719 (merged as #734).

What was wrong

The exporter picked its free parameters out of the raw config by matching key names:

_VAR_DECL = re.compile(r'(_var$|^var$|^logvar$)')
...
if not _VAR_DECL.search(keyword):
    continue

ploop stores an edition-2 parameter: record under a ('parameter', id) key, which matches
none of the three alternatives — so the record was skipped, not refused. No row reached
_parameter_rows, parameters.tsv was written without it, and nothing downstream noticed the
id was gone (an unbound free parameter is exactly what _resolve_free_to_model has nothing to
say about). The export finished cleanly, exit 0, no warning.

Two consequences, both contradicting the module's own stated contract — "Everything else
raises NotImplementedError (the boundary is in code, not silent)"
:

  • a parameter written the new-era way silently vanished from the exported problem;
  • a conf whose parameters are all records reported No exportable free parameters found in the config (expected one of [...]), naming only the *_var keywords, pointing the user away
    from the syntax the edition-2 docs teach.

The damage was widest on truncated priors. A record is the only grammar carrying
lower/upper, which is why import_._free_parameter_conf_line emits one for a prior
truncated to a box (ADR-0020/0047). So a PEtab problem with bounded priors imported fine and
then exported to a table missing precisely those parameters. On the tutorial's own PEtab priors
problem (lesson 15) that is three of four:

before after
kon (log-normal, bounds 0.001–10) dropped round-trips
koff (gamma, bounds 0.001–10) dropped round-trips
R0 (normal, bounds 1–100) dropped round-trips
L0 (plain uniform) survives survives

The re-import then produced a fit over one parameter instead of four.

This is the same defect #603 found on the coherence gate, reached from the other side: keying
on the config key name instead of on what the declaration builds makes the record syntax
invisible to a rule the positional line goes through.

What changed

_free_parameters_from_conf reads both spellings in one pass, in declaration order, so the
two may be mixed and the table follows the conf.

A record is built by free_parameter_from_record — the same mapping
Configuration._load_variables loads a job with. It moved out of Configuration into
pybnf/parameter_record.py for the purpose: an exporter that re-derived the record grammar for
itself would drift from the one the fitter actually runs, and the whole point of the adapter is
that a PEtab row and a native declaration land on the same object (ADR-0004).
Configuration._free_parameter_from_record stays as a delegating method, so the references in
ADR-0043 and ADR-0047 and the tests that exercise the record grammar directly still name
something real. The move is behavior-neutral — the body is unchanged and the config and
start-point suites pass untouched.

Because a record resolves to an ordinary *_var keyword, it meets the same boundaries the
positional line does, reached by a different spelling. The exportability gate is now one
function keyed on that keyword, whose message names what the declaration built:

record builds refused because
no prior:, no box var a flat improper prior is not a PEtab family
parameter_scale: ln lnnormal_var PEtab has no natural-log sampling scale
prior: student_t student_t_var no three-parameter PEtab family
prior: cauchy, parameter_scale: log10 logcauchy_var PEtab defines no log- form for cauchy

The no-prior keyword set is derived from PRIOR_KEYWORD_MAP rather than listed, so a family
added to the registry cannot quietly land in the wrong arm of the message.

A record's initial_value: is honoured as the start point it is (#719), reaching the row's
nominalValue like a start_point line does. Where both spellings name one parameter the rule
is Configuration._load_start_point's: accepted when they agree, refused when they disagree.
An out-of-box initial_value is a PybnfError, as at the Configuration loader's own call site,
not the bare OutOfBoundsException that reaches users as "an unknown error … please report this
bug".

Tests

Fourteen in TestExportParameterRecord; thirteen fail against the old code.

  • The drop itself: one parameter written both ways exports to a byte-identical table; a
    record-only conf exports all three of its parameters; records and *_var lines mixed keep
    declaration order.
  • Truncated priors: lesson 15's three survive import → export → re-import with their
    declarations unchanged; the re-exported problem has four rows and passes petab's own
    validator (the row count first — a table that dropped three parameters is still perfectly
    valid PEtab, so validity alone is not the guard); a half-bounded truncation (ADR-0047) writes
    a finite lowerBound and an infinite upperBound, a shape the exporter could not previously
    emit at all.
  • Start point: nominalValue from initial_value:; agreement accepted; contradiction
    refused; out-of-box refused.
  • Boundaries: the no-prior refusal, plus the three inexpressible priors parametrized over
    the keyword each record builds.

Full suite green: 5157 passed, 25 skipped. ruff@0.15.14 clean; the -W --keep-going
Sphinx build succeeds (the new module gets an API page, docs/modules/parameter_record.rst).

ADR-0043 carries an amendment recording the move and why the key-name match was the defect.

…free parameter no longer vanishes from the problem (#733)

The exporter picked its free parameters out of the raw config by matching key
NAMES against `_VAR_DECL` = `(_var$|^var$|^logvar$)`. `ploop` stores a new-era
`parameter:` record under a `('parameter', id)` key, which matches none of the
three alternatives, so the record was *skipped* rather than refused: no row
reached `_parameter_rows`, `parameters.tsv` was written without it, and nothing
downstream noticed the id was missing, because an unbound free parameter is
exactly what `_resolve_free_to_model` has nothing to say about. The export
finished cleanly having dropped the parameter.

Two consequences. A conf whose parameters are all records reported "No
exportable free parameters found in the config (expected one of [...])", naming
only the `*_var` keywords and so pointing the user away from the syntax the
edition-2 documentation teaches. And both outcomes contradict the module's own
contract, stated in its docstring: "Everything else raises NotImplementedError
(the boundary is in code, not silent)."

The damage was widest on truncated priors. A record is the only grammar carrying
`lower`/`upper`, which is why `import_._free_parameter_conf_line` emits one for a
prior truncated to a box (ADR-0020/0047) -- so a PEtab problem with bounded
priors imported fine and then exported to a table missing precisely those
parameters. Measured on the tutorial's own PEtab priors problem (lesson 15,
`examples/tutorial/15_petab_priors`), that was three of four: the log-normal
`kon`, the gamma `koff` and the normal `R0` all disappeared and only the plain
uniform `L0` survived, so the re-import produced a fit over one parameter
instead of four. This is the same failure #603 found on the coherence gate,
reached from the other side: keying on the config key name instead of on what
the declaration builds makes the record syntax invisible to a rule the
positional line goes through.

`_free_parameters_from_conf` now reads both spellings in one pass, in
declaration order, so the two may be mixed and the table follows the conf.
A record is built by `free_parameter_from_record`, the same mapping
`Configuration._load_variables` loads a job with -- moved out of `Configuration`
to `pybnf/parameter_record.py` for the purpose, because an exporter that
re-derived the record grammar for itself would drift from the one the fitter
actually runs, and the whole point of the adapter is that a PEtab row and a
native declaration land on the same object (ADR-0004).
`Configuration._free_parameter_from_record` stays as a delegating method, so the
references in ADR-0043 and ADR-0047, and the tests that exercise the record
grammar directly, still name something real. The move is behavior-neutral: the
method body is unchanged and the config and start-point suites pass untouched.

Because a record resolves to an ordinary `*_var` keyword, it meets the same
boundaries the positional line does, reached by a different spelling. The
exportability gate is now one function keyed on that keyword, with a message that
names what the declaration built: a no-prior point start (`var`/`logvar`/`lnvar`,
or a record with an `initial_value:` and no `prior:` and no box), a natural-log
sampling scale, a three-parameter family such as student_t, and the `log-` forms
PEtab defines for no family. The no-prior keyword set is derived from
`PRIOR_KEYWORD_MAP` rather than listed, so a family added to the registry cannot
quietly land in the wrong arm of the message.

A record's `initial_value:` is honoured as the start point it is (#719), so it
reaches the row's `nominalValue` like a `start_point` line does. Where both
spellings name one parameter the rule is `Configuration._load_start_point`'s:
accepted when they agree, refused when they disagree, since silently preferring
one would reintroduce the class of failure the start-point work exists to remove.
An out-of-box `initial_value` is a `PybnfError`, as at the Configuration loader's
own call site, not the bare `OutOfBoundsException` that reaches users as "an
unknown error ... please report this bug".

Tests: fourteen, of which thirteen fail against the old code. That one parameter
written both ways exports to a byte-identical table; that a record-only conf
exports all three of its parameters; that records and `*_var` lines mixed keep
declaration order; that lesson 15's three truncated priors survive import ->
export -> re-import with their declarations unchanged, and that the re-exported
problem has four rows and passes petab's own validator; that a half-bounded
truncation (ADR-0047) writes a finite lowerBound and an infinite upperBound --
a shape the exporter could not previously emit at all; the start-point pair
(nominalValue from `initial_value:`, agreement accepted, contradiction refused,
out-of-box refused); and the four refusals, parametrized over the keyword each
inexpressible record builds.

ADR-0043 carries an amendment recording the move and why the key-name match was
the defect.

Signed-off-by: Bill Hlavacek <hlavacek@lanl.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PEtab export skips edition-2 parameter: records, so a free parameter declared the new-era way silently vanishes from the exported problem

1 participant