fix(petab): read edition-2 parameter records on export, so a new-era free parameter no longer vanishes from the problem (#733) - #735
Merged
Conversation
…free parameter no longer vanishes from the problem (#733) The exporter picked its free parameters out of the raw config by matching key NAMES against `_VAR_DECL` = `(_var$|^var$|^logvar$)`. `ploop` stores a new-era `parameter:` record under a `('parameter', id)` key, which matches none of the three alternatives, so the record was *skipped* rather than refused: no row reached `_parameter_rows`, `parameters.tsv` was written without it, and nothing downstream noticed the id was missing, because an unbound free parameter is exactly what `_resolve_free_to_model` has nothing to say about. The export finished cleanly having dropped the parameter. Two consequences. A conf whose parameters are all records reported "No exportable free parameters found in the config (expected one of [...])", naming only the `*_var` keywords and so pointing the user away from the syntax the edition-2 documentation teaches. And both outcomes contradict the module's own contract, stated in its docstring: "Everything else raises NotImplementedError (the boundary is in code, not silent)." The damage was widest on truncated priors. A record is the only grammar carrying `lower`/`upper`, which is why `import_._free_parameter_conf_line` emits one for a prior truncated to a box (ADR-0020/0047) -- so a PEtab problem with bounded priors imported fine and then exported to a table missing precisely those parameters. Measured on the tutorial's own PEtab priors problem (lesson 15, `examples/tutorial/15_petab_priors`), that was three of four: the log-normal `kon`, the gamma `koff` and the normal `R0` all disappeared and only the plain uniform `L0` survived, so the re-import produced a fit over one parameter instead of four. This is the same failure #603 found on the coherence gate, reached from the other side: keying on the config key name instead of on what the declaration builds makes the record syntax invisible to a rule the positional line goes through. `_free_parameters_from_conf` now reads both spellings in one pass, in declaration order, so the two may be mixed and the table follows the conf. A record is built by `free_parameter_from_record`, the same mapping `Configuration._load_variables` loads a job with -- moved out of `Configuration` to `pybnf/parameter_record.py` for the purpose, because an exporter that re-derived the record grammar for itself would drift from the one the fitter actually runs, and the whole point of the adapter is that a PEtab row and a native declaration land on the same object (ADR-0004). `Configuration._free_parameter_from_record` stays as a delegating method, so the references in ADR-0043 and ADR-0047, and the tests that exercise the record grammar directly, still name something real. The move is behavior-neutral: the method body is unchanged and the config and start-point suites pass untouched. Because a record resolves to an ordinary `*_var` keyword, it meets the same boundaries the positional line does, reached by a different spelling. The exportability gate is now one function keyed on that keyword, with a message that names what the declaration built: a no-prior point start (`var`/`logvar`/`lnvar`, or a record with an `initial_value:` and no `prior:` and no box), a natural-log sampling scale, a three-parameter family such as student_t, and the `log-` forms PEtab defines for no family. The no-prior keyword set is derived from `PRIOR_KEYWORD_MAP` rather than listed, so a family added to the registry cannot quietly land in the wrong arm of the message. A record's `initial_value:` is honoured as the start point it is (#719), so it reaches the row's `nominalValue` like a `start_point` line does. Where both spellings name one parameter the rule is `Configuration._load_start_point`'s: accepted when they agree, refused when they disagree, since silently preferring one would reintroduce the class of failure the start-point work exists to remove. An out-of-box `initial_value` is a `PybnfError`, as at the Configuration loader's own call site, not the bare `OutOfBoundsException` that reaches users as "an unknown error ... please report this bug". Tests: fourteen, of which thirteen fail against the old code. That one parameter written both ways exports to a byte-identical table; that a record-only conf exports all three of its parameters; that records and `*_var` lines mixed keep declaration order; that lesson 15's three truncated priors survive import -> export -> re-import with their declarations unchanged, and that the re-exported problem has four rows and passes petab's own validator; that a half-bounded truncation (ADR-0047) writes a finite lowerBound and an infinite upperBound -- a shape the exporter could not previously emit at all; the start-point pair (nominalValue from `initial_value:`, agreement accepted, contradiction refused, out-of-box refused); and the four refusals, parametrized over the keyword each inexpressible record builds. ADR-0043 carries an amendment recording the move and why the key-name match was the defect. Signed-off-by: Bill Hlavacek <hlavacek@lanl.gov>
This was referenced Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #733. Found while fixing #719 (merged as #734).
What was wrong
The exporter picked its free parameters out of the raw config by matching key names:
ploopstores an edition-2parameter:record under a('parameter', id)key, which matchesnone of the three alternatives — so the record was skipped, not refused. No row reached
_parameter_rows,parameters.tsvwas written without it, and nothing downstream noticed theid was gone (an unbound free parameter is exactly what
_resolve_free_to_modelhas nothing tosay about). The export finished cleanly, exit 0, no warning.
Two consequences, both contradicting the module's own stated contract — "Everything else
raises
NotImplementedError(the boundary is in code, not silent)":No exportable free parameters found in the config (expected one of [...]), naming only the*_varkeywords, pointing the user awayfrom the syntax the edition-2 docs teach.
The damage was widest on truncated priors. A record is the only grammar carrying
lower/upper, which is whyimport_._free_parameter_conf_lineemits one for a priortruncated to a box (ADR-0020/0047). So a PEtab problem with bounded priors imported fine and
then exported to a table missing precisely those parameters. On the tutorial's own PEtab priors
problem (lesson 15) that is three of four:
kon(log-normal, bounds 0.001–10)koff(gamma, bounds 0.001–10)R0(normal, bounds 1–100)L0(plain uniform)The re-import then produced a fit over one parameter instead of four.
This is the same defect #603 found on the coherence gate, reached from the other side: keying
on the config key name instead of on what the declaration builds makes the record syntax
invisible to a rule the positional line goes through.
What changed
_free_parameters_from_confreads both spellings in one pass, in declaration order, so thetwo may be mixed and the table follows the conf.
A record is built by
free_parameter_from_record— the same mappingConfiguration._load_variablesloads a job with. It moved out ofConfigurationintopybnf/parameter_record.pyfor the purpose: an exporter that re-derived the record grammar foritself would drift from the one the fitter actually runs, and the whole point of the adapter is
that a PEtab row and a native declaration land on the same object (ADR-0004).
Configuration._free_parameter_from_recordstays as a delegating method, so the references inADR-0043 and ADR-0047 and the tests that exercise the record grammar directly still name
something real. The move is behavior-neutral — the body is unchanged and the config and
start-point suites pass untouched.
Because a record resolves to an ordinary
*_varkeyword, it meets the same boundaries thepositional line does, reached by a different spelling. The exportability gate is now one
function keyed on that keyword, whose message names what the declaration built:
prior:, no boxvarparameter_scale: lnlnnormal_varprior: student_tstudent_t_varprior: cauchy, parameter_scale: log10logcauchy_varlog-form for cauchyThe no-prior keyword set is derived from
PRIOR_KEYWORD_MAPrather than listed, so a familyadded to the registry cannot quietly land in the wrong arm of the message.
A record's
initial_value:is honoured as the start point it is (#719), reaching the row'snominalValuelike astart_pointline does. Where both spellings name one parameter the ruleis
Configuration._load_start_point's: accepted when they agree, refused when they disagree.An out-of-box
initial_valueis aPybnfError, as at the Configuration loader's own call site,not the bare
OutOfBoundsExceptionthat reaches users as "an unknown error … please report thisbug".
Tests
Fourteen in
TestExportParameterRecord; thirteen fail against the old code.record-only conf exports all three of its parameters; records and
*_varlines mixed keepdeclaration order.
declarations unchanged; the re-exported problem has four rows and passes petab's own
validator (the row count first — a table that dropped three parameters is still perfectly
valid PEtab, so validity alone is not the guard); a half-bounded truncation (ADR-0047) writes
a finite
lowerBoundand an infiniteupperBound, a shape the exporter could not previouslyemit at all.
nominalValuefrominitial_value:; agreement accepted; contradictionrefused; out-of-box refused.
the keyword each record builds.
Full suite green: 5157 passed, 25 skipped.
ruff@0.15.14clean; the-W --keep-goingSphinx build succeeds (the new module gets an API page,
docs/modules/parameter_record.rst).ADR-0043 carries an amendment recording the move and why the key-name match was the defect.