Skip to content

Validation audit fixes, each model's build released after its phases (0.3.1) - #8

Merged
JKRT merged 7 commits into
masterfrom
msl-gaps
Oct 14, 2026
Merged

JKRT merged 7 commits into
masterfrom
msl-gaps

Conversation

@JKRT

@JKRT JKRT commented Oct 14, 2026

Copy link
Copy Markdown
Member

Validation fixes from an audit of the comparison, and each model's build released after its phases.

  • Validation compares every listed reference signal at 101 points and fails on NaN or a missing signal; quote-aware CSV headers and // lines; a run that terminate()d is compared up to its end; reference end times to 8 significant digits.
  • BevelGear1D (MSL 3.2.3) compared over 0.4 s, as in 4.1.0 (it diverges after 0.4 s).
  • Runner: each result line names its model (they came after the headers of models dispatched since); OMBackend.forgetModel after each model's phases, so a worker running hundreds of models does not keep every build and its tables (compat OMBackend 0.5.5).

Checked locally: tests 78/78; full coverage run with OMBackend 0.5.5 and OMFrontend 1.7.3: MSL 3.2.3 validate 347/425, 4.1.0 366/534 (CI 314/343).

Version 0.3.1; OpenModelica/OMFrontend.jl#104 and OpenModelica/OMBackend.jl#86 (same branch).

🤖 Generated with Claude Code

JKRT added 7 commits October 13, 2026 21:50
From an audit of the comparison: NaN passed, a signal missing from the solution
was skipped (validated on a subset), quoted header names with commas shifted the
columns, // lines in signal files were read as names. The reference signals are
kept observable (_reference_signal_filter); 101 sample points instead of 21.

Assisted-by: Claude Opus 5.5
The reference ends there too (MSL 4 RobotR3); an end that differs from the reference's is
an error.

Assisted-by: Claude Opus 5.5
Result lines print when a worker finishes, after other models' headers.

Assisted-by: Claude Opus 5.5
Assisted-by: Claude Opus 5.5
Its motion diverges after 0.4 s; MSL 4.1.0's reference ends there.

Assisted-by: Claude Opus 5.5
A worker kept every model's build and tables for the whole run.

Assisted-by: Claude Opus 5.5
@JKRT
JKRT merged commit 245550e into master Oct 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant