Coverage site on GitHub Pages: a page per run, history, badge (0.1.2)

.github/workflows/coverage.yml

@@ -3,7 +3,9 @@ name: MSL coverage
 # The coverage of the Modelica Standard Library examples through OM.jl
 # (flatten, translate, simulate, validate against the reference results),
 # published as documentation: docs/COVERAGE.md and the "Latest coverage"
-# section of README.md (scripts/publish_coverage.jl).
+# section of README.md (scripts/publish_coverage.jl), and on the coverage site
+# (GitHub Pages from the gh-pages branch, scripts/publish_site.jl): one page
+# per measured date, an index over all runs and the README badge.
 #
 # The run is a no-skip run (OMJL_COVERAGE_NOSKIP=1): the registry's
 # skipPhases and broken marks are removed, so every model goes through every
@@ -15,7 +17,8 @@ name: MSL coverage
 # on the runner; record_experiments() refreshes the file).
 #
 # Manual runs (Actions > MSL coverage > Run workflow) commit the documentation
-# back when `commit` is set; the weekly run always does. A schedule only fires
+# back and publish the run on the site when `commit` is set; the weekly run
+# always does. A schedule only fires
 # from the default branch, so it measures that branch's line.
 
 on:
@@ -38,7 +41,7 @@ on:
         required: false
         default: "600"
       commit:
-        description: Commit docs/COVERAGE.md and README.md back to the branch
+        description: Commit docs/COVERAGE.md and README.md back to the branch and add the run to the coverage site
         type: boolean
         required: false
         default: false
@@ -169,3 +172,31 @@ jobs:
           if git diff --cached --quiet; then echo "Coverage unchanged"; exit 0; fi
           git commit -m "Coverage of $(date -u +%Y-%m-%d) [skip ci]"
           git push origin "HEAD:${GITHUB_REF_NAME}"
+
+      # The run on the coverage site: runs/<date>/, the index and the badges on
+      # the gh-pages branch (created on the first run), which Pages serves.
+      - name: Publish the coverage site
+        if: steps.publish.outcome == 'success' && (github.event_name == 'schedule' || inputs.commit)
+        working-directory: OM.jl/OMLibraryTesting.jl
+        env:
+          NOTE: >-
+            No-skip run on GitHub Actions (ubuntu-latest, Julia ${{ inputs.julia || '1.13' }}),
+            ${{ inputs.workers || '1' }} workers, ${{ inputs.timeout || '600' }} s per model
+        run: |
+          export OMJL_SITE_URL="https://${GITHUB_REPOSITORY_OWNER,,}.github.io/${GITHUB_REPOSITORY#*/}/"
+          site="$RUNNER_TEMP/site"
+          if git fetch --depth=1 origin gh-pages; then
+            git worktree add -B gh-pages "$site" FETCH_HEAD
+          else
+            git worktree add --orphan -b gh-pages "$site"
+          fi
+          results=$(ls -t logs/partial_ci_*.jls | head -1)
+          report=$(ls -t reports/coverage_ci_*.html | head -1)
+          julia --color=yes --project=. scripts/publish_site.jl "$site" "$results" "$report" docs/COVERAGE.md "$GITHUB_WORKSPACE/commits.txt" "$NOTE"
+          cd "$site"
+          git add -A
+          if git diff --cached --quiet; then echo "Site unchanged"; exit 0; fi
+          git -c user.name="github-actions[bot]" -c user.email="41898282+github-actions[bot]@users.noreply.github.com" \
+            commit -q -m "Coverage of $(date -u +%Y-%m-%d)"
+          git push origin gh-pages
+          echo "Coverage site: $OMJL_SITE_URL" >> "$GITHUB_STEP_SUMMARY"

Project.toml

@@ -1,6 +1,6 @@
 name = "OMLibraryTesting"
 uuid = "4866894a-ca78-4bef-9a5c-0d35f2f81ce1"
-version = "0.1.1"
+version = "0.1.2"
 authors = ["John Tinnerholm <john.tinnerholm@liu.se>"]
 
 [deps]
@@ -13,6 +13,7 @@ Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
 DoubleEnded = "1bc4e7c2-c1e1-11e9-03fe-315bbef43fde"
 ImmutableList = "4a558cac-c1ed-11e9-20da-3584bcd8709a"
 ListUtil = "b2a72b7e-c1ef-11e9-3a87-6ddc5900c366"
+Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
 MetaModelica = "9d7f2a79-07b5-5542-8b19-c0100dda6b06"
 OM = "2f925a0b-2436-437b-858e-49aee461894b"
 OMBackend = "9400bd5f-7561-42f6-8760-03756753da50"

README.md

@@ -1,5 +1,7 @@
 # OMLibraryTesting.jl
 
+[![MSL coverage](https://img.shields.io/endpoint?url=https%3A%2F%2Fopenmodelica.github.io%2FOMLibraryTesting.jl%2Fbadge.json)](https://openmodelica.github.io/OMLibraryTesting.jl/)
+
 A Julia package for running MSL (Modelica Standard Library) coverage tests against [OM.jl](https://github.com/OpenModelica/OM.jl). It auto-discovers all experiment models in the MSL using `omc`, runs each through the OM.jl pipeline (flatten -> translate -> simulate -> validate), and generates HTML coverage reports.
 
 ## Overview
@@ -21,6 +23,8 @@ Results are compared against Dymola-generated reference trajectories from the MA
 Measured 2026-10-04 on 425 MSL example models: frontend 423 (99.5 %), backend 398 (93.6 %), simulate 324 (76.2 %), validate 299 (70.4 %). Per domain and what stops the rest: [docs/COVERAGE.md](docs/COVERAGE.md).
 <!-- coverage:end -->
 
+Every published run, by date, with the trend: the [coverage site](https://openmodelica.github.io/OMLibraryTesting.jl/).
+
 ## Directory Structure
 
 ```
@@ -34,6 +38,7 @@ OMLibraryTesting.jl/
 │   ├── comparison.jl         # CSV trajectory comparison
 │   ├── analysis.jl           # Error categorization and analysis utilities
 │   ├── report.jl             # HTML report generation
+│   ├── site.jl               # Coverage site: one page per run, index, badges
 │   └── testsuite.jl          # Fetch + run the upstream OpenModelica testsuite (rtest / OM.jl modes)
 ├── models/
 │   └── models.toml           # Per-model overrides (atol, reltol, expected phase, signal mapping)
@@ -78,11 +83,14 @@ OMLibraryTesting.print_error_analysis(results)
 The "Latest coverage" section above and docs/COVERAGE.md are published by the `MSL coverage` workflow
 (.github/workflows/coverage.yml: weekly, or by hand from the Actions tab with `commit` set). It runs
 `scripts/run_full_coverage.jl` with `OMJL_COVERAGE_NOSKIP=1` (every model through every phase) and passes the
-serialized results to the publisher. The same by hand:
+serialized results to the publisher. It also adds the run to the coverage site, served by GitHub Pages from
+the `gh-pages` branch (`scripts/publish_site.jl`): `runs/<date>/` with the analysis and the per-model report, an
+index over all runs, and the badges (`badge.json`, models that validate; `badges/<phase>.json`). The same by hand:
 
 ```bash
 OMJL_COVERAGE_NOSKIP=1 julia --project=. scripts/run_full_coverage.jl
 julia --project=. scripts/publish_coverage.jl logs/partial_full_msl_<timestamp>.jls [commits.txt] ["how the run was made"]
+julia --project=. scripts/publish_site.jl <gh-pages checkout> logs/partial_full_msl_<timestamp>.jls reports/<report>.html docs/COVERAGE.md [commits.txt] ["how the run was made"]
 ```
 
 ## Hot Validation Loop (`scripts/warm_validate.jl`)

scripts/publish_coverage.jl

@@ -26,8 +26,8 @@ const NOTE = length(ARGS) >= 3 ? ARGS[3] : ""
 const ROOT = joinpath(@__DIR__, "..") |> abspath
 const MEASURED = Dates.format(Dates.unix2datetime(mtime(ARGS[1])), "yyyy-mm-dd")
 
-ok(r) = r.highest != OLT.BROKEN && r.highest != OLT.UNKNOWN
-reached(r, p) = ok(r) && r.highest >= p
+const ok = OLT.coverage_ok          # the definitions the coverage site uses (src/site.jl)
+const reached = OLT.coverage_reached
 domainOf(r) = join(split(r.spec.domain, '.')[1:min(2, end)], '.')
 short(name) = replace(name, "Modelica." => "")
 function firstError(r)

scripts/publish_site.jl

@@ -0,0 +1,32 @@
+#=
+Add a coverage run to the coverage site (GitHub Pages, gh-pages branch): runs/<date>/ with the
+analysis and the per-model report, the index over all runs and the badges (src/site.jl).
+
+    julia --project=. scripts/publish_site.jl <site dir> <results.jls> [report.html] [COVERAGE.md] [commits.txt] [note]
+
+<results.jls> is a serialized Vector{ModelResult} (the partial_*.jls of scripts/run_full_coverage.jl);
+the date is the one in its name (partial_<tag>_<yyyy-mm-dd>_<HHMM>.jls), else its modification date.
+[report.html] is the run's generate_report output, [COVERAGE.md] the analysis of
+scripts/publish_coverage.jl, [commits.txt] "<repository> <commit>" lines, [note] one line on how
+the run was made. An empty argument skips that part. OMJL_SITE_URL (optional) is the site's address,
+shown in the badge snippet.
+=#
+
+using Dates
+using Serialization
+
+import OMLibraryTesting
+const OLT = OMLibraryTesting
+
+length(ARGS) >= 2 || error("usage: publish_site.jl <site dir> <results.jls> [report.html] [COVERAGE.md] [commits.txt] [note]")
+arg(i) = length(ARGS) >= i ? ARGS[i] : ""
+const RESULTS_PATH = ARGS[2]
+const RESULTS = deserialize(RESULTS_PATH)::Vector{OLT.ModelResult}
+const NAMED = match(r"_(\d{4}-\d{2}-\d{2})_\d{4}\.jls$", RESULTS_PATH)
+const DATE = NAMED === nothing ? Date(Dates.unix2datetime(mtime(RESULTS_PATH))) : Date(NAMED[1])
+const COMMITS = isfile(arg(5)) ?
+    [String(strip(l)) for l in eachline(arg(5)) if occursin(r"^\S+\.jl \S+$", strip(l))] : String[]
+
+dir = OLT.publish_site(ARGS[1], RESULTS; date = DATE, report = arg(3), analysis = arg(4),
+                       commits = COMMITS, note = arg(6), url = get(ENV, "OMJL_SITE_URL", ""))
+@info "Coverage site updated" run = dir funnel = OLT.coverage_funnel(RESULTS)

src/OMLibraryTesting.jl

@@ -34,6 +34,7 @@ module OMLibraryTesting
 using TOML
 using Dates
 using Distributed
+using Markdown
 import OM
 import OMBackend
 import OMFrontend
@@ -45,6 +46,7 @@ include("discovery.jl")
 include("runner.jl")
 include("analysis.jl")
 include("report.jl")
+include("site.jl")
 include("rerun_failed.jl")
 include("testsuite.jl")
 
@@ -56,6 +58,7 @@ export load_reference_csv, load_comparison_signals, validate_against_reference
 export discover_experiments, record_experiments, merge_overrides!, clear_discovery_cache!, auto_detect_references!
 export run_coverage, run_coverage_and_report, run_model, WorkerManager, WorkerPool, ensure_pool!
 export generate_report, print_summary
+export coverage_funnel, publish_site, site_history
 export categorize_error, categorize_results, print_error_analysis
 export models_with_error, error_for_model, run_frontend_coverage
 export subcategorize_meta_errors