From e7212ca022928964f92b9779c90a994b23868963 Mon Sep 17 00:00:00 2001 From: ewowi Date: Fri, 16 Oct 2026 11:15:50 +0200 Subject: [PATCH 1/8] MoonWalk: every process on one map, and one way to measure live MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MoonWalk draws every MoonLight process, from a first reader to a release, on one zoomable map in the docs, with search, filters by actor and by system, and done marks; every docs page opens it at its own place. The scenario runners keep one measurement, the per-target contract, and the docs that disagreed with the code or each other now agree. KPI: 256lights | Desktop:2327KB | tick:8/8/7/2/8/8/284/7/7/10us(FPS:125000/125000/142857/500000/125000/125000/3521/142857/142857/100000) | src:331(83067) | test:244(55912) | functions-over:350 Commit: src +0/-0 lines | test +167/-79 lines | duplicated lines 3586 -> 3586 (+0) Checks: check_specs, check_prose, build_docs --strict, check_docgen, test_host --python (413) and --js, check_devices, check_firmwares, check_code, build_desktop, test_desktop (2571), run_scenario (32: 17 passed, 15 skipped), check_esp32_built (esp32, esp32s3-n16r8, esp32p4rev3-eth) and collect_kpi passed. Skipped: check_taglines (no front page), build_desktop --no-jit (no MoonLive change), improv_smoke_test (provisioning path untouched). Repo-health: the only firmware move, esp32p4rev3-eth -144 B, comes from a rebuild with no source change. **Scripts/MoonDeck** - MoonWalk: `moonwalk/NN-.yaml`, eleven stage files holding 48 processes (three with their steps: installing on the desktop, the first light show, the commit), validated against `moonwalk/moonwalk.schema.json` by `gen_moonwalk.py`, which the docs build serves as `moonwalk/moonwalk.json`. A step has an actor, conditions on the device, the change or the reader's choice, an action by intent, and expectations; its runners and its systems follow from its action. - The systems' one list is the architecture's Part table, which gains MoonTube and MoonWalk; a process names only systems the table names. - The live runner drops the baseline generation: `--update-baseline`, `--compare-baseline`, `test/scenario-baseline.json` and every `bounds` check. The contract and the observations are the measurement. - `collect_kpi` runs `check_specs` through `uv run`, so the KPI report no longer reads FAIL for a missing package. - The P4 eth-wifi firmware's description says what holds on IDF v6.1: Ethernet works, WiFi over the C6 associates on some boots; it keeps shipping so users can try it. **Tests** - The in-process runner: `live_only` is the one rule keeping a scenario out; a mutate scenario with neither `top` nor `fixture` that does not say so fails; `skip_on` and `bounds` are gone; an op it does not run prints SKIP rather than passing in silence. The three scenarios that relied on the inferred rule say `live_only`. - `test_scenario_runner.py` pins the SKIP and the new failure against the built runner; `test_gen_moonwalk.py` pins the generator's 14 rules. **Docs/CI** - The MoonWalk page and its map button on every page; the model's spec in `docs/work/future/moonwalk.md`; the processes inventory, step 1 of the processes plan, which gains the pilot, the device's state, MoonCloud, GitHub and the backlog as steps. - Fixes the inventory found: Docker is amd64 and arm64 (the compose file's platform pin goes), the IDF pin is v6.1, the workflow headers count what they build, testing.md runs MoonDeck's scripts rather than ctest, the MoonBase variants, the `{ }` popup, the preset pads, restore applying live, and check_prose as a gate. **Reviews** - ๐Ÿ‘พ the hooks imported jsonschema for every check that imports them โ†’ done, the generator imports it inside `build()`. - ๐Ÿ‘พ the commit process copied a partial gate list โ†’ done, one step points at CLAUDE.md until step 7 moves the gates. - ๐Ÿ‘พ an expectation accepted a misspelt key โ†’ done, each branch refuses unknown keys, and a test pins it. - ๐Ÿ‘พ testing.md said the runner skips `device/` wholesale โ†’ done, each carries `live_only`. - ๐Ÿ‘พ the spec promised runner filters โ†’ done, it says they wait for the runners' adapters. - ๐Ÿ‘พ a comment named process.json, a dead guard, an edge id that could collide with a step id, an amd64-only package name, the schema's page description, British spellings โ†’ done. - ๐Ÿ‘พ the runner change rides with MoonWalk in one commit โ†’ accepted: CLAUDE.md asks one commit per cycle and 51 files is inside CodeRabbit's coverage; the change is named here and pinned by a test. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/release.yml | 17 +- .github/workflows/test.yml | 5 +- .gitignore | 4 - docker-compose.yml | 8 +- docs/assets/extra.css | 13 + docs/assets/moonwalk.js | 255 +++++++ docs/explanation/architecture/index.md | 8 +- docs/gettingstarted.md | 2 +- docs/how-to/building.md | 8 +- docs/how-to/logging-an-issue.md | 7 +- docs/how-to/presets.md | 2 +- docs/how-to/updating-firmware.md | 2 +- docs/moonmodules/core/ui.md | 3 +- docs/moonwalk.md | 15 + docs/reference/integrating.md | 2 +- docs/reference/metrics/code.md | 18 +- docs/reference/metrics/docgen.md | 14 +- docs/reference/metrics/prose.md | 10 +- docs/reference/metrics/repo-health.json | 24 +- docs/reference/metrics/repo-health.md | 56 +- docs/reference/testing.md | 55 +- docs/work/future/backlog-core.md | 2 +- docs/work/future/index.md | 2 + docs/work/future/moonwalk.md | 136 ++++ docs/work/future/processes-inventory.md | 671 ++++++++++++++++++ ...an-20261015 - Processes, described once.md | 69 +- mkdocs.yml | 4 + moondeck/MoonDeck.md | 8 +- moondeck/build/build_esp32.py | 14 +- moondeck/check/collect_kpi.py | 3 +- moondeck/docs/_test_metadata.py | 3 +- moondeck/docs/build_docs.py | 1 + moondeck/docs/gen_moonwalk.py | 156 ++++ moondeck/docs/generate_test_docs.py | 24 +- moondeck/docs/mkdocs_hooks.py | 5 +- moondeck/scenario/run_live_scenario.py | 141 +--- moondeck/scenario/run_scenario.py | 3 - moondeck/test/test_host.py | 4 +- mooninstaller/firmwares.json | 2 +- moonwalk/01-read.yaml | 10 + moonwalk/02-install.yaml | 67 ++ moonwalk/03-set-up.yaml | 11 + moonwalk/04-first-light.yaml | 70 ++ moonwalk/05-everyday.yaml | 35 + moonwalk/06-advanced.yaml | 35 + moonwalk/07-maintain.yaml | 28 + moonwalk/08-report.yaml | 14 + moonwalk/09-contribute.yaml | 144 ++++ moonwalk/10-merge.yaml | 26 + moonwalk/11-release.yaml | 11 + moonwalk/moonwalk.schema.json | 492 +++++++++++++ test/python/test_gen_moonwalk.py | 111 +++ test/python/test_scenario_runner.py | 40 ++ test/scenario_runner.cpp | 92 +-- ...oint_first_setup_from_its_own_network.json | 1 + ...o_AccessPoint_protected_and_always_on.json | 1 + ...iFi_known_networks_fall_back_in_order.json | 1 + 57 files changed, 2534 insertions(+), 431 deletions(-) create mode 100644 docs/assets/moonwalk.js create mode 100644 docs/moonwalk.md create mode 100644 docs/work/future/moonwalk.md create mode 100644 docs/work/future/processes-inventory.md create mode 100644 moondeck/docs/gen_moonwalk.py create mode 100644 moonwalk/01-read.yaml create mode 100644 moonwalk/02-install.yaml create mode 100644 moonwalk/03-set-up.yaml create mode 100644 moonwalk/04-first-light.yaml create mode 100644 moonwalk/05-everyday.yaml create mode 100644 moonwalk/06-advanced.yaml create mode 100644 moonwalk/07-maintain.yaml create mode 100644 moonwalk/08-report.yaml create mode 100644 moonwalk/09-contribute.yaml create mode 100644 moonwalk/10-merge.yaml create mode 100644 moonwalk/11-release.yaml create mode 100644 moonwalk/moonwalk.schema.json create mode 100644 test/python/test_gen_moonwalk.py create mode 100644 test/python/test_scenario_runner.py diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f111b2947..765cb232d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,12 +1,9 @@ name: Release -# Tag-triggered release pipeline. Builds 4 ESP32 firmwares + 2 desktop binaries -# in parallel, generates 4 ESP Web Tools manifests, attaches everything to the -# GitHub release, and deploys the installer page to GitHub Pages. +# The release pipeline. Builds every shipped ESP32 firmware (the matrix from mooninstaller/firmwares.json), the Teensy firmware and the desktop packages (macOS arm64, Windows x64, Linux x64 and arm64) in parallel, writes an ESP Web Tools manifest per firmware, attaches everything to the GitHub release, publishes the container image, and deploys the installer page to GitHub Pages. # # Two triggers produce a release: -# - push to main (src/esp32/CMakeLists.txt changed) โ†’ tags/publishes as `latest` -# (a moving prerelease, always the newest build from main) +# - push to main touching a path below โ†’ tags/publishes as `latest` (a moving prerelease, always the newest build from main) # - push of a vX.Y.Z tag โ†’ stable/RC release # # Manual replay: workflow_dispatch with an existing tag, useful for retries @@ -179,15 +176,7 @@ jobs: - name: Build firmware uses: espressif/esp-idf-ci-action@v1 with: - # Match the local development IDF (v6.1-rc1, commit 44f0c59f). - # The project uses v6-era APIs โ€” `esp_eth_phy_new_generic`, the new - # mDNS component manager, `int clock_gpio` in the EMAC config (v5.x - # had a strongly-typed enum). v5.4 fails to compile platform_esp32.cpp. - # `v6.1-rc1` is the espressif/idf Docker tag pinned to the same - # tagged pre-release the local pin uses โ€” fixed and signed (not the - # rolling `release-v6.1` branch tag), so CI and local build the exact - # same tree. This tracks the v6.1 line toward GA; re-pin to the `v6.1` - # tag once it ships (a deliberate re-test pass, see docs/how-to/building.md). + # The espressif/idf Docker tag of the local pin (PINNED_IDF_COMMIT in build_esp32.py), a fixed release tag rather than the rolling `release-v6.1`, so CI and local build the same tree; v5.x lacks the v6 APIs platform_esp32.cpp uses, and moving the pin is a re-test pass (docs/how-to/building.md). esp_idf_version: v6.1 # The IDF target is the firmware's declared chip, carried by the matrix from firmwares.json. target: ${{ matrix.chip }} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1935ccebd..fa53a6f6b 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -1,9 +1,6 @@ name: Test -# Fast host-side unit tests for the Python (MoonDeck / build scripts) and JS -# (web installer) code that the C++ ctest/scenario suites can't reach. Runs on -# every PR and on pushes to main/next-iteration. Scoped to the paths these tests -# cover so a docs-only or pure-firmware change doesn't spend a runner here. +# Fast host-side unit tests for the Python (MoonDeck / build scripts) and JS (web installer) code that the C++ unit and scenario suites can't reach. Runs on every PR and on pushes to main, scoped to the paths these tests cover so a docs-only or pure-firmware change doesn't spend a runner here. # # Today this pins the Improv frame wire format (test/python + test/js assert a # shared golden vector so the device C++, Python, and JS builders can't drift). diff --git a/.gitignore b/.gitignore index 371b6d0a6..0d28116fe 100644 --- a/.gitignore +++ b/.gitignore @@ -108,10 +108,6 @@ moonbase/dependencies.lock moonbase/build/ esp32/monitor.log -# Generated live-scenario baseline cache (regenerated by run_live_scenario.py -# --update-baseline; per-target observations live in the scenario JSONs instead). -test/scenario-baseline.json - # IDE .vscode/ .idea/ diff --git a/docker-compose.yml b/docker-compose.yml index 4c7276dfa..48b43fea1 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -11,13 +11,7 @@ services: moonlight: build: . - # The released binary is amd64 only, so this line is REQUIRED on an arm64 host (an Apple-silicon - # Mac, an ARM server). Without it compose builds for the host's own architecture and the - # container dies with "rosetta error: failed to open elf" the moment it starts. It is harmless - # on an amd64 host, where it is what would have happened anyway. It goes when the release grows - # an arm64 Linux build. - platform: linux/amd64 - # Or, once images are published, drop `build:` and use one of these instead: + # Or drop `build:` and use a published image, which carries amd64 and arm64: # image: ghcr.io/moonmodules/moonlight:latest # the rolling prerelease # image: ghcr.io/moonmodules/moonlight:6.0.0 # a fixed version container_name: moonlight diff --git a/docs/assets/extra.css b/docs/assets/extra.css index e2fce221e..2c5eeb5b1 100644 --- a/docs/assets/extra.css +++ b/docs/assets/extra.css @@ -145,3 +145,16 @@ .mm-deck-compact .md-typeset .mm-pcard p, .md-typeset .mm-deck-compact .mm-pcard p { -webkit-line-clamp: 2; } /* A screenshot of a module's controls shows whole, scaled into the frame; an animation fills it. */ .mm-pcard-front img[src$=".png"] { object-fit: contain; } +/* MoonWalk's journey map (assets/moonwalk.js): the controls, the canvas, and the panel beside it, which moves below on a narrow screen. */ +.mm-map { display: grid; grid-template-columns: 1fr 20rem; grid-template-rows: auto 75vh; gap: 0.5rem; } +.mm-map-bar { grid-column: 1 / -1; display: flex; flex-wrap: wrap; gap: 0.5rem; align-items: center; font-size: 0.75rem; } +.mm-map-bar input[type="search"] { flex: 1 1 12rem; } +.mm-map-bar input[type="search"], .mm-map-bar select, .mm-map-bar button { border: 1px solid var(--md-default-fg-color--lightest); background: var(--md-default-bg-color); color: inherit; border-radius: 4px; padding: 4px 8px; font: inherit; } +.mm-map-canvas { border: 1px solid var(--md-default-fg-color--lightest); border-radius: 6px; min-height: 0; } +.mm-map-panel { overflow: auto; font-size: 0.75rem; border: 1px solid var(--md-default-fg-color--lightest); border-radius: 6px; padding: 0 0.75rem; } +.md-typeset .mm-map-panel h3 { margin-top: 0.75rem; } +.mm-map-meta { color: var(--md-default-fg-color--light); } +.mm-map-chips { flex-basis: 100%; display: flex; flex-wrap: wrap; gap: 4px; } +.mm-map-chip { border: 1px solid var(--md-default-fg-color--lightest); background: var(--md-default-bg-color); color: inherit; border-radius: 999px; padding: 2px 10px; font: inherit; cursor: pointer; } +.mm-map-chip.on { border-color: var(--md-accent-fg-color); background: var(--md-accent-fg-color--transparent); } +@media (max-width: 760px) { .mm-map { grid-template-columns: 1fr; grid-template-rows: auto 60vh auto; } } diff --git a/docs/assets/moonwalk.js b/docs/assets/moonwalk.js new file mode 100644 index 000000000..ea8f58612 --- /dev/null +++ b/docs/assets/moonwalk.js @@ -0,0 +1,255 @@ +// MoonWalk's journey map: every process from moonwalk/*.yaml, drawn from moonwalk.json as a story map, stages (or systems) as columns and each process's steps inside it. The model is docs/work/future/moonwalk.md. +// On every page it adds a map button to the header that opens the map at that page; on the map page it draws the map, through Material's document$ so instant navigation mounts it too. + +const ROOT = new URL("../", import.meta.url); +const MODEL = new URL("moonwalk/moonwalk.json", ROOT); +// Cytoscape.js, the graph library: compound nodes give the three levels, and it never enters the firmware. +const CYTOSCAPE = "https://cdnjs.cloudflare.com/ajax/libs/cytoscape/3.30.4/cytoscape.min.js"; +const LS_DONE = "mm_moonwalk_done"; // the processes this reader marked done, which the state cannot show on a website +const ACTORS = ["user", "contributor", "product-owner", "agent", "ci"]; +const COL = 340, PROC_H = 64, STEP_H = 36, GAP = 28; + +function readDone() { + try { return new Set(JSON.parse(localStorage.getItem(LS_DONE) || "[]")); } catch { return new Set(); } +} +function writeDone(done) { + try { localStorage.setItem(LS_DONE, JSON.stringify([...done])); } catch { /* a private window keeps marks for the visit only */ } +} + +// The page's address under the site root, the key moonwalk.json's byPage index uses. +function here() { + const path = location.href.split(/[?#]/)[0]; + return path.startsWith(ROOT.href) ? decodeURIComponent(path.slice(ROOT.href.length)) || "index.html" : ""; +} + +function mapButton() { + const header = document.querySelector(".md-header__inner"); + if (!header) return; + let a = header.querySelector(".mm-map-link"); + if (!a) { + a = document.createElement("a"); + a.className = "md-header__button mm-map-link"; + a.title = "This page on MoonWalk, the journey map"; + a.setAttribute("aria-label", a.title); + a.textContent = "\u{1F5FA}๏ธ"; + header.insertBefore(a, header.querySelector(".md-search") || null); + } + a.href = new URL(`moonwalk.html?focus=${encodeURIComponent(here())}`, ROOT).href; +} + +function loadScript(src) { + if (window.cytoscape) return Promise.resolve(); + return new Promise((resolve, reject) => { + const s = document.createElement("script"); + s.src = src; + s.onload = resolve; + s.onerror = () => reject(new Error(`could not load ${src}`)); + document.head.appendChild(s); + }); +} + +// The map's columns: the journey's stages, or the architecture's systems, where a process touching several stands in each of their columns. +function columns(model, by) { + if (by !== "system") return model.stages.map((s) => ({ id: s.id, title: s.title, processes: s.processes })); + const cols = model.systems.map((name) => ({ id: name, title: name, processes: [] })); + const none = { id: "none", title: "No system", processes: [] }; + for (const stage of model.stages) + for (const pid of stage.processes) { + const named = model.processes[pid].systems || []; + if (!named.length) none.processes.push(pid); + for (const name of named) cols.find((c) => c.id === name).processes.push(pid); + } + return [...cols, none].filter((c) => c.processes.length); +} + +// The story map's grid: a column each, its processes stacked, a process's steps inside it, one below the other. Every node carries its process id, since in the system view one process has several. +function elements(model, by) { + const out = []; + columns(model, by).forEach((column, x) => { + out.push({ data: { id: `col:${column.id}`, label: column.title, kind: "stage" } }); + let y = 0; + for (const pid of column.processes) { + const p = model.processes[pid]; + const steps = p.steps || []; + const nid = `${column.id}::${pid}`; + out.push({ data: { id: nid, pid, parent: `col:${column.id}`, label: p.title, kind: "process", actor: p.actor, stepped: steps.length > 0 }, + position: { x: x * COL, y } }); + steps.forEach((s, k) => { + out.push({ data: { id: `${nid}/${s.id}`, pid, parent: nid, label: s.say, kind: "step", actor: s.actor }, + position: { x: x * COL, y: y + (k + 1) * STEP_H } }); + if (k) out.push({ data: { id: `next:${nid}/${k}`, source: `${nid}/${steps[k - 1].id}`, target: `${nid}/${s.id}`, kind: "next" } }); + }); + y += steps.length ? (steps.length + 1) * STEP_H + GAP : PROC_H; + } + }); + // Dependencies are the journey's order, so only the stage view draws them. + if (by !== "system") + for (const [pid, p] of Object.entries(model.processes)) + for (const dep of p.dependsOn || []) + out.push({ data: { id: `dep:${dep}>${pid}`, source: `${model.processes[dep].stage}::${dep}`, target: `${p.stage}::${pid}`, kind: "dep" } }); + return out; +} + +// A theme color as rgb(), since Material writes hsla() with degrees, which Cytoscape does not parse; the browser resolves it. +function themeColor(variable, fallback) { + const probe = document.createElement("span"); + probe.style.color = `var(${variable}, ${fallback})`; + document.body.appendChild(probe); + const rgb = getComputedStyle(probe).color; + probe.remove(); + return rgb; +} + +function style() { + const fg = themeColor("--md-default-fg-color", "#222"); + const bg = themeColor("--md-default-bg-color", "#fff"); + const accent = themeColor("--md-accent-fg-color", "#526cfe"); + const actor = { user: "#2e7d32", contributor: "#1565c0", "product-owner": "#6a1b9a", agent: "#ef6c00", ci: "#546e7a" }; + return [ + { selector: "node", style: { label: "data(label)", color: fg, "font-size": 12, "text-wrap": "wrap", "text-valign": "center", "text-halign": "center", "min-zoomed-font-size": 7 } }, + { selector: "node[kind='stage']", style: { shape: "round-rectangle", "background-color": fg, "background-opacity": 0.04, "border-width": 1, "border-color": fg, "border-opacity": 0.2, "text-valign": "top", "font-size": 22, "font-weight": "bold", padding: 24, "min-zoomed-font-size": 4 } }, + { selector: "node[kind='process']", style: { shape: "round-rectangle", width: 280, height: 44, "text-max-width": 260, "background-color": bg, "border-width": 3, "border-color": (n) => actor[n.data("actor")] || fg } }, + { selector: "node[kind='process'][?stepped]", style: { "text-valign": "top", "font-weight": "bold", padding: 10 } }, + { selector: "node[kind='step']", style: { shape: "round-rectangle", width: 260, height: 28, "text-max-width": 250, "font-size": 10, "background-color": fg, "background-opacity": 0.06, "border-width": 2, "border-color": (n) => actor[n.data("actor")] || fg, "min-zoomed-font-size": 9 } }, + { selector: "edge", style: { width: 1.5, "line-color": fg, "line-opacity": 0.35, "curve-style": "bezier", "target-arrow-shape": "triangle", "target-arrow-color": fg, "arrow-scale": 0.8 } }, + { selector: "edge[kind='next']", style: { width: 1, "line-style": "dotted", "target-arrow-shape": "none" } }, + { selector: ".done", style: { opacity: 0.4 } }, + { selector: ".faded", style: { opacity: 0.08 } }, + { selector: "node:selected", style: { "border-color": accent, "border-width": 5 } }, + ]; +} + +function control(tag, attrs, text) { + const el = document.createElement(tag); + Object.assign(el, attrs); + if (text) el.textContent = text; + return el; +} + +// What the panel adds after a step: who does it when that is not the process's actor, and what it waits on. +function stepNote(s, p) { + const notes = []; + if (s.actor && s.actor !== p.actor) notes.push(s.actor); + for (const g of s.given || []) { + if (g.input) notes.push(`${g.input}: ${g.equals}`); + else if (g.changed) notes.push(`changes ${g.changed.join(", ")}`); + } + return notes.length ? ` (${notes.join("; ")})` : ""; +} + +// The panel beside the map: the clicked process's goal, its steps, its page, and the done mark. +function showPanel(panel, model, pid, done, onDone) { + const p = model.processes[pid]; + panel.replaceChildren(); + if (!p) return; + const meta = [model.stages.find((s) => s.id === p.stage)?.title, p.actor, ...(p.systems || [])].join(" ยท "); + panel.append(control("h3", {}, p.title), control("p", { className: "mm-map-meta" }, meta), control("p", {}, p.goal)); + if (p.url) { + const line = control("p"); + line.append(control("a", { href: new URL(p.url, ROOT).href }, "Read the page")); + panel.append(line); + } + if (p.steps?.length) { + const ol = control("ol"); + for (const s of p.steps) ol.append(control("li", {}, s.say + stepNote(s, p))); + panel.append(ol); + } + const label = control("label", { className: "mm-map-done" }); + const box = control("input", { type: "checkbox", checked: done.has(pid) }); + box.addEventListener("change", () => onDone(pid, box.checked)); + label.append(box, " Done"); + panel.append(label); +} + +async function drawMap(host) { + if (host.dataset.mounted) return; + host.dataset.mounted = "1"; + const bar = control("div", { className: "mm-map-bar" }); + const search = control("input", { type: "search", placeholder: "Search processes, steps, modules" }); + const actor = control("select"); + actor.append(control("option", { value: "" }, "Every actor"), ...ACTORS.map((a) => control("option", { value: a }, a))); + const hide = control("input", { type: "checkbox" }); + const hideLabel = control("label"); + hideLabel.append(hide, " Hide done"); + const by = control("select", { title: "Columns" }); + by.append(control("option", { value: "stage" }, "Columns: stage"), control("option", { value: "system" }, "Columns: system")); + const all = control("button", { type: "button" }, "Whole map"); + const chips = control("div", { className: "mm-map-chips" }); + bar.append(search, actor, by, hideLabel, all, chips); + const canvas = control("div", { className: "mm-map-canvas" }); + const panel = control("aside", { className: "mm-map-panel" }); + panel.append(control("p", {}, "Click a process for its goal, its steps and its page.")); + host.append(bar, canvas, panel); + + let model; + try { + await loadScript(CYTOSCAPE); + model = await (await fetch(MODEL)).json(); + } catch (e) { + canvas.textContent = `The map could not load: ${e.message}`; + return; + } + const done = readDone(); + const picked = new Set(); // the system chips switched on; none means every system + // A chip for each system some process touches, so none filters to an empty map. + const used = new Set(Object.values(model.processes).flatMap((p) => p.systems || [])); + for (const name of model.systems.filter((n) => used.has(n))) { + const chip = control("button", { type: "button", className: "mm-map-chip" }, name); + chip.addEventListener("click", () => { + picked.has(name) ? picked.delete(name) : picked.add(name); + chip.classList.toggle("on", picked.has(name)); + fit(picked.size ? apply() : (apply(), cy.collection())); + }); + chips.append(chip); + } + const cy = window.cytoscape({ container: canvas, elements: elements(model, by.value), style: style(), layout: { name: "preset" }, minZoom: 0.05, maxZoom: 3 }); + const nodesOf = (pid) => cy.nodes().filter((n) => n.data("pid") === pid); + + function apply() { + const q = search.value.trim().toLowerCase(); + const visible = cy.collection(); + cy.batch(() => { + for (const [pid, p] of Object.entries(model.processes)) { + const nodes = nodesOf(pid); + const text = [p.title, p.goal, ...(p.systems || []), ...(p.steps || []).map((s) => `${s.say} ${JSON.stringify(s.when)}`)].join(" ").toLowerCase(); + const show = (!actor.value || p.actor === actor.value) && !(hide.checked && done.has(pid)) && (!q || text.includes(q)) + && (!picked.size || (p.systems || []).some((name) => picked.has(name))); + nodes.toggleClass("faded", !show).toggleClass("done", done.has(pid)); + if (show) visible.merge(nodes.filter((n) => n.data("kind") === "process")); + } + }); + return visible; + } + const fit = (eles) => { cy.fit(eles.length ? eles : cy.elements(), 40); if (cy.zoom() > 1.2) cy.zoom({ level: 1.2, position: eles.length ? eles.position() : undefined }); }; + + search.addEventListener("input", () => fit(search.value.trim() ? apply() : (apply(), cy.collection()))); + actor.addEventListener("change", () => fit(apply())); + hide.addEventListener("change", () => apply()); + all.addEventListener("click", () => fit(cy.collection())); + by.addEventListener("change", () => { + cy.elements().remove(); + cy.add(elements(model, by.value)); + fit(picked.size || search.value.trim() ? apply() : (apply(), cy.collection())); + }); + const mark = (id, on) => { on ? done.add(id) : done.delete(id); writeDone(done); apply(); }; + cy.on("tap", "node[kind='process'], node[kind='step']", (ev) => showPanel(panel, model, ev.target.data("pid"), done, mark)); + apply(); + + // Open where the reader came from: a page's processes, or one process by id. + const focus = new URLSearchParams(location.search).get("focus") || ""; + const ids = model.byPage[focus] || (model.processes[focus] ? [focus] : []); + const target = cy.collection(ids.map((id) => nodesOf(id))).filter((n) => n.data("kind") === "process"); + if (target.length) { + target.select(); + showPanel(panel, model, ids[0], done, mark); + fit(target.union(target.children())); + } else fit(cy.collection()); +} + +function mount() { + mapButton(); + const host = document.getElementById("mm-map"); + if (host) drawMap(host); +} + +if (window.document$) window.document$.subscribe(mount); else mount(); diff --git a/docs/explanation/architecture/index.md b/docs/explanation/architecture/index.md index 2d7e6b8ee..ef6031c8f 100644 --- a/docs/explanation/architecture/index.md +++ b/docs/explanation/architecture/index.md @@ -30,11 +30,15 @@ flowchart LR inst["MoonInstaller ยท firmware, deviceModel, board"] cloud["MoonCloud ยท opt-in stats and talk"] deck["MoonDeck ยท the developer console"] + tube["MoonTube ยท clips that are UI tests"] + walk["MoonWalk ยท every process on one map"] core -.-> base core -.-> inst core -.-> cloud core -.-> deck + core -.-> tube + core -.-> walk classDef po fill:#2d3561,stroke:#7b88c9,color:#fff classDef agent fill:#3d2d61,stroke:#a07bc9,color:#fff @@ -43,7 +47,7 @@ flowchart LR class mm po class core,light agent class live check - class base,inst,cloud,deck gate + class base,inst,cloud,deck,tube,walk gate ``` | Part | What it decides | @@ -56,6 +60,8 @@ flowchart LR | [MoonInstaller](mooninstaller.md) | Firmware, deviceModel and board: three words for three different things | | [MoonCloud](mooncloud.md) | The opt-in server side, and the only server a device talks to | | [MoonDeck](moondeck.md) | One script per task, two front ends | +| [MoonTube](../../../moondeck/moontube/moontube.md) | One run file per clip: performed as a UI test, filmed as a documentation video | +| [MoonWalk](../../moonwalk.md) | Every process, from a first reader to a release, described once and drawn on one map | ## Core and light domain diff --git a/docs/gettingstarted.md b/docs/gettingstarted.md index 8d9867f33..64e241898 100644 --- a/docs/gettingstarted.md +++ b/docs/gettingstarted.md @@ -152,7 +152,7 @@ Learning one card teaches you all of them. Each carries the same five buttons in -โป turns a module off, โœŽ swaps it for another type, ร— deletes it, and ? opens that module's page in the documentation. { } opens the card's own JSON, which is what to paste into an issue when something misbehaves. +โป turns a module off, โœŽ swaps it for another type, and ร— deletes it. { } opens the card as JSON, to copy, to save as a preset, or to open its diagnostics, which is what to paste into an issue when something misbehaves. ### The system modules diff --git a/docs/how-to/building.md b/docs/how-to/building.md index 1209a510a..77b7b38a4 100644 --- a/docs/how-to/building.md +++ b/docs/how-to/building.md @@ -131,8 +131,7 @@ networking. Discovery and the broadcast or multicast output modes do not cross a **Several instances** run side by side with no conflict: each container has its own port space, so they all listen on 8080 internally with different published ports, and each generates its own identity so they are distinguishable on the network. CPU is the practical limit rather than memory (measured at ~5 MB and about one core each, since the desktop build renders as fast as it is allowed); cap it with `cpus:` in compose when running a fleet. -**amd64 only** for now: the release ships no arm64 Linux binary. On an Apple-silicon Mac or an ARM -server the compose file's `platform: linux/amd64` runs it under emulation, which works but is slower than native. +**amd64 and arm64**: the release builds a package for each, and the build picks the host's own, so an Apple-silicon Mac or an ARM server runs it natively. ## ESP32 @@ -168,7 +167,7 @@ uv run moondeck/build/flash_esp32.py --firmware esp32 --port /dev/tty.usbserial- uv run moondeck/run/monitor_esp32.py --port /dev/tty.usbserial-XXXX ``` -On the variants that opt into it (`esp32`, `esp32-16mb`, `esp32-wrover`, `esp32-eth`, `esp32s3-zero`, and `qemu`, which is emulated rather than installable) the build also produces +On the variants that opt into it (`esp32`, `esp32-eth`, `esp32-wrover`, `esp32-pico`, `esp32-16mb`, `esp32s3-zero`, `esp32c3`, and `qemu`, which is emulated rather than installable) the build also produces **MoonBase**, the second boot image ([MoonBase](../explanation/architecture/moonbase.md)), and `flash_esp32.py` writes the corrected layout in one pass: app in the big `ota_0` slot, MoonBase in `factory`, and an otadata that boots the app directly. A device on the older dual-OTA table adopts this layout only through such a full serial flash, OTA never rewrites the partition table. @@ -297,8 +296,7 @@ The Ethernet PHY type and pin map are runtime config, not baked into the build: A board already on the network is updated over HTTP, with no cable. Which route to use depends on whether the variant carries [MoonBase](../explanation/architecture/moonbase.md): a board cannot rewrite the partition it is executing from, so on a MoonBase variant the app hands over to MoonBase and MoonBase does the writing. -**On a MoonBase variant** (`esp32`, `esp32-16mb`, `esp32s3-zero`, and any variant `build_esp32.py` -builds MoonBase alongside), it is two requests: +**On a MoonBase variant** (the list above), it is two requests: ```sh # 1. the app reboots into MoonBase, with nothing staged. Back in ~4 seconds. diff --git a/docs/how-to/logging-an-issue.md b/docs/how-to/logging-an-issue.md index 70f46fe8d..3e3da958a 100644 --- a/docs/how-to/logging-an-issue.md +++ b/docs/how-to/logging-an-issue.md @@ -1,6 +1,6 @@ # Log an issue -Something not working? Tell us. You do not need to diagnose it, describe what you saw, and hand us the state of the device it happened on. The **`{ }`** link on every module card gives you that in one click, and it is the single most useful thing you can attach. +Something not working? Tell us. You do not need to diagnose it, describe what you saw, and hand us the state of the device it happened on. The **`{ }`** link on every module card leads you there in two clicks, and it is the single most useful thing you can attach. ๐Ÿ‘‰ **[Open an issue on GitHub](https://github.com/MoonModules/MoonLight/issues)** @@ -42,9 +42,8 @@ Actual behavior: **How to get it:** -1. Find the card that misbehaved, the effect that went dark, the driver that - isn't lighting, the network card that won't connect -2. Click the **`{ }`** link at the end of its title row +1. Find the card that misbehaved, the effect that went dark, the driver that isn't lighting, the network card that won't connect +2. Click the **`{ }`** link at the end of its title row, then **diagnostics** in the popup that opens 3. Select all in the new tab (Ctrl/Cmd+A), copy, and paste it into the issue **Which cards to include:** diff --git a/docs/how-to/presets.md b/docs/how-to/presets.md index e358b6248..088703c23 100644 --- a/docs/how-to/presets.md +++ b/docs/how-to/presets.md @@ -8,7 +8,7 @@ They live on the Control card as a grid of 64 pads. Click one to apply it. ## Save one -Right-click a pad, or press and hold it on a touchscreen. The pad editor opens. +Click an empty pad, or right-click a full one, or press and hold it on a touchscreen. The pad editor opens. Type a name, choose which container it saves, and press **save current state here**. The name, pad and source in the editor are input for the next save, not configuration, so none of them is written to flash. diff --git a/docs/how-to/updating-firmware.md b/docs/how-to/updating-firmware.md index c66dc5213..54a621edd 100644 --- a/docs/how-to/updating-firmware.md +++ b/docs/how-to/updating-firmware.md @@ -21,7 +21,7 @@ The card checks for both channels and tells you when one is newer, and a stable ## On a device that carries MoonBase -Some variants carry [MoonBase](../explanation/architecture/moonbase.md), a small recovery image in the factory slot. The 4 MB classic, `esp32-16mb` and the S3-Zero use it today. +Some variants carry [MoonBase](../explanation/architecture/moonbase.md), a small recovery image in the factory slot. `esp32`, `esp32-eth`, `esp32-wrover`, `esp32-pico`, `esp32-16mb`, `esp32s3-zero` and `esp32c3` use it. The card looks slightly different there. It gains an **image** selector, choosing whether the version shown and the install performed apply to the app or to MoonBase itself, and a **Restart in MoonBase** button. diff --git a/docs/moonmodules/core/ui.md b/docs/moonmodules/core/ui.md index e509ba9b7..9572f8ada 100644 --- a/docs/moonmodules/core/ui.md +++ b/docs/moonmodules/core/ui.md @@ -88,8 +88,7 @@ Each MoonModule renders as a card, with **child cards nested inside their parent - The parent's own controls render **above** its children; `+ add child` renders **below** them. Child cards live in a `.card-children` wrapper appended into the parent card's DOM node (not flat siblings); `renderModuleTree` recurses into the parent card, not into `main`. -- **`{ }`** opens `GET /api/modules/{name}` in a new tab, that one module's live JSON, for issue - reports (see [Log an issue](../../how-to/logging-an-issue.md)). On EVERY card, unlike `โœŽ`/`ร—` (user-editable children only). +- **`{ }`** opens a popup holding the card as a state document, with **copy**, **save as preset** where a preset module exists, and **diagnostics**, which opens `GET /api/modules/{name}` in a new tab, that module's live JSON for issue reports (see [Log an issue](../../how-to/logging-an-issue.md)). On EVERY card, unlike `โœŽ`/`ร—` (user-editable children only). - **Enabled toggle** in the right-hand action cluster mirrors `MoonModule::enabled()`, a styled ` +
diff --git a/src/ui/style.css b/src/ui/style.css index 7b9eb3b96..388c00d76 100644 --- a/src/ui/style.css +++ b/src/ui/style.css @@ -1938,6 +1938,25 @@ body.help-hidden .control-row.has-help { margin-bottom: 6px; } background: var(--card-bg-0); color: var(--fg); } .mb-ask::backdrop { background: rgba(0,0,0,0.5); } + +/* MoonWalk's map, hosted from the docs site in the app's own dialog: a window's worth over the cards, resized from its corner, and the map fills whatever size it is given. */ +.walk-dialog { + width: min(960px, 92vw); height: min(640px, 85vh); min-width: min(420px, 92vw); min-height: 360px; max-width: 98vw; max-height: 96vh; + resize: both; overflow: hidden; padding: 12px 14px; + border: 1px solid var(--border); border-radius: 10px; + background: var(--card-bg-0); color: var(--fg); +} +.walk-dialog[open] { display: flex; flex-direction: column; } +/* The map reads the docs theme's color variables; the dialog maps the app's colors onto them, so the map's script knows no name of the app's. */ +.walk-dialog { + --md-default-fg-color: var(--fg); --md-default-bg-color: var(--card-bg-0); --md-accent-fg-color: var(--accent); + --md-default-fg-color--lightest: var(--border); --md-default-fg-color--light: var(--fg-muted); +} +.walk-dialog::backdrop { background: rgba(0,0,0,0.5); } +.walk-head { display: flex; align-items: center; justify-content: space-between; margin-bottom: 8px; } +.walk-head h3 { margin: 0; font-size: 1rem; } +.walk-body { flex: 1; min-height: 0; } +.walk-dialog .mm-map { height: 100%; grid-template-rows: auto 1fr; --mw-panel: 16rem; } /* the text's default share in the dialog; the handle moves it */ .mb-ask h3 { margin: 0 0 10px; font-size: 1rem; } .mb-ask p { margin: 0 0 10px; font-size: 0.9em; line-height: 1.45; } /* The reassurance reads quieter than the warning above it: the point is that a mistake is cheap, diff --git a/test/js/moonwalk-device.test.mjs b/test/js/moonwalk-device.test.mjs new file mode 100644 index 000000000..ef5338761 --- /dev/null +++ b/test/js/moonwalk-device.test.mjs @@ -0,0 +1,64 @@ +// MoonWalk on a device: a process is done when the device's own state shows it, and what comes next is the first process not done whose dependencies are. +// Run: `node --test test/js/moonwalk-device.test.mjs`. + +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { doneOnDevice, nextOf, choicesOf, applies } from "../../docs/assets/moonwalk.js"; + +const model = { + stages: [{ id: "install", processes: ["install"] }, { id: "light", processes: ["grid", "effect", "status-only"] }], + processes: { + install: { actor: "user", steps: [] }, + grid: { actor: "user", dependsOn: ["install"], steps: [{ id: "w", then: [{ module: "Grid", control: "width", equals: 16 }] }] }, + effect: { actor: "user", dependsOn: ["grid"], steps: [{ id: "a", then: [{ module: "Rainbow", exists: true }] }] }, + "status-only": { actor: "contributor", steps: [{ id: "s", then: [{ status: 200 }] }] }, + }, +}; +const device = (width, extra = []) => ({ modules: [{ name: "Layouts", children: [{ name: "Grid", controls: [{ name: "width", value: width }] }] }, ...extra] }); + +test("a process is done when every expectation the device's state answers holds", () => { + assert.deepEqual([...doneOnDevice(model, device(16))], ["grid"]); +}); + +test("one expectation that does not hold leaves the process to do", () => { + assert.equal(doneOnDevice(model, device(8)).has("grid"), false); +}); + +test("a module that exists is an expectation the state answers", () => { + assert.equal(doneOnDevice(model, device(16, [{ name: "Rainbow" }])).has("effect"), true); +}); + +test("a process the state cannot answer is never done by the device, only by the reader's mark", () => { + assert.equal(doneOnDevice(model, device(16)).has("status-only"), false); + assert.equal(doneOnDevice(model, device(16)).has("install"), false); +}); + +test("next is a process not done whose dependencies all are, in the journey's order", () => { + assert.deepEqual(nextOf(model, new Set(["install"])), ["grid", "status-only"]); + assert.deepEqual(nextOf(model, new Set(["install", "grid"])), ["effect", "status-only"]); +}); + +test("next can be narrowed to the reader's actor", () => { + assert.deepEqual(nextOf(model, new Set(["install"]), "user"), ["grid"]); +}); + +test("a list control counts as its number of rows", () => { + const wifi = { stages: [{ id: "set-up", processes: ["wifi"] }], processes: { wifi: { steps: [{ id: "pick", then: [{ module: "WiFi", control: "known", notEquals: 0 }] }] } } }; + const known = (rows) => ({ modules: [{ name: "WiFi", controls: [{ name: "known", value: rows }] }] }); + assert.equal(doneOnDevice(wifi, known([])).has("wifi"), false); + assert.equal(doneOnDevice(wifi, known([{ ssid: "home" }])).has("wifi"), true); +}); + +test("the choices are what is here and what comes next, without what is done or listed twice", () => { + const choices = choicesOf(model, new Set(["install"]), "", ["grid", "effect", "install"]); + assert.deepEqual(choices.here, ["grid", "effect"]); // install is done, so it is no choice + assert.deepEqual(choices.next, ["status-only"]); // grid is next too, but already here +}); + +test("a step shows for the chosen option, and a step for every option always shows", () => { + const chosen = { platform: "Windows" }; + assert.equal(applies({ id: "download" }, chosen), true); + assert.equal(applies({ id: "setup", given: [{ input: "platform", equals: "Windows" }] }, chosen), true); + assert.equal(applies({ id: "dmg", given: [{ input: "platform", equals: "macOS" }] }, chosen), false); + assert.equal(applies({ id: "after", given: ["install-desktop"] }, chosen), true); // a process precondition is no choice +}); diff --git a/test/js/ui-help.test.mjs b/test/js/ui-help.test.mjs index 138095d63..74681ae26 100644 --- a/test/js/ui-help.test.mjs +++ b/test/js/ui-help.test.mjs @@ -56,13 +56,13 @@ test("a link or picture from the file must be https and stay inside its attribut assert.equal(safeUrl(undefined), null); }); -// toggleHelp out of app.js, with the browser around it stubbed: what it asks, stores and fetches. +// toggleHelp out of app.js, with the consent it shares with MoonWalk, and the browser around them stubbed: what it asks, stores and fetches. import { fnSource } from "./app-source.mjs"; function toggleWorld({ consent = "", answer = false } = {}) { const store = { mm_helpConsent: consent }; const w = { asked: 0, loads: 0, shown: null, hidden: true }; const toggle = new Function("document", "localStorage", "confirm", "lsRead", "applyHelpShown", "loadHelp", "helpModules", - "const LS_HELP_CONSENT = 'mm_helpConsent', LS_HELP_SHOWN = 'mm_helpShown';\n" + fnSource("toggleHelp") + "\nreturn toggleHelp;")( + "const LS_HELP_CONSENT = 'mm_helpConsent', LS_HELP_SHOWN = 'mm_helpShown';\n" + fnSource("docsSiteConsent") + "\n" + fnSource("toggleHelp") + "\nreturn toggleHelp;")( { body: { classList: { contains: () => w.hidden } } }, { setItem: (k, v) => { store[k] = v; } }, () => { w.asked++; return answer; }, @@ -121,3 +121,29 @@ test("a link outside the help, or a modified click, is left to the browser", () assert.equal(linkWorld({ inHelp: false }).opened.length, 0); assert.equal(linkWorld({ meta: true }).opened.length, 0); }); + +test("a list renders as one, its items formatted", () => { + assert.equal(helpMarkdown("- **Pin**: a strip\n- *Network*: Art-Net"), "
  • Pin: a strip
  • Network: Art-Net
"); +}); + +test("a heading renders one level under the view's own", () => { + assert.equal(helpMarkdown("## What you built\n\nA layout."), "

What you built



A layout."); +}); + +test("a clip on its own line plays, a picture shows, and anything not https stays text", () => { + assert.equal(helpMarkdown("![A grid](https://moonmodules.org/a.webm)"), ''); + assert.equal(helpMarkdown("![A card](https://moonmodules.org/a.png)"), 'A card'); + assert.equal(helpMarkdown("![x](javascript:alert(1))"), "![x](javascript:alert(1))"); +}); + +test("a clip that ends the text still plays", () => { + assert.equal(helpMarkdown("Watch it.\n\n![A wall](https://moonmodules.org/b.webm)\n"), 'Watch it.

'); +}); + +test("a fenced block keeps its text as written, escaped and unformatted", () => { + assert.equal(helpMarkdown("```\nxattr -dr **x** \n```"), "
xattr -dr **x** <app>
"); +}); + +test("a fenced block with a blank line inside stays one block", () => { + assert.equal(helpMarkdown("Run:\n\n```\nfirst\n\nsecond\n```"), "Run:

first\n\nsecond
"); +}); diff --git a/test/python/test_check_esp32_built.py b/test/python/test_check_esp32_built.py new file mode 100644 index 000000000..f295de9fd --- /dev/null +++ b/test/python/test_check_esp32_built.py @@ -0,0 +1,24 @@ +"""The ESP32 freshness check marks a firmware stale when anything it is built from changed, the UI included: the UI reaches the image as a generated header whose own inputs no compiler reads.""" + +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent.parent +sys.path.insert(0, str(ROOT / "moondeck" / "check")) + +import check_esp32_built as check # noqa: E402 + + +def test_the_ui_inputs_are_the_files_the_header_is_generated_from(): + inputs = check.ui_inputs() + assert {"src/ui/app.js", "src/ui/help.js", "src/ui/index.html", "src/ui/style.css"} <= inputs + assert all((ROOT / f).exists() for f in inputs) + + +def test_a_build_reading_the_ui_header_depends_on_the_ui_files(): + deps = check.with_ui({"src/main.cpp", "src/ui/ui_embedded.h"}) + assert "src/ui/app.js" in deps + + +def test_a_build_without_the_ui_header_gains_nothing(): + assert check.with_ui({"src/main.cpp"}) == {"src/main.cpp"} diff --git a/test/python/test_gen_moonwalk.py b/test/python/test_gen_moonwalk.py index 78bd92431..7fe695345 100644 --- a/test/python/test_gen_moonwalk.py +++ b/test/python/test_gen_moonwalk.py @@ -24,7 +24,7 @@ def test_the_repo_stage_files_build_into_one_map(): model = gen_moonwalk.build() assert [s["id"] for s in model["stages"]][:2] == ["read", "install"] assert model["processes"]["first-light-show"]["stage"] == "first-light" - assert "first-light-show" in model["byPage"]["tutorials/first-light-show.html"] + assert "run-linux-board" in model["byPage"]["how-to/installing-on-linux.html"] def test_a_step_runs_in_the_runners_its_action_allows(): @@ -109,3 +109,34 @@ def test_a_misspelt_expectation_key_is_refused_rather_than_never_checked(tmp_pat " - {id: s, say: Set it., when: {set: {module: Grid, control: width, value: 16}}, then: [{module: Grid, control: width, equal: 16}]}\n") with pytest.raises(ValueError, match="then/0"): gen_moonwalk.build(root) + + +def test_the_prose_is_plain_text_under_the_map_with_the_process_id_as_its_address(): + text = gen_moonwalk.render_text(gen_moonwalk.build()) + assert "## Build your first light show {#first-light-show}" in text + assert "### 1. Say where the lights are" in text and "### 5. Keep it" in text + assert text.index("**Open Layouts. A fresh device has a Grid.**") < text.index("A grid is the shape most rigs start as") # the instruction, then its detail + assert '