This document is a file-by-file map of the repository: what each file is, what it does, and how the pieces fit together. It was written with the help of Claude Code.
Parakeet Web is a browser speech-to-text app. NVIDIA's Parakeet TDT model (in ONNX form) runs entirely client-side via ONNX Runtime Web (WebGPU, with a WASM fallback). The repository is four layers:
app/src/— the forked, framework-agnostic inference engine (parakeet.js): model loading, mel front-end, decoding, phrase boosting.app/ui/— the Preact/React single-page UI that drives the engine, plus the "phone as mic" remote feature and its end-to-end crypto.signaling/— a small Express server that brokers the WebRTC handshake for the remote microphone (it never sees plaintext audio).docker/,scripts/,test/, CI — packaging, operator tooling, and the three-tier test suite.
A high-level data flow for a single transcription:
audio (file or mic/phone)
-> PCM 16 kHz (audio.js / pcm-recorder-worklet.js / remote-webrtc.js)
-> log-mel spectrogram (mel.js, or preprocessor.js ONNX variant)
-> encoder ONNX session (parakeet.js + backend.js + ONNX Runtime Web)
-> TDT greedy / beam decode, with optional phrase boosting (parakeet.js + phraseBoost.js)
-> token ids -> text (tokenizer.js)
-> dictation regex post-processing (App.jsx)
-> rendered transcript with word timestamps
| Path | What it is |
|---|---|
README.md |
User-facing overview: features, quick start, per-feature docs. |
README_fr.md |
French translation of README.md, kept in lockstep with it (each links to the other; the About modal points here when the UI language is French). |
CHANGELOG.md |
Release notes, starting at 10.0.0 (earlier releases live only in the git history). Written for readers, not as a commit dump: each entry says what was measured and on what. |
CHANGELOG_fr.md |
French translation of CHANGELOG.md, kept in lockstep with it exactly like the two READMEs (each links to the other). |
ARCHITECTURE.md |
This file. |
CLAUDE.md |
Instructions for Claude Code / contributors (version bump, screenshot, vendored-dep and Caddy refresh procedures). |
LICENSE |
AGPLv3 for the combined work. |
package.json / package-lock.json |
Root dev/test harness package (not published). Defines the test:unit / test:http / test:e2e scripts and the prepare hook that installs the git hooks path. |
.npmignore |
Files excluded when the inference engine is packed for npm. |
.dockerignore |
Build-context filter so host node_modules/, dev TLS keys and .env secrets never enter the Docker image. |
.gitignore |
Standard ignores. |
benchmark_reports/sync.sh |
Operator tool: two-way, ADD-ONLY sync of the collected benchmark reports between this folder (gitignored data, only the script is tracked) and the VPS, driven by the same VPS_USERNAME/VPS_IP/VPS_PORT environment as a deploy. Reports are immutable server-named files, so the merge is the union of both sides: no --delete either way, --ignore-existing both ways (a name already on the receiving side is never overwritten), only top-level report-*.json moves, pull before push, then it verifies (checksums of pre-existing local files, listing diff of both sides) and exits non-zero naming any report a side is missing. --dry-run touches nothing. SYNC_LOCAL_REMOTE swaps the VPS for a local folder so test/unit/benchmark-reports-sync.test.mjs runs the real script end to end. |
icon.svg |
App logo (used in README and as a source for the favicon). |
image.png |
README screenshot, refreshed via shot-scraper (see CLAUDE.md). |
This folder is a long-diverged fork of ysdede/parakeet.js.
It is first-party source maintained in-tree, not a clean vendor; see
app/src/SOURCE.md for the fork point and the manual upstream-sync runbook.
Imports resolve through the Vite alias parakeet.js -> app/src/index.js.
| File | Role |
|---|---|
index.js |
Public entry point of the engine. Re-exports ParakeetModel, the hub loaders, and the fromUrls / fromHub convenience factories. |
parakeet.js |
The heart of the engine (~1.3k lines). ParakeetModel: holds the encoder + decoder/joiner ONNX sessions, runs the combined TDT step, and implements both decode paths — greedy (beam width 1) and MAES beam search — plus word-timestamp/confidence extraction and the stateful-streaming hooks used by live transcription. Opt-in collectBeamStats (default off, greedy/production path unchanged) returns per-step beam expansion sizes on result.beamStats. encodeBatch() folds N EQUAL-LENGTH chunks into one encoder run (the ONNX has dynamic batch + time axes) for a WebGPU throughput win; transcribeChunked uses it when this.maxEncoderBatch > 1 (set in fromUrls: WASM=1 so that path is byte-identical; on WebGPU resolveMaxEncoderBatch auto-adapts to the GPU from adapter memory limits + encoder weight size, floor 2, ceiling 4), grouping only equal-length chunks because padding unequal ones leaks through the conformer convs. transcribeChunked also accepts two injected async hooks, opts.decodeChunk(encoded, meta, decodeOpts) and opts.encodeChunk(pcm, meta, encodeOpts): decodeChunk alone runs a producer/consumer pipeline (encode ahead on this thread, off-load decode, drain oldest-first so stitching stays in chunk order) so App.jsx can overlap GPU encode with worker WASM decode; encodeChunk alone drives the encode pool (pooled encode, in-thread decode); BOTH together COMPOSE (pooled encodes with bounded look-ahead feed the off-thread decodes, and the model's own encode paths are never used), which is the WASM composed mode. The single-pass path runs encodeChunk whenever it is present and never calls decodeChunk (nothing to overlap with one chunk). With neither hook (Node/CLI) the loop is byte-identical to before. ParakeetModel.decoderOnlyFromUrls builds the decode-only model the worker uses. The constructor logs, once, what the loaded decoder declares ([Parakeet.js] Decoder in-graph outputs: log-partition=... top-K=...): that runtime signal, never a filename, is how the app and the specs tell a promoted decoder from a stock upstream one. When it declares the in-graph top-K outputs (topk_logits/topk_ids/duration_logits, added by the model repo's optimize-decoder-graph.py) AND the lse ones, the GREEDY loop fetches only those (TOPK_FETCHES, _readTopkStep) instead of reading the whole ~8.2k-float outputs row back per joint call, logging [Parakeet.js] TopK decoder outputs engaged. It engages only at temperature 0, without a phrase-boost trie, and when the row holds the candidates the step reads; the beam path never asks for it (it needs the blank's logit and arbitrary extension-token logits, which a top-K row cannot serve). useTopkOutputs: false (model option or transcribe opt) is the A/B switch back to the full row. |
backend.js |
ONNX Runtime Web initialisation. Picks the WebGPU or WASM backend and integrity-verifies the ORT WASM/MJS runtime it hands to ORT against /ort/manifest.json (defence against a tampered serving path swapping in a malicious ML runtime). Only the PINNED pair for the variant actually loaded is fetched (selectOrtRuntimeAssets over ORT_RUNTIME_ASSETS / ORT_RUNTIME_ASSETS_JSPI, handed over as blob URLs via wasmPaths so ORT requests nothing else). ORT_VARIANTS declares both distributions. The DEFAULT is jspi, ORT's native C++ WebGPU execution provider, which suspends the WASM stack through JavaScript Promise Integration instead of crossing into JS at every GPU/CPU partition like JSEP; jsep is the fallback on a browser without JSPI (everything not Chromium-based) and the escape hatch ?ortep=jsep, per page load, never persisted, no UI (?ortep=jspi still parses as a no-op so older harnesses keep working). Each variant is pinned to its OWN runtime pair so the integrity check covers the bytes ORT actually executes, and jspiSupported/resolveOrtVariant are pure and unit-tested: note downgraded means an EXPLICIT request was refused, so the default landing on jsep warns nobody. The switch was made on SIZE, not speed (16 MB runtime against 27, 112 KB bundle against 404, so ~11 MB off every load): two A/Bs of 3 interleaved reps with the runtime verified per run put the two within noise (GPU fp32 wall 13.0 s jsep / 13.8 s jspi; WASM int8 3-minute clip 125 s / 114 s). Because ORT pins one runtime PER JS CONTEXT, the variant is plumbed into every context that builds a session (both *.worker.js model workers via their init payloads, and perf.worker.js, whose own contract is to time the runtime the app will actually use); a main-thread-only flag would leave the encode pool, which does the encoding, on the other runtime and would not be an escape hatch at all. loadOrtModule is the SINGLE owner of the ORT module instance in a JS context and every consumer must come through it, initOrt included: a module that imports onnxruntime-web itself gets a second instance the moment the two specifiers resolve to different variant entry points, and only the one this loader configured holds the verified blob wasmPaths, so the other fetches its runtime from wherever the bundle sits, gets the SPA fallback HTML and dies with no available backend found (app/ui/src/lib/speakerEmbedding.js did exactly this: invisible while every specifier resolved to the same instance, but forcing the main thread onto the other runtime killed voice matching SILENTLY while diarization kept producing turns). A variant asked for after one is live is warned about, not honoured, since ORT cannot host two runtimes in one context. The import-graph guard in test/unit/ort-asset-verify.test.mjs is what keeps that true, because whenever both specifiers happen to resolve alike the bug has no behavioural symptom at all. A sibling guard there covers the other half, the plumbing: a *.worker.js that builds an ORT session but carries no ortVariant fails tier-1 by name. That one is a tripwire for a HALF-switch, which was a real intermediate state here (a page running three jsep contexts and two jspi ones at once, found by counting log lines by hand) and which nothing behavioural detects, since transcripts come out correct either way and only the probe's timing and the escape hatch quietly stop meaning what they say. It logs `ORT runtime variant: (main |
integrity.js |
The sha384 primitives both verify-then-load paths share: sha384Base64 (the sha384-<base64> form the build-time manifests store), ASSET_INTEGRITY_HARD_FAIL (a missing pin is fatal in a production build and a loud warning in dev, so the vite dev server and the Node unit tests still boot) and integrityError (the IntegrityError name callers match on to tell wrong-bytes apart from a network failure). backend.js pins the ORT runtime and app/ui/src/lib/asset-integrity.js pins the loose assets; each used to carry its own copy of all three, which is how the two would eventually disagree about what counts as production. |
models.js |
Central model registry: per-variant metadata (vocab size, mel bins, prediction-network shape, supported languages) plus LANGUAGE_NAMES. Adding a model version is a one-object change. |
hub.js |
HuggingFace Hub download + browser caching (IndexedDB). Supports a local-base-URL fallback for when HF is firewalled; HubDownloadError lets the UI distinguish "HF blocked" from other failures and offer the local model. It asks for exactly ONE name per quant: the model repo ships one build per precision, so its graph work (the constant-folded encoder from optimize-encoder-graph.py fold, the decoder's in-graph log-partition + top-K outputs from optimize-decoder-graph.py) lives INSIDE encoder-model*.onnx / decoder_joint-model*.onnx. There is deliberately no filename-preference chain here any more (it cost six HEAD probes per local-mirror load and could only ever describe our own repo): the decoder fast paths are detected at RUNTIME from the loaded session's outputNames, so a stock upstream build simply does not engage them. Download-resume segments are persisted BY VALUE (ArrayBuffer, never Blob) and the freshly cached file is served from its IndexedDB readback rather than the in-memory composite, so no handle can alias blob data that later gets reclaimed (unit-tested in test/unit/hub-cache-readback.test.mjs). Files too big to cache whole (the fp32 encoder shards: reading a value back throws once the origin holds more than ~2^31 aggregate bytes of large values, whatever the record size) instead keep a deliberately PARTIAL cache, prefixCacheBytes bytes of resume state that a Range request completes on the next load, which takes the fp32 transfer from ~2.4 GB to ~0.95 GB per load (unit-tested in test/unit/shard-prefix-cache.test.mjs, proved in a real browser by test/e2e/transcription-fp32-wasm.spec.js). A content-encoded response (what a precompressed zstd mirror returns) is handled explicitly: its headers describe the COMPRESSED entity while resp.body yields decoded bytes, so the length is treated as unknown rather than adopted, and the variant etag is not persisted for a later If-Range (unit-tested in test/unit/precompressed-response.test.mjs). |
modelLayout.js |
The single source of truth for where a model file lives inside a repo, shared by every JS loader (hub.js, scripts/transcribe.mjs, test/e2e/serve.mjs, test/e2e/dangling-links.mjs, scripts/fetch-e2e-models.mjs). Basenames never change, so a file's directory is a pure function of its basename: fp32/ for the fp32 encoder/decoder graphs and their .data / .data.NNN sidecars, int8/, int8-lite/, w4a8/, fp16/ for the quantized graphs, and the repo root for vocab.txt, config.json, nemo128.onnx, README.md. layoutDirFor(basename) gives that directory, candidatePaths(basename) the ordered probe list (layout dir, then the flat root, then sharded/) which is what keeps the two LEGACY layouts loadable: everything flat at the root (upstream istupakov), and flat plus a sharded/ folder holding the fp32 shards (the optimized repo before the move). findRepoFile(repoFiles, basename, { preferDir }) picks from a repo LISTING using the same order, with preferDir so the fp32 graph is taken from whichever directory holds the shards rather than from a stale flat monolith. Nothing else in the tree may hardcode a model subdirectory. Unit-tested in test/unit/model-layout.test.mjs; the Python side has the one mirror copy in scripts/wer-quants.py (onnx-asr resolves by glob, so it needs its own). |
idb.js |
Tiny shared IndexedDB helper (memoised open), used by both the model cache and the UI settings store. openIdb self-heals a DB that exists without its expected store (the empty shell a versionless open racing a deleteDatabase leaves behind, after which a versioned open never fires onupgradeneeded again): it bumps the version to force an upgrade that creates the store. E2e-covered by test/e2e/settings-db-storeless-shell.spec.js. |
tokenizer.js |
vocab.txt parsing and decode (id -> text): SentencePiece ▁->space, blank/<unk> skipping, punctuation cleanup. parseVocabText is shared with the server-side boost prebuild. |
mel.js |
Pure-JS log-mel spectrogram (~845 lines) matching the NeMo/onnx-asr preprocessor exactly, with an incremental/streaming API. Ported from upstream after the fork point. |
preprocessor.js |
OnnxPreprocessor: the alternative ONNX-model-based mel front-end (vs. the pure-JS mel.js). De-duplicates concurrent session creation. |
fbank.js |
Pure, dependency-free kaldi-compatible 80-dim log-mel fbank (computeFbank): 25 ms/10 ms povey-windowed frames, pre-emphasis, power spectrum, 80 mel bins, log, plus the CAM++ global-mean per-dim time normalization. Matches kaldi-native-fbank so it can feed the 3d-speaker CAM++ speaker-embedding model. Shared by the browser embedding path and scripts/speaker-embedding-check.mjs; unit-tested in test/unit/fbank.test.mjs. Distinct from mel.js (Parakeet's 128-bin log-mel). |
bpeEncoder.js |
BPE encode (text -> token ids), the reverse of tokenizer.js. Reimplements the upstream HuggingFace tokenizers BPE pipeline closely enough to match token ids for realistic boost phrases. Needed only by phrase boosting. |
phraseBoost.js |
Phrase boosting / context biasing: a token-level trie that injects an additive logit-space reward to bias decoding toward (or away from) user phrases. Drives both the greedy and MAES-beam decode paths in parakeet.js. |
boostCompile.js |
Shared "compile" pipeline that turns a boost-phrase .txt into the serialized token-id artifact (the expensive BPE encode done once). Single source of truth for both the container prebuild and the operator CLI compiler. Node-only. |
SOURCE.md |
Provenance: fork point, divergence notes, and the manual upstream-sync procedure. |
LICENSE.upstream |
The upstream MIT license that still covers the forked portions. |
A Vite-built Preact app (using the React-compat layer). It has two HTML entry points: the main app and the remote-microphone phone page.
| File | Role |
|---|---|
index.html |
Main app HTML entry. |
remote-mic.html |
Separate HTML entry for the phone "remote mic" page. |
vite.config.js |
Vite build config: the parakeet.js alias to app/src, the two entry points, optional local HTTPS, and the COOP/COEP setup. |
postbuild.mjs |
Post-build SRI injector: adds integrity= to the content-hashed <script>/<link> refs in dist/*.html, and emits the asset-integrity / ORT manifests that backend.js and asset-integrity.js verify against at runtime. |
package.json / package-lock.json |
UI dependencies and build scripts. |
| File | Role |
|---|---|
main.jsx |
React root bootstrap for the main app. Installs a global error/rejection banner so nothing fails silently in production. |
App.jsx |
The application (~7.3k lines, and being pulled apart one feature hook at a time, see hooks/ below): UI state and orchestration — model load, file/mic/phone recording, decode options (beam, boosting), dictation-regex post-processing, word-timestamp rendering, settings persistence, and wiring of every lib/ helper. Also owns the sidebar Benchmark section's orchestration: it pushes each planned backend/precision into React state, waits for the change to be LIVE (liveSettingsRef, refreshed every render, polled with a real sleep) so loadModelRef/runTranscriptionRef run closures that see the new settings, then feeds lib/benchmark.js's driver its own load and transcribe paths (runTranscription's benchmark: true mode keeps the results out of the history/clipboard and rethrows failures as data). It restores the user's combination afterwards, reloading only when the run ended on a different model. Owns the SOURCE-CAPABILITY probe too: on every change of model source or repo it lists whichever source a load would really use (the same local-vs-hub order, with the same both-ways retry) and hands the listing to lib/encoderQuants.js, which is what lets a precision this deployment does not host be greyed out with a reason BEFORE it is picked instead of after a failed load. The listing is a hint, never a gate: no answer means every precision stays offered. And when a precision turns out to be unservable anyway, the QuantUnavailableError catch now tries the cheapest OTHER GPU precision this source can serve before it considers moving to the CPU, because a visitor who chose the GPU would rather change file than change backend. |
App.css |
Styles for the app. Includes the html.gpu-run rule pausing every CSS animation while a WebGPU transcription runs (runTranscription toggles the class): compositor frames gate WebGPU callback delivery process-wide, so the animating spinner taxed every JSEP yield inside encoder session.run ~50x (2026-08-11). |
config.js |
Build-time/runtime config indirection. Reads window.__CONFIG__ (written by the Docker entrypoint) or falls back to Vite import.meta.env. Every operator-settable VITE_* key must be listed here. |
i18n.jsx |
Translation tables + I18nProvider / useI18n / LanguageSwitcher. |
remote-mic-entry.jsx |
React root + UI for the phone page: captures mic audio (or decodes a saved audio file on the phone), encrypts it, and streams PCM to the desktop over WebRTC (with pause/resume, multi-recording, wake-lock). The "Send an audio file" action decodes + downmixes the file to mono locally (no phone-side resample, for iOS robustness; the desktop resamples) and pumps it through the same audio-config->Int16-chunks->audio-end framing as the live mic, paced via RemoteMicRTC.drain(). |
phraseBoost.worker.js |
Module worker that runs the whole boost-list compile chain (parse -> warnings/conflicts -> augmentation expansion -> BPE encode, i.e. compileBoostList) off the main thread so the UI does not freeze on large clinical lists. encode: false asks for a parse-only pass (count + warnings, no expansion, no BPE). A second request kind, kind: 'prebuilt', fetches and parses the server-prebuilt encoding (<list>.json) via parsePrebuiltBoost and transfers back the packed arrays: on the French clinical lexicon that artifact is ~37 MB parsing to ~478k entries, and doing it on the main thread froze the tab for 2.7 s at page load on a 6x-throttled CPU (the ?mode=med stall). |
decode.worker.js |
Module worker that runs the Parakeet TDT decoder/joiner (WASM) off the main thread. On WebGPU its CPU decode overlaps the main thread's GPU encode; on WASM it engages only COMPOSED with the encode pool (pooled encodes feed worker decodes), so a single-pass clip or a gated-off pool keeps the in-thread decode. Builds a DECODE-ONLY ParakeetModel (ParakeetModel.decoderOnlyFromUrls: joiner + tokenizer, no encoder/preprocessor) and calls the SAME transcribe() fed opts.encoded, so no decode logic is duplicated. Rebuilds the phrase-boost trie from cloneable token ids (BoostingTrie.buildFromEncoded) since the live trie cannot cross postMessage. Decodes are serialized FIFO (one stateful joiner session); the encoder output crosses as a TRANSFERRED buffer (zero-copy). App.jsx wires it to transcribeChunked's injected opts.decodeChunk (on WASM alongside opts.encodeChunk, which the driver COMPOSES: pooled encode feeding worker decode); any failure falls back to in-thread decode. Gating: WebGPU always; on WASM it is operator OPT-IN (VITE_WASM_DECODE_PIPELINE=true, no rebuild) and then engages only when the encode pool engaged for that run. Default off on evidence: composed measured +1.6% wall at beam 1 and +0.3% at beam 5 versus pool-only (2026-08-12 in-browser A/B), because the pool already overlaps decode with encode, so it is kept for the main-thread responsiveness it buys rather than for throughput. The WebGPU pipeline is not exercised by the WASM e2e (WebGPU-gated, so headless-CI can never hit it) but IS validated on a real GPU by scripts/webgpu-check.mjs --fp32, which asserts the [Decode] pipeline engaged marker end to end; the WASM composed pipeline is fully headless-covered by test/e2e/transcription-composed-pipeline.spec.js. |
encode.worker.js |
Module worker (WASM only) for chunk-parallel encoding: App.jsx spawns a POOL of these (size + per-worker threads from encodePoolPlan in lib/cpuThreads.js, gated on cores/RAM and the parallelEncode toggle) and wires them into transcribeChunked's injected opts.encodeChunk, so two workers encode different chunks concurrently while the decode runs on the main thread (or, when the decode worker is up too, in that worker: the driver composes the two hooks). The encoder's thread scaling saturates near the physical core count and chunks are independent, which is what the pool converts into throughput. Deliberately NOT used for WebGPU: a worker-side GPU encoder was tried (2026-08-11) and measured ~3x worse than the main thread, since WebGPU callback delivery is gated by the page's compositor activity process-wide; the rendering-coupling fix is the html.gpu-run animation pause instead (App.jsx/App.css). Builds an ENCODE-ONLY ParakeetModel (ParakeetModel.encoderOnlyFromUrls: encoder + mel preprocessor, no joiner/tokenizer) from the main thread's pre-verified weights (blob URL/bytes; never a second unverified fetch) and calls the SAME encode(). PCM crosses in and the encoder output crosses back as TRANSFERRED buffers; encodes are chained FIFO per worker (parallelism comes from the pool). Any failure (gate, init, crash, mid-run reject) falls back to the serial in-thread path. Fully covered headless: driver in test/unit/chunk-stitch.test.mjs, plumbing end to end in test/e2e/transcription-parallel-encode.spec.js. |
| File | Role |
|---|---|
useDiarization.js |
Speaker diarization as one hook: the "Speakers" view's state (15 slots + four refs), the run itself (diarizeEntry/cancelDiarizeEntry, silence excision, the single-vs-piecewise choice, cross-recording voice matching), the auto-run and model-prefetch effects, and the speaker rename/merge/persist helpers. The first cluster lifted out of App(), which had grown to one 8800-line function holding 231 state slots in a single scope; diarization was a clean seam because almost nothing outside it reads its state. Everything only diarization touches (run progress, the model-download error, the silence-cut cache, the embedding/name mirror refs) is private to the hook; what crosses the boundary is the caches the persist effect writes and the handful of values the entry UI renders. diarizedPlainText and renderDiarizedTranscript deliberately stayed in App.jsx, because both compose the diarized view with the DICTATION regex layer, a different feature with its own state. |
usePipelineWorkers.js |
The two off-main-thread pipeline workers as one hook: the DECODE worker and the chunk-parallel ENCODE pool. One subject because they are two halves of the same lever and share one gate: on WebGPU the decode worker is independent and always runs, hiding the WASM decode behind the GPU encode, while on WASM it only ever runs COMPOSED with the pool, so the parallelEncode toggle owns both and both start or stop WITHOUT a model reload (each stashes its init params from the last successful load). Owns the refs, the two bridges handed to transcribeChunked (decodeChunkViaWorker, encodeChunkViaPool), the boost sync that ships the cloneable token ids because the live BoostingTrie cannot cross a postMessage, the hardware gate via encodePoolPlan, and the toggle and unmount effects; the request-id counters, pending maps, round-robin cursor and the two raw init functions are private. Worth isolating precisely because every failure in here is a FALLBACK, never an error: a worker that cannot be created, a failed init, a crash mid-run and a toggle-off all reject what is pending so the in-thread path re-runs the clip, which means a break is masked by a healthy transcript. loadModel (still in App.jsx) stashes the params and calls start/stop after a load. |
usePhraseBoost.js |
Phrase boosting (context biasing) as one hook: the phrase text and the strength / min-p / depth-scaling knobs, the /boost-phrases/ manifest and the per-list server-prebuilt encodings, the lazy encode worker that keeps BPE and the multi-megabyte prebuilt parse off the main thread, the render-time size gates that decide whether a list is mounted in a textarea at all, the debounced trie rebuild, and waitForBoostReady, the gate a run polls so a transcription can never start decoding against a stale trie. Six scattered regions of App.jsx, and of the 41 names they defined only 22 are read outside, so the build-key stamp, the strength/min-p/verbose live mirrors, the prebuilt and BPE-encoder caches and the worker plumbing are private now. Two things stayed in App.jsx: the sidebar JSX that renders it all, and the decode worker, which reads the hook's boostEncodedRef because the live BoostingTrie cannot cross a postMessage and only the token ids can. One coupling had to become explicit rather than move: the ?mode= medical preset and this hook's own one-shot source resolution both fire once settings and the manifest have loaded, and in App() the preset won by being declared first and claiming a shared ref. A hook's effects all run before the effects declared after its call site, so that no longer holds; the hook now takes skipInit and stands down entirely when a med-mode link opened the page. |
useRemoteMic.js |
The remote microphone ("phone as mic") as one hook: the encrypted WebRTC session with the phone, the ECDH handshake and its fingerprint verification, the PCM the phone streams in, the elapsed timer, the SRI-pinned QR script loader, and the batch handed to the transcription core as if it were an upload. The most contiguous seam in App.jsx: startRemoteMic .. cancelRemoteMic was one unbroken 635-line run, and of the 52 names it defines only 22 are read anywhere else, so the RTC handle, the ECDH key, the sample-rate/format negotiation, the handshake-in-progress and verify-resolver refs and the QR loader are all private now. The PCM slab buffer stayed in App.jsx because local recording fills the same one, and the three audio preferences (noise suppression, AGC, gain) stayed because they are persisted sidebar settings; the hook takes them as inputs. It also stopped the shared clearPcmChunks from reaching into remote-mic state: the two counters it used to reset now live behind the hook's own resetRemoteMicBuffer. |
| File | Role |
|---|---|
Banner.jsx |
Tone-styled banner (info/danger/...). |
Button.jsx |
Variant-styled button. |
Card.jsx |
Tone-styled card container. |
CollapsibleSection.jsx |
One collapsible settings group in the sidebar. The body UNMOUNTS when closed, so the drawer stays a short list of titles; open/onToggle are controlled props because the open set is persisted per section. |
InfoTooltip.jsx |
The ? help icon and its popup. Rendered through a PORTAL into <body>, not inside the icon's span: a dimmed ancestor (.disabled-option, opacity 0.5, used for a greyed-out precision row) would otherwise multiply into the popup and make the very explanation of WHY the option is greyed out unreadable, and a portal also keeps it out of any ancestor stacking context or transform that would break position: fixed. Width is viewport-clamped rather than measured, since shrink-to-fit inside the narrow sidebar produced a thin, very tall column on phones. |
DecodeDebugView.jsx |
Per-entry "Debug" base mode (shown when the entry carries a decodeDebug payload, i.e. it was transcribed with the sidebar "Add decoder debug view" checkbox on): one clickable pill per decoded token (confidence-tinted, boost-highlighted, per-chunk groups on chunked runs) opening an inline card with that emission's evidence (true logit, log-prob, boost bonus, TDT duration, confidence, joint score) plus the top-k alternatives and, on beam runs, the surviving MAES beam at that frame. |
Modal.jsx |
Modal primitive + a module-level "any modal open" counter (useAnyModalOpen) that disables background controls to thwart keystroke-injection attacks. |
settings/SettingsSidebar.jsx |
The settings drawer itself: the backdrop, the close button, the language row and the "Mode Dictee Medical" shortcut above the groups, and the footer below them (dictation-device connect, clear history, reset everything, About, version). The six collapsible groups arrive as children, not as props: they already own their own state and between them take about 150 props, so threading those through the shell would rebuild the monolith one level down and make this file change every time a single group gains a checkbox. onMedMode is one callback for the same reason EngineSection takes onAutoconfigure: the click reruns the performance probe and can arm a model reload, and those rules stay next to the rest of the load machinery in App.jsx. |
settings/RecordingSection.jsx |
The sidebar's Recording group: the browser's capture shaping (noise suppression, auto gain), the remote-mic gain (shown only while a phone is acting as the mic), and live transcription with its context window. The capture-shaping controls are disabled DURING a recording rather than hidden, because they are applied when the stream is opened and a mid-take change would claim something the running capture is not doing. |
settings/EngineSection.jsx |
The sidebar's Engine group, and the largest of the six: model repo, chunking, the loaded-vs-requested row, backend radios, autoconfigure, the encoder-precision radios, thread count, the encode-pool toggle, frame stride, beam width and the MAES knobs. Three handlers arrive as single callbacks rather than as the state they touch (onRepoChange, onAutoconfigure, onCpuThreadsCommit): each arms a model reload and reads refs only App.jsx owns, so passing those refs down would put the reload rules in two places. The loaded-vs-requested row exists because hub.js is allowed to resolve a request differently (the WASM int8 pin, the GPU-to-WASM fallback, a switch to the /models mirror) and every one of those used to be invisible: the controls kept showing the request, so a station could sit on "WebGPU / fp32" while an int8 CPU model did the work. |
settings/GeneralSection.jsx |
The sidebar's General group: the keyboard-shortcut opt-in and its cheat sheet, what happens to a finished transcript (auto-copy, spelled numbers to digits, whether history survives a reload), and which view a transcript opens in by default. The display-mode options are DISABLED rather than hidden when they cannot be served: a Speakers default with no diarization models greys out and names the reason on hover, and the dictation options only exist when rules are configured. |
settings/BoostingSection.jsx |
The sidebar's Phrase boosting group: which list is loaded, its strength, the list itself, and the two advanced knobs (boostMinp, boostDepthScaling). The phrase area has four mutually exclusive faces, which is what the branch chain encodes: boosting off, a curated list too long to edit in place, the user's own list too long to edit in place (with an explicit "edit anyway" escape), or the editable textarea. Only the Custom slot is the user's own, so edits made while a curated file is selected live for the session and are never written over it. The advanced knobs are hidden with no phrases loaded, because the trie is inert then. Owns BOOST_HINT_PANEL_STYLE, the dashed hint panel shared by three sibling branches: they differ only in text colour, and the copy-paste is how one of them once ended up reading a surface token defined in no stylesheet, at about 1.4:1 contrast. |
settings/BenchmarkSection.jsx |
The sidebar's Benchmark group: pick the backend/precision combinations this device can run, measure them on a clip that ships with the app, and read the anonymised report before copying or sending it. Nothing leaves the browser without an explicit action, so whether a Send button exists at all is the operator's VITE_BENCHMARK_UPLOAD, passed in as uploadEnabled rather than read here, so the button and App.jsx's auto-send path agree by construction. The table shows speed as AUDIO PER SECOND OF COMPUTE, not the conventional rtf: same measurement, but this way round reads without translation ("6x" is an hour of audio in ten minutes, bigger is better) and matches every other speed figure in the app, while the report still stores rtf so older reports and scripts/benchmark-throughput.mjs stay comparable. |
settings/DebugSection.jsx |
The sidebar's Debug group: logging level, the per-token decode trace opt-in, and the copyable support report. The logging select drives TWO stored settings (showAdvancedInfo for extra numbers in the UI, verboseLog for engine console output) because a support request needs both or neither. First of the settings-sidebar sections to be pulled out of App.jsx; the rest follow the same shape (props are the state the section actually needs, nothing else). |
VerificationModal.jsx |
Blocking fingerprint-compare modal for the remote-mic handshake — the human MITM check on the swapped-key attack. Non-selectable code + confirm delay. |
| File | Role |
|---|---|
audio.js |
Shared audio helpers: PCM resample to 16 kHz (resamplePcmTo16k), an RMS level monitor (createLevelMonitor), buildRecordingRateCandidates (the ordered sample-rate list for opening the LOCAL recording AudioContext: native/reported rate and the browser default before the SpeechMike-specific low rates, so a Firefox mic that reports no rate is not forced into a 16 kHz context and slowed down), and pickRemoteMicCaptureRate (the PHONE remote-mic capture rate: force 16 kHz to keep the WebRTC wire small when the browser reports the mic rate and resamples correctly, else native rate on Firefox so the mic-vs-context mismatch does not slow the stream). Used by both local and remote recording. Both rate helpers unit-tested in test/unit/recording-rate-candidates.test.mjs. Also createWavBlob, the 16-bit PCM WAV writer behind the inline history player, the recording download and the remote-mic batch (which is handed to the transcription core as if it were an uploaded file, so all three carry the same bytes); it moved out of App() to be reachable from the hooks, and test/unit/wav-blob.test.mjs is the first coverage it has had. |
audioDecode.js |
Decodes an uploaded file to mono 16 kHz PCM for the transcribe pipeline. decodeToPcm16kFfmpeg runs the vendored ffmpeg.wasm (ffmpeg -i <file> -ac 1 -ar 16000 -f f32le) for byte-for-byte parity with the CLI (scripts/transcribe.mjs), including the AAC encoder-delay/priming trim that the browser's decodeAudioData skips (this is what fixed "Venlafaxine" mis-hearing as "Velnafacine" on uploads). decodeToPcm16kWebAudio is the single-pass OfflineAudioContext fallback (decode straight into a 16 kHz context, no 48 kHz intermediate). decodeToPcm16k is the shared entry point (ffmpeg first, Web Audio on any failure, returns which decoder ran) used by BOTH App.jsx's processAudioFile and the sidebar benchmark, so a benchmark measures the same decode an upload gets. ffmpeg core is lazy-loaded on first upload. |
loadProgress.js |
planLoadProgress(event, ctx): one hub progress event turned into what the load UI should show. The hub reports two unrelated things through one callback, and they behave differently: ATTEMPT events fire before any bytes flow so a stalled connection still shows "Retry 2/3" (silent at maxAttempts 1, since "Retry 1/1" makes an ordinary first try look like a recovery, and only the FIRST attempt rewinds the bar because a resume picks up from the prefix already on disk), while BYTE events carry the file line, the percentage, the trailing-10s rate and the ETA. It also owns the two rules that are easy to get subtly wrong: a resumed download credits only what the CONNECTION was asked for (the cached prefix comes back off) and the credit is monotonic per file so a restarted stream cannot shrink the accounted total; and a byte event is the ONLY proof that lets the app say "downloading", because a load answered entirely from IndexedDB streams nothing and must not claim a download it never made. Pure: now is passed in and the caller keeps the four refs. Unit-tested in test/unit/load-progress.test.mjs. |
modelRequest.js |
buildDownloadOpts(...): what precision, and from which source, one model-load attempt ASKS FOR. hub.js decides what a repo can serve; this decides what to request of it, and the rules are the sharp ones. The app rewrites NO GPU precision locally (fp16 used to be degraded to fp32 whenever the adapter lacked shader-f16, a 2.35 GB download nobody asked for), so an unhonourable request is sent as-is and refused by hub.js, which is what raises the banner. int8lite/w4a8 pass straight through instead of collapsing to int8, so hub.js can tell a hand-picked build this repo does not ship apart from the default: that distinction IS the no-silent-downgrade signal. A first (HuggingFace) attempt still names /models, but as localUpgradeBaseUrl rather than localFallbackBaseUrl (hub.js may switch to the mirror BEFORE downloading rather than fetching downgraded weights and throwing them away); the two keys are mutually exclusive. Returns wantWebgpu and wasmEncoderRequest alongside the options because the encode-pool gate in App.jsx needs them again after the download. Unit-tested in test/unit/model-request.test.mjs. |
pipelinePlan.js |
planPipelineWorkers(...): which off-thread halves a freshly loaded model gets, and which of them start now. The encode pool is WASM-only and refused at fp32 (each worker holds its own copy of the encoder weights: fine at int8 ~850 MB, not at fp32 ~2.4 GB), then gated again by hardware via encodePoolPlan. The decode worker is unconditional on WebGPU and, on WASM, only ever eligible COMPOSED with the pool (alone it is a measured slight loss, since the pool already overlaps decode with encode), which is why it asks the pool's hardware gate in advance. Eligibility and starting are separate answers: the init-params STASH is what lets the sidebar parallelEncode toggle start or stop a half later without a model reload, so only startPool/startDecodeWorker follow the toggle. Also buildWorkerInitParams(...): the payload each allowed half is handed. Separate from the plan because deciding WHETHER a half runs and deciding WHAT to hand it are different questions, and this one has its own failure mode: every worker is its own JS context with its own ORT runtime, so ortVariant has to travel in BOTH payloads or a ?ortep= choice reaches only some of the contexts that build sessions (that has happened once, 3 jsep / 2 jspi in one page). Pure and unit-tested (test/unit/pipeline-plan.test.mjs) precisely because every failure here falls back to the in-thread path and produces a perfectly healthy transcript, so nothing else can notice a gate that drifted. |
loadFailure.js |
What a failed model load does next, as pure policy: planLoadFailure returns one of five outcomes (retry against HuggingFace, retry against the local mirror, flip a GPU backend to WASM and retry, name an unservable precision in a banner, or give up loudly) from the error kind, the source the attempt used, the backend and precision in play, and which retries are already spent; shouldProbeLocalMirror says when the one piece of I/O the decision needs (a HEAD of /models) is worth making, which is never when the operator already enabled fallback. App.jsx keeps only the doing. The local-retry gate itself is NOT restated here: it calls hub.js's shouldRetryLocally, so the two cannot drift. Extracted because every branch used to be reachable only by building the deployment that produces it (a mirror that 404s the GPU shards, a repo with no fp32 shards, a HuggingFace that only LOOKS unreachable), i.e. a tier-3 spec loading real weights, and several combinations had no spec at all. Unit-tested in test/unit/load-failure.test.mjs, including an exhaustive sweep of all 4096 input combinations asserting the invariants that hold across the whole ladder: the banner and the blocking popup never fire together (one says "change your pick", the other says nothing is left to pick), a retry never announces a final outcome alongside itself, and the GPU flip is the only thing that changes the backend and always retries. |
loadPhase.js |
The three phases of a model load and how to report them. MODEL_LOAD_STATUSES/isModelLoading are the set every "is a load in flight?" gate asks about, hoisted out of App.jsx because the pair they used to be was spelled out at six call sites and a third phase would have had to be edited into each. formatLoadTiming is the per-load console line ([Load] ready in 9m12s: fetch 9m04s (2331 MB), sessions 8.1s): before it only the SUM existed, as the benchmark's loadMs, so a slow load could not be attributed to a connection rather than a GPU shader compile without a rerun. A zero-byte load prints cached and a driver that reports nothing prints transfer unknown, deliberately distinct, since reading the second as the first is how a cold load gets mistaken for a warm one. Unit-tested in test/unit/load-phase.test.mjs. |
format.js |
Pure display formatters (formatTime, formatDuration, formatBytes) plus relativeAge/isFresherThanDays, the coarse "how long ago did this instance restart" pair behind the dev-mode banner and its 5-day expiry (both fail permissive on an unparseable or absent timestamp, so the dev server, which stamps none, keeps its warning), plus wavNameFor, the download filename for a history entry's stored audio (the blob is always the 16 kHz mono WAV the model heard, so the source extension is swapped for .wav), and boldRuns, which splits a translated string on **bold** markers into { text, bold } runs so a label owned by i18n (the encoder-precision radios' "recommended") can carry emphasis without dangerouslySetInnerHTML; an unclosed marker stays a literal asterisk, since a translator's typo must not emphasise the rest of the sentence. Also transcribeErrorMessage, the one formatter behind every transcription/diarization alert, which used to live inside App() even though it closes over nothing: errors reach the UI from three layers that throw differently (an Error with a message, a DOMException whose name is the only useful part, a bare string), and only the ones carrying a stack point the user at the console. And sanitizeDeviceName, which makes an untrusted WebHID productName safe to render: a bidi override (U+202E) in a USB descriptor can make "SpeechMike" render with its suffixes swapped, which is enough to fool someone confirming they paired the right device (F-52). |
boostConfig.js |
Module-level phrase-boost configuration shared by App.jsx and the usePhraseBoost hook: the two source sentinels (__custom__, __disabled__, neither a valid .txt manifest entry, so neither can collide with a served list), the slider defaults (BOOST_STRENGTH_DEFAULT, BOOST_MINP_DEFAULT, each asserted in four places, which is why they are named rather than repeated), the size thresholds that decide when a list is collapsed rather than mounted in a textarea or slow enough to warrant a spinner, and two pure helpers: normalizeBoostName (what a ?phrase_boost= link or VITE_PHRASE_BOOST_DEFAULT means; a bare name gains .txt, a blank value is null so a saved choice stands) and boostBuildKey (the identity of one trie build, which waitForBoostReady compares against before letting a transcription start; strength and min-p are absent by design because they mutate the live trie). Unit-tested in test/unit/boost-config.test.mjs. |
fetchCapped.js |
fetchTextCapped(url, maxBytes) and the 5 MB SERVED_FILE_MAX_BYTES default: streamed, byte-capped text fetch for anything served out of the operator's static content directory (dictation-regex CSVs, boost-phrase TXTs and their prebuilt JSON). F-102 defence in depth against a poisoned upstream feeding the tab a multi-GB body. Shared by App.jsx and phraseBoost.worker.js, which fetches the prebuilt boost encoding itself. |
keepalive.js |
Ref-counted keepalive: screen Wake Lock + a silent looping audio element to dodge background-tab throttling during long inference. |
workerInit.js |
workerReady(worker, initParams, {timeoutMs, label}): the shared init handshake for decode.worker.js/encode.worker.js in App.jsx, and for the sherpa-onnx diarization worker in diarizer.js. Folds all three failure signals into one always-settling Promise: the init-scoped error message, the worker error EVENT (a worker whose SCRIPT asset fails to load posts no message at all, which used to leave readiness pending forever and hang every WebGPU transcription gated on it before chunk 1), and a 120 s watchdog for a hung init. Unit-tested in test/unit/worker-init.test.mjs. |
modelWorker.js |
createModelWorker({initModel, runType, run, handlers}) plus errorMessage: the WORKER side of that same contract, shared by decode.worker.js/encode.worker.js. The two were structural twins (same modelPromise, same init -> ready/error handshake, same FIFO .then() chain serialising runs on the one ORT session, same String(e?.message ?? e) shaping, same {type:'result', id, chunkIndex, ...} envelope), so each now supplies only a model factory and one run body. The message shapes are load-bearing on the main thread: workerReady resolves on {type:'ready'} and fails on an init-scoped {type:'error'} with NO id, while a per-request error always carries one. Unit-tested in test/unit/model-worker.test.mjs. |
perf.worker.js |
Throwaway module worker holding ONE arm (WASM int8 or WebGPU fp32) of the autoconfigure probe; App.jsx spawns one per arm, interleaves their timed runs and terminates both. One worker PER ARM is the whole point: ORT-web initialises its WASM runtime once per JS context and picks a single binary while doing it (plain vs JSEP), so probing on the main thread would pin that choice before the real model load and silently pin that choice for the real model load, while probing both arms in one worker would force the second to reuse the first's binary. The GPU arm uses a STRICT webgpu provider list (no wasm fallback, unlike the app's webgpu-hybrid): with a fallback, an adapter that cannot run the graph would quietly execute it on the CPU and report a CPU time as a GPU time, the one lie that would recommend a 2.4 GB download for nothing. Fetches nothing itself (the main thread hands in the bytes) and settles through the shared workerReady handshake, so no failure can hold up a load. |
liveTranscriber.js |
Streaming transcriber: runs the model over a sliding PCM window, emits committed vs. pending words with absolute timestamps, and adapts step/window size to bound latency. |
captureQueue.js |
Pure createCaptureQueue({canRun, runJob, onCountChange}): a gated FIFO buffer for audio captured before the model is ready. App.jsx routes the record / phone / upload paths through one instance so a clip finished while the model is still downloading is buffered (not dropped) and transcribed in enqueue order once canRun() (model loaded, no live recording, no transcription already running) turns true. Unlike writeQueue.js's createSerialQueue (runs tasks immediately, just in order), this HOLDS tasks until they may run. Unit-tested in test/unit/capture-queue.test.mjs. |
cpuThreads.js |
Pure policy for the WASM inference thread-count setting: restoreCpuThreads restores the persisted slider value, clamped to the core count, with a ONE-TIME migration of the legacy hardwareConcurrency - 2 default (which oversubscribed hyperthreaded CPUs and, with ORT-WASM's spin-waiting pool, could be slower than 1 thread) to the ORT-style defaultWasmThreads() (min(4, ceil(hc/2)), exported by app/src/backend.js). The migration flag persisted by App.jsx keeps a later deliberate re-pick of that same number honoured. encodePoolPlan is the chunk-parallel encode pool's gate + thread split: 2 workers each granted half the USER's thread budget (the slider stays the one CPU knob; the pool redistributes it, never multiplies it), refused below 12 logical cores (hyperthreads are counted, so 8 cannot tell a 4C/8T laptop from an 8C/8T desktop and an unknown count is refused too), below 8 GB deviceMemory (undefined passes: Chrome-only API) or on an unsplittable 1-thread budget. Unit-tested in test/unit/cpu-threads.test.mjs. |
chunkDuration.js |
Pure policy for the persisted long-audio chunk window: restoreChunkDuration honours a stored value (clamped to [MIN, MAX_CHUNK_DURATION_SEC]) with a ONE-TIME migration of the legacy 20 s default (which usePersistedSetting wrote back on every pre-bump install's first boot) to the current DEFAULT_CHUNK_DURATION_SEC 60 s, measured better on long audio. Same flag pattern as cpuThreads.js: App.jsx persists chunkDurationMigrated, so a later deliberate re-pick of 20 s is honoured. Unit-tested in test/unit/chunk-duration.test.mjs. |
beamWidth.js |
Pure policy for the auto-coupled beam width default. The 2026-08 French-medical grid sweep showed the beam effect FLIPS SIGN with the lexical prior (unboosted accuracy degrades monotonically as the beam widens on term-dense audio; boosted improves monotonically), so while the user has never chosen a width, resolveAutoBeamWidth follows the boost state: greedy (1) with no active phrase list, the device-tier default with one. restoreBeamWidthAuto restores the persisted beamWidthAuto flag with a legacy-install inference (a stored width equal to this device's tier default is treated as "never chosen"; any other value was picked on purpose and is honoured). Editing the width in the UI turns the coupling off for good. Unit-tested in test/unit/beam-width-auto.test.mjs, wiring end to end in test/e2e/beam-width-auto.spec.js. |
supportReport.js |
The sidebar Debug section's copyable support report: collectEnvironment probes ONLY guarded browser APIs (UA + high-entropy client hints, hardware/screen/JS-heap, WASM feature detection via WebAssembly.validate byte-modules, WebGPU adapter info/features/limits, connection, storage estimate, audio-input count, and the live globalThis.ort env once a model has loaded) so it never throws anywhere from bare Node to any browser; buildSupportReport assembles it plus the app/settings/model state App.jsx passes in into stable fixed-key-order JSON (BigInt folded) that a user can paste into an issue so "support my hardware" reports arrive with the exact context. Unit-tested in test/unit/support-report.test.mjs, probe truth + copy round-trip in test/e2e/support-report.spec.js. |
benchmark.js |
Pure logic behind the sidebar Benchmark section (the one-click "measure every backend/precision this machine can run" report). planBenchmark builds the candidate matrix (WASM int8 / fp32, plus WebGPU fp32 when an adapter exists and is not disabled), marks the >1.5 GB fp32 rows heavy so they stay opt-in, and sorts the visitor's CURRENT combination last so the run ends on the model the single-model cache already holds; estimatedDownloadMB prices a selection. Given servableQuants (from lib/encoderQuants.js) it also DROPS the combinations this deployment cannot serve, so a mirror that ships no fp16 offers no fp16 row rather than a row that spends a minute finding out. planBenchmarkRows then turns the plan into pending placeholder rows so the table is drawn in full before the first measurement, markBenchmarkRowRunning marks the row being measured (and which phase it is in), and mergeBenchmarkRow folds each result in as it lands: a profile-less row (a whole-combination failure) claims only rows still pending, so a finished number is never overwritten by a later verdict about its sibling. runBenchmarkPlan is the driver: per combination it calls the injected applyCombo/loadModel/transcribe (App.jsx wires those to the app's OWN paths, so the numbers describe what a real user gets), times each run, medians repeats, reports each finished row through onResult as soon as it exists (which is what fills the live table), and NEVER throws (a load or decode failure becomes a failed row, a QuantUnavailableError an unavailable one, a Cancel a cancelled one). Benchmark loads deliberately run with App.jsx's precision substitution DISABLED: everywhere else, answering an unservable precision with a servable one is a kindness, but here it would put another precision's numbers under this row's name, so the row has to say unavailable instead. tilePcm repeats the shipped 11 s clip to ~90 s for the optional chunked profile so one small asset yields an identical multi-chunk workload everywhere; transcriptSimilarity (word LCS) sanity-checks the short profile against the clip's known transcript, which is what catches a backend that returns silence. anonymizeEnvironment is an ALLOWLIST over supportReport.js's probe: it keeps cores/RAM/heap-limit, coarse brand+major version, WASM capabilities, the WebGPU adapter/features/limits and the ORT env, and drops the raw user agent, high-entropy client hints, languages, time zone, screen geometry, storage estimate, audio-input count and network RTT. buildBenchmarkReport/formatBenchmarkReport emit fixed-key-order parakeetweb-benchmark-report/1 JSON. Unit-tested in test/unit/benchmark.test.mjs. |
encoderQuants.js |
The single source of truth for which encoder precisions exist, what they weigh, and which of them a given deployment can actually serve. WASM_ENCODER_QUANTS/WEBGPU_ENCODER_QUANTS are the per-backend whitelists (moved here out of App.jsx so the radios, the benchmark plan and the tests read the same list), ordered for display, and QUANT_DOWNLOAD_MB prices each one. DEFAULT_WASM_ENCODER_QUANT (int8) and DEFAULT_WEBGPU_ENCODER_QUANT (fp16) are what a visitor who has never chosen gets and what a nonsense saved value is coerced back to; they are also the only two precisions the app is ever allowed to reach for on its own: fp32 and w4a8 are hand picks, since one nearly triples an unrequested download and the other is the weakest encoder on long audio. fp16 became the GPU default on 2026-09-11: it is half of fp32's bytes at the same accuracy, which is why the model repos publish the file at all. servableEncoderQuants({repoFiles, shaderF16}) asks hub.js's own quantSatisfiable per backend/precision, so the answer is the same predicate the loader will apply rather than a second guess at it, and returns null for an absent or empty listing: null means NO OPINION and every consumer has to fail PERMISSIVE, since a probe that has not answered must never grey out a precision that would have loaded. effectiveEncoderQuant({backend, selected, servable, shaderF16}) answers the question the radios and the loaded-model row both need, which precision a load would REALLY use: the selection when all three questions pass, else the backend's default if THAT passes, else null, meaning this backend has nothing it may use unasked. On the GPU null is the answer for an adapter with no shader-f16 and for a source hosting no fp16 file, and App.jsx takes it to WASM int8 rather than to fp32, WITHOUT rewriting the stored preference, so a visitor who moves to a machine that can run fp16 gets it back. On WASM a null means the deployment cannot serve even int8, which is the blocking popup. It lived in App.jsx at component scope until 2026-09-11, where it could only be tested through the rendered app. gpuBackendAutoUsable({servable, shaderF16}) answers the matching question about the BACKEND, whether the GPU may be selected for somebody who did not ask for it, and gates the performance probe: without it a machine with no shader-f16 would spend two timed runs proving its GPU is faster and be moved to WASM on the next load anyway. encoderQuantRows({backend, repoFiles, shaderF16, order}) turns all of that into the rows the sidebar renders, and the three questions get three different answers: a precision the BACKEND has no kernel for and one the SOURCE does not host are not rendered at all (they describe a different backend, or a different deployment, and a greyed row claiming otherwise reads as breakage), while a precision only this MACHINE cannot run keeps a greyed row with its reason, since the visitor's own adapter is the only thing that decided it. A null listing still means no opinion, so everything the backend offers is shown; a source that hosts nothing runnable falls back to greyed rows rather than an empty control. Imported by relative path (never the parakeet.js Vite alias) so it stays runnable under bare node, like cpuThreads.js. Unit-tested in test/unit/encoder-quants.test.mjs, including against a verbatim listing captured from the maintainer's own deployment. |
perfProbe.js |
Pure policy for the autoconfigure performance probe: whether a GPU is worth using ON THIS MACHINE, which no capability check can answer. Holds the probe geometry (768x1024, the encoder's own shapes, chosen because GPU per-node overhead swamps a smaller graph and understates the GPU), the decision rule pickBackendFromProbe (WebGPU only on a >= PROBE_MARGIN 2.0x win, because it costs 1.2-2.4 GB of weights against ~600 MB, and EVERY degenerate case resolves to wasm), verdictStillValid (a stored verdict expires on an app update, a changed GPU signature, a changed sourceQuantSignature of what the model source can serve, or 90 days, since driver updates move GPU speed silently; the source half is compared only when BOTH sides are known, so a deployment that publishes no listing keeps its answer instead of re-measuring every load), shouldAutoProbe (never over a hand-picked backend, once per machine, never re-entrant) and the arm watchdog bounds. App.jsx runs the arms through perf.worker.js. Unit-tested in test/unit/perf-probe.test.mjs. |
modelRepos.js |
Pure policy for the model-repo picker. parseModelRepos turns VITE_MODEL_REPO (one repo id, or several comma-separated) into an ordered list, dropping malformed entries rather than offering an option that only fails later as a 404; the FIRST entry is the default for a visitor who has never chosen. shortRepoLabel strips owner, shared model stem and -onnx so a narrow sidebar row reads optimized / UltiMed, keeping product casing. matchModelRepo resolves ?model=ultimed in tiers from exact id down to substring, returning null on an AMBIGUOUS tier instead of guessing (a coin-flip model still transcribes fluently, so a wrong guess is invisible). The URL's choice is resolved at App.jsx MODULE scope rather than inside the settings restore, because that restore has four paths that boot without reading a saved value (?reset, a version mismatch, the watchdog, the catch) and a FIRST-TIME visitor takes the version-mismatch one, i.e. exactly the person a shared ?model= link is for. Unit-tested in test/unit/model-repos.test.mjs, with the precedence itself covered end to end in test/e2e/model-picker.spec.js. Also holds diarizationFileName, which reduces a VITE_DIARIZATION_*_FILE to the bare filename it is documented to be: that value is joined UNDER its repo, so a full path written there was appended whole and could only 404, and because a mirror answers an unknown path with the SPA page the miss read as "this mirror lacks the diarization models" and sent the download back to HuggingFace, silently killing the speakers feature on exactly the networks that motivate self-hosting. |
browserFamily.js |
Pure engine-family detection behind the slow-browser warning popup: the WASM engine is ~9x slower on Firefox than on any Chromium browser on the same machine (SpiderMonkey SIMD codegen, not fixable app-side), so App.jsx shows a dismissable, NEVER-persisted popup recommending Brave/Chrome/Edge on every non-Chromium load. isChromiumFamily trusts userAgentData.brands containing "Chromium" when present (every Chromium derivative ships it; Firefox/Safari implement no userAgentData), falls back to the Chrome/NN UA marker, and resolves anything unknowable (no navigator, empty UA, hostile getters) to true so the popup can never nag spuriously. Unit-tested in test/unit/browser-family.test.mjs, popup behaviour end to end in test/e2e/slow-browser-popup.spec.js. |
deviceClass.js |
Pure phone/tablet detection behind the "made for a computer" warning popup: the app downloads hundreds of MB of weights and runs the model on the device itself, which is a desktop workload, and a backgrounded mobile tab is suspended mid-transcription. isHandheldDevice trusts userAgentData.mobile === true when present, otherwise matches phone/tablet UA tokens (an Android tablet reports mobile:false, so a false hint must not short-circuit) and catches iPadOS 13+, which claims a Macintosh UA and is only betrayed by maxTouchPoints. Anything unknowable resolves to false (stay quiet when unsure, same rule as browserFamily.js). App.jsx uses it for a dismissable, NEVER-persisted popup on load, and to gate a second popup explaining that Phone Mic pairs a phone with a COMPUTER that has no microphone. |
numberWords.js |
Pure spelled-out-cardinals-to-digits conversion (inverse text normalization) for English and French, behind the ON-by-default "numbers as digits" setting: numberWordsToDigits rewrites a plain string, numberWordsToDigitsInWords does the same over the decoder's word-timestamp array (a number spelled over several words collapses into ONE word spanning the run, so the plain and speaker views cannot disagree); both share the findNumberSpans core. Decimals are supported with a dot in both languages ("one point five" -> 1.5, "un virgule trente" -> 1.30); the separator is only read as one BETWEEN two numbers, so "the point is" and a dictated punctuation "virgule" are untouched, and the fraction is read group by group so "deux virgule zéro cinq" is 2.05. Conservative by construction: a run only converts if it spells ONE grammatical number ("two two" stays "2 2"), and words that are numbers only sometimes (English "one", French "un"/"une") never convert alone (a decimal counts as being part of a number, so "un virgule trente" does convert). French compounds ("quatre-vingt-dix-sept", "soixante et onze", hyphenated or spaced) are handled explicitly. Known limitation, documented in the tooltip: a spoken year ("nineteen eighty-four") is two numbers that cannot combine, so it converts as "19 84". App.jsx applies it once, after the benchmark early-return so measurement sees raw model output, and to the live preview so the text does not respell itself. |
medMode.js |
Pure policy behind "Mode Dictée Médical", the one-shot preset that turns the generic transcriber into a French medical dictation station: the UltiMed model, the french_medical.txt phrase list at its default tuning, chunking on at 30 s, the dictation display, auto-copy to the clipboard (the ONE default it flips on rather than restores: dictate-then-paste is the whole workflow, and it is worth the clipboard-exposure trade here), int8 on WASM / fp16 on WebGPU (fp16 since 2026-09-11, matching the app-wide defaults: it was fp32 because fp16 needs BOTH a shader-f16 adapter and a source hosting the fp16 file and missing either one used to fail the load over to WASM and PERSIST that flip, but the adapter half now degrades to fp32 before hub.js is asked and the source half clears its own gpuWeightsUnservableSig, leaving only the reason to prefer fp16 here more than anywhere else, a station that reloads all day on a locked-down network). Writing BOTH precisions is the point rather than a detail: the link is a setup instruction for the whole station, so a heavy precision picked by hand months ago is RESET by it, which is the invisible half of that rule (every other preset value looks applied while the station keeps fetching 2.35 GB), and a French UI. MED_MODE_PRESET is the single table; medModeRequested/isMedModeValue/foldModeValue decide whether a ?mode= value asks for it (accepting med/medecin/medical/doc/doctor/ultimed, case- AND accent-insensitively, since the param exists for links people type from memory). It lives here, and both entry points go through ONE applier in App.jsx, because the two would otherwise drift into slightly different stations: the ?mode= link (applied on page load, and STICKY unlike ?model= — a "medical mode" link is a setup instruction, not a one-visit override) and the sidebar button pinned above the collapsible groups (the preset rewrites settings across four of them, so it belongs in none). An unrecognised ?mode= means "no opinion" and changes nothing, the same contract as matchModelRepo returning null. App.jsx also fires the autoconfigure probe on PAGE LOAD in this mode instead of at the Load-model click, so the backend is settled before the clinician touches anything. It passes userPickedBackend: false into shouldAutoProbe on purpose, and ignores the stored backendUserPicked flag: that refusal is right for an ordinary page load, where an unrequested measurement must not overrule a deliberate choice, and wrong here, because ?mode=med IS the request and the same preset already overwrites the model, the window, the display, the language and BOTH precisions, every one of which the visitor may equally have set by hand. Reusing the gate unchanged is what made the link apply six of its seven promises on a real station: backendUserPicked persists the moment anyone touches the backend radios, so on any machine that had ever been fiddled with, the link's measurement was suppressed forever while the sidebar button (which calls runPerfProbe directly, past every gate) still measured every time. The stored-verdict branch is exempted for the same reason. shouldAutoProbe itself is unchanged: the gate is correct, it was being asked the wrong question. Missing pieces degrade rather than fail: an instance offering no UltiMed repo, or serving no french_medical.txt, keeps that one piece and warns. Unit-tested in test/unit/med-mode.test.mjs, end to end in test/e2e/med-mode.spec.js. |
hubReachability.js |
Pure policy behind the background HuggingFace preflight. probeHubReachable asks the HF API (/api/models/<repo>) with a HEAD in mode: 'cors' and its own 4 s abort, and answers a single bit: did this machine reach HuggingFace. The CORS mode is not a style choice and must not be "simplified" to the cheaper opaque no-cors fetch of a static asset: this app ships Cross-Origin-Embedder-Policy: require-corp (it needs SharedArrayBuffer for WASM threads), under which a cross-origin no-cors response is blocked unless it carries Cross-Origin-Resource-Policy, and huggingface.co sends no CORP on its static assets, so that probe reports EVERY visitor's network as blocked; a CORS response that passes the CORS check satisfies COEP, and the HF API echoes the Origin. preferLocalFirst then decides whether a load goes local-first: only a definite negative counts, so an unanswered probe behaves exactly as before. It deliberately does NOT also require a verified /models mirror, and that condition was REMOVED rather than never written: it gated the whole feature behind a mirror check that is easy to miss (a mount serving another repo, a layout the probe cannot attribute), which is how a deployment on a network that blocks HF still spent every load talking to HF. The reasoning it rested on does not hold either, since once HF is unreachable the HF attempt cannot succeed and there is no slow success left to trade away. What makes it safe is that the reorder is REVERSIBLE: App.jsx retries HF when the local-first attempt fails with a HubDownloadError (hubRetryTried), mirroring the HF -> local retry that already existed, so a false negative costs one fast same-origin miss. lib/diarizationModels.js takes the same answer through its own localFirst option, because those two models live in their own repos with their own HF-first order and otherwise went on paying a connect timeout each after the ASR load had learned better. App.jsx runs it from a mount effect keyed on [modelSource, repoId] and stores the answer in localFirstRef, which loadModel reads as its useLocalFallback default (a ref, not state, because that default is evaluated against a closure). Unit-tested in test/unit/hub-reachability.test.mjs. |
loadedModel.js |
Pure policy behind the sidebar's "Currently loaded" row: describeLoadedModel builds the one-line summary (backend, the precision that really mounted, the source, and the repo ONLY when it is not the selected one) and loadedModelDiverges decides whether to call the divergence out. It exists because every other model control in the panel shows a REQUEST while the app is allowed to resolve one differently: the WASM int8 pin and the GPU->WASM fallback change the precision, the /models quant upgrade and the reachability preflight change the source, and a picker change outlives the loaded model until a reload. None of that was reported anywhere, so a station could sit on a "WebGPU / fp32" selection while an int8 CPU model did the work, with neither the UI nor the transcript saying so. Two deliberate refusals: the SOURCE is never treated as a divergence (weights from the local mirror disagree with nothing the visitor picked, and flagging it on every offline-capable deployment teaches people to ignore the flag), and a load that could not report its precision claims nothing rather than claiming a mismatch. The comparison is against the EFFECTIVE selected precision (what the radios show, effectiveEncoderQuant in lib/encoderQuants.js, resolved once at component scope so the row and the radios share one definition of the rules), not the raw stored one, or every fp16 pick on a GPU without shader-f16 would report a divergence the visitor cannot see. App.jsx fills it from getParakeetModel's quantisation/servedFrom/resolvedBackend at the end of a successful load and clears it on every dispose, and it is deliberately NOT persisted (a value restored from disk would assert something about a model that is not loaded). reconcileSelection closes the same loop from the other side: a fallback has to move the CONTROLS, not just the console, and the backend half already did (the GPU->WASM fallback flips it through applyBackend before retrying, because the retry reads the new value) while the precision stayed on the request forever, the radios showing a display-only effective value over a stored setting that said something else. It writes the loaded precision back, under two limits that keep it from fighting the visitor: only the precision of the backend that was actually loaded AND only while that is still the selected one (moving the backend radio arms its own reload, so a finishing load describes a configuration they have left), and never the repo (a picker change outlives the loaded model on purpose; reconciling it would silently cancel the switch just requested, which is what the row reports instead). In practice it fires only on WebGPU, where a precision the machine or source cannot honour resolves to fp32, so it is tier-1 territory by construction: the headless tier is WASM-only, and on WASM a request now either loads or throws. Unit-tested in test/unit/loaded-model.test.mjs; the row is asserted end to end by test/e2e/gpu-quant-fallback.spec.js. |
asset-integrity.js |
Verify-then-load for loose runtime assets that bypass the HTML SRI chain (the PCM worklet, the sherpa-onnx diarization glue/wrapper/wasm). Hashes bytes against the build-time pin before AudioWorklet.addModule (verifiedAddModule) or returns the verified bytes/blob (fetchVerifiedAsset). verifiedAddModule is a thin blob-URL handoff ON TOP of fetchVerifiedAsset (it used to inline its own copy of the manifest lookup, hard-fail branch, fetch, hash, compare and throw, so the worklet's pin and the sherpa engine's pin could drift apart), and the digest, the production hard-fail flag and the IntegrityError tag come from app/src/integrity.js, shared with backend.js's ORT-runtime pinning. |
diarizer.js |
Main-thread CLIENT for speaker diarization: fetches + sha384-verifies the vendored sherpa-onnx engine bytes (glue/wrapper/wasm) and brokers them to diarizer.worker.js, which runs the heavy synchronous WASM process() OFF the main thread so it never freezes the UI. Sends the ~34 MB model bytes only when they change (a count-change re-run reuses the worker's cached diarizer). Exposes runDiarization(pcm16k, opts) (segments) and cancelDiarization() (hard-terminates the worker mid-run; the pending run rejects with cancelled). One run at a time per client, enforced synchronously: pending is a single slot, so a second concurrent run() used to overwrite it and orphan the first caller's promise forever (nothing ever settled it). The piecewise pool is safe because its clientLoop awaits, but the default client behind runDiarization() is shared by every caller of it. Init goes through the shared workerReady() handshake (so a hung init hits the same watchdog the model workers get), and the worker error handler is PERSISTENT rather than scoped to init: a worker that dies afterwards (uncaught throw, WASM OOM, pthread failure) posts no message at all, and the run's single pending slot used to stay unsettled for the life of the tab. Unit-tested in test/unit/diarizer-client.test.mjs. |
diarizer.worker.js |
Classic Web Worker that runs the sherpa-onnx diarization engine. Receives the already-verified engine + model bytes from diarizer.js, importScripts-loads the glue/wrapper from blob: URLs, feeds the wasm via Module.wasmBinary (pthread sub-workers spawn from the verified glue blob via mainScriptUrlOrBlob), caches the built diarizer by model identity, and runs process() here so the page stays responsive (spinner animates, run is cancellable). Integrity verification stays on the main thread; the worker only evaluates verified bytes. |
diarizationModels.js |
Downloads the two diarization models (pyannote segmentation + CAM++ embedding) through the same hub as the ASR model (HF first, local /models fallback, IndexedDB-cached, memoised). Exports getDiarizationModels() and diarizationModelProtectKeys() (the cache keys the orphan sweep must keep). Repo/file defaults come from the VITE_DIARIZATION_* config. Takes a localFirst flag (the answer from hubReachability.js, threaded down from App.jsx) that swaps the order to local-then-HuggingFace, still falling back to the hub if the mirror cannot serve the file: these two models live in their own repos with their own HF-first order, so without it a network that blocks HuggingFace paid a connect timeout PER MODEL here even after the ASR load had already learned better. |
speakerAssign.js |
Pure helpers mapping diarization output onto the transcript: assignSpeakersToWords (each word gets the max-overlap speaker, gaps go to the nearest), groupWordsIntoTurns (consecutive same-speaker words -> turns), resolveSpeakerRoot + canonicalizeTurns (apply user speaker-merges via union-find and renumber to gap-free display positions, so renaming a speaker into another merges their colour/label and the diarizer's non-contiguous indices never leave a gap), speakerCount, and turnsToLabeledText (turns -> Name: text blocks for copy/export, via a nameFor(speaker, position) resolver so renamed speakers and the gap-free default ordinal names come through). Unit-tested in test/unit/speaker-assign.test.mjs. |
speakerEmbedding.js |
Computes one CAM++ voice embedding per diarized speaker in-browser (the sherpa-onnx engine exposes no embedding API): gathers each speaker's segment audio from the in-memory PCM, runs the shared app/src/fbank.js front-end, and feeds the same CAM++ model diarizationModels.js already downloaded through the app's onnxruntime-web (x=[1,T,80] -> embedding=[1,192]). embedSpeakers(pcm16k, segments, embeddingBytes) -> { speakerIndex -> Float32Array(192) }. Embeddings stay in memory only (voiceprints are biometric, never persisted); quality is validated by scripts/speaker-embedding-check.mjs. |
speakerMatch.js |
Pure cross-recording speaker-matching logic (session-only): cosineSimilarity, buildProfiles (group the session's embeddings by user-assigned name into per-name centroids, derived not accumulated so a rename stays consistent), matchProfile (best centroid above a cosine threshold), and autoNameSpeakers (label a recording's unnamed speakers from the OTHER recordings' profiles, never overwriting a user name). Lets a speaker named in one recording be auto-labelled in a later one when the voice matches. DEFAULT_MATCH_THRESHOLD = 0.5. Unit-tested in test/unit/speaker-match.test.mjs. |
diarizePiecewise.js |
Parallel piecewise diarization for long clips. shouldPiecewise(durationSec, numSpeakers) gates it (only above PIECEWISE_MIN_SEC 900 s AND in auto-detect mode). planPieceRanges cuts silence-aligned pieces via planChunks/createEnergySampler (app/src/parakeet.js). runPiecewiseDiarization dispatches pieces across a pool of createDiarizerClient() workers (least-outstanding, so a slow piece never blocks others), embeds each piece with embedSpeakers after all pieces resolve, then reconcilePieces folds per-piece speaker labels into one global space using cosineSimilarity centroids from speakerMatch.js (DEFAULT_MATCH_THRESHOLD), erring toward over-splitting (the UI can merge but not un-merge), and stitches + seam-merges the timeline. App.jsx diarizeEntry uses it (composed after silenceCut.js) and falls back to a single full run on any non-cancel failure. Unit-tested in test/unit/diarize-piecewise.test.mjs. |
silenceCut.js |
Pure silence-excision helpers for the diarization pipeline: findSilenceCuts(pcm, sampleRate) (dense per-hop energy via createEnergySampler.hopProfile in app/src/parakeet.js, adaptive threshold ceilinged below the speech level, returns sample runs of silence >= minSilenceSec keeping a padSec margin), excisePcm(pcm, cuts, sampleRate) (condensed PCM + a kept-span offset map, with a short anti-click fade at each splice), and remapSegments(segments, map, sampleRate) (condensed-timeline diarizer segments back to the original timeline, SPLITTING any segment that bridges an excised gap). App.jsx diarizeEntry uses these so the diarizer sees a shorter clip on long recordings while everything downstream still gets original-timeline seconds. Unit-tested in test/unit/silence-cut.test.mjs. |
writeQueue.js |
Pure createSerialQueue(): a tiny serial task queue so a burst of async writes runs strictly in enqueue order (a later task starts only after the previous settles, and a rejection does not wedge the chain). App.jsx routes ALL transcripts-DB mutations (save / wipe-and-rewrite / forget) through one instance so back-to-back saves (diarize then rename) cannot race as independent IndexedDB transactions and leave stale data on disk. Unit-tested in test/unit/write-queue.test.mjs. |
remote-crypto.js |
The E2E crypto for the remote mic: ECDH (P-256) key exchange -> HKDF -> AES-GCM, all via Web Crypto. |
remote-webrtc.js |
RemoteMicRTC: WebRTC peer-connection lifecycle, signaling, and the data channel that carries encrypted PCM. Includes the HTTPS-relay fallback for UDP-blocked networks. |
remote-relay-transport.js |
The two HTTPS relay transports (WebSocket + long-poll) used as last resort when WebRTC cannot connect. Same ciphertext frames, same interface as the data channel. Each exposes a drain() (buffered-amount / queue-depth) so the saved-file pump can pace itself to the link. |
remote-mic-handshake.js |
Shared handshake logic used by both desktop and phone, so both sides hash the public keys in the same byte order for the fingerprint compare. |
remote-mic-link.js |
Pure parser/validator for a scanned remote-mic QR payload (parseRemoteMicLink). Accepts only a same-origin /remote-mic.html#roomId:secret link, the trust boundary for the in-page camera re-scan. Unit-tested in test/unit/remote-mic-link.test.mjs. |
persistStorage.js |
Asks the browser to promote this origin's IndexedDB to the "persistent" bucket so Chromium does not evict the multi-GB model cache under disk pressure (which looked like "the version bump wiped my model"). Idempotent, called on every load. |
| File | Role |
|---|---|
favicon.svg |
Favicon. |
pcm-recorder-worklet.js |
AudioWorklet processor that captures raw PCM (bypassing MediaRecorder's Opus priming delay). Integrity-checked at load by asset-integrity.js. |
js/eruda-loader.js |
Opt-in (?debug=1) loader for the vendored eruda mobile devtools; externalised so the CSP can stay strict. |
js/eruda.min.js |
Vendored eruda devtools bundle. |
js/qrcode.min.js |
Vendored QR-code generator (renders the phone-pairing QR). |
js/jsqr.min.js |
Vendored QR-code scanner (jsQR 1.4.0, Apache-2.0). Loaded lazily behind an SRI pin by the phone page's in-page camera re-scan, so a dropped phone can re-pair by scanning the desktop's QR without leaving the page. |
benchmark/jfk.mp3 |
The 11 s clip the sidebar Benchmark section transcribes on every backend/precision it measures (lib/benchmark.js). A JFK inaugural-address excerpt: a US Government work, public domain, the same source as test/fixtures/jfk.mp3 (kept as a separate copy because one is a shipped app asset and the other a test golden). Its known transcript is what the report's similarity score is measured against, which is how a backend that returns silence is caught. |
probe/probe-encoder.{fp32,int8}.onnx |
The two ~5 MB graphs the autoconfigure probe times (fp32 for the GPU arm, int8 for the WASM arm, matching what each backend really loads: timing both in fp32 would hand the GPU the 2-3x int8 buys the CPU). Prefetched at idle only on a machine that has an adapter, so there is something to decide. Generated reproducibly by scripts/make-probe-model.py. |
tokenizer/bpe-merges.json |
Distilled BPE merges + added-token list for the phrase-boost encoder (loaded lazily only when boosting is on). |
tokenizer/SOURCE.md |
Provenance + refresh recipe for that asset. |
ort/* |
Mirror of the ONNX Runtime Web WASM/MJS runtime files, served same-origin and integrity-verified via the manifest. |
ffmpeg/ffmpeg-core.{js,wasm} |
The @ffmpeg/core single-thread emscripten build (glue + ~31 MB wasm), served same-origin (mirrored like ort/*) so the upload decoder loads it under the strict CSP + COEP require-corp with no CDN. Lazy-fetched by lib/audioDecode.js. Provenance in vendor/ffmpeg/SOURCE.md. |
sherpa-onnx/sherpa-onnx-wasm-main-speaker-diarization.wasm |
The sherpa-onnx diarization engine's WebAssembly binary (bundles its own ONNX Runtime). Loaded and integrity-verified by diarizer.js. |
sherpa-onnx/sherpa-onnx-wasm-main-speaker-diarization.js |
Emscripten glue for that wasm, with the baked-in .data model loader stripped out (the app loads its own models instead). Injected as a classic blob-URL script. |
sherpa-onnx/sherpa-onnx-speaker-diarization.js |
sherpa-onnx's small JS API wrapper (verbatim upstream), defines OfflineSpeakerDiarization / createOfflineSpeakerDiarization over the emscripten module. |
Locally vendored npm packages, served same-origin instead of from a CDN. Each
has a SOURCE.md recording the pinned version and tarball hash. Refreshed via
scripts/update-vendored.sh (except dictation_support, which is upstream
git-only). Not documented file-by-file here:
preact/— the UI framework (with thecompatReact shim).onnxruntime-web/— the ONNX Runtime Web distribution (the inference runtime).dictation_support/— SpeechMike / dictation-device support (GoogleChromeLabs/dictation_support).sherpa-onnx-diarization/— provenance only (SOURCE.md+LICENSE) for the prebuilt sherpa-onnx speaker-diarization WASM artifacts; the runtime files themselves live underpublic/sherpa-onnx/(above) because they are integrity-pinned and served same-origin. Not refreshed byupdate-vendored.sh; refresh procedure is in itsSOURCE.md.ffmpeg/— the@ffmpeg/ffmpegESM wrapper (aliased invite.config.js) used bylib/audioDecode.jsfor the in-browser upload decode, plus aSOURCE.mddocumenting both the wrapper and the@ffmpeg/coreemscripten build (glue + wasm) that ships underpublic/ffmpeg/(above). Not refreshed byupdate-vendored.sh; refresh procedure is in itsSOURCE.md.
| File | Role |
|---|---|
server.js |
Express server (~1.3k lines): room management, SDP offer/answer relay, ICE trickle, and time-limited TURN credential generation. Brokers the handshake only; it never sees plaintext audio (everything is E2E-encrypted by remote-crypto.js). Also hosts POST /api/benchmark-report, the receiver for the sidebar Benchmark section's anonymised reports: disabled (503) unless the operator points BENCHMARK_REPORTS_DIR at a writable folder, format-checked, size-capped (32 KB), file-count-capped, stored one report per server-named JSON file (no request byte ever reaches a path, and immutable files are what makes the operator's add-only two-way sync in benchmark_reports/sync.sh safe), and deliberately storing nothing about the sender. Runs as a sidecar inside the Docker image. |
package.json / package-lock.json |
Server dependencies (Express). |
| File | Role |
|---|---|
Dockerfile |
Multi-stage build (Node builder -> Caddy runtime). Base images pinned to immutable digests; optional npm audit build gate. |
Caddyfile |
Production reverse proxy: serves the built bundle, sets the COOP/COEP/security headers, and proxies /api/signal/* to the Node sidecar. Both /srv handlers serve through file_server { precompressed br } and the /models mirror through file_server { precompressed zstd }, so a <file>.br / <file>.zst sidecar from scripts/precompress.mjs is sent with the matching Content-Encoding and the browser decodes it natively; no sidecar (or no support for the encoding) falls back to encode's on-the-fly compression exactly as before. Inside the /models handler two mutually exclusive handle blocks split the mirror manifest (model-manifest.json, served from the /var/model-manifests tmpfs the entrypoint writes, matched for both the flat and the <owner>/<repo> shapes) from the weights themselves, so the manifest cannot fall through to the read-only mount it is deliberately not written into. |
docker-compose.yml |
One-command deployment; wires env vars and the bind-mounted fallback-model folder. |
entrypoint.sh |
Container boot: verifies the fallback model (VITE_MODEL_REPO may be a comma-separated LIST feeding the sidebar picker: every entry is validated, not just the first, and with several configured the mount is NOT collapsed by descending into one repo, since that would hide the rest and make each picker entry load the descended repo's weights under its own name; each repo is then reported present/absent, absent being a warning that falls back to HuggingFace), populates dictation regex, generates config.js (runtime VITE_* -> window.__CONFIG__), runs the boost prebuild and then REWRITES manifest.txt from what is on disk so the prebuilt <name>.json files it just produced are listed alongside the .txt sources (the app treats that manifest as authoritative and only asks for a .json the manifest names; before this, every visitor's browser fired a doomed request for a prebuilt file that only exists when the container had a vocab to compile against, and a 404 in the console is exactly the kind of noise that makes a real error invisible), starts the signaling sidecar, then execs Caddy. Also runs scripts/precompress.mjs --models --check over the mounted model dir, which reports any .zst sidecar older than (or orphaned from) its source file, since Caddy would serve that stale copy to every zstd-capable visitor. Report-only by default because the container runs unprivileged and the model dir is usually mounted read-only; PRECOMPRESS_MODELS=1 switches it to actually generate them at boot (needs a writable mount). Also runs scripts/model-manifest.mjs over the mount at every boot, writing each repo's model-manifest.json into the /var/model-manifests tmpfs (the mount is read-only, and Caddy serves it back under /models/) so the app reads a listing instead of HEAD-probing for paths it can only guess at. Regenerated from what is on disk each start, so it cannot go stale; a failure is a warning, never a refused boot, since a mount with no manifest is probed exactly as before. |
prebuild-boost.mjs |
Boot-time phrase-boost prebuild: when the operator ships boost lists and the vocab is on disk, encodes each list to token ids once (via app/src/boostCompile.js) so visitors' browsers skip the BPE work. |
env.example |
Documented template for docker/.env (all operator-settable knobs). |
| File | Role |
|---|---|
transcribe.mjs |
CLI transcription harness: runs the real engine modules under Node (ORT WASM) to reproduce the browser transcript from the terminal. Also produces the E2E golden transcript. --quant sets the encoder quant; --decoder-quant (default fp32) picks the fused decoder_joint quant independently, so the heavy encoder can stay int8 while the small decoder runs full precision. resolveFiles asks for the canonical name per quant (mirrors hub.js), with one legacy alias kept for the int8 encoder. Prints the resolved encoder/decoder basenames ([transcribe] files:) so a timing quoted from this harness says which files produced it. |
compile-boost.mjs |
Compiles a boost .txt into a .pwc artifact so the container skips re-encoding on boot (operator-run counterpart of prebuild-boost.mjs). |
distill-bpe-merges.py |
Distills the small bpe-merges.json asset from the upstream tokenizer.json. |
gen-bpe-fixture.py |
Emits the BPE cross-check fixture (ground-truth ids from real HuggingFace tokenizers) consumed by the unit tests. |
make-probe-model.py |
Generates the two committed autoconfigure-probe graphs in app/ui/public/probe/ (uv run, self-contained deps in the shebang). Emits a chain of SiLU + LayerNorm blocks at the real encoder's shapes (768x1024) sharing one weight set, then an int8 dynamic quantisation of it, so each arm times what its backend actually runs. Deterministic: same seed and defaults reproduce the committed bytes. Its header records the calibration that fixed the geometry (a smaller 256x512 graph read 1.5-2.8x on a box whose true gap is 5.4x, because GPU per-node overhead swamps small GEMMs). Run it only to regenerate the artifacts. |
gen-fleurs-fixtures.mjs |
One-time local tool that builds the FLEURS regression fixtures (test/fixtures/fleurs/): samples en+fr validation clips, transcodes them to mp3, transcribes each with the int8 pipeline (reusing transcribe.mjs), keeps the ones the model reproduces well, stitches them into one long clip, and writes manifest.json with both the human reference and the model golden. --decoder-quant (default fp32) sets the decoder_joint quant; warns when it is not int8, since the e2e app decodes int8. |
gen-jfk-moon-fixtures.mjs |
One-time local tool that builds the long-audio chunking fixture: downloads the public-domain JFK "We choose to go to the Moon" speech (Internet Archive) into the gitignored cache, crops the first 3 min to test/fixtures/jfk-moon-3min.mp3, and transcribes it with the int8 pipeline for the golden. --decoder-quant (default fp32) sets the decoder_joint quant; warns when it is not int8, since the e2e app decodes int8. Exports the download/transcode helpers (and a full-speech clip in the cache) reused by webgpu-check.mjs. |
gen-medical-val-sets.mjs |
Builds the French-medical validation sets under benchmark_datasets/french_medical/ (gitignored) by sampling the UltiMed-ASR-FR corpus: a seeded draw of N clips per subset (dictionary/drugs/PARHAF from their val split, PARROT from test since that subset ships eval-only), QC failures dropped and one clip per group_id so no single term dominates. Copies the FLACs and writes one NeMo manifest per subset with audio_filepath relative to the output dir, plus README.md/sample.json provenance. Kept as SEPARATE manifests on purpose: grid_search_benchmark.mjs takes --manifest repeatedly and breaks every grid cell down per dataset, which is what shows whether a knob tuned on one medical domain costs accuracy on another. The audio is not committed; the script + recorded seed is what makes the set reproducible. |
wer-bench.mjs |
WER bench that drives the repo's OWN JS pipeline (transcribe.mjs + the chunked TDT decode) to A/B encoder quantisations across chunk windows. Built to confirm fp16 holds long chunks where the stock int8 dropped content; runs on native onnxruntime-node (--ort node) so fp16/fp32 load. fp16 is no longer shipped to browsers (withdrawn 2026-08-23) but stays supported here, since native ORT has fp16 CPU kernels and this is the only way to score a regenerated fp16 build. The --configs quant is the encoder quant; --decoder-quant (default fp32) sets the fused decoder_joint quant for every config. Appends each run to bench_wer.md. |
wer-quants.py |
Small Python WER+timing+RAM bench across int8/fp16/fp32, built on the UPSTREAM onnx-asr library (the lib this app is a port of) rather than the JS pipeline. Self-contained uv run script. Used to validate the SmoothQuant int8 encoder. --quants sweeps the encoder quant; --decoder-quant (default fp32) holds the fused decoder_joint at a fixed precision by swapping only its InferenceSession (resolved via onnx-asr's own resolver, so no second encoder loads and the RAM figure stays honest); the oracle reference stays matched at --reference-quant. --audio takes a single file OR a folder (e.g. the model repo's calibration_audio/ speeches): a folder is analysed file-by-file and capped by a final cross-file overall-WER summary. Runs on CPU by default; --cuda re-launches once under onnxruntime-gpu via uv (with the local CUDA-12/cuDNN-9 wheel libs on LD_LIBRARY_PATH) to run on an NVIDIA GPU. --manifest mode: instead of the long-pass/oracle analysis, score whole FLEURS-style validation splits (<lang>/validation.json + wavs_validation/) against their HUMAN labels as one corpus WER per quant; references/hypotheses are normalised (case+punctuation folded, accents kept; --no-normalize for raw WER). --manifest is REPEATABLE: pass it once per language and every language is scored in a SINGLE model load (a tqdm bar per language on stderr). Each language emits a __WER_JSON__ line tagged with --run-label so a driver can build a model x language matrix. (The gitignored fallback_models/Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx/wer-fleurs-validation.sh driver evaluates a roster of models -- istupakov fp32/int8, this repo's int8, and the models_in_testing/ candidates -- each loaded once over all languages, with a pre-flight that skips unloadable model dirs, and prints the matrix + per-model MICRO/MACRO.) |
test_wer-quants.py |
Self-contained uv run unit tests (T1-T8) for the model-free helpers of wer-quants.py's --manifest mode: normalize_for_wer (case/punctuation folding, accents kept), load_manifest (basename wav resolution, missing/limit/blank-line handling, explicit audio dir), and corpus_wer (aggregate not per-clip mean, empty-reference drop, normalise toggle). No model/onnxruntime needed; main() runs every test sequentially. |
quantize-nbits.py |
Builds a quantised encoder from an fp32 one at either width: --bits 4 gives the w4a8 encoder (~383 MB), --bits 8 gives this repo's int8 one (~650 MB). Block-wise weights (--block-size, default 32, symmetric) through ORT's MatMulNBits quantizer, with accuracy_level=4 so the kernel also quantises activations to int8 at run time. Converts the 265 MatMuls that have a constant weight; the other 72 multiply two activations and are left alone, as are the remaining convolutions and LayerNorms. That count was 217 until the model repo's optimize-encoder-graph.py pointwise pass began rewriting the 48 pointwise Conv1d into MatMul first, which is what moved 288.0 MiB of weights out of fp32 Conv nodes and into reach of this script. One fp32 scale is stored per block, so --block-size 64 halves the scale payload (72.5 MiB to 38.8 MiB at 8 bits) for identical packed weights; the shipped int8 encoder uses 64, w4a8 still uses 32. No calibration data at either width, so it runs in seconds to minutes on CPU. That is what separates this int8 from the static QDQ int8 upstream ships: there the activation ranges are frozen by a calibration campaign, here they are recomputed per run inside the kernel. Tries the modern matmul_nbits_quantizer entry point and falls back to the older matmul_4bits_quantizer one, since the class moved between ORT releases (the fallback is 4-bit only, so --bits 8 errors there rather than silently producing int4). Saves through external data, then collapses the result to one self-contained file since both widths fit under protobuf's 2 GB message cap, which is what the app's filename probe expects. |
check-nbits.py |
Acceptance gates for a quantize-nbits.py output at either width (--bits says which the graph should carry), run before promoting one. Checks the MatMulNBits node attributes are what was asked for, that the web-export invariants survived the rewrite (the padded-batch NaN tripwire still fires, batch-1 at len = T-1 is finite, an equal-length batch of 2 is finite), and that the encoder output still correlates with the fp32 encoder's above 0.90. The correlation gate feeds it REAL mel features on purpose: white noise scores 0.71 through an unmodified model, so a noise-driven gate measures the input, not the quantisation. Correlation is a smoke check either way, not the acceptance criterion; WER on the medical sets is. |
benchmark-throughput.mjs |
Drives the app's own sidebar Benchmark section from the CLI against an arbitrary --model-dir, so backends and precisions can be compared in one command through the exact paths a user gets. --combos wasm:int8,webgpu-hybrid:w4a8 picks the rows, --repeats N medians N timed runs (use at least 3 on WebGPU: a single run pays shader compilation and reads several times slower than steady state), --long swaps the 11 s clip for a 90 s tiled profile. A webgpu row implies a headed Chromium, since headless has no adapter. Re-reads the plan after a reload up to 3 times before declaring a row unavailable, because the adapter query behind the plan intermittently comes back empty on a first boot even on a box whose adapter probes fine. |
grid_search_benchmark.mjs |
Grid-search WER bench over NeMo jsonl manifest(s): reuses the production decode + phrase-boost trie unchanged and sweeps encoder-quant x decoder-quant x beam-width x boost-strength (--quant int8,fp16,fp32 benchmarks each encoder quant, the outer dimension, with its own model load + encoder cache), printing WER/Levenshtein per combination. --decoder-quants int8,fp16,fp32 (default fp32) sweeps the fused decoder_joint quant independently of the encoder quant, nested under each encoder quant so the cached encoder output is reused across the decoder sweep (no re-encode); the accuracy table gains a dec column. Sorts by CER by default; multi-dataset overall is size-weighted (micro-average). A manifest's RELATIVE audio_filepath resolves under --audio-root first and falls back to the manifest's own directory, which is what lets one run mix manifests living under DIFFERENT audio roots (the flag is global); --audio-root still wins when both hold the file. Besides the end-of-cell load5 point sample, each cell also reports load_avg/load_max, the mean and peak OS 1-min load sampled once per utterance DURING the cell, so a cell slowed by unrelated work on the box is identifiable afterwards (a spike that starts and ends mid-cell is invisible to load5). Both covered by test/unit/grid-search-audio-root-and-load.test.mjs. --ort is REQUIRED and has no default: wasm and node yield identical transcripts but very different timings, and only wasm is what the web app ships, so defaulting it would silently decide whether a run's proc_t/dur_t/dec_t/aud describe real user-facing decode cost or a native-only number no browser ever sees. Omitting it errors out with that tradeoff spelled out, and --ort wasm is rejected for fp16/fp32 (no fp16 CPU kernels, 2 GiB per weight file). --commitment-scaling 0,0.5,1 is a SWEPT axis (it used to be one value per whole run): like depth-scaling it is baked into the trie at build time, so each value costs one extra trie build, and the values appear side by side in the cscale column. A null (unswept) value appends nothing to a cell's resume key, so existing benchmark_results.jsonl files stay resumable; covered by test/unit/grid-search-commitment-scaling.test.mjs. --chunk-duration off,20,40 (plus --chunk-overlap/--chunk-snap/--chunk-energy-ms) sweeps the long-audio chunking itself: off is the whole-clip reference cell (pre-sweep resume keys unchanged), numeric windows run the real transcribeChunked seam path on raw PCM (bypassing the whole-clip encoder cache, so each chunked cell pays a full re-encode), the off cell collapses instead of multiplying with sub-knob lists, and unset sub-knobs are omitted from the decode opts so the engine defaults stay authoritative; covered by test/unit/grid-search-chunk-sweep.test.mjs. When the decoder runs with collectBeamStats, the table also reports two DISTINCT beam-search series that must not be conflated: the true beam occupancy (beam_med/beam_max, surviving hypotheses after merge+prune, from the kept series) and the joiner batch size (batch_med/batch_max, hypotheses due per frame, from the expansion series, the per-call CPU-cost driver behind the beam-on-CPU question in murmure#338, much smaller than the occupancy because TDT durations scatter the beam across frames) plus steps, and the 5-min system load (load5) at the end of each cell plus a per-run average, so a cell whose decode timing was inflated by an unrelated process is visible. Diagnostic knobs for the beam-vs-greedy study (murmure#338), each constant across a run: --merge-duplicates on|off (NeMo merge_duplicate_hypotheses log-sum-exp recombine vs Viterbi keep-best), --length-norm-prune on|off (rank the per-frame survival prune by length-normalized score, the candidate fix for the wide-beam deletion bias), --force-beam on|off (run the beam decoder even at width 1, to compare against the dedicated greedy loop), and --oracle-nbest N (each beam decode also returns its top-N distinct paths so the harness scores the best-achievable oracle WER/CER); the per-utterance records and summary additionally carry the NIST substitution/deletion/insertion split of the 1-best word edits (levenshteinCounts). |
speaker-embedding-check.mjs |
Validation spike for cross-recording speaker matching: computes CAM++ speaker embeddings (shared app/src/fbank.js + native onnxruntime-node) for several windows of test/fixtures/two-speakers.wav and prints pairwise cosine similarities, asserting same-speaker pairs sit clearly above cross-speaker pairs (PASS/FAIL exit). Proves the embedding front-end is faithful enough before any browser feature code; a faithful proxy for the browser ORT path. |
webgpu-check.mjs |
Manual WebGPU harness (NOT a test tier; run by hand on a real GPU box, or opt into it from the .githooks/pre-push prompt). The WebGPU analog of the wasm long-audio-chunking e2e, and the ONLY thing that exercises the real GPU path (encoder batching + the encode/decode worker pipeline), which CI and headless-CI cannot (no GPU). Reuses serve.mjs + seed.mjs on webgpu-hybrid. It runs the fp32 encoder (via shards), the WebGPU default since the fp16 build was withdrawn on 2026-08-23; fp32 needs no shader-f16 and so validates on GPUs whose Dawn build omits that feature. The GPU path also accepts w4a8 (MatMulNBits has a WebGPU kernel), but this harness deliberately stays on fp32: it is the precision a WebGPU visitor gets by default, and the one whose ~2.3 GB shard load is the fragile part worth guarding. --fp32 is accepted and ignored so the npm run webgpu:check:fp32 alias keeps working. It asserts the [Transcribe] animations paused marker (the WebGPU rendering-coupling guard: without it wall time blows up ~15x while content still passes), the [Decode] pipeline engaged marker and batch>=2. Reports which decoder BUILD ran (from parakeet.js's in-graph-outputs marker), since that is decided by what the mirror serves and a wall time with no build attached cannot be compared. --full (npm run webgpu:memcheck) runs the FULL ~17 min speech and watches JS heap (via CDP) for a leak. Fails on OOM/crash, silent WASM fallback, content miss, or unbounded heap growth; SKIPs (exit 2) when no real WebGPU GPU is present (rejects software/SwiftShader adapters). Defaults --channel chromium; accepts both --flag value and --flag=value. |
probe-check.mjs |
Manual real-GPU check of the autoconfigure probe (npm run probe:check; not a test tier). test/e2e/perf-probe.spec.js can only prove the probe stays out of the way, since headless CI has no GPU and its GPU arm can never win; this asserts the half that needs real hardware: both artifacts prefetched at idle, the probe running on the Load model click with animations paused, and the verdict reaching the app and IndexedDB without being recorded as a human choice (which would suppress every future probe). It also re-checks that the ?webgpu=0 kill switch still fetches and runs nothing. Prints the verdict either way and says so loudly when a real GPU is NOT picked, since that is the reading worth a second look. Reuses lib/browser-app.mjs; SKIPs (exit 2) without a real GPU, and also without adapter shader-f16, because fp16 is the only GPU precision the app ever selects unasked, so a box without it (this one) never auto-probes at all and there is nothing here to measure. This is the tool for measuring the probe on GPUs other than the one reference box. |
transcribe-browser.mjs |
Manual browser-driving transcription CLI (npm run transcribe:browser). Runs the BUILT app in a headed, WebGPU-enabled Chromium (Playwright) and automates it end to end, so it delivers the two things the pure-Node transcribe.mjs cannot: real WebGPU compute (the fp32 encoder via shards) AND speaker diarization (the in-browser sherpa-onnx engine + pyannote/CAM++ models), then writes the result as Markdown. Defaults to the high-quality recipe (webgpu-hybrid, fp32, beam 5, no boost, forced 2-speaker diarization). `--ortep jsep |
lib/browser-app.mjs |
Shared glue for Node harnesses that drive the built app in a real browser: spawnAppServer (spawns test/e2e/serve.mjs), launchWebGpuBrowser (Chromium with --enable-unsafe-webgpu; maps channel 'chromium' to undefined like webgpu-check.mjs so headless runs use the headless shell, because the full binary's blob-storage paging breaks multi-GB model loads with ERR_BLOB_REFERENCED_BLOB_BROKEN), bootApp (force local model source + seedSettings + reload; keeps ?webgpu=1 on the webgpu path as a no-op guard against the app-wide pin returning, since it used to coerce every seeded webgpu backend to WASM, and takes an ortep option that pins the ORT distribution for the run: it builds the query here rather than leaving it to a caller's goto because the reload() has to preserve it, ORT being chosen once per JS context), loadModelAndWaitReady (throws immediately on the app's Failed status instead of masking it as a timeout), probeRealWebGpu (reject software/SwiftShader adapters), waitForServer. Used by transcribe-browser.mjs; webgpu-check.mjs predates it and still inlines the equivalent glue (migratable later). |
lib/sample.mjs |
Seeded-sampling helpers shared by the dataset/fixture generators: mulberry32 (seedable PRNG) and shuffled (Fisher-Yates on a copy). Used by gen-fleurs-fixtures.mjs and gen-medical-val-sets.mjs so a given --seed reproduces the same draw forever, which is what makes "regenerate the set" a no-op when nothing changed. |
model-manifest.mjs |
Writes the file list a local model mirror uses to describe itself: model-manifest.json, a JSON array of repo-relative paths at each repo root, read back by hub.js listLocalRepoFiles. A folder behind a static file server cannot be listed, so without one the app HEAD-probes the paths modelLayout.js predicts, which finds every layout anyone wrote down and no others: the optimized repo keeps a complete second model under istupakov_smoothquant/ and moved the lite int8 encoder into it, which the app resolves fine from the HuggingFace listing and no probe can ever name locally. Derived from the filesystem every time, never hand-edited, so it cannot go stale: scripts/fetch-e2e-models.mjs writes it after downloading and docker/entrypoint.sh runs this file directly at every boot (hence the CLI at the bottom and the node-builtins-only rule). writeMirrorManifests writes one per repo on a shared mount and only falls back to the mirror root for the flat single-repo layout, using the same vocab.txt marker the entrypoint uses; a separate output directory covers the container's read-only model mount. Symlinks are followed (a maintainer's mirror is symlinks into the model repo), dangling ones and .git-shaped bulk directories are left out, and an unwritable destination warns rather than throwing, since a mirror with no manifest only pays the probing it paid before. discoverRepoRoots finds the roots when no list is given, which is what the local npm run e2e:manifest uses so the repo list lives in one place rather than four. Unit-tested against real temp trees in test/unit/model-manifest.test.mjs. |
fetch-e2e-models.mjs |
Downloads the model files the tier-3 E2E needs into the E2E model dir (skips files already present): the int8 ASR weights (the two canonical files plus the vocab, since the model repo's graph work ships inside them), plus the two speaker-diarization models (pyannote segmentation + CAM++ embedding) that transcription-diarization.spec.js needs. Entries name the CANONICAL layout path (where the file must LAND, so serve.mjs and the local-mirror probes find it), while remotePathFor resolves where the repo actually SERVES it from that repo's live listing, exactly as hub.js does: the optimized repo moved the lite int8 encoder into its nested istupakov_smoothquant/ sub-repo, which 404ed the hardcoded path while the app went on loading it fine. Every entry is REQUIRED, so a broken model URL fails loudly; download's optional escape hatch is unused but kept and tested for the window where a file is committed to the model repo before it is pushed to HF. The CI cache key hashes this file, so editing the list re-keys the cache. Entry contract unit-tested in test/unit/fetch-e2e-models.test.mjs. |
run_all_tests.sh |
Convenience runner for the full three-tier suite: rebuilds app/ui/dist (the e2e tier tests the built app, so a stale dist would test an old UI), then runs tier 1 (unit) -> tier 2 (http) -> tier 3 (e2e), fail-fast. --no-build / --no-e2e flags. Excludes the GPU/WebGPU diagnostics and WER benches by design. |
download-dictation-regex.sh |
Fetches dictation regex CSVs from Murmure for non-Docker local dev. |
update-vendored.sh |
Refreshes the npm-vendored deps (version query, download, SHA verify, rewrite SOURCE.md). Run only on explicit request. |
update-caddy.sh |
Refreshes the pinned Caddy base-image digest in the Dockerfile. |
check-hf-cdn-hosts.mjs |
Resolves a real HuggingFace download hop by hop and reports any origin the CSP allowlist (_HF_HOSTS_DEFAULT in docker/entrypoint.sh) no longer covers. HF moves its download hosts (repos migrated to Xet storage left cdn-lfs*.huggingface.co for the regional *.aws.cdn.hf.co CDN), and a missing origin fails in the browser: hub.js only sees a failed download, falls through to the local mirror, and the deployment reports whatever THAT lacks, so the real cause appears in no app log. Network-dependent, hence a script; test/unit/csp-hf-hosts.test.mjs pins the list itself offline. Exit 1 names the line to add. |
precompress.mjs |
Generates the precompressed sidecars Caddy serves via file_server { precompressed ... }, and prunes any that went stale. Two modes, because the two payloads want different compressors: --static <dir> emits <file>.br for the built bundle (brotli q11, built into Node, run by the Docker builder right after vite build: dist is ~138 MB with ~130 MB of WASM that Caddy would otherwise re-compress on EVERY request; measured 26 MB -> 3.5 MB on the ORT jsep build, 31 MB -> 6.9 MB on ffmpeg-core), and --models <dir> emits <file>.zst for a self-hosted model mirror (zstd -9, since brotli on hundreds of MB of weights would take tens of minutes; measured 841 MB -> 643 MB on the int8 encoder, ~11 s with the zstd binary and ~32 s through Node's own zstd when it is absent). Model weights are application/octet-stream, which Caddy's encode directive deliberately skips, so without a sidecar they cross the wire raw. Models mode deduplicates by resolved path: a maintainer's tree reaches the same bytes through both the nested model repo and the flat root symlinks (and through the symlinked per-precision directories, fp32/ and friends, or sharded/ on an older mirror), so each file is compressed ONCE next to the real file and every other view gets a relative symlink to that sidecar; without that the flat path Caddy actually serves would have no sidecar at all and a second full copy would be written per duplicate view. The walk never descends into a directory named local or .git (the model repos' gitignored working folder, and a checkout's object/LFS store): neither is uploaded or served, but both hold many GB of weight-shaped files, so without the skip precompress would spend hours writing sidecars next to orphan builds no request can reach. Staleness is the hazard it exists to prevent (a sidecar older than its source is served INSTEAD of it, silently, to part of the audience only): it regenerates what it can, DELETES what it cannot, and sweeps orphaned or dangling sidecars. --check reports without writing, which is what the container runs at boot. Idempotent, never fatal. Unit-tested in test/unit/precompress.test.mjs. |
openai-like-server/ |
OpenAI/whisper-compatible HTTP API in front of this pipeline (own section below). |
A self-contained HTTP server (default port 8002) that speaks the OpenAI
audio-transcription API plus the whisper.cpp / whisper-asr-webservice dialects,
so any client written against those can transcribe against a local Parakeet
model. It imports the pipeline rather than reimplementing it
(scripts/transcribe.mjs for the model/ffmpeg/boost glue, app/src/ for the
engine, app/ui/src/lib/speakerAssign.js for speaker labelling), so its
transcripts are byte-identical to the CLI's for the same options. The default
--ort wasm backend has no npm dependency at all (it uses the vendored
onnxruntime-web Node build); onnxruntime-node is installed only for the
node/cuda backends.
| File | Role |
|---|---|
README.md |
API reference: endpoints, the request-field compatibility matrix (honoured / aliased / ignored / rejected), the verbose_json field mapping and its honest constants, wordlists, diarization, backends, auth, limits, client examples, troubleshooting. |
server.mjs |
Boot sequence: resolve options -> build the engine -> listen -> drain on SIGTERM/SIGINT. Distinct exit codes (2 config, 3 model, 4 port in use, 5 server) and the keyless-bind warning. |
lib/options.mjs |
THE table: every knob's CLI spellings, env var, type/range, default, and (when per-call safe) the multipart field that overrides it. Also the two whisper-compatibility tables (ACCEPTED_NOOP warn-and-ignore, UNSUPPORTED fatal-with-alternative) and --help rendering. A unit test iterates it against docker-compose.yml/env.example, so a knob cannot exist in code but be unreachable in the container. |
lib/app.mjs |
HTTP layer: routing (incl. the /inference alias and --request-path prefix), constant-time bearer auth, CORS, security headers, the transcript-free access log, and the error envelope. Takes the engine as an argument, which is what lets tier 2 drive every route with a double. |
lib/engine.mjs |
The inference side: loads the model via transcribe.mjs's loadParakeetModel, decodes uploads with ffmpeg, runs transcribeChunked, and owns the wordlist registry + diarizer. Model-load failures carry the hf download hint. |
lib/params.mjs |
One request's form -> the parameter set for a run, layered over the launch options; alias mapping, unknown-field 400s, --lock-params, granularity resolution. |
lib/formats.mjs |
Words -> segments (pause / sentence end / speaker change / soft char cap) -> json/text/srt/vtt/verbose_json, incl. the real gzip compression_ratio and avg_logprob. |
lib/queue.mjs |
Single-slot strict-FIFO queue: 429 + Retry-After when full, 504 on the deadline (a waiting job is dropped; a running one cannot be cancelled and the error says so). |
lib/wordlists.mjs |
Boot-time snapshot of --wordlist-dir (so a crafted name cannot traverse out), .pwc-over-.txt preference, and the LRU trie cache keyed by name/inline/depth-scaling. |
lib/multipart.mjs |
Capped body read (Content-Length and a running counter, so chunked cannot bypass 413), Request.formData() parsing (no dependency), the accepted file-part names, and random-named temp files that are always unlinked. |
lib/diarize.mjs |
Resolves the pyannote/CAM++ models and fronts the worker; the same engine the browser app uses, so labels match. |
lib/diarize.worker.mjs |
Runs the vendored sherpa-onnx WASM glue under worker_threads (its process() call is synchronous), evaluating a runtime .cjs copy because the vendored .js sits under an ESM package.json. |
lib/errors.mjs |
ApiError + the OpenAI error envelope helpers (400/401/404/413/429/501/503/504). |
lib/constants.mjs |
Shared constants (sample rate), kept separate so app.mjs never has to import the ORT-loading engine. |
Dockerfile |
Two-stage build from the REPO ROOT: lockfile-integrity gate, npm ci --ignore-scripts, optional npm audit gate, ORT_NODE_VARIANT pruning (none/cpu/cuda), then a digest-pinned Node slim runtime with ffmpeg, a non-root UID 1000 user, and only the files the server imports. |
docker-compose.yml |
Hardened stack mirroring docker/docker-compose.yml: non-root, cap_drop: ALL, no-new-privileges, read-only rootfs + noexec tmpfs, pids limit, init, log caps, read-only model/wordlist mounts, port published to 127.0.0.1 only, and a commented GPU block. |
env.example |
Documented template for .env: the mandatory host MODEL_DIR, auth/exposure, limits, model+runtime, decoding, boosting, output, diarization, behaviour, and the build/resource knobs. |
package.json, package-lock.json |
The single pinned dependency (onnxruntime-node, matching the vendored ORT generation), needed only for --ort node|cuda. |
Tier 1 (unit) and tier 2 (http) run on pre-push and in CI; tier 3 (E2E) is the slow, model-loading tier run separately (offered as a pre-push opt-in; a CI job).
WebGPU test coverage (read this before assuming "WebGPU can't be tested").
The deciding factor is a real GPU, NOT headless-vs-headed. Automated tier-3 /
CI Chromium has no GPU, so it always falls back to WASM int8 and can never
exercise WebGPU: the single-file/WebGPU-fp32 paths are consequently
outside the e2e tier (the one exception the e2e tier DOES cover is sharded
fp32 on WASM). The real GPU path (encoder batching + the encode/decode worker
pipeline) is instead validated by scripts/webgpu-check.mjs, which drives the
built app on a real webgpu-hybrid session and runs fine headless on a GPU
box (--headless). On a GPU whose Dawn build omits shader-f16 (this repo's
box), the withdrawn fp16 build loaded but computed empty; fp32 needs no such
feature and is what the harness now runs. It also asserts the decode-worker
pipeline engaged. Opt into it from
the .githooks/pre-push prompt, or run it by hand. It needs a FREE GPU.
| Path | Role |
|---|---|
test/unit/*.test.mjs |
Tier 1, pure-logic unit tests (no model download). Decode/front-end: beam-decode, topk-decoder-outputs (in-graph top-K decode path: the exact reduced fetch list when it engages, NO fetches argument when it must not (switched off, phrase boosting, beam, a decoder without the outputs, a too-short row), and result equivalence with the full-row path including the tie case), decode-debug (opt-in collectDecodeDebug payload: greedy/beam per-token records with true logit, log-prob, boost bonus and top-k alternatives, the beam keptHyps timeline, and transcribeChunked's per-chunk aggregation), bpe-encoder, chunk-default, chunk-stitch (overlap stitch + createEncodeProducer, the one bounded look-ahead encode loop both pipelined drivers share (window bound, refill-before-return, in-order hand-back under out-of-order completion, the birth-time catch, and the out-of-order/exhausted throws) + the injected-decodeChunk/encodeChunk pipelined drivers: out-of-order completion, in-order consumption, bounded dispatch, failure paths, the COMPOSED mode where both hooks are injected (encode identity handed to the decoder, own encode paths untouched, both windows bounded), and the single-pass path routing a short/unchunked clip through an injected encodeChunk while never calling decodeChunk), execution-providers (executionProvidersFor: the ORT EP list shared by fromUrls and encoderOnlyFromUrls so the two session builders can never drift, plus encoderOnlyFromUrls' unsupported-backend guard), session-options (baseSessionOptions/withExternalData: the rest of that same contract, pinned so an option added to fromUrls cannot leave decoderOnlyFromUrls/encoderOnlyFromUrls on the old config and mix numerics inside one clip), worker-init (workerReady init handshake: script-load failure, init error message, watchdog on a hung init, first-signal-wins), model-worker (createModelWorker, the worker side of that handshake shared by the decode/encode workers: ready and init-scoped-error shapes, a synchronous initModel throw reported like an async one, the result envelope and transfer list, per-request errors carrying id/chunkIndex, FIFO serialisation of runs on the one session, a run arriving before init, and the non-serialised extra handlers), cpu-threads (defaultWasmThreads/restoreCpuThreads/encodePoolPlan), model-repos (the repo-picker policy: list parsing, short labels, ?model= closest match incl. ambiguity returning nothing, and the URL > saved > default precedence), entrypoint-model-repo (the shipped _validate_model_repo extracted from docker/entrypoint.sh and run under /bin/sh: every list entry validated rather than just the head, so good/repo,../../evil/repo is refused, plus the empty-entry typo shapes), entrypoint-served-manifest (the shipped _write_served_manifest, likewise run under /bin/sh: the listing the browser reads to learn what an instance serves, asserted over real temp trees for the multi-extension case the boost prebuild needs (a <list>.json that exists is named, one that does not is NOT, which is the whole fix for the 404 every visitor used to collect), the manifest never naming itself despite matching its own *.txt glob on the second pass, an unmatched extension contributing nothing rather than a literal glob, and the temp file it builds through not being left served), chunk-duration (restoreChunkDuration: persisted chunk-window restore, clamping, and the one-time legacy-20 s-default rescue to the current 60 s default), loaded-model (the "Currently loaded" row: that a real fallback (a WebGPU/fp32 selection answered by a WASM int8 model) reads as a divergence, that a repo or precision difference alone does too, and the two refusals that keep the row worth reading, the source never counting as a divergence and an unreported precision claiming nothing), hub-reachability (the background HuggingFace preflight: that the probe never throws whatever fetch does, that a hung request is abandoned on its own schedule rather than waited out, the request shape with mode: 'cors' pinned as a CONTRACT (a no-cors probe is blocked outright by this app's own COEP header and would report every visitor as offline), and every preferLocalFirst branch, including the pinned refusal to reinstate the local-mirror precondition that once kept the whole feature from firing), med-mode (the medical dictation preset: which ?mode= values are recognised — and, more importantly, that an unknown one is NOT, since the preset is sticky — plus every preset value asserted literally, because a partly-applied preset still transcribes happily and nothing says so), beam-width-auto (restoreBeamWidthAuto/resolveAutoBeamWidth: the boost-coupled beam width default + legacy-install inference), benchmark-reports-sync (runs the real benchmark_reports/sync.sh against a local stand-in for the VPS: union of both sides, a name on both sides keeps its own bytes on each, only top-level reports move, dry run changes nothing), support-report (the Debug-section support report: guarded environment collector in bare Node and against a stubbed browser incl. denied/throwing probes, BigInt-folding fixed-key-order JSON builder), benchmark (the self-service benchmark harness: matrix planning incl. the heavy-row and shader-f16 gates and the current-combination-last ordering, PCM tiling for the chunked profile, word-LCS transcript similarity, the fake-driven run loop over load failures / unavailable quants / cancellation / medianed repeats, and the anonymiser's allowlist asserted from BOTH sides, keeps and drops, plus the live table: placeholder rows planned up front, results merged in as they land, and a finished row never clobbered by a later profile-less one), encoder-quants (which encoder precisions a given deployment can actually serve: servableEncoderQuants asked over three real repo listings AND a verbatim listing captured from the maintainer's own instance, the null-means-no-opinion rule that keeps an unanswered probe permissive, effectiveEncoderQuant answering null rather than degrading to fp32 or w4a8 when the backend has nothing it may use unasked, and gpuBackendAutoUsable deciding whether the GPU may be chosen for somebody who did not ask for it), encoder-quant-rows (the precision radios: that the rows are exactly the union of the per-backend whitelists, that they run smallest-download-first as measured from the sizes the LABELS quote, and that the label and QUANT_DOWNLOAD_MB, which prices the benchmark rows and the estimate above the Load button, quote the same size for the same file), perf-probe (the autoconfigure probe's decision rule: the >=2.0x margin that prices the bigger GPU download, every degenerate timing and failed GPU arm resolving to wasm, verdict expiry on app update / changed GPU / age, the auto-run gate never overriding a hand-picked backend, and the slow-hardware sample-count plan), browser-family, device-class (phone/tablet detection: client hint, tablet UA, iPadOS-as-Mac, and the quiet-when-unsure fallbacks), number-words (English/French cardinal conversion, the ambiguity and non-combining guards, and text-vs-word-array agreement) (the slow-browser popup's engine gate: client-hint brands decide, UA fallback, unknowable environments never nag), encode-batch-equivalence (real int8 encoder, self-skips without local weights: proves encodeBatch on equal-length chunks is byte-identical to standalone encode(), that N=1 delegates exactly, that mixed lengths throw, and that a maxEncoderBatch=2 transcribeChunked transcript equals the un-batched one), max-encoder-batch (resolveMaxEncoderBatch GPU-adaptive batch sizing with a stubbed WebGPU adapter: WASM=1, floor/ceiling, fp32-vs-int8, guarded failure paths), mel, fbank (kaldi 80-dim fbank geometry, global-mean normalization, mel-bin localization for the speaker-embedding front-end), phrase-boost, tokenizer, boost-compile, boost-spec-file, boost-config (the boost knobs that were module-level in App.jsx until the usePhraseBoost extraction: normalizeBoostName, where a bare ?phrase_boost=medical becomes a manifest entry while a blank value stays null so a saved choice is not overridden and a sentinel passes through rather than becoming __custom__.txt, and boostBuildKey, the build identity a transcription waits on, with the phrase text placed LAST so a text containing the separator cannot forge another vocab's key). Hub/cache/quant selection: resolve-quant, get-parakeet-model-files, list-local-repo-files, resolve-local-model-base, hub-cache-validate, should-retry-locally, load-failure (the whole failure ladder around it: which retry is spent when, the local-first rescue that costs one same-origin miss rather than a failed load, the GPU-to-WASM flip the benchmark is forbidden from making, and an exhaustive 4096-combination sweep of the invariants, chiefly that the change-your-pick banner and the nothing-left-to-pick popup can never fire together), model-corruption-recovery, sweep-orphans (cache-GC orphan selection, incl. the protected-key carve-out that keeps diarization models across loads), stream-to-memory (fp32 shard byte-assembly), shard-cache-gate (the size gate that decides whether a streamed noCache file is written to IndexedDB after all: cached and re-served when small enough, untouched and re-downloaded when over the limit that Chromium's blob readback fails at), shard-prefix-cache (the PARTIAL cache for the files too big to cache whole: the prefix written and capped at the budget, the next load resuming with a Range request for exactly the missing tail, a file that fits inside the budget served entirely from disk, a budget of zero keeping the never-touch-IndexedDB behaviour byte for byte, and the three ways a prefix is thrown away rather than trusted (a server that ignores Range, a file whose length changed under an unchanged etag, an unreadable segment)), precompress (the sidecar generator's rules: which files earn a .br/.zst, the never-keep-a-sidecar-older-than-its-source test, the minimum-gain drop, and the canonical/alias planning that gives the flat path Caddy actually serves its own sidecar from one copy of the bytes, walk order and symlinked directories included, plus end-to-end runs on a real temp tree with real symlinks), precompressed-response (a Content-Encoding: zstd model download: the compressed content-length is not adopted as the total, progress never overshoots, the variant etag is not replayed as If-Range while an identity one still is, and a resume that only learns the real length on the retry keeps the bytes already streamed), preprocessor-opts (the ONNX preprocessor's constructor: the caller's options object left unmutated, so one object reused across a wasm and a webgpu preprocessor no longer leaks the first one's graph-capture default into the second, plus the per-backend default and the explicit override), external-data, resolve-files (per-quant encoder/decoder/vocab resolution: canonical names only, incl. the int8 SmoothQuant encoder-name fallback, a loud failure on a variant-only dir, and the independent decoderQuant so an int8 encoder can pair with an fp32 decoder). Diarization: speaker-assign (word -> speaker max-overlap mapping + turn grouping), embedding-session (the CAM++ session cache, with an injected ORT loader: two concurrent callers sharing ONE session (the memo holds the PROMISE, so a second in-flight pass no longer builds and leaks a duplicate ~28 MB session), the superseded model released on a key change, and a failed build not memoised), speaker-match (cross-recording cosine matching: name-profile centroids, threshold matching, auto-naming a recording's unnamed speakers from prior recordings without clobbering user names), silence-cut (silence-excision cut-finding + condensed PCM + condensed->original segment remap with joint-crossing splits, plus the dense createEnergySampler.hopProfile block-prefix energy vs a direct block-aligned recompute), diarizer-client (the diarization client's single-run contract: a second concurrent run refused rather than orphaning the first, the guard synchronous so a re-entry during the init await is caught too, and the client released again after a settled, cancelled or failed run; plus the crash paths, where the worker dies with an error event and no message at all: mid-run the pending run rejects instead of hanging forever, during init it surfaces as a failed init, and either way the next run rebuilds the worker and re-sends the models), diarize-piecewise (cross-piece speaker-label reconciliation: same-voice convergence, below-threshold/missing-embedding minting, non-contiguous labels, centroid drift, seam merge/stitch, shouldPiecewise gating, and the per-piece claim rule that keeps two locals of ONE piece off the same global: the diarizer already called them different people, and the UI can merge two speakers but cannot un-merge one, so the strongest match takes the global, the next one falls back to its second choice, and the loser mints rather than merges). Bench/misc: gen-val-sets (drawOrder: seeded-shuffle vs --longest duration-descending draw of gen-medical-val-sets.mjs, tie-break determinism, input non-mutation), grid-search-datasets, grid-search-eta, grid-search-chunk-sweep (the chunking sweep axis: off token parsing, off-cell collapse vs cross-product in buildChunkConfigs, chunk resume-key back-compat and distinctness in tagOf), grid-search-oracle (levenshtein S/D/I decomposition; oracle-vs-1-best edit accumulation in newAcc/addScore/buildDatasets), ort-runtime-config (the --ort backend -> executionProviders/from-path mapping, incl. the opt-in cuda GPU backend), ort-asset-verify (the ORT runtime integrity loader: pins the request SET to the manifest plus the one variant it hands ORT, never the three unused ones, one object URL per pinned file, a tampered pinned runtime still throwing, every fallback path fetching no assets, the resolveOrtVariant truth table, the one-ORT-import-site graph guard, and the guard that every session-building worker forwards the runtime variant), transcribe-browser (the browser-driving CLI's pure logic: parseArgs defaults/validation and the turnsToMarkdown/buildMarkdown builders; the GPU/diarization path is out of CI like webgpu-check), level-monitor (createLevelMonitor: stop() disconnecting the AnalyserNode as well as ending the rAF loop, since one was built per recording at four call sites and they used to accumulate for the life of the tab, plus the 0..100 clamp), recording-rate-candidates (recording AudioContext sample-rate ordering: browser default before the SpeechMike low rates so a Firefox mic reporting no rate is not forced to 16 kHz and slowed), load-progress (the download line itself: what a resumed download may count as a network byte, the monotonic per-file credit, the byte event that is the only licence to say "downloading", the silent single attempt, and the rate/ETA appearing only once a window exists), model-request (what one load attempt asks for: fp16 never rewritten locally, int8lite/w4a8 never collapsed to int8, the upgrade and fallback mirror keys never both set, and an absent revision pin left off the object rather than sent as undefined), pipeline-plan (the off-thread pipeline gates: fp32 refused a pool, WebGPU refused a pool but always given a decode worker, a WASM decode worker only ever eligible composed with the pool, and the stash surviving a toggle that is off, plus an invariant sweep over every backend/quant/hardware combination), load-phase (the model-load phase set and its timing line: every phase counts as loading and a settled status does not, the m/ss vs decimal-seconds switch, and cached staying distinct from transfer unknown), bold-runs (the **bold** splitter behind the encoder-precision labels: the marked run, an unmarked string as one plain run, and an unclosed marker staying literal), theme-tokens (the design-token contract between App.css and the inline styles that read it: every var(--token) referenced anywhere under app/ui/src is really defined, which a var() fallback otherwise hides forever (three phrase-boost hint panels sat on a --surface-muted that exists in no stylesheet, so they stayed near-white under dark-mode --text-muted at about 1.4:1), and no token exists only inside the dark-mode block, which is the same mistake pointed the other way), wav-blob (createWavBlob: the canonical 44-byte mono header field by field, the declared rate following the argument, and the asymmetric int16 scale where -1.0 lands on -32768 and +1.0 on 32767 rather than wrapping, plus clamping and the header-only empty clip), fetch-capped (the F-102 byte cap on operator-served text: the declared-length refusal before streaming, the streamed overrun, and the no-streaming-body fallback measuring UTF-8 BYTES rather than UTF-16 code units, which used to let a non-ASCII body twice the cap through), persist-storage, write-queue (serial write-queue ordering: enqueue-order execution despite faster later tasks, rejection isolation), live-transcriber-lifecycle (the live loop's start/stop, driven by a fake clock: stop() abandoning a wedged transcribe() at a finite drain bound rather than hanging the UI in the recording state, and a tick left in flight across a stop/start pair not queueing a second self-rescheduling chain beside the new one), capture-queue (gated capture-queue: holds jobs until canRun(), FIFO drain, single-flight serialization, mid-drain pause when the gate closes, rejection isolation), format, remote-crypto, remote-relay-drain (transport backpressure drain for the saved-file pump), caddy-permissions-policy (asserts the production Caddy Permissions-Policy grants camera=(self) for the remote-mic QR re-scan and pins the self-allowlist), strict-weights (the tier-3 missing-weights gate: env precedence of PARAKEET_E2E_STRICT_WEIGHTS over the CI default, and that requireWeightsOrSkip fails-vs-skips accordingly), model-probe (the tier-3 optional-weight probe: the mirror's own manifest consulted first and treated as authoritative for the base that has one (a directory no candidate path names is found, a file it omits is reported unserved WITHOUT probing, a flat mirror's manifest is only asked once the repo root has missed, and something that is not a listing is ignored rather than believed), then every repo-nested candidate before every flat one, the URL list pinned to candidatePaths() mapped over both layouts so the probe cannot grow a narrower private idea of the layout, an fp32 shard found in fp32/ or the older sharded/, and a diarization model looked for under its own repo AND at the mirror root), repo-layout-detection (the loaders resolved against VERBATIM captures of the three reference repo listings in test/fixtures/repo-listings/: an exhaustive literal table of where every model basename lands in each layout, which quants each repo can really serve (upstream istupakov stays fully usable on int8 and pins back rather than guessing for what it lacks, the lite encoder is still found after moving into the optimized repo's nested sub-repo), and parseEncoderShards picking ONE directory so a nested sub-repo's identically-named shards can never double the download into a corrupt encoder), dangling-links (the model dir's broken-symlink walk over a real temp tree: which links resolve, the .git skip, the served-vs-deeper split that decides fatal-vs-warning, and that a missing dir yields findings rather than throwing), pipeline-trouble (the shared encode-pool/decode-worker failure patterns, asserted against the LITERAL log templates the app emits plus a scan of App.jsx/workerInit.js that fails on any uncovered [Encode]/[Decode] warning, since a retyped string would have passed while the real one did not match), csp-hf-hosts (the HuggingFace origins the CSP lets the browser reach: the Xet CDN weights are really redirected to, the API origin and the legacy LFS CDNs, and that every entry is a bare https origin so the baked-in default cannot be the one place a wildcard enters the header), routes-gpu-encoders (the tier-3 helper that models a deployment with no GPU-runnable encoder: the shard bytes 404 AND the same entries are stripped from the mirror's model-manifest.json, because hub.js believes a manifest over any probing, so hiding only the files left the shards announced and the GPU-to-WASM fallback with nothing to fire on), serve-range-requests (the tier-3 static server answering like a real one: Content-Length on GET and HEAD, open-ended and closed Range as 206 with the right window, If-Range resuming on a match and sending the whole entity on a stale one, a past-the-end Range refused with 416, an empty file served rather than erroring), serve-manifest-sidecar (the tier-3 server serving model-manifest.json out of PARAKEET_E2E_MANIFEST_DIR while weights still come from the model dir, and 404ing a repo the sidecar does not describe), model-manifest-plumbing (the container half of the mirror manifest asserted end to end, because every break in it is silent: the reader and writer naming the same file, the generator present in the runtime image, the entrypoint running it with mount + repo list + writable destination, that failing to generate it can only warn, the /var/model-manifests tmpfs existing under the read-only rootfs, and Caddy routing both mirror shapes to it ahead of the weights via mutually exclusive handles), model-manifest (the mirror self-description writer, over real temp trees: a configured repo the mount does not serve warned about by name while a flat single-repo mount is not, since the app silently falls back to HuggingFace either way and only the first is a mistake, nested sub-repos listed, symlinked files and directories followed, dangling links and .git bulk left out, paths the reader would refuse never written, the manifest kept out of its own listing, a separate writable destination for a read-only mount, one manifest per repo on a shared mount with none at its root, and an unwritable destination warning instead of throwing), fetch-e2e-models (the CI model fetch's entry contract: every entry is required so a broken URL fails loudly, the optional escape hatch stays 404-tolerant, present files short-circuit, and every file lands under its own repo prefix rather than loose at the mirror root, with the repo NOT leaking into the HuggingFace request path; plus remotePathFor following a file into a nested sub-repo, leaving an already-canonical path alone, finding it in a flat repo, falling back to the canonical path on an unusable listing, and the download requesting the resolved path while still landing at the canonical one), openai-server-options (the API server's option table: CLI/env precedence, ranges, the whisper flag tables, the keyless non-loopback refusal, plus the plumbing gate asserting every option's env var reaches docker-compose.yml/env.example and the compose/Dockerfile hardening properties), openai-server-formats (words -> segments -> srt/vtt/text/verbose_json, one case per break rule, exact timecodes, real gzip ratio, avg_logprob null-vs-log(mean)), openai-server-params (per-request field resolution: aliases from other whisper servers, granularities, unknown/ignored/rejected fields, --lock-params, and the FIFO queue's 429/504/slot-release contract). |
test/http/*.test.mjs |
Tier 2, integration tests over real loopback HTTP. Against the real signaling server spawned on a random port: config, origin, rate-limit, rooms, validation, benchmark-report (the benchmark receiver: off without a configured folder, format/size/file-count refusals, server-owned filenames that a path-shaped payload cannot influence, re-serialised storage, and no sender data written). Against the real OpenAI-like API server (openai-server, started in-process on a random port with a fake engine, so no weights and no minutes of CPU): every route, status code, response format, auth mode, CORS mode, the 413/429 limits and the diarization labelling. |
test/http/helpers.mjs |
Spawn/teardown helper for the signaling server, shared by the tier-2 tests. |
test/e2e/transcription.spec.js |
Tier 3 Playwright happy-path: loads the WASM int8 model in real headless Chromium and transcribes each clip in a fixture list (French sample.aac + English jfk.mp3) end to end against its golden. |
test/e2e/transcription-upload-ffmpeg-parity.spec.js |
Tier 3 proof that an uploaded file is decoded in-browser by the vendored ffmpeg.wasm (lib/audioDecode.js) for CLI parity: injects the production CSP on the document, uploads sample.aac (raw ADTS AAC whose encoder-delay/priming decodeAudioData doesn't trim), and asserts (1) the decode went via ffmpeg.wasm (so its worker/core/wasm all loaded under script-src/worker-src/connect-src 'self' blob: + COEP; a CSP block would fall back to web-audio), (2) no same-origin CSP violation, and (3) the drug name decodes as "Venlafaxine" (CLI spelling), not the browser-front-end artefact "Velnafacine". |
test/e2e/transcription-diarization.spec.js |
Tier 3 in-browser proof that the vendored sherpa-onnx WASM speaker-diarization engine loads and runs: transcribes the two-speaker fixture (two-speakers.wav: JFK + a FLEURS English clip, loudness-normalised lossless PCM so both speakers transcribe), clicks the per-entry Speakers button, and asserts >= 2 colour-coded speaker turns (first != last speaker, non-empty text) plus a Raw <-> Speakers toggle that reuses the cached result. Then exercises the interactive controls: forcing a speaker count from the entry kebab (re-segments down to one turn, then back to Auto for >= 2) and renaming a speaker inline (label button -> text input). Finally proves persistence: with persistTranscripts seeded on, it asserts the grouped turns + custom name (and ONLY those, no per-word timings or raw segments, per F-130) reach the transcripts DB and survive a page reload, where the Speakers view + the renamed label reappear from disk with the in-memory audio gone. When the two diarization models are not served the HEAD-probe miss fails locally / skips in CI via strict-weights.mjs; npm run e2e:models fetches them. |
test/e2e/transcription-diarization-model-failure.spec.js |
Tier 3 proof of the diarization model-load FAILURE UX: routes the CAM++ embedding model to a 404 so getDiarizationModels always rejects (needs no diarization weights, never skips), loads the ASR model, and asserts the background prefetch failure greys out BOTH the per-entry Speakers button (display-mode-button--unavailable + aria-disabled + a reason tooltip) and the sidebar's "Speakers" default-display option (disabled + title), with NO browser alert raised and a click on the greyed button a no-op. |
test/e2e/transcription-speaker-match.spec.js |
Tier 3 in-browser proof of session-only cross-recording speaker matching: uploads two-speakers.wav, diarizes it, renames its first speaker (JFK) to "Alice", then uploads the SAME clip again (newest entry prepends to the top) and diarizes it, asserting the second recording's matching voice auto-labels "Alice" with no manual rename while the other (un-named) speaker stays a default label. Exercises the full in-browser embedding chain (app/src/fbank.js + onnxruntime-web CAM++ -> speakerMatch.js) that the unit tests and scripts/speaker-embedding-check.mjs only cover piecewise. When the diarization models are not served the HEAD-probe miss fails locally / skips in CI via strict-weights.mjs; npm run e2e:models fetches them. |
test/e2e/chunking.spec.js |
Tier 3 long-audio path: seeds a 10 s chunk window (the minimum allowed) and feeds the ~11 s jfk.mp3 so transcribeChunked splits into >1 chunk, asserting chunking engaged and the stitched transcript recovers the golden content. |
test/e2e/fleurs-regression.spec.js |
Tier 3 multilingual regression: loads the model ONCE and loops the 10 en + 10 fr FLEURS clips through the file input, asserting each transcript against both the committed int8 golden and the FLEURS human reference (word-overlap). |
test/e2e/long-audio-chunking.spec.js |
Tier 3 realistic long-audio path: feeds the committed 3 min JFK "moon speech" crop (jfk-moon-3min.mp3, one continuous speech, so seams land mid-sentence) at a seeded 20 s chunk window (deliberately below the 60 s default, giving ~a dozen chunks on a 3 min clip); asserts chunking engaged, content recovered, and no runaway seam duplication. (Replaced the stitched-FLEURS clip, now used only by scripts/wer-bench.mjs.) |
test/e2e/transcription-fp32-wasm.spec.js |
Tier 3 in-browser proof that the sharded fp32 encoder loads and transcribes on WASM in real headless Chromium (the single 2.4 GB sidecar can't; the scripts/shard-fp32.py pieces each < 2 GB can). Gated behind the allowWasmFp32 opt-in; when the local shards are absent (upstream ships none) the HEAD-probe miss fails locally / skips in CI via strict-weights.mjs. The probe asks for the bare shard name and serve.mjs resolves it through app/src/modelLayout.js, so the shards may sit in fp32/, flat at the root, or in sharded/ on an older mirror. The model repo ships ONE fp32 build (graph-optimized) under the canonical shard names and the fold is bit-exact, so there is no build to disambiguate: the spec asserts the shards mounted and that the opt-in did not fall back to the int8 pin. |
test/e2e/transcription-int8-lite-wasm.spec.js |
Tier 3 in-browser proof that the lite int8 encoder (encoder-model.int8.lite.onnx, same SmoothQuant calibration with --exclude-worst 0.05 so 11 MatMuls stay fp32 instead of 18) is what actually loads when the "int8 lite" precision radio is picked. Fully coverable headless (plain WASM int8, just a different file). The failure it guards is silent: a regression collapsing int8lite back to int8 would still transcribe perfectly, so the spec pins WHICH FILE hub.js fetched (lite name present, default int8 name absent) on top of the usual transcript check. CI does not fetch the lite build, so a HEAD-probe miss fails locally / skips in CI via strict-weights.mjs. |
test/e2e/transcription-lse-decoder.spec.js |
Tier 3 in-browser proof that the decoder's in-graph log-partition outputs (lse_token/lse_duration, added by the model repo's optimize-decoder-graph.py and shipped inside the canonical decoder_joint-model.int8.onnx) transcribe correctly at a seeded beam width 5 (that path is beam-only, and their consumption is silent by design, so an unchanged golden IS the assertion). Checks no FILENAME: gated on the runtime capability marker [Parakeet.js] Decoder in-graph outputs: log-partition=yes, so it skips against a stock upstream decoder and fails instead under PARAKEET_E2E_STRICT_WEIGHTS via strict-weights.mjs. |
test/e2e/transcription-topk-decoder.spec.js |
Tier 3 in-browser proof for the decoder's in-graph top-K outputs (topk_logits/topk_ids/duration_logits, same script, same canonical file) and the reduced-fetch decode path they enable. Gated on the runtime capability marker (log-partition=yes top-K=yes), then asserts [Parakeet.js] TopK decoder outputs engaged (the greedy loop actually took the fast path, which a transcript alone could never show) on the zero-configuration default run, plus the golden overlap. A second test pins the negative gate: with phrase boosting seeded the engaged marker must NOT appear (boosting reads arbitrary vocab ids, so the full row is required) while the transcript is still produced. Skips against a stock upstream decoder; fails under PARAKEET_E2E_STRICT_WEIGHTS. |
test/e2e/transcription-fp32-wasm-autoupgrade.spec.js |
Tier 3 proof of the local auto-upgrade: user picks WASM fp32, the HF repo ships no shards, so hub.js (given localUpgradeBaseUrl='/models') probes the local mirror, finds the shards, and switches the whole load to local. Routes the HF listing to the shard-less istupakov set; needs the local shards (else fails locally / skips in CI via strict-weights.mjs). |
test/e2e/transcription-fp32-wasm-no-downgrade.spec.js |
Tier 3 negative counterpart: when NEITHER source can serve fp32, hub.js throws QuantUnavailableError instead of silently falling back to int8, and the UI shows a banner + Failed status. 404s the local shard probes; needs no weights, so never skips. (Seeds wasmEncoderQuant:'fp32' directly rather than driving the settings UI.) |
test/e2e/backend-webgpu-gating.spec.js |
Tier 3 proof of how the WebGPU backend is gated now that it is available app-wide and the autoconfigure probe decides per machine (it replaces the old backend-webgpu-disabled spec, which pinned the app-wide kill switch that no longer applies): with an adapter present WebGPU is selectable and a persisted webgpu-hybrid SURVIVES a reload (it used to be coerced to WASM on every boot, so this is what would catch the pin returning by accident); on WebGPU the int8 precision gets no row at all (there is no GPU int8 encoder kernel), fp16 is what a capable adapter loads, and an adapter with no shader-f16 greys fp16 out and gets NOTHING selected in its place, with a hint saying the load would run on the processor at int8: fp32 and w4a8 stay on screen and pickable but are never chosen for anyone, which is the 2026-09-11 owner rule (the fp16 gate itself is older, and exists because ORT's fp16 kernels silently yield an EMPTY transcript without the feature); with no adapter WebGPU is greyed out and WASM int8 stays the default; and ?webgpu=0 still forces WASM and coerces a persisted webgpu choice, which is the support/diagnostic kill switch. navigator.gpu is stubbed so each test pins one machine shape. Needs no weights, so never skips. |
test/e2e/gpu-quant-fallback.spec.js |
Tier 3 proof that a model source shipping no encoder the GPU may use unasked falls back to WASM int8 instead of failing the load. This became load-bearing when WebGPU was re-enabled: the performance probe can put a visitor on the GPU backend without them choosing it, so a deployment pointed at a CPU-only model repo would strand every probe-winning visitor on Failed. hub.js still refuses to silently downgrade the quant (that guard is what makes the failure legible, see the no-downgrade spec); App.jsx catches the resulting QuantUnavailableError on a webgpu backend and switches to WASM int8, warns, and retries ONCE. WASM int8 is the ONLY substitution the app ever makes: fp32 and w4a8 are hand picks, so the spec also asserts no fp32 or w4a8 encoder is fetched on the way. Stubs an adapter with shader-f16 so fp16 has to come back unservable from the SOURCE rather than from the machine, serves the local mirror with every GPU-runnable encoder routed away (routeLocalMirrorWithoutGpuEncoders: fp32 shards in both layouts, w4a8 and fp16, so the premise cannot be satisfied by a precision the spec forgot to hide), and asserts the load actually reaches ready on int8 rather than merely not throwing. Also the end-to-end coverage of the "Currently loaded" row (lib/loadedModel.js): after the fallback the row must name the precision that really mounted and the source it came from, AND must not be flagged as a divergence, since the fallback flipped the backend selection too and a warning that fires when everything went right is one people learn to ignore. Needs the int8 weights, so it skips like the other loading specs. |
test/e2e/precision-radio-source-gate.spec.js |
Tier 3 (model-free) proof that the encoder-precision radios describe THIS deployment rather than the precisions the app knows in the abstract. The report behind it: a visitor on the maintainer's instance picked fp16 on the GPU, sat through a load, and was then told the source hosts no GPU-executable fp16, so the CPU version was loaded instead. All true (that mirror carries fp32 shards, int8 and w4a8 but no fp16, and its network blocks HuggingFace so the mirror is the only source) and all far too late. Three different things can rule a precision out and they point at different people: the backend cannot run it (the build), the adapter reports no shader-f16 (the machine, unfixable), or the source does not host the file (the DEPLOYMENT, which the operator can fix), so the spec stubs an adapter that DOES report shader-f16 to rule the machine out, states the mirror's whole file set through routeSyntheticLocalMirror (so it behaves identically on CI, which has no weights, and on a developer box, which may have any subset), and asserts fp16 comes back disabled and NAMED as a source gap rather than a hardware one. Both halves matter: the precisions the mirror does host stay pickable, so a probe that simply failed and greyed everything out cannot pass. The WASM half pins the same rule on the backend most visitors use, via int8lite. |
test/e2e/perf-probe.spec.js |
Tier 3 proof that the autoconfigure probe stays out of the way. Headless has no GPU, so no real GPU verdict is reachable here (as with WebGPU generally); what this tier guards is that a probe sitting in FRONT of the Load model button costs nothing where it cannot help: with WebGPU disabled app-wide nothing is fetched, run or shown even on a machine that HAS a GPU; with WebGPU selectable but no adapter nothing is fetched or run; with an adapter that enumerates but cannot produce a device the probe runs, the GPU arm fails and the verdict is wasm (the safety direction, and a canary for the arm watchdogs, since a hang would stop the verdict arriving); and with an adapter that cannot do fp16 neither the artifacts nor the two timed runs are spent, since fp16 is the only precision the app will pick a GPU backend at unasked, so a win there could never be acted on. navigator.gpu is stubbed so each test pins one machine shape whatever the box has, and weight fetches are stalled, so it needs no weights and never skips. |
test/e2e/recording-section.spec.js |
Tier 3 (model-free) cover for the sidebar's Recording group, which previously had NONE: nothing drove noise suppression, auto gain, live transcription or its context window, so the whole group could have been wired to the wrong setting keys with every other spec still green. Test 1 seeds the opposite of every default, checks each control restores, flips it, and reads the value back out of IndexedDB. Test 2 reaches a REAL recording (the only file in the tier that does: it launches Chromium with a fake audio device, and holds the HF listing pending so the capture controls exist without a model) and pins that the capture-shaping controls are disabled DURING the take and enabled again after. That half is not cosmetic: those flags are read once, when getUserMedia opens the stream, so a live control would describe a capture that is not running. |
test/e2e/transcription-parallel-encode.spec.js |
Tier 3 chunk-parallel encoding end to end, pinning the DEFAULT WASM shape (pooled encode, in-thread decode: it also asserts the decode worker stayed out, which is what keeps the pool-only driver covered now that composed mode exists): seeds a 10 s chunk window + parallelEncode on, feeds jfk.mp3 (>1 chunk) and asserts the [Encode] pool engaged marker fired with ZERO pool-failure/fallback logs (the serial fallback would otherwise mask a broken pool behind a healthy transcript), plus the same stitched-overlap and no-duplication checks as chunking.spec.js. Self-skips on machines that cannot pass the encodePoolPlan hardware gate (< 8 cores / low deviceMemory). Headless-coverable because it is pure WASM, unlike the WebGPU decode pipeline. |
test/e2e/transcription-composed-pipeline.spec.js |
Tier 3 COMPOSED WASM pipeline (encode pool + decode worker together) end to end, opting in with VITE_WASM_DECODE_PIPELINE='true' (default off): same recipe as transcription-parallel-encode.spec.js (10 s chunk window + parallelEncode on, jfk.mp3 so the clip splits into >1 chunk) but asserts BOTH engagement markers, [Decode] pipeline engaged: pooled encode overlapping WASM decode in worker (composed) and [Encode] pool engaged, with ZERO pool/decode failure or fallback logs (the in-thread retry would otherwise mask a broken stage behind a healthy transcript), plus the golden-overlap and no-duplication checks of chunking.spec.js. Self-skips under the same encodePoolPlan hardware gate (< 8 cores / low deviceMemory), which also gates the WASM decode worker. Fully headless-coverable: composed mode is pure WASM, unlike the WebGPU decode pipeline. |
test/e2e/slow-browser-popup.spec.js |
Tier 3 (needs no model weights, so it never skips) slow-browser warning popup: Chromium must NEVER show it (a false positive would nag every normal user), while a MANUALLY launched Playwright Firefox against the same webServer must show it, dismiss it via its button, and see it AGAIN after reload (the dismissal is deliberately not persisted, so this fails if anyone "helpfully" remembers it). The Firefox context is created with the explicit Desktop Firefox device preset because a @playwright/test worker injects the active project's Desktop Chrome context options (including its Chrome/NNN userAgent) into library-launched browsers, which would correctly suppress the popup. |
test/e2e/seed-survives-first-boot.spec.js |
Tier 3 guard on the SEEDER itself (seed.mjs), not on the app: seeds a NON-default wasmEncoderQuant: 'fp32', reloads, and asserts it survived both in the settings DB and in the booted UI (the fp32 radio is checked). A seeder that loses its writes does not fail loudly, it makes every seeded spec run on defaults and quietly assert nothing (that is how the fp32-no-downgrade spec came to load int8 weights and pass in isolation while failing under full-suite load). Needs no weights and no network. |
test/e2e/controls-available-during-load.spec.js |
Tier 3 (model-free): record / upload / phone controls appear as soon as a load has STARTED (not only once ready), so audio can be captured during the download and queued. Holds every HF request open so the app parks in loadingModel and asserts the controls are present + usable there (idle still hides them). |
test/e2e/capture-queued-during-load.spec.js |
Tier 3: a file uploaded while the model is still loading is queued and transcribed automatically once ready (not dropped/refused). Delays ONLY the encoder fetch (~15 s) to make the loading window deterministic, uploads mid-load, asserts the queued-capture banner then the recovered transcript. Uses the local WASM-int8 weights (self-serves via serve.mjs). |
test/e2e/capture-queued-during-transcription.spec.js |
Tier 3: the upload / record / phone controls stay usable while a transcription is RUNNING, so more audio can join the capture queue mid-run. Transcribes the 3-minute JFK moon clip (a minutes-wide mid-run window), asserts the three controls are enabled mid-inference, uploads the French clip on top, asserts the queued banner + untouched status line, then both transcripts against their goldens in queue order. |
test/e2e/decode-debug-view.spec.js |
Tier 3: the decode-debug introspection chain end to end: enables the sidebar "Add decoder debug view" checkbox through the real UI, transcribes jfk.mp3, opens the entry's Debug mode, and asserts the token pills render, a pill click opens the detail card with a numeric alternatives table and exactly one chosen row, the card toggles closed, and Raw view still works. |
test/e2e/model-params-live-swap.spec.js |
Tier 3: the backend / precision / CPU-threads controls stay editable after the model loads, and changing one disposes the live model and reloads with the new setting. Loads int8, asserts the controls are enabled, then swaps CPU threads and observes the lock/unlock reload cycle + the dispose-then-reload log. |
test/e2e/model-unservable-popup.spec.js |
Tier 3 (model-free): the end of the line. When WASM int8, the one configuration the app picks for itself, fails from every source, the load raises a blocking popup (model-unservable-modal) instead of a Failed status under a page that still looks usable. The distinction it pins is between a failure with a setting to revisit and one without: a hand-picked fp32 with no shards anywhere keeps its banner naming fp32 (second test here, and transcription-fp32-wasm-no-downgrade.spec.js), while int8 has nowhere left to go, since fp32 and w4a8 are hand picks and the GPU-to-WASM fallback already lands here. Lists the repo honestly and then aborts the downloads with no local mirror, so the failure happens where it really happens, at the bytes, after every retry is spent. Asserts the popup names int8, that the status is Failed, and that it dismisses like the handheld notice (an undismissable popup would hide the settings and the support report at the moment somebody needs them). |
test/e2e/model-picker.spec.js |
Tier 3, model-free (repo selection is settled before any weight is fetched, so it needs no ONNX files and runs in ~13 s). Covers the sidebar model picker and its ?model= override: every configured repo offered in order with short labels, the picker absent on a single-repo instance, a pick surviving a reload, ?model= beating the saved choice but NOT being written back over it, a hand pick after such a link saving again, an unknown or ambiguous ?model= being ignored rather than guessed, and a repo removed from VITE_MODEL_REPO no longer loading for someone who had selected it. Every one of those failures is silent in production: the wrong repo still yields a fluent transcript from a real model. |
test/e2e/remote-mic-button-creates-room.spec.js |
Tier 3 regression: clicking "Phone Mic" must MINT a new room (createRoom path), not the re-arm path. Guards the onClick={() => startRemoteMic()} wiring (forwarding the click event as existingRoom made the first click POST /rooms/undefined/rearm -> 401 -> "Phone disconnected"). Fakes /api/signal/* to reach the QR/"waiting" state and asserts no /rearm or /undefined/ request. |
test/e2e/keyboard-shortcuts-opt-in.spec.js |
Tier 3 (model-free): global single-letter shortcuts (R/S/F/Space/Enter) are opt-in and OFF by default; exercises the 'S' settings-toggle before and after opting in. |
test/e2e/keyboard-shortcuts-guards.spec.js |
Tier 3 (model-free): once enabled, the global shortcuts never swallow a browser chord (Ctrl+R/S/F, Cmd+R) nor a focused <select>'s own keys, and the advertised shortcut table matches the implemented bindings. |
test/e2e/handheld-warning.spec.js |
Tier 3 (model-free): fakes a phone/desktop UA and asserts the "made for a computer" popup shows only on a handheld and dismisses, and that Phone Mic there first explains it pairs a phone with a computer (cancel backs out, the warning returns, confirming pairs). |
test/e2e/settings-watchdog.spec.js |
Tier 3 (model-free): startup must not hang when the settings IndexedDB never opens (a blocking versionchange in another tab). Stubs indexedDB.open to never settle and asserts the restore watchdog boots on defaults. |
test/e2e/settings-db-storeless-shell.spec.js |
Tier 3 (model-free): the app must recover from a settings DB that exists WITHOUT its object store (the empty shell a versionless indexedDB.open racing the first-boot purge leaves behind; every versioned open then skips onupgradeneeded and settings transactions threw NotFoundError forever). Plants the shell from a script-free same-origin page (/favicon.svg), then asserts the app boots, seedSettings completes, and a seeded value survives a reload (exercises the openIdb version-bump self-heal + the non-creating seed poll). |
test/e2e/med-mode.spec.js |
Tier 3 (model-free, seconds — the preset is settled before any weight is fetched): the French medical dictation preset through BOTH entry points, ?mode=med and the sidebar "Mode Dictée Médical" button, asserted to land on the same station. Checks each value separately (French UI, UltiMed repo, 30 s chunking, dictation display, auto-copy, int8 on the CPU and fp16 on the GPU, the loaded lexicon, boost knobs back at defaults) because a partly-applied preset is invisible: it still transcribes perfectly, just not as promised. Also pins the two contracts that are easy to break in opposite directions: the preset PERSISTS (survives a reload with no param, unlike ?model=), and an unrecognised ?mode= changes nothing at all. A last pair covers the measurement from both sides: with an adapter the app would actually use (one reporting shader-f16) and a hand-picked backend seeded, the link still MEASURES the machine, because the regression it covers was reported from a deployed station and left no trace (everything else about the preset landed while the backend stayed put, and backendUserPicked persists, so it was permanent on any machine anyone had touched the radios on); with an adapter that lacks the feature, nothing is measured at all, since fp16 is the only precision the app will put on a GPU unasked and the load would end up on the processor whatever the timings said. |
test/e2e/settings-url-reset.spec.js |
Tier 3 (model-free): the ?reset (and #reset fallback) URL escape hatch purges saved settings and boots on defaults, then strips the directive from the address bar. Recovery path when a persisted value wedges the app. |
test/e2e/benchmark-section.spec.js |
Tier 3 (loads real WASM int8 weights): drives the sidebar Benchmark section end to end — plans the matrix (no WebGPU row in headless, fp32 never pre-selected), runs the single wasm:int8 row for real, and asserts the produced parakeetweb-benchmark-report/1 JSON carries a genuine load/transcribe timing plus a similarity score against the shipped clip's known sentence. Also the privacy contract on a REAL probe: the raw report text must contain no user agent, time zone, languages, screen geometry, storage estimate or transcript. Finally the consent contract with uploading ENABLED and every POST intercepted: a finished run transmits nothing, and only the explicit button posts, byte for byte, the text the user was shown. |
test/e2e/support-report.spec.js |
Tier 3 (model-free, seconds): opens the sidebar Debug section and asserts the support-report textarea fills with valid parakeetweb-support-report/1 JSON describing the running browser. Chromium is the reference truth for the WASM probe byte-modules: simd and threads (COOP/COEP server, so crossOriginIsolated) must both read true, guarding the probes themselves against bit-rot. Also exercises the copy button end to end (granted clipboard permission, clipboard content parses back to the same format) and requires zero console errors. |
test/e2e/boost-default-source.spec.js |
Tier 3 (model-free): a curated phrase-boost list can be pre-selected via ?phrase_boost=<name> or the operator default, but NEITHER overrides a returning user's saved choice. |
test/e2e/boost-rebuild-on-status.spec.js |
Tier 3 regression: the phrase-boost trie rebuilds once per real model change, NOT on every status transition (which used to refreeze the UI on large curated lists). Counts [Boost] rebuilding trie logs across a full transcription. |
test/e2e/boost-prebuilt-offthread.spec.js |
Tier 3 (model-free) regression: the server-prebuilt boost encoding is fetched and parsed by phraseBoost.worker.js, not the page. Asserts the mechanism rather than a timing bound (a small fixture would not stall either way): a dedicated worker keeps its own performance timeline, so the list .txt must appear in the window's resource entries and its .json sibling must not, while still being requested. |
test/e2e/boost-unk-preview-before-model.spec.js |
Tier 3 (model-free) regression: the "untokenizable terms" warning for a curated list appears as soon as the list loads (from the prebuilt artifact's skipped), not only after a model is loaded. |
test/e2e/boost-applies-to-queued-capture.spec.js |
Tier 3 regression for the boost-trie race: a run that starts the instant the model turns ready (a capture queued during load; the queue drains in the tick that publishes the vocab signature) must decode WITH the configured phrase boost, not race the async trie rebuild and silently run boost-less. Delays the encoder fetch to queue sample.aac mid-load, then asserts the run's decode-debug summary counts boosted tokens (runTranscription awaits waitForBoostReady). |
test/e2e/boost-knobs-persist.spec.js |
Tier 3 (model-free): the advanced boost knobs (min-p gate override, depth scaling) restore from saved settings, hide while the phrase list is empty, and persist UI edits across a reload. |
test/e2e/beam-width-auto.spec.js |
Tier 3 (model-free): the auto-coupled beam width default (lib/beamWidth.js + its App.jsx wiring): greedy while no phrase list is loaded, the device-tier default once one is typed, back to greedy when it is cleared; an explicit width edit ends the coupling (hint gone, beamWidthAuto persisted false, boost state no longer moves it, survives reload); a legacy profile's persisted non-default width is honoured as a deliberate choice. |
test/e2e/boost-custom-slot-not-polluted.spec.js |
Tier 3 (model-free) regression: a curated list's text is never persisted under boostPhrases and never migrated into the user's editable "Custom" slot (only a saved Custom source seeds it). Stops a 75k-line lexicon from silently becoming the user's own text, which froze the sidebar for ~1 s every time Custom was selected or the section reopened. |
test/e2e/boost-large-custom-lazy-editor.spec.js |
Tier 3 (model-free): an oversized Custom phrase list is collapsed to a summary card (with "Edit as text" / "Clear") instead of being mounted in a textarea, the editor re-collapses on a source switch and on a section reopen, and an ordinary small list stays editable inline. |
test/e2e/seed.mjs |
Shared seedSettings(page, extra) helper: writes the app's settings IndexedDB so a spec boots with a known config (local WASM model source + spec-specific keys). Waits for the first boot to stamp version AND to run its default-persist storm (every usePersistedSetting re-writes its default the moment settingsLoaded flips), then writes the seed and re-writes it until a read-back holds across consecutive polls. Without that hold the seed was silently overwritten and the spec ran on DEFAULTS; pinned by seed-survives-first-boot.spec.js. |
test/e2e/routes.mjs |
Shared network-routing helpers for the "quant unavailable" specs: routeHfRepoListing() (serve a file set as the HF listing), abortHfDownloads(), routeNoLocalMirror() (404 the local /models probes), routeLocalMirrorWithoutGpuEncoders() (hide EVERY GPU-runnable encoder (the fp32 shards in both layouts, w4a8 and fp16), so the mirror can serve WASM but not WebGPU, which is the deployment the GPU-to-WASM fallback exists for; it 404s the shard bytes AND strips the same entries from the mirror's model-manifest.json, because hub.js believes a manifest over any probing, so 404-ing the files alone left the shards announced and the fallback under test with no reason to fire; hiding fp32 ALONE stopped being enough the day App.jsx learned to answer an unservable GPU precision with another GPU precision, since a mirror still serving w4a8 would then have kept the visitor on the GPU and the spec would have gone on passing while testing something else), and routeSyntheticLocalMirror() (state the mirror's whole file set outright instead of subtracting from whatever the box holds, which is what the model-free precision-radio spec needs to behave identically on CI and on a developer machine). Pinned by test/unit/routes-gpu-encoders.test.mjs, which asks quantSatisfiable about EVERY GPU precision rather than fp32 alone. The local route is a regex ANCHORED to the loopback origin: the old '**/models/**' glob also matched https://huggingface.co/api/models/..., and since Playwright resolves overlapping routes most-recently-registered-first it shadowed the HF listing route and 404'd it, so the specs tested "listing unreachable" instead of "listing lacks the variant". |
test/e2e/text-overlap.mjs |
Shared transcript-comparison helpers (words(), overlap(), and order/count-sensitive wer()) used by the transcription + chunking specs and the WER benches. |
test/e2e/model-probe.mjs |
Where a tier-3 spec looks for an OPTIONAL model file, given that the mirror can be laid out either way: <base>/<repo>/<path> (what scripts/fetch-e2e-models.mjs builds for CI, and what a mount serving several repos must use) or flat <base>/<path> (the single-repo LOCAL_MODEL_PATH contract, and what a maintainer's fallback_models checkout has historically been). modelProbeUrls(repo, basename) lists every candidate nested-first, probeModelUrl(request, repo, basename) HEAD-probes them and returns the one that answered, after first asking the mirror's model-manifest.json (scripts/model-manifest.mjs) if it has one: a present manifest is the WHOLE answer for that base, since hub.js returns it verbatim and never probes behind it, so a file it omits is one the app cannot load however many HEADs would find it. Mirrors hub.js resolveLocalModelBase on purpose, so the harness and the app agree on what counts as served. Holds no opinion of its own about directories: it maps candidatePaths() over both layouts, so a shard in sharded/ rather than fp32/ keeps resolving. The failure it prevents is silent and expensive: a probe that misses a file which IS present reads as "weights missing", which strict-weights.mjs turns into a local FAILURE blaming the checkout. Unit-tested in test/unit/model-probe.test.mjs. |
test/e2e/strict-weights.mjs |
Shared gate deciding whether a spec whose OPTIONAL weights (fp32 shards, diarization models) are not served should FAIL or self-skip. requireWeightsOrSkip(test, missing, msg) throws (fail) when strict, else calls test.skip. strictWeights(env) is strict by default on a maintainer checkout and lenient in CI (!env.CI), overridable with PARAKEET_E2E_STRICT_WEIGHTS. So the operator's local checkout (which is meant to serve every quant) cannot green on a silent skip, while CI (which fetches only the int8 + diarization set) keeps skipping the rest. The pre-push hook exports PARAKEET_E2E_STRICT_WEIGHTS=1. Pure logic, unit-tested in test/unit/strict-weights.test.mjs. |
test/e2e/pipeline-trouble.mjs |
The console patterns that mean an off-thread stage (encode pool, decode worker) failed, shared by transcription-parallel-encode.spec.js and transcription-composed-pipeline.spec.js. Both specs assert the absence of any trouble log, because every failure falls back in-thread and still yields a good transcript, so the log is the only evidence. The lists were copy-pasted into both specs and both copies missed the same string (workerReady logs [Encode] worker init failed, the patterns matched only [Encode] pool worker init failed), so a pool worker that timed out during init was invisible and the spec failed with "expected the marker" and no cause. One copy here plus test/unit/pipeline-trouble.test.mjs, which scans App.jsx and workerInit.js and fails if either grows an uncovered [Encode]/[Decode] warning. |
test/e2e/dangling-links.mjs |
Dangling-symlink walk serve.mjs runs before it listens. The served model dir follows the layout in app/src/modelLayout.js (each weight in the directory its precision names, vocab.txt and the preprocessor at the root), which is both the documented LOCAL_MODEL_PATH contract and the shape fetch-e2e-models.mjs builds in CI from three different repos, but a maintainer's ASR weights live in a nested folder that is its own git repo, bridged by symlinks. Rename or move that folder and every link dangles, which the harness would otherwise report as "weights missing", sending you after the wrong problem. partitionDangling splits findings by whether serving can reach them: a broken link is fatal when it sits somewhere serve.mjs would actually serve it from, which partitionDangling asks candidatePaths() rather than restating (the root, plus the layout directory and sharded/ for that basename), and anything deeper (the model repo's local candidates/ A/B farm) only warns. Kept out of serve.mjs because importing that starts a server. Unit-tested in test/unit/dangling-links.test.mjs. |
test/e2e/serve.mjs |
Static server for the E2E (serves the built UI + weights with the cross-origin-isolation headers ORT needs). PARAKEET_E2E_DIST_DIR overrides the served build (default app/ui/dist) so A/B harnesses can serve two builds side by side; unit-tested in test/unit/serve-dist-override.test.mjs. A request for model-manifest.json is answered from PARAKEET_E2E_MANIFEST_DIR (default <model dir>/.manifests, written by npm run e2e:manifest) before the model dir, mirroring the container, where Caddy serves it from a tmpfs because the mount is read-only: locally the model dir is a maintainer's symlinks INTO the model repo, which is a separate checkout and not ours to write a generated file into. Unit-tested in test/unit/serve-manifest-sidecar.test.mjs. It answers with a Content-Length and an ETag and honours Range (206 + Content-Range, If-Range, 416), which is not politeness: the app treats a length-less response as one whose size it cannot know and then skips buffer preallocation, byte progress AND the resumable prefix cache, so a harness that piped a length-less stream (as this one did) silently hid all three from every tier-3 spec. PORT=0 asks the OS for a free port and the startup line reports the bound one. Unit-tested in test/unit/serve-range-requests.test.mjs. |
test/e2e/playwright.config.js |
Playwright config that boots serve.mjs. |
test/support/bpe-fixture.mjs |
Loader for the BPE cross-check fixture. |
test/support/fake-indexeddb.mjs |
Minimal fake IndexedDB (open + get/put/delete/getAllKeys/clear, callbacks on a later turn) for the tier-1 tests that exercise hub.js's caching paths under Node. Instruments what type a value was stored AS and whether get() hands back the same object, which is what two of those tests are built to observe, and can fail writes on demand. |
test/support/load-browser-module.mjs |
Helper to unit-test browser files that attach to a bare window (evaluated in a vm context). |
test/fixtures/ |
Committed test inputs/goldens: bpe-fixture.json, sample.aac + sample.expected.txt (French clinical clip), jfk.mp3 + jfk.expected.txt (public-domain JFK English clip), jfk-moon-3min.mp3 + jfk-moon-3min.expected.txt (+ .meta.json provenance) for the long-audio chunking e2e, built by scripts/gen-jfk-moon-fixtures.mjs. Audio goldens are produced by the int8 weights via scripts/transcribe.mjs. repo-listings/ holds verbatim captures of the real HuggingFace file listings of the three REFERENCE model-repo layouts (ultimed.json the canonical v2 one, optimized.json v2 plus a complete nested sub-repo under istupakov_smoothquant/, istupakov-flat.json upstream's fully flat tree), which test/unit/repo-layout-detection.test.mjs resolves against. |
test/fixtures/fleurs/ |
FLEURS regression set built by scripts/gen-fleurs-fixtures.mjs: 10 en + 10 fr validation clips (mp3) + a stitched long clip, with manifest.json carrying each clip's human reference and int8 golden. |
| File | Role |
|---|---|
.githooks/pre-push |
Runs the fast tiers (unit + http) on every push, then (when a terminal is attached) offers two heavy opt-ins asked UP FRONT so the run is unattended: tier 3 (Playwright WASM E2E, rebuilds dist first) and scripts/webgpu-check.mjs on a real GPU (--fp32 by default; override via PREPUSH_WEBGPU_ARGS). A webgpu-check exit-2 (no GPU) is a SKIP, not a failure. No terminal (CI) skips both. Activated by the root package.json prepare script. |
.github/workflows/test.yml |
PR CI gate: mirrors the pre-push fast tiers and adds tier-3 E2E as a separate job. |