Skip to content

Play MIDI through a SoundFont synth with one MuseScore General bank - #203

Merged
ctoth merged 3 commits into
masterfrom
feat/sf2-synth
Oct 21, 2026
Merged

ctoth merged 3 commits into
masterfrom
feat/sf2-synth

Conversation

@ctoth

@ctoth ctoth commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Replaces the MIDI.js sample renderer with a SoundFont synthesizer (spessasynth_lib) that plays one bank, MuseScore General (MIT), and adds the script that builds that bank.

Client

  • src/midi/SoundfontOutput.ts is now a WorkletSynthesizer behind the same MidiOutputSink (send, sendAt, close).
    • ready() (:36) checks that the client audio output exists. It then loads the worklet with addModule from a Vite ?url import of spessasynth_lib/dist/spessasynth_processor.min.js (:2, :39), creates the synth on virtualMidiService.getAudioOutput().context and connects it to that output's destination (:41-42).
    • The bank downloads in the background from https://mongoose.world/sounds/soundfonts/MuseScore_General.sf3, with credentials: 'omit' (:49).
    • Until it has loaded, every message goes to the built-in synth (:87). Program, controller and pitch-bend messages sent in that time are kept, the latest per channel and controller, and replayed into the synth once the bank loads (:59).
    • After the switch, note-offs and CC 64/120/123 also go to the built-in synth, so notes it started still end (:93).
    • sendAt(data, s) calls sendMessage(data, 0, { time: context.currentTime + s }) (:95).
    • addSoundBank never resolves on a bank the worklet rejects; the worklet posts soundBankError instead (lib basic_synthesizer_core.ts:386-391). So the load also listens for that event (:55-56).
    • If the bank fails to load, the built-in synth stays in use and the status says so.
    • close() aborts the download, destroys the synth and closes the built-in synth (:110-115).
  • src/midi/soundfonts.ts is down to SOUNDFONT_OUTPUT (device id unchanged) and SOUNDFONT_NAME.
  • Removed: the three MIDI.js collections, src/midi/programs.ts, SoundfontPicker.tsx and .css, and the picker in preferences.tsx.
  • The MIDI tab's sampled-instruments card now describes the one bank and links the MuseScore General license (MidiStatus.tsx:599-611).
  • src/stores/preferencesStore.ts: removed the midiJsSoundfont field and its normalisation (:174). There is no migration and no STORAGE_VERSION change; a leftover midiJsSoundfont key in saved data is ignored. The enabled default, STORAGE_VERSION 3 and the migration from Make MIDI automatic: on by default, output connects on the first MOO message #202 are untouched (this branch merges master at f955956).
  • MidiService.ts:294,300,314: new constructor, and the device name is now Soundfont Synthesizer · MuseScore General.
  • vite.config.ts:37: the worklet file is a ?url asset, so optionalChunks (which only drops chunks, optionalChunks.ts:33) can't keep it out of the **/*.js precache. A globIgnores entry keeps it out of the service-worker install. It is 402,596 bytes.
  • package.json pins spessasynth_core 4.3.22, spessasynth_lib 4.3.14 and stb-vorbis 0.0.6 exactly. Both libs declare their own dependency as "latest"; npm ls shows both nested dependencies deduped onto these pins.
  • scripts/check-soundfonts.mjs (run in CI by test:audio-cache) now serves core's one-preset sample bank (BasicSoundBank.getSampleSoundBankFile(), :30) at the bank URL (:31, :38). The first request returns 503 and the check expects "MuseScore General could not load". After Reconnect it expects "MuseScore General ready." The collection loop is gone. I have not run this script locally; CI runs it.

Bank build: scripts/build-soundbank.mjs

node scripts/build-soundbank.mjs <outDir> [--source-dir <dir>] [--quality <q>]. Needs ffmpeg with libvorbis. It writes to <outDir>:

  • MuseScore_General.flac.sf3, the FLAC master: core's setSampleFormat({format:"compressed"}) with ffmpeg FLAC, written to a temp file so STREAMINFO is filled in (:216-218). This is SF3-shaped with FLAC payloads, for rebuilding, not playback.
  • MuseScore_General.sf3: ffmpeg libvorbis at each sample's own rate, quality 6 (:247-249).
  • MuseScore_General_License.md, copied from the upstream directory (:344).

Gates (any failure exits 1):

  • every sample really compressed (core's compressSample swallows encoder errors);
  • the FLAC master decodes byte-identical to the source PCM;
  • the structure of both outputs (presets, zones, generators, modulators, samples, loop points) matches the source;
  • every SF3 sample's stb-vorbis decoded length equals the source length.

A longer decode is allowed only when both of these hold (loopOnlyModes :171, alignment :288):

  • offset 0 has the lowest RMS error of all shifts in [-64, 64];
  • every instrument zone using the sample has sampleModes & 3 == 1 and generators 1/12/3/50 all 0.

This covers an stb-vorbis 0.0.6 final-frame trim bug (wasm/stb_vorbis.c:3413-3443): ffmpeg's last granule and ffmpeg's own decode both equal the source length for the three samples below.

Run on MuseScore_General.sf2 0.2.0 (215,614,036 bytes), exit 0:

source: MuseScore_General.sf2 215614036 bytes, 309 presets, 204 instruments, 1246 samples
GATE PASS: every sample FLAC-encoded
GATE PASS: FLAC master decodes byte-identical to source PCM (1246/1246 samples identical)
GATE PASS: FLAC master structure matches source
GATE PASS: every sample Vorbis-encoded
GATE PASS: SF3 presets, generators, modulators and loop points match source
  allowed longer decode: sample 79 Cello-G#1: source 37067, decoded 37376, rms at offset 0 -45.5 dBFS, best other offset 1 -32.3 dBFS, zones: Cello_F/Cello_M/Cello_P mode 1 offsets 0/0/0/0
  allowed longer decode: sample 80 Cello-F1: source 33493, decoded 33920, rms at offset 0 -46.2 dBFS, best other offset 1 -34.1 dBFS, zones: Cello_F/Cello_M/Cello_P mode 1 offsets 0/0/0/0
  allowed longer decode: sample 599 OcarE1: source 60416, decoded 60544, rms at offset 0 -43.9 dBFS, best other offset -1 -34.1 dBFS, zones: Ocarina mode 1 offsets 0/0/0/0
GATE PASS: every SF3 sample decodes (stb-vorbis) to the source length, or longer only where it is aligned and never played past the loop end (1246/1246 samples, 3 allowed longer)
MuseScore_General.flac.sf3: 95679054 bytes (91.25 MiB)
MuseScore_General.sf3: 32986146 bytes (31.46 MiB)
all gates passed

The structure counts all match (modulators 1769, samples 1246, loop points 1246, instruments 204, zones 3859, generators 11854, presets 309). The worst whole-sample Vorbis error is Applause at -29.8 dBFS RMS; the worst within 64 samples of a loop point is Applause at -32.3 dBFS.

The bank is not uploaded by this PR. Until MuseScore_General.sf3 is at the URL above, Soundfont Synthesizer stays on the built-in synth and says the bank could not load.

Line endings

src/MidiService.ts had mixed CRLF and LF line endings on master. Editing it rewrote every line as CRLF, so the diff shows about 160 changed lines. git diff --ignore-cr-at-eol shows the real change is 9 lines.

Tests

src/midi/SoundfontOutput.test.ts is rewritten against a mocked spessasynth_lib: it failed 8 of 8 against the old renderer and passes 8 of 8 now. After the merge, SoundfontOutput.test.ts and preferencesStore.test.ts pass 26 of 26. I ran only those two files; CI runs the rest.

ctoth added 3 commits October 20, 2026 20:38
Replace the MIDI.js sample renderer with a spessasynth_lib WorkletSynthesizer
that renders into the client's audio output and plays
https://mongoose.world/sounds/soundfonts/MuseScore_General.sf3. The built-in
synth plays until the bank loads. scripts/build-soundbank.mjs builds that bank
(and a lossless FLAC master) from MuseScore_General.sf2, with gates.

Remove the three MIDI.js collections, the font picker and the midiJsSoundfont
preference. Pin spessasynth_core 4.3.22, spessasynth_lib 4.3.14, stb-vorbis 0.0.6.
# Conflicts:
#	src/stores/preferencesStore.test.ts
After a successful Connect, handleConnectOutput calls loadDevices, whose
updateReconnectableDevices read connectionState from that handler's render:
still disconnected, while the service was connected and had just saved the
device as lastOutputDeviceId. So it connected the same output a second time.
With the SoundFont output that meant a second bank download. Read the live
midiService.connectionStatus instead.
@ctoth
ctoth merged commit 000c4b1 into master Oct 21, 2026
2 checks passed
@ctoth
ctoth deleted the feat/sf2-synth branch October 21, 2026 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant