[botandbang]

What and why

A set destination plugin is handed a set's recording (often 100 MB to 2 GB for a DJ mix) to upload, but net/fetch could only send a file from the request's scratch folder, and the host read it into memory first. This lets a plugin send the recording, part of it, or a form with it, while the host reads it from disk as it goes, and shows the upload's progress.

Added:

• body_range {offset, length?} on net/fetch (crates/loclib-plugin/src/protocol.rs) sends part of a body_blob, for uploads in parts. A range past the file's end is refused before anything is sent.
• multipart on net/fetch sends a multipart/form-data form: {name, value} text parts (with {secret:} filled in) and {name, file, range?, filename?, content_type?} file parts. The host makes the boundary and the Content-Type.
• crates/locallibrary/src/plugin/net/body.rs builds the body: text, a file or part of one, or a form through reqwest's streaming multipart support (its multipart feature is now on). Files are read as they're sent with an exact Content-Length, opened again when a 307 or 308 resends the body, and sending stops at the next read once the fetch is cancelled. A request that sends files gets 30 s plus the time 128 KiB/s would take, instead of a flat 30 s.
• Phase::Upload in crates/loclib-progress, shown as "Uploading" in the CLI and "uploading" in the TUI. The host reports the bytes it sends as that phase's progress, in order with the plugin's own phase_start and phase_end (Pending::reporter() in crates/loclib-plugin/src/peer.rs).
• Slot::call_lending (crates/locallibrary/src/plugin/process.rs): publishing a set lends the hook its recording, so net/fetch may send it while the hook runs. The dry run lends nothing.

Changed:

• Scratches (crates/locallibrary/src/plugin/blobs.rs) tracks each hook call under way: its scratch folder, the files lent to it, and where its upload progress goes. A body or form file may be a lent file or one in a scratch folder under way, nothing else.
• net/fetch ignores a Content-Length the plugin sets, and for a form, its Content-Type.
• ProgressEvent is read by hand. Its own items count had been taking a phase_start's items while it was read, so the host skipped every plugin's phase start as an unknown event and a plugin's phase never showed. This is its own commit.
• apps/locallibrary/docs/rpc/ is regenerated for the new phase. crates/loclib-plugin/README.md, apps/locallibrary/docs/plugins.md and the CHANGELOG describe the new fields and what a plugin may send.

Removed:

• net/fetch no longer reads a body_blob into memory before sending it.

The fields are additive and protocol 0.2 hasn't been released, so the version stays. Once it is, a change like this needs a feature flag: Fetch accepts unknown fields, so an older host would send a whole file where a plugin asked for a range.

How it was tested

• crates/locallibrary/tests/it/plugins/net.rs: ranges as a Hegel property over any size, offset and length, plus the edges (offset 0, a last partial part with and without a length, past the end, an empty part at the end); forms as a Hegel property on the exact bytes, boundary and Content-Length; redirects that resend a file (307, 308) or drop it (303); six new refusals in the wrong-request property.
• crates/locallibrary/tests/it/plugins/destinations.rs: a recording sent in parts while a set is published, with each part's bytes and Content-Length and progress going phase start, bytes summing to the file, phase end; refused in the dry run and once publishing is done; a 32 MiB upload to a slow server stopped partway by a cancel.
• crates/loclib-plugin/src/protocol/tests.rs: a phase start keeps its item count.
• Each new test was checked by breaking the code it covers.
• Local: cargo fmt --check, workspace clippy with -D warnings (and x86_64-pc-windows-gnu for the touched crates), nextest for the touched packages, and coverage over loclib-plugin, locallibrary and loclib-progress (lines 97.39, functions 95.35, regions 94.91).

Checklist

• Tests cover the change
• For the website: Storybook stories cover any new UI states
• For locallibrary: apps/locallibrary/CHANGELOG.md has a line under Unreleased, if people using it will notice
• The READMEs, docs/ and --help still say what the code does (apps/locallibrary/docs/rpc/ too, for a sidecar change)

[github-actions]

Size

Compared with main. KiB are 1,024 bytes.

	This pull request	main	Change
Client JavaScript	619.2 KiB	619.2 KiB	0 B (0%)
Client JavaScript, brotli	204.5 KiB	204.5 KiB	0 B (0%)
Client CSS	87.6 KiB	87.6 KiB	0 B (0%)
Client CSS, brotli	12.5 KiB	12.5 KiB	0 B (0%)

[github-actions]

Rust coverage

From run 471. Both are held to the thresholds in .cargo/config.toml.

Tests	Lines	Functions	Regions
All	97.48% (79487/81541)	96.64% (12674/13114)	95.78% (174608/182300)
Hegel property tests alone	95.45% (77833/81541)	93.95% (12320/13114)	93.5% (170444/182300)