Storage backends: dumb-store WIT, the cryptographic pull layer, and the S3-compatible provider spike

[lannbot]

Design recorded in NOTES.md — Storage backends and the cryptographic pull layer (from the 2026-08-16 design discussion; leaning, not ruling). Summary of what this issue tracks:

The design

• Backend contract: authenticated owner PUT/DELETE + GET by unguessable name. No ACLs, no LIST, no conditional writes (content-addressed chunks; per-device signed head manifests; readers merge). Admits S3-anything, consumer drives as dumb stores, static hosts, CDNs, IPFS.
• Cryptographic pull layer: per (doc × epoch) name-keys; object name = HMAC(name-key, cref); optional outer AEAD hiding keyhive envelope metadata. Name-keys travel over the E2E channel as capabilities; Access::Relay ≈ name-key possession, uniform with the realtime pull tier. Read keys stay in keyhive's op stream (also stored as blobs).
• Cooperative fetch revocation (K_p): per-recipient-device wrapped name-key at a pairwise-derived location, deleted on revocation ingest; honest clients never persist pull-layer material. Four-layer revocation: BeeKEM rotation (read, hard) / name-key rotation (pull-forward, hard) / K_p deletion (pull-now, cooperative) / compaction relocation (pull-past, hard, eventual) — plus storage-credential rotation for the owner's own devices.
• Motivating scenario: stolen device cracked offline after revocation — compromise narrows from everything the device could reach to everything it had reached; the layers act at revocation time, independent of when the crack lands.
• Accepted losses (threat-model entries): no retroactive fetch-denial against modified clients until relocation; HNDL surface widens to name-holders (bounded by rotation; expiring-URL minting as leak hygiene where cheap); egress abuse on paid-egress backends (private buckets + minted URLs there); provider sees recipient counts via K_p objects.

Deliverables

• polymorph:blobstore-shaped WIT: core dumb-blob interface (put/get/delete by name); optional share-mint (expiring URL) capability interface, class-D-style declared
• Pull layer in the framework core (backend-independent): name-key derivation, per-device manifests, K_p wrap/pickup/delete, rotation hooks into the existing revocation path, relocation folded into compaction
• S3-compatible provider spike (first): SigV4 via polymorph:webcrypto HMAC in-guest, network grant scoped to the backend host; R2 + B2 as documented defaults, MinIO for self-host; scenario = two devices converge through the bucket, a recipient fetches via name-keys with no backend account, K_p deletion goes dark for a stock client
• Google Drive provider as a dumb store (fast follow): drive.file scope, appDataFolder; start the OAuth-verification clock early (wall-time dependency)
• Provider-config conformance notes: object-versioning off for the K_p prefix (non-load-bearing; see NOTES), CORS recipe per backend
• Recovery-bundle placement (Key lifecycle: disposable device keys, recoverable data root (recovery phrase / WebAuthn PRF), last-device loss #11): public-fetch mode, name + KEK from the recovery phrase
• Threat-model entries (Threat model: the adversary table and the "nothing is both live and trusted" invariant #1): cooperative-revocation classification, HNDL widening, egress abuse, provider metadata (incl. recipient counts)

Related: #8 (data services; the storage path is its non-realtime half), #11 (recovery bundle), #1 (threat model), #6/#7 (provider confinement is the dogfooded plugin model).

[lannbot]

The provider spike is executed and passing (commit 297aab0, spikes/storage/, just run — MinIO fetched once, run as a user process).

Green, in order: SigV4 signed in-guest via polymorph:webcrypto against real MinIO (bucket create, policy put, object put/delete); the dumb-store contract confirmed sufficient — unsigned LIST and PUT probed and refused, public read per-object via an in-guest bucket policy; device convergence through the bucket (signed per-device manifests); an account-less recipient reading via K_p → name-keys → readkeys → manifests → chunks, all unsigned GETs; and the stolen-device scenario as an executable assertion: a cracked-image resurrection reads successfully before revocation (image sufficiency — the darkness is caused by the mechanism, not missing data) and retrieves nothing after it (K_p 404, no derivable names), while the owner's second device rides the epoch rotation. The no-persist discipline is structural: all pull-layer material in read-shared is function-local, so the cracked image is exactly an honest client's disk.

Numbers: recipient session ~1.8 ms (4 unsigned GETs + derivations), revoke (delete K_p + rotate + republish) ~2.7 ms, 12 HTTP requests per party for the whole scenario.

Findings:

• Runtime-version isolation via composition: wasip3's wasi:http client pins wit-bindgen 0.57; webcrypto-guest pins 0.59; they cannot share a component — so the HTTP client is its own fetcher component composed via wac plug, which also makes the fetch import the exact seam where the per-destination network grant attaches. Zero friction across the boundary. This is the standing pattern for any dependency dragging its own bindings runtime.
• wasi:http@0.3 bodies stream chunked: MinIO's PutBucketPolicy demands Content-Length (fixed with an explicit header from the buffered fetcher) and its CreateBucket dislikes bodyless requests (fixed with an explicit CreateBucketConfiguration). Both worth carrying into R2/B2 interop testing.
• First provider-checker entries validated: public-read-no-list policy + the two refusal probes.

Remaining on this issue: R2/B2 against the same component, TLS endpoints, share-mint (expiring URLs as leak hygiene), the real keyhive op-stream/device-group integration (this spike's readkeys/epoch blobs are labeled stand-ins), relocation/GC, credential rotation, and the Drive-as-dumb-store provider.

[lannbot]

Generalizing the provider contract (design discussion 2026-08-17, ahead of the Drive fast-follow): digging into "Drive as a dumb store" surfaced that the interesting variation across consumer backends is not naming quirks — it is which pull-tier mechanics each backend can enforce. Recording the requirements shape here before the next provider dig. All of this is pull-tier only; BeeKEM read crypto is untouched in every variant.

Drive facts verified against current docs

• Shortcuts do not carry access: a shortcut's ACL inherits from the folder the shortcut sits in, and the shortcut breaks when the holder loses access to the target. Per-peer folders of shortcuts grant nothing; "linked contents" can only ever be a discovery index.
• Folder access is expansive and inheritance is live: everyone with folder access reaches all items inside; moving an item out drops the inherited grant. So Drive has revocation-by-reparenting — one metadata op, no re-upload.
• Scope tiers: drive.file (+ Google Picker grants) and drive.appdata are non-sensitive; drive/drive.readonly are restricted (CASA security assessment). Recipient-side API visibility under drive.file requires picker ceremonies; whether a picked folder covers dynamically added future children is undocumented (empirical).
• No client-chosen addresses: fileIds are server-assigned; anonymous GET-by-derived-name does not exist (resolution requires an authenticated list or a listable folder).

Three Drive shapes fell out

• GAS adapter: an Apps Script web app ("execute as: me", access: anyone) as a personal dumb-store endpoint over Drive quota. Restores GET-by-derived-name exactly (the script resolves names privately); no OAuth client anywhere, which kills the verification-clock dependency. Costs: a deploy ceremony, base64 framing (ContentService has no binary), GAS quotas/latency, platform quirks (302 to googleusercontent, no OPTIONS preflight — auth in query/body, not headers).
• D1 — account ACLs: per-doc content folder; grant = permissions.create(type=user, role=reader), revoke = permissions.delete. The only shape with hard retroactive fetch denial against modified clients; no K_p machinery needed on this backend. Costs: recipients need Google accounts + OAuth'd clients + picker ceremonies; Google sees the collaboration graph (a real metadata widening vs anonymous GETs); the pull tier forks per backend.
• D2 — link-shared folders as capabilities: per-doc folder shared type=anyone; the unguessable folderId travels E2E exactly like a name-key; recipients stay account-less (anonymous fetch needs an API key — CORS to verify). Revocation = new folder + move contents: old fileIds die via inheritance (hard pull-past in one metadata op). K_p generalizes to a stable pickup file updated in place, deleted on revoke. Leak class identical to name secrecy (bearer identifiers).

The generalization: a backend capability profile

• N: client-chosen names + overwrite-in-place at a stable address
• A: anonymous fetch under a container capability (no recipient account)
• D: derivable addresses (offline HMAC names resolve)
• R: server-revocable bearer capability
• I: per-identity grants (recipient account required)
• E: expiring minted URLs (leak hygiene)
• L: owner-only listing (GC/compaction)

	S3-class	GAS	Drive	OneDrive	Dropbox
N	yes	yes	no (emulable)	yes (paths)	yes (paths)
A	yes (public-read policy + name secrecy)	yes	partial (link + list indirection + API key)	yes? (folder link)	yes? (folder link)
D	yes	yes	no	yes? (link + path)	yes? (link + path)
R	no	self-impl.	yes (perm delete / move-out)	yes (link delete)	yes (link revoke)
I	no	no	yes	yes	yes
E	yes (presigned)	self-impl.	no	yes (downloadUrl)	paid tier
L	yes	yes	yes	yes	yes

("?" = needs an empirical probe run. S3 column spike-verified; Drive column doc-verified 2026-08-17; OneDrive/Dropbox from API-doc knowledge, unverified.)

Interface consequences (feeds the polymorph:blobstore WIT deliverable)

1. Required floor: authenticated put/delete + overwrite-in-place at a stable address (stable pickup objects depend on it; all five backends have it), fetch-under-capability, owner-only listing for GC. Backends without N emulate name resolution provider-side. Capability holders never need listing — and within-container listing by holders is harmless by construction (everything in a per-doc×epoch container shares one readership), so the S3 no-list policy is a per-strategy requirement, not a floor requirement.
2. The pull tier becomes a strategy; the E2E-travelling capability becomes a tagged union (s3-name-secrecy{endpoint, bucket, name-key} | share-link{url} | drive-folder{id} | ...). Name-secrecy + K_p is the floor strategy for R-less backends; where R exists, revoke-pull maps to native link/permission deletion.
3. Operations report their guarantee class (hard/cooperative × immediate/eventual). The four-layer revocation model keeps one semantics with backend-varying strength: on R-capable backends, pull-now upgrades cooperative→hard and pull-past upgrades eventual-re-upload→immediate-metadata-op. The NOTES rule "UX must not imply hard denial" becomes machine-carried metadata instead of prose.
4. K_p generalizes to "stable pickup object"; on link backends its transport is a per-recipient revocable link, so even pickup revocation is hard there.
5. Per-identity grants (I) stay a declared optional extension (recipient accounts, provider-visible social graph, identity plumbing in contact cards) — never engine-required.
6. Recovery bundle (Key lifecycle: disposable device keys, recoverable data root (recovery phrase / WebAuthn PRF), last-device loss #11) requires class D specifically: a recovery phrase can derive a name, never a link. Link-only backends fatten the recovery kit to phrase + printed link.
7. Confinement finding: pre-authenticated download hosts differ from API hosts (OneDrive downloadUrl, Dropbox usercontent) — per-destination network grants must enumerate them per provider; conformance-checker entries.

Provider-order update

OneDrive/Dropbox look easier than Drive (path-addressable + revocable anonymous links ⇒ derived names work under a folder-link capability; no GAS shim; app-folder scopes on both sides no picker drama for owners). Drive is the odd one out, not the template. Next: OneDrive dig, with its empirical probe list (anonymous fetch mechanics under /shares, CORS on download hosts, link-permission semantics on move, app-folder × createLink interaction, tenant-policy interference on business accounts).

[lannbot]

Dropbox probe run executed 2026-08-17 — every decision-critical fact confirmed against the live API (free "basic" account, app-folder app, scoped files.content.* + sharing.*; ad-hoc HTTP probes, ~20 requests total). Follow-up to the capability-profile comment above; this validates the Dropbox column and hardens the general interface claims.

Confirmed, in probe order

• N (client-chosen names + overwrite-in-place): files/upload at arbitrary paths, parent folders created implicitly, mode=overwrite keeps the path stable. ~1.2 s/op (consumer-API latency class; MinIO was ~2 ms — non-realtime path only, as designed).
• Container capability: create_shared_link_with_settings on a folder → public /scl/fo/...?rlkey=... link (two unguessable components), can_revoke: true.
• A + D (the decisive one): account-less recipient fetch by derived relative name works. get_shared_link_file with app auth only (Basic key:secret, no user token, no recipient account) + folder link + path=/chunk-....bin → bytes. Fetch resolves paths at fetch time (served post-overwrite content). Fully unauthenticated is refused (401) — the app key is the anonymous-fetch mediator, same class as a Google API key but spec-documented (auth = "app, user"). Unknown names → 409 shared_link_access_denied (indistinguishable from denial; no existence oracle).
• R (hard revocation, immediate and retroactive): revoke_shared_link → the fetch that succeeded moments earlier returns 409 0.7 s later. Re-minting on the same folder yields a different URL. So on Dropbox the pull-revocation layers collapse as predicted: pull-now + pull-past = one revoke (hard, works against modified clients, no K_p cooperation, no relocation/re-upload); pull-forward = re-mint + distribute via pickup objects. The S3 design needs three mechanisms to approximate this.
• Ancestor-link rule, as an executable assertion: a link minted on the parent folder serves the child's content even while the child's own epoch link is revoked (per the revoke_shared_link docstring). Layout rule: mint links only on leaf containers (per-doc×epoch folders, per-recipient pickup folders/files), never on the store root or intermediate dirs. Provider-conformance-checker entry.
• Pickup-object pattern: a file-level link survives mode=overwrite and serves the new content — per-recipient pickup objects work directly as file links (stable capability, rotate contents in place, revoke per-recipient).
• E gating: expires setting on free tier → settings_error/not_authorized (paid feature, as expected). Expiring-URL hygiene is unavailable on free Dropbox; revocability substitutes.
• CORS (browser recipient path): content.dropboxapi.com preflights cleanly (reflects Origin; allows Authorization + Dropbox-API-Arg; exposes Dropbox-API-Result) — the app-auth fetch works from a browser. Nuance: the tokenless ?dl=1 file-link path (which works credential-free in native contexts, redirecting to *.dl.dropboxusercontent.com, ACAO *) is CORS-broken at hop 1 — the www.dropbox.com 302 carries no ACAO header. Browser recipients use the API path; tokenless is a native/CLI nicety only.
• Tokenless folder links yield a whole-folder zip (dl=1), not per-child access — consistent with the API being the per-child path.
• Network-grant enumeration for the provider component: api.dropboxapi.com, content.dropboxapi.com (+ www.dropbox.com and *.dl.dropboxusercontent.com only if the tokenless path is ever used).

Consequences

• Dropbox is the strongest consumer-drive fit for the contract and the cheapest to start: instant app creation, no verification clock for development (production review is a lightweight form; 500-user dev cap until then), console-minted tokens for spikes. Weak spots: 2 GB free quota; app-secret-in-client degrades to a public identifier when shipped in a PWA (threat-model note: it mediates anonymous fetch quota/abuse, not confidentiality).
• Matrix row finalized: N yes · A yes (app-key-mediated) · D yes · R yes (hard, ~1 s) · I yes (unused: mounts consume recipient quota) · E paid · L yes.
• The interface claims from the previous comment that this run executes: capability-as-tagged-union (share-link{url} variant), guarantee-class metadata (Dropbox reports hard/immediate where S3 reports cooperative/eventual), leaf-only link minting as a conformance rule, pickup objects as overwrite-in-place stable addresses.

Remaining for a proper spikes/dropbox/ (per repo convention): script these probes as executable assertions, then the provider component behind the same fetch seam as the S3 spike (SigV4 ladder replaced by bearer/Basic headers — strictly simpler), and the full storage-spike scenario (two devices, account-less recipient, revocation) against live Dropbox. OneDrive remains attractive for its redeemSharingLink durable-grant mechanism (no picker, no owner-side identity management) but is gated on Azure signup friction; its Stage-0 tokenless probe needs only a hand-minted share link.

[lannbot]

Dropbox spike executed 2026-08-17 and passed — PR #24 (spikes/dropbox/, just probe + just run against live consumer Dropbox), plus the first cut of the polymorph:blobstore WIT draft (wit/blobstore.wit). This executes the link-capability strategy from the capability-profile comment above.

Green, in order: the probe suite — the earlier ad-hoc probes scripted as 27 executable assertions (path-addressed writes, overwrite-in-place, app-auth account-less child fetch under a folder link, hard ~600 ms revocation, fresh-URL re-mint, the ancestor-link rule, no existence oracle, free-tier expiring-link gating, CORS posture). The component spike — same two-component wac plug architecture as the S3 spike, first run over real TLS (wasmtime-wasi-http p3's rustls path, zero host changes): owner devices converge on the Bearer tier with the pull tier not involved at all; two account-less recipients read via pickup link → sealed container link → readkeys → signed manifests → chunks; revocation goes dark for the stock client in 458 ms and for a cracked image in 880 ms — including through the container link the image deliberately hoarded (a labeled no-persist violation modeling the maximally dishonest client). That is the assertion the S3 name-secrecy strategy structurally cannot make: its equivalent hoard keeps fetching until relocation. The remaining recipient rides the rotation through the same standing pickup link she was granted at the start (pickup object overwritten in place; file links survive overwrite); the owner's second device rides it on the Bearer tier. ~50 s wall for the full scenario; owner 23 / recipient 12 HTTP requests.

Findings for the running lists:

• The four-layer revocation model collapses on link backends: pull-now + pull-past = one revoke_shared_link (hard, immediate, against modified clients); pull-forward = re-mint on the same folder. Zero data movement; relocation/compaction is a no-op for revocation purposes on this backend class. Name-keys stop paying rent (plain derivable paths under the capability; within-container listing by holders is harmless — one readership per container).
• Leaf-only link minting is a provider-conformance rule with an executable assertion behind it (ancestor links out-survive child revocation).
• Assert refusal classes, not status codes: the same bad-auth write was refused 401 and 400 on consecutive live runs.
• App secret = public identifier once shipped in a client: it mediates anonymous-fetch quota/abuse, never confidentiality (threat-model entry, same class as a Drive API key).
• Consumer-API latency class is ~1.2–3 s/op (vs ~2 ms on MinIO) — comfortably the non-realtime path.
• Not validated here: refresh-token flow (console tokens live ~4 h), 429 behavior under fan-out, >150 MB upload sessions, the production-approval path past 500 users.

The WIT draft carries the generalization: required floor (store: put/delete with overwrite-in-place at stable addresses, owner get, owner-only list-names) + profiled pull (mint/revoke/rotate/relocate) + recipient-side pull-fetch + profile. The two probe-hardened design points are in the types: pull-capability is a tagged union (name-secrecy | share-link), and revocation-shaped operations return a guarantee record (hard/cooperative × immediate/eventual) so the NOTES rule "the UX must not imply hard denial" is machine-carried. Structural consequence worth review: the name-secrecy strategy is a framework-core component implementing pull/pull-fetch above any floor-only provider — two worlds (provider, capable-provider), one pull-tier semantics, strategy chosen by profile. Status: leaning, not ruling; the OneDrive dig and the K_p/keyhive integration should bend it before anything pins.

Remaining on this issue: OneDrive probe (Stage 0 needs only a hand-minted share link; the redeemSharingLink durable-grant path and the AppFolder×createLink scope tension are the open questions), the Drive decision (GAS adapter vs link-folder designs, recorded above), R2/B2 against the S3 component, refresh-token lifecycle in the provider surface, and folding the guarantee-class vocabulary into the #1 threat-model entries.

[lannbot]

Both storage strategies now run under the engine, in the browser (PR #24, second commit) — the provider contract from the capability-profile comment above stops being a document and becomes a variant in the engine's WIT.

The engine surface, generalized

variant store-config { s3(s3-config), dropbox(dropbox-config) }
init-store:   async func(config: store-config) -> result<_, string>;
store-grant:  async func(doc-id: list<u8>, member-id: list<u8>) -> result<option<string>, string>;
store-revoke: async func(doc-id: list<u8>, member-id: list<u8>) -> result<string, string>;
bucket-pull:  async func(doc-id: list<u8>, owner-id: list<u8>, pickup: option<string>) -> result<string, string>;

Three shapes here are the blobstore draft leaking into practice, and all three earned their place: store-grant returns an optional pull capability (none under name secrecy — the K_p location is derivable; the minted pickup link under Dropbox — it is not); store-revoke returns its guarantee note as prose ("K_p deleted (cooperative now); name-key epoch rotated (hard forward)" vs "revoked server-side (hard, retroactive); container link re-minted") so the UI can be honest without knowing the provider; bucket-pull takes an optional pickup link so a link-tier recipient — no storage account, no token — pulls at all. The S3 path is behaviorally unchanged and the full MinIO+relay scenario still passes (grant/revoke/flush/pull, identity-bundle restart, 8-task convergence).

The Dropbox strategy is the spike's, ported: plain derivable paths under a per-doc folder link, pickup objects as per-member sealed files with their own revocable links, oplog/manifest/chunk contents byte-identical to the S3 path (only addressing and transport differ), plus refresh-on-401 for the token. Owner-tier reads list the doc folder for the device set; link-tier reads ride the pickup link with app auth.

Verified live in the page (real Dropbox, free account)

Three replicas converge over Dropbox + iroh; the tablet cold-boots from the bucket with iroh conns: 0; a collaborator pulls the bucket through his standing pickup link under app auth alone (pulled dropbox(link) devices=3 events=71); after Revoke Bob, the same button reports pickup link refused (409): revoked or never granted — hard, provider-enforced — while that same peer holds live-wire ciphertext he cannot decrypt (undecryptable: 1). Both exclusions, both tiers, in one UI: the crypto tier and the pull tier failing differently and visibly.

Timings in-page (consumer API, non-realtime path by design): grant ~1.5–2 s, flush ~1.5 s, link-tier pull ~2 s, revoke ~5 s (five API calls: revoke pickup link, delete pickup, revoke container link, re-mint, rewrite remaining pickups).

Notes for the WIT draft

• The guarantee class currently crosses as prose, which is right for a spike and wrong for the framework — wit/blobstore.wit already models it as a record; the demo shows where it lands (a status line the user reads at the moment of revoking), which argues for the record plus a provider-supplied human string, not one or the other.
• The demo's storage config is a provider-opaque blob chrome carries from panel to engine (see the Framework chrome: trusted code, position+secret authentication, input suspension - the shell's own UI #22 comment). That keeps chrome provider-agnostic, but an opaque blob is precisely what a compound consent prompt cannot describe — a real tension for Permission model: linking as enforcement - deny-by-unlink, prompt-on-first-use, revocation semantics, durable grants #6/Network capabilities: per-destination grants, the app manifest, proxied fetch and the audit log, exfiltration stance #7 that the blobstore profile interface is the natural place to resolve.
• Per-session root namespacing ({root}/run-xxxxxxxx) papered over re-run collisions on minted links; a real provider needs a container lifecycle story (GC of abandoned doc folders and their links) that the draft's destroy-container only gestures at.