Upstream iroh crate as endpoint core: feasibility and the ecosystem-compatibility ruling

[lann]

Driving goal: crate-level compatibility with iroh's protocol ecosystem
(iroh-blobs, iroh-gossip, ...) — stock crates running on our endpoint — not
just wire compatibility. Protocol crates bind concrete iroh::Endpoint /
endpoint::Connection / tokio io types (iroh = "1.0" non-optional in
iroh-blobs; ProtocolHandler::accept(Connection)); there is no trait seam
to substitute an endpoint through. Consequences:

• Wire compat (the current README design) lets us talk to blobs/gossip
  nodes only by porting each protocol's wire logic — a permanent
  parallel-maintenance treadmill that drifts from the ecosystem it chases.
• An API-compatible iroh facade over the WIT surface is a whole-crate
  fork; strictly worse than upstream + patches.
• So ecosystem compat is achievable only by running the upstream crate
  in-guest. The question becomes where that is feasible and what it costs
  the architecture.

All findings below verified 2026-08-05 against n0-computer/iroh v1.0.3
(tokio 1.52.3, rustc 1.97.0), iroh-blobs v0.103.0, iroh-gossip v0.101.0.
File paths are upstream paths unless marked local. Compile claims are
cargo check only — nothing linked or run.

Rationale check: README "Port the protocol, not the crate"

README claim	Status at v1.0.3
"quinn assumes OS sockets"	No longer true at the API boundary. unstable-custom-transports ships poll-based CustomTransport/CustomEndpoint/CustomSender (iroh/src/socket/transports/custom.rs), Builder::add_custom_transport + path_selector, clear_ip_transports()/clear_relay_transports(). TransportAddr::{Ip,Relay,Custom} is first-class; custom paths participate in multipath selection; discovery records round-trip custom addrs (iroh-dns/src/endpoint_info.rs:492). Feature is explicitly "may change without notice".
"tokio's WASI support is partial"	Materially weakened by tokio-rs/mio#1931 (merged 2026-02-24: wasip2 via wasi-libc poll(2), single-thread Waker) + tokio-rs/tokio#7933 (wasip2 networking, v1.51.0+), both gated behind --cfg tokio_unstable (stabilization: tokio-rs/tokio#4827). See empirical status below.
"existing browser build targets wasm-bindgen, not components"	Still true; wasm_browser cfg = wasm && os=unknown (iroh/build.rs:7), so wasip2 takes the native branch. The browser branch hard-requires JS (wasm_bindgen_futures::spawn_local, iroh/src/runtime.rs). Not componentizable; irrelevant to fix.

What upstream provides vs what stays coupled (v1.0.3)

Pluggable today: custom transports (above); rustls CryptoProvider is
builder-injectable (iroh/src/endpoint.rs:149) — the polymorph-tls
provider slots into the crypto split's in-guest half as-is; AddressLookup
is a public trait (custom publishers avoid the pkarr signing path).

Still coupled:

• Identity is concrete. SecretKey (seed in linear memory) is wired
  through the cert resolver (iroh/src/tls/resolver.rs:12), the relay
  handshake challenge, and pkarr record signing. rustls's
  SigningKey/Signer machinery is right there; an Arc<dyn SigningKey>
  refactor in iroh + iroh-relay is small and PR-able, but without it the
  upstream path violates the non-extractable-identity ruling.
• Runtime is pub(crate) and cfg-hardwired two ways
  (iroh/src/runtime.rs): tokio (TaskTracker + noq::TokioRuntime
  timers) vs wasm_bindgen_futures + web timers. With custom transports
  only, tokio-as-I/O disappears and the residue is exactly two seams —
  spawn (noq driver tasks: endpoint driver, per-connection driver, path
  filter; plus iroh's actor mesh) and timers (noq new_timer,
  n0_future::time = tokio::time on native) — plus executor-agnostic
  tokio::sync/tokio-util and tokio io traits on streams (public API:
  tokio::* is in allowed external types). Upstream motion covers time
  only (Allow host-provided QUIC time sources n0-computer/iroh#4459); a spawn-side sibling plus a relay-dialer
  seam would complete it.
• Relay client is compile-time split (native tokio-websockets/hyper vs
  browser ws_stream_wasm); no dialer trait. Upstream peers are reachable
  only through real Ip/Relay transports (routing is by addr variant),
  so a port needs the internal relay transport working, not a
  relay-as-custom-transport workaround (workable outbound, but inbound
  reachability and internal lookup-driven dialing fight it).
• noq-proto note (relevant to Direct UDP as an upgrade target #12 independently of this issue):
  n0_nat_traversal.rs, address_discovery.rs (QAD), and multipath live
  at the sans-I/O proto layer of the noq fork — inheritable by the local
  core as a dependency, without the tokio-coupled socket layer.

Empirical wasip2 compile status

cargo check -p iroh --no-default-features --features tls-ring,unstable-custom-transports --target wasm32-wasip2:

• Without tokio_unstable: dies at tokio's guard (iroh's native branch
  demands net,fs,io-std; iroh's own src has zero tokio::net/tokio::fs
  uses — trimming is a one-line iroh PR).
• With --cfg tokio_unstable: tokio + mio compile; ring,
  tokio-websockets, hickory-resolver, hyper, reqwest all pass. Remaining
  failures — exactly three:
  1. noq-udp: platform imp module selection matches nothing for wasi
     (posix_minimal.rs exists in-tree; needs selection or a std fallback);
  2. netdev (via netwatch): IFF_* libc flags — needs the stub netwatch
     already has for wasm_browser;
  3. iroh-dns: one line, hickory read_system_conf() cfg'd out on wasi
     (iroh-dns/src/dns.rs:774).

Interpretation: on socket-bearing hosts, tokio-as-executor becomes coherent
inside a component (current-thread rt parking on poll(2) → wasi-libc →
wasi:io/poll). What looked like a family-wide port campaign is now ≈3
small patches + an iroh cfg/feature PR. Caveats: tokio_unstable carries
no semver guarantee; open wasip2 bugs (tokio-rs/tokio#8082,
tokio-rs/tokio#7936).

Browser leg

Two vehicles, different gaps:

Upstream wasm_browser build + WebRTC custom transport (wasm-bindgen):
compiles today with unstable-custom-transports (checked, exit 0), and has
no executor problem — the JS event loop is the single wake path. Missing:
a signaling story (WebRTC needs SDP/ICE exchange; the natural shape is our
upgrade model — dial via relay, signal over an ALPN protocol, migrate — so
"custom transport only" is a fiction; relay stays for bootstrap/fallback);
a public mid-connection addr-injection API (mechanism exists —
resolve_remote merges per-remote and open_path_on_all_conns
(iroh/src/socket/remote_map/remote_state.rs:309) reaches live
connections — but only insert_relay is public, and Custom-addr
end-to-end is unverified); Send + Sync trait bounds vs !Send web-sys
types (send_wrapper/channel confinement); identity in linear memory; no
per-transport MTU clamp (only GSO segment count). And it is not a
component: no WIT surface, no composition, JS host only — a legitimate
upstream contribution, not this repository's vehicle.

wasip2 component under jco: CM-async import completions (waitables)
cannot wake tokio parked in poll(2) — wasi-libc's poll sees fds only, and
p2 has no pollable→fd bridge, so polymorph:webrtc events go unnoticed
between timer ticks (bounded polling again, now under iroh's ~15-task
actor mesh). Browsers have no wasi:sockets, so relay/DNS/pkarr compile
but are dead at runtime; signaling must ride polymorph:websocket. Plus the
known jco CM-async scheduler defects (#10, #9, #6) that already block our
much less demanding pump. tokio-rs/mio#1931's own text defers unification
to p3 wasi-libc (pipes, threads); p3 CM-async is where waitables become
the single wake path.

Ecosystem crates measured

More portable than iroh itself; n0 engineered the family to the same
two-branch pattern, both crates CI wasm32:

• iroh-gossip v0.101: protocol core has zero target cfg; net (the
  iroh binding) and rpc are features; common tokio {io-util,sync}.
• iroh-blobs v0.103: common tokio sync only; only target-split deps
  are chrono/getrandom; fs-store (redb + reflink-copy + bao-tree/fs) and
  rpc (noq + irpc) are opt-out features; wasm_browser cfg confined to
  store/mem.rs + store/util.rs. MemStore portable by construction;
  FsStore plausible on wasip2 (redb is std::fs-based, no mmap) —
  unverified; absent on browser.
• Long tail unverified: irpc noq_endpoint_setup, iroh-tickets, iroh-io,
  iroh-util.

Composition-model consequence

#3's "protocols as plain consumer components composed via wac plug" is
crate-incompatible by construction — stock protocol crates cannot consume
a WIT endpoint import. The ecosystem-compatible shape is the fat
guest: upstream iroh + stock protocol crates linked into one wasip2
guest, WIT exported at the application boundary (blobs/gossip interfaces
and/or the endpoint surface), transports/crypto still injected where the
invariants demand. WIT composition remains for protocols written natively
as components. Both models coexist; "stock ecosystem crate as a
separately-composed component" cannot exist without ports.

Per-target status under the ecosystem goal

Target	Stock blobs/gossip via upstream iroh
wasmtime-class (cloud, devices)	Plausible now: 3 patches + tokio_unstable, blobs MemStore (FsStore worth one experiment)
Browser as component (jco)	Blocked on the CM-async→executor wake path, same as bare iroh; p3 is the durable fix; protocol crates add no new blockers
Browser via upstream wasm-bindgen	Works today upstream; not a component — n0's lane

The ruling this forces

Ecosystem-compat-now collides with one-implementation-across-targets:

• (a) Declare crate-level ecosystem compat an optional capability of
  socket-bearing targets (the portability ladder's last rung — declared,
  gated), with p3 named as the convergence point where the browser joins.
  Browser keeps the purpose-built endpoint — which is also the
  WebRTC-direct story upstream lacks. Recommended.
• (b) Dual endpoint implementations behind one WIT surface,
  conformance-gated — the shape the README's central ruling exists to
  forbid.
• (c) Defer ecosystem compat wholesale to p3 — cedes a year-plus for
  no architectural gain over (a).

Resolving this issue means amending the README ruling by PR, not sliding
into an implementation.

Next steps if (a) is adopted

1. Fat-guest spike (own issue): iroh + blobs MemStore on wasip2 under
   wasmtime, the 3 local patches, fetch a blob from an upstream native
   node; record binary size + handshake numbers (the budget policy will
   demand them).
2. Upstream patches, each independently justified: noq-udp wasi impl
   selection; netwatch/netdev wasi stub; iroh-dns one-liner; signer
   abstraction (Arc<dyn SigningKey> in tls/resolver + relay handshake)
   for the identity ruling.
3. Direct UDP as an upgrade target #12 tie-in regardless of outcome: evaluate switching the local core to
   noq-proto to inherit n0_nat_traversal/QAD/multipath sans-I/O.
4. Re-evaluation triggers: tokio wasi stabilization (meta: Stabilize WASI support tokio-rs/tokio#4827);
   unstable-custom-transports stabilization; upstream runtime injection
   (Allow host-provided QUIC time sources n0-computer/iroh#4459 + a spawn-side sibling); a relay-dialer or
   HTTP-client seam; wasi-libc p3 (pipes/threads) + jco CM-async fixes
   (jco: scheduler stops delivering waitable events once a detached task holds in-flight imports across export calls #10). Two or more firing flips the browser-leg calculus.

[lann]

Probe result (#18): the browser-leg wake-path boundary is bridgeable on p2. The issue states CM-async import completions cannot wake a tokio parked in poll(2). True at the mechanism level — but under jco the host owns wasi:sockets, so it can synthesize the fd world: a synthetic UDP socket whose receive queue is fed from arbitrary JS events makes those events visible to mio as datagram readiness, resolving the suspended wasi:io/poll#poll (JSPI) and waking the reactor.

Measured (just udp-wake, Node 24.18, unmodified tokio 1.53 current-thread rt+time+sync+net on wasm32-wasip2, 200–500 events):

• inject→echo p50 0.34ms, p99 ~1.1ms, max ≤3.8ms — event-driven, not tick-polling (which would center on half a poll interval)
• ~2 poll calls per wake (mio armed-waker zero-timeout re-poll); idle poll rate tracks the guest's own timers; pollable.block never called
• timers stay accurate while parked; a cross-task mpsc hop rides the same wake cycle

Consequences for the analysis above:

1. The fat-guest browser leg no longer waits on p3 for the executor question. An in-guest tokio under jco can be woken by host events today. What p3 still buys is doing this without a synthetic-shim layer and without JSPI availability constraints.
2. For a WebRTC transport the bridge is the data path, not just a doorbell: the host adapter can deliver RTCDataChannel messages as datagrams from a synthetic address — which is UDP-shaped exactly like iroh's CustomTransport/AsyncUdpSocket expectations. QUIC-over-data-channel then runs against an unmodified socket layer.
3. The relay leg is the remaining seam. Synthetic sockets deliver host-mediated events; they don't let the guest TCP-connect to real internet hosts from a browser. Either the upstream relay-dialer injection seam (as argued above), or a host adapter that owns the wss connection and exposes relay datagram frames as synthetic datagrams — the latter moves relay protocol/auth host-side, a per-host reimplementation with its own divergence cost.
4. Applies to the current architecture too: the endpoint's bounded clock-polling workaround for cross-task wakeups (README, issues jco: scheduler stops delivering waitable events once a detached task holds in-flight imports across export calls #10/Design the exported endpoint WIT surface #3 context) has the same shape as tick-polling this probe was designed to rule out; a synthetic-socket doorbell is a candidate replacement even without adopting the upstream crate.

Caveats: measured on Node with --experimental-wasm-jspi; browser JSPI availability varies by engine and is a real deployment gate. Transpiled with the repo's pinned jco fork (asyncWasiImports/asyncWasiExports porcelain). Latency only — throughput and per-datagram copy costs unmeasured. Single socket, single instance; multi-socket and TCP synthesis untested.

[lann]

A coupling fact for step 3 (the noq-proto evaluation), verified against the v1.0.3 checkout under .deps/iroh: noq preserved quinn's crypto seam. noq-proto exposes the same crypto::Session/PacketKey trait surface (iroh/src/endpoint/quic.rs maps noq_proto::crypto::* types; iroh/src/tls/misc.rs imports noq_proto::crypto), and noq's rustls glue — noq::crypto::rustls::{QuicClientConfig, QuicServerConfig} (iroh/src/tls.rs:11) — is feature-gated on ring/aws-lc exactly as upstream quinn's is (tls-ring/tls-aws-lc-rs in iroh/Cargo.toml).

Consequences:

• Switching the local core to noq-proto means porting polymorph-tls-quinn from quinn-proto 0.11's traits to noq-proto 1.x's (or dual-targeting during a transition). Mechanical: it is the same re-hosting of the feature-gated rustls glue that crate's session.rs already performed for quinn-proto.
• polymorph-tls#11's provider-agnostic-glue upstreaming gains a second candidate target (noq), plausibly the more receptive one given n0's wasm interest.
• noq/noq-proto/noq-udp are published crates.io crates (1.1.0 at the iroh v1.0.3 pin), not a git fork — pinning/patching mechanics are ordinary.

[lann]

Step 3 is executed as adoption: the local core now runs on noq-proto 1.1.0 (#21, with polymorph-tls#24/#25 renaming the crypto crate to polymorph-tls-quic and porting it to noq's traits). Consequences for this issue's remaining scope:

• The Direct UDP as an upgrade target #12 tie-in is now live: n0_nat_traversal, QAD, and multipath exist at our proto layer as inheritable dependency surface (the port stubs their events explicitly; nothing is wired). Direct UDP as an upgrade target #12's implementation can build on noq-proto instead of hand-building the extension.
• The crypto-seam coupling noted above resolved cheaply: noq-proto's rustls feature is provider-agnostic (initial suite from the config's provider, RustCrypto retry tag), so the port deleted the 518-line re-hosted session glue rather than re-porting it, and added the multipath _for_path nonce to the packet module (picoquic KAT pinned).
• Steps 1–2 (whole upstream crate in-guest) are unaffected and remain open — but the delta shrinks: the version skew between our stack and upstream's is gone (one noq-proto 1.1.0 in the lock serves our components and the pinned upstream checkout), and the upstream-interop matrix legs now exercise noq↔noq.

[lann]

Relay-leg spike (#24, stacked on #18): the unmodified upstream iroh crate runs relay-only on wasm32-wasip2 under jco/JSPI, against a stock relay, with the relay websocket on the polymorph-websocket host module. Two endpoints in one component: dial-through-relay 23–30ms, echo RTT over the relay p50 ~0.5ms, guest 7.3MB.

What it took — the full wasi-enablement inventory (vendored, iroh logic untouched):

1. noq-udp → posix_minimal on wasi + non-vectored recv_from (functional UDP over wasi:sockets, not a stub — the 'direct UDP on socket-bearing hosts' path is now plausible too);
2. netwatch → its netdev-free branch (2 lines);
3. iroh-dns → tolerate missing system resolver config (1 cfg arm);
4. iroh-relay → a wasm_wasi connect() mirroring the browser branch: DatagramPipe replaces WsBytesFramed over the synthetic socket (one datagram = one ws message = one relay frame), in-band challenge handshake unchanged;
5. iroh → net_report's HTTPS relay probe short-circuited on wasi (home-relay selection gates on it; upstream's browser build only passes it because reqwest becomes fetch).

Finding worth recording: home-relay selection is probe-gated, so any non-HTTP host needs either a probe path or the short-circuit — this is the concrete content of the 'relay dialer seam' upstream ask: a pluggable relay dial + a probe that rides it, rather than reqwest.

With #18 (wake path, portable virt) and this (relay tier on the real crate), the browser leg of the fat-guest route is demonstrated through QUIC-over-relay end-to-end. Remaining for the full story: the WebRTC custom transport (datagram-shaped by construction, per the CustomTransport analysis above) and the identity/webcrypto split (SecretKey still in linear memory here).

[lann]

Housekeeping: the upstream-iroh-crate validation now lives on the long-lived branch upstream-iroh (draft PR #25, stacked on #18). Vendored/pinned deps stay on that branch while features are proven out; nothing vendored reaches main — the endgame is upstreaming the wasi patches (noq-udp's already landed upstream independently as noq#773) and porting minimal glue. Next targets on the branch: WebRTC as a CustomTransport, the identity/webcrypto split, iroh-blobs on the fat guest, and budget measurements.

[lann]

Correction to the browser-leg section from the WebRTC CustomTransport spike on #25: the mid-connection addr-injection line ("mechanism exists — resolve_remote merges per-remote and open_path_on_all_conns reaches live connections — but only insert_relay is public, and Custom-addr end-to-end is unverified") is now verified false — open_path_on_all_conns is only reachable from the CID-exhaustion retry path, resolve_remote records without opening, and the NAT-traversal candidate exchange is SocketAddr-typed, so custom addrs handed to a live remote are never probed by any mechanism. The upgrade model (dial relay → signal → migrate) needs an upstream seam. Full inventory with file:line refs and the working QUIC-over-data-channel demo: #25 (comment)