Support Client ID Metadata Documents (CIMD) as a registration mode

[aterga]

Summary

Both directories steer servers to Client ID Metadata Documents over DCR. Anthropic's connector auth doc recommends CIMD for directory listings ("DCR causes Claude to register a new client on every fresh connection"); OpenAI's Apps SDK auth doc says ChatGPT prioritises it. Each selects CIMD when our AS metadata advertises client_id_metadata_document_supported: true alongside "none" in token_endpoint_auth_methods_supported. We already had the second; this PR adds the first and the implementation behind it. Advertising the flag alone would have been an outage: Claude would send client_id=https://claude.ai/oauth/…, validate_client would find nothing in the DCR store, and every Claude connection would fail with invalid_client.

With CIMD the client_id is an https URL, and the RFC 7591-shaped JSON at that URL is the client's registration. Nothing is stored per client, so a directory client connecting thousands of times mints no registrations. Both vendors' live documents are the parsing test's fixtures:

Client	client_id	redirect_uris	auth method
ChatGPT	https://chatgpt.com/oauth/client.json	["https://chatgpt.com/connector_platform_oauth_redirect"]	prefers private_key_jwt, lists none
Claude Code	https://claude.ai/oauth/claude-code-client-metadata	["http://localhost/callback", "http://127.0.0.1/callback"]	none

ChatGPT's document declares exactly the redirect #189 pinned Exact, which is a nice confirmation of that call.

Design basis: the scoping in PR #143

This follows the design PR #143 lays out (docs/scoping-cimd.md there; unmerged, so referenced rather than linked), which I had not read when the PR was first opened. Where it stands against that scoping:

• Trust-policy-gated (§3.1, §3.6) — a URL client_id is fetched only when its origin is a vetted vendor's: a host on or under a domain of DEFAULT_ALLOWED_REDIRECTS (plus OAUTH_ALLOWED_REDIRECT_PREFIXES entries), on the default https port (cimd_origin_trusted). One source of truth for "who is a vetted vendor", as §8.2 recommends. An explicit :8443 is refused before any lookup, as §3.1 asks. The new outbound-fetch surface on the unauthenticated /oauth/authorize is therefore a finite set of vetted hosts, not any URL.
• Reject, don't fall back (§8.1) — a URL client_id off the policy gets the same treatment as a hosted redirect off the allow-list: 403 invalid_client naming the contact, or the "not approved" page for a browser. Nothing is fetched, nothing about the URL is reflected.
• Fetcher (§3.3, Phase 0) — the discovery module's SSRF guard (its classifiers tightened along the way to default-deny: the IPv6 side refuses every native address outside the allocated global-unicast 2000::/3, default-denies the IETF protocol-assignment block 2001::/23 within it admitting only the IANA registry's six reachable exceptions by name, and carves out documentation, 6to4 and 3fff::/20; the IPv4 side also refuses the deprecated 6to4 relay block 192.88.99.0/24 bar its reachable 192.88.99.2), exposed as imcp2_core::public_fetch, with the strict reader §3.3 asks for (only a 200 OK is the document — a 206 fragment or any other 2xx is refused like a 3xx — and an over-cap, cut-off, or non-UTF-8 body is an error, never a truncated document), Accept: application/json and a required JSON media type, redirects disabled entirely (the option §3.3 recommends), and no proxy from the environment (through a proxy the address pin would bind nothing). Tighter than proposed where the real documents allow it: 8 KiB cap (they are under 1 KB), one 5 s deadline including DNS (Claude gives our authorize endpoint 10 s).
• Document-intrinsic vs per-request (§3.4, §3.5) — the split is implemented exactly. Document-intrinsic failures (404, a redirect, not JSON, client_id ≠ URL as a plain string match, too large, over MAX_REDIRECT_URIS or MAX_REDIRECT_URI_LEN, a flow this server does not run, no redirect a DCR registration could have registered) are negative-cached for 60 s; the redirect membership and allow-list checks run per request against the positively cached document and never produce a negative entry, so a bad-redirect probe cannot lock out a real client. Transient failures (a resolver that did not answer, deadline, connection, 5xx, the 4xx a client may retry — 408, 421, 425, 429 — the in-flight bounds full) are not cached; the SSRF guard reports a resolver failure apart from its refusals (ResolveError) so the two cannot be confused.
• Outbound-DoS control (§5, Phase 1 acceptance criterion) — the concurrency cap: 4 fetches in flight per host, 16 overall, an excess request told to retry rather than queued, plus the negative cache. All of it is one per process (CimdState::shared), shared by every store the binary mounts (/mcp, /mcp-beta), so the limits hold as documented rather than multiplying with the mounts; hosts are keyed by one spelling (lower-case, no trailing dot) so no spelling buys a second per-host quota; a client_id is bounded at 2 KiB before it becomes a cache key, so the cache's memory is bounded too; and a request dropped mid-fetch gives back its host slot and permit, while the flight it was in stays for its waiters (the last one out retires it), so cancelled connections neither grow the map nor split one document's requests over two fetches. Concurrent misses for one document share one fetch. Logging is bounded too: the per-request diagnostics on the unauthenticated path are debug-level, and the invalid/unavailable outcomes are logged where the fetch happened, at warn at most once a minute per vetted domain (warn_permitted) and at debug otherwise — without a rate cap, the fetch rate alone no longer bounds them, and the vetted set is finite. The rate cap §5 also asks for is deliberately not in this PR. Earlier revisions had one (a token bucket per process and per vetted domain); it is taken out again at the author's request, to keep the PR simple — and sea-snake's review made the case against it as it stood: a spent per-minute budget was itself a lever a flood of made-up URLs on a vetted host could pull against that vendor's real clients, whereas an in-flight slot frees within the 5 s deadline. A request-rate limit belongs in front of the server (the README already says the server does none of its own) or in a follow-up.
• Display fields never trusted (§4) — nothing from the document is displayed; the section comment records that a future consent screen may show the client_id host only.
• Divergence: no TTL floor (§3.5 asks for one). A floor overrides an origin's no-store / max-age=0 and can keep a withdrawn redirect authorized; review round 1 flagged exactly that. There is a 24 h ceiling and a 10 min default (less the age the response already has); the in-flight caps and the negative cache are what bound the fetches, not a floor. ETag revalidation is not implemented; the documents are tiny and the cache is in-memory (§8.3: a miss just re-fetches).

How it works

• Detection — cimd_client_id: an https URL (the scheme in any case) of at most 2 KiB with a host, a path beyond /, no fragment or userinfo. It is taken as given — the string its document must repeat byte for byte, and the cache key — so it must also be what the parser will fetch: serialising the parsed URL must give the identifier back (parsed_as_given), bar the scheme's and the host's ASCII case and an explicit :443, which the parser normalises and the identifier may spell either way. That one rule refuses everything the WHATWG parser silently rewrites — tab/newline/CR stripped, leading or trailing controls trimmed, a backslash read as a slash, an empty @ erased, a space, control, quote, angle bracket, brace, backtick or non-ASCII character percent-encoded, a percent-encoded or IDNA host decoded, . and .. segments resolved — so https://ChatGPT.com/… or an explicit :443 is a client like any other provided its document says the same, and https://chatgpt.com/oauth/cl ient.json is no client at all. Only the host is normalised beyond that, for the trust policy and the per-host bound. Anything else is an ordinary DCR id; that path is unchanged.
• Trust policy — cimd_origin_trusted, before anything else (above).
• Allow-list before fetch — the requested redirect_uri must pass the hosted-redirect allow-list before the document is asked for, so even a vetted host is not fetched on behalf of a redirect that could never be used.
• Fetch — fetch_public_document (above), once a host slot and a permit are held. Freshness follows HTTP for the SHARED cache this is: Vary: * means no reuse at all; every Cache-Control line is combined (a line that cannot be decoded counts as forbidding reuse) and split into directives only at commas outside a quoted-string, private forbids reuse like no-store, s-maxage takes precedence over max-age, a directive given more than once is honoured at its most restrictive value, one given without a valid number is stale rather than the default lifetime, Expires (relative to Date) decides where Cache-Control grants no freshness, and the response's current age — the larger of Age (every line counted, the greatest winning, an unparseable one counting as the greatest) and the time since Date — is subtracted, and reported alongside so the caller's own default lifetime is net of it too. Failures are typed (FetchError::{Refused, Unreachable, Answered{status}, TooLarge, NotUtf8}) so the caller can make the §3.4 split.
• Validation — the document is deserialised into ClientMetadataDocument (the RFC 7591 members this server reads; a member of the wrong type — an explicit null included — is a malformed document, only an absent member an omission), then parse_client_metadata applies the policy, per the draft and Anthropic's reference server: client_id equals the URL exactly; no client secret; can authenticate as a public client (none — absent means none — or none listed in token_endpoint_auth_methods_supported, which is ChatGPT's case); can run this server's one flow (grant_types absent or including authorization_code, response_types absent or including code, as DCR requires of a registration); redirect_uris within what a DCR registration may send (16 entries of at most 2 KiB each); and of those only the ones a DCR registration could have registered (redirect_uri_permitted: loopback, or https on an allow-listed host and pinned path, never with query or fragment) that are loopback or same-origin with the document URL are kept, so a self-asserted document cannot point the code at another party nor slip in a redirect DCR would refuse.
• Redirect check — exactly what a DCR registration gets: redirect_allowed over the document's URIs (loopback port-agnostically, per RFC 8252 §7.3 — Claude Code needs localhost as well as 127.0.0.1) and the hosted-redirect allow-list.
• Single-flight — concurrent misses for one document share the one fetch and its outcome (document, uncacheable document, invalid, or unavailable), so nobody re-fetches serially behind a slow origin. The fetcher retires the flight before publishing, still under the flight's lock, so no later request can join it to reuse an outcome the origin said not to reuse; every request in the flight holds a FlightGuard, and for a flight whose fetcher was cancelled before publishing the last holder out retires it, so a waiter takes over the fetch and nothing is left behind.
• Errors — a transient fetch failure is temporarily_unavailable (retry, not "re-add the connector"); an invalid document is invalid_client; an off-policy origin is invalid_client with the contact. None reflects the caller-supplied URL to the browser; the cause is logged where the fetch happened.
• Token endpoint — unchanged: it already compares the request's client_id with the one bound into the grant, and a URL binds fine.

Opt-in, and rollout

CIMD is off unless the deployment sets OAUTH_CIMD_ENABLED=1: unset, the metadata does not advertise it and a URL client_id is an unknown client, so a deploy of this PR changes no behaviour by itself. The variable is wired through the checked-in deployment path — deploy/native/imcp2.service → deploy.sh → deploy-native.yml, which takes vars.OAUTH_CIMD_ENABLED from the GitHub Environment (see deploy/native/README.md). Set it to 1 on the staging Environment and deploy, connect from Claude web and ChatGPT while watching for client metadata document unavailable / client metadata document is invalid at warn in the logs (at most one a minute per vendor; the rest at debug, never per request), then production. To roll back, unset the variable and redeploy (workflow_dispatch with the same ref is enough; no rebuild): the value is rendered into the systemd unit at deploy time and read once at start-up, so changing the variable alone changes nothing on the host. Once the process restarts without it, Claude's discovery cache (~5 minutes) has clients back on DCR within minutes. The status dashboard's as-metadata check reports CIMD=on|off.

What this PR could not verify: hosted Claude's own document URL is not published (the doc names only Claude Code's, and guesses at it returned 403), so its shape could not be checked the way the other two were; Anthropic's reference server enforces the same rules this PR does, so their client should pass them, but "should" is not "verified". (An earlier revision of this description reported ChatGPT's document as 404: that was this sandbox's egress, not the document, which is served fine elsewhere.)

What this does not do

There is still no consent screen: /oauth/authorize hands the browser straight to Internet Identity, for CIMD clients exactly as for DCR ones. Phase 2 of #143 (branding keyed on the verified domain, coordinated with II) and Phase 3 (opening CIMD beyond the trust policy) are not here. There is no fetch rate cap (see §5 above: deliberately left out; in front of the server, or a follow-up). Nor does it serve a stale document when its origin starts failing (stale-if-error). The discovery crawl's own site_client still takes a proxy from the environment as reqwest does by default; that is pre-existing and left for a follow-up.

Related issues

Follows #189. Design per #143 (unmerged scoping). Rebased onto #190, which checked in rustfmt.toml and made the formatting check part of CI. main merged in again on 2026-09-22 (#172, #194, #195): the one conflict, in deploy/native/deploy.sh between this PR's OAUTH_CIMD_ENABLED substitution on the unit_mcp line and #194's SERVE_STATUS block around the caddyfile line, is resolved by keeping both. Both submission docs are updated (docs/anthropic-directory-submission.md had CIMD down as a follow-up "if usage grows").

Changes

• crates/imcp2-core/src/public_fetch.rs (new) — fetch_public_document with typed FetchError; the strict SSRF-guarded, proxy-free GET, split into sending and accept so the acceptance rules are testable on synthetic responses; PublicDocument carries the remaining freshness and the response's current age; freshness (Vary: *, combined Cache-Control with undecodable lines read as no reuse, Expires fallback) and current_age (every Age line, conservatively, and Date), cache_directives (quoted-string-aware splitting), cache_max_age (shared-cache semantics: private, s-maxage, most-restrictive duplicates, malformed values stale, directives matched by name) and delta_seconds (HTTP's 1*DIGIT, so a signed value the integer parser would take is malformed). discover.rs gains read_capped_bytes (the lossy read_capped_inner is now built on it), a typed ResolveError for resolve_public_url (the crawl keeps its string errors via From), makes both pub(crate), and makes the classifiers default-deny (ipv6_is_global: nothing native outside 2000::/3, 2001::/23 denied with ietf_protocol_assignment_is_global for the registry's exceptions, documentation and 6to4 carved out; ipv4_is_global: the 6to4 relay block bar its reachable exception); lib.rs exports the module (additive public API on the published crate — no version bump here, that's yours to schedule); Cargo.toml adds tokio's time feature, httpdate (already in the lockfile through hyper) as a dependency, and http as a dev-dependency (two edges in Cargo.lock, no new crate).
• src/auth.rs — the CIMD section: constants, cimd_enabled_by_env / cimd_enabled_by (the opt-in), ClientMetadata and ClientMetadataDocument (the RFC 7591 members read, a serde Deserialize struct the document is parsed into; its optional members go through present, so an explicit null is a type error rather than an omission), cimd_client_id / parsed_as_given (the shape, and the round-trip rule), cimd_origin_trusted / allow_listed_domain / vetted_domain / host_key (the trust policy), parse_client_metadata, is_json_media_type, cimd_ttl (the origin's remaining freshness, or the default less the response's age), CimdState (the process-wide cache, single-flight map and in-flight bounds; retire_flight; warn_permitted, the once-a-minute-per-vendor sampling of the fetch-failure warnings), HostSlot and FlightGuard (the guards that give a slot back and retire an unpublished flight, however the request ends), fetch_and_validate_client_metadata, classify_fetch_error (5xx and the retryable 4xx — 408, 421, 425, 429 — are the moment; every other answer is the URL), fetch_client_metadata_document with a #[cfg(test)] fixture registry (answering with a document or any status, aged, failing, or hanging). AuthStore gains cimd: Arc<CimdState> and cimd_enabled; validate_client returns a ClientCheck verdict (Allowed / Refused / MetadataUnavailable / UntrustedClientOrigin); client_metadata_for / fetch_and_cache_client_metadata / remember_client_metadata do single-flight, bounds, fetch, validate, cache, keyed by the identifier as given, with the failure logging sampled as above; /oauth/authorize maps MetadataUnavailable to a retry and UntrustedClientOrigin to the not-approved page or its JSON; the metadata advertises the flag per cimd_enabled.
• deploy/native/imcp2.service, deploy/native/deploy.sh, .github/workflows/deploy-native.yml, deploy/native/README.md — OAUTH_CIMD_ENABLED wired from the GitHub Environment variable to the unit, with the rollback (unset and redeploy) spelled out.
• monitoring/mcp-status/checks.js — the as-metadata detail line reports CIMD=on|off (reported, not required); its test fixture and assertion updated.
• Tests — cimd_client_id_shape (including non-canonical spellings — scheme and host case, :443, a trailing dot, percent-encoding as given, a doubled slash — accepted, the exact length cap, and everything the parser would rewrite — an empty userinfo, tab/newline/CR, edge controls, backslashes, an internal space or control, DEL, a quote, angle bracket, brace, backtick or non-ASCII character, a ' in the query, a percent-encoded or IDNA host, a renumbered IPv6 literal, :0443, dot segments — refused), cimd_client_id_is_taken_as_given, client_metadata_parsing (both vendors' real documents, every refusal, the redirect_uris count and length bounds, members of the wrong type and explicit nulls, the grant and response types, an off-origin port, a loopback entry with a fragment and an unpinned own-origin path dropped), cimd_fetch_error_classification (5xx, 408, 421, 425 and 429 the moment; redirects and the other 4xx the URL), cimd_host_key_is_one_spelling_per_host, cimd_opt_in_values, cimd_origin_trust_policy (real identifiers and subdomains trusted; a stranger, a look-alike, a vetted name under a stranger, and a non-default port refused with no fetch; CIMD off makes a URL id an unknown client), cimd_cache_ttl_is_bounded (including the default net of the response's age), cimd_media_type, cimd_warnings_are_sampled_per_vendor, cimd_client_authorization (the authorize path end to end without network: allow-list before fetch, caching, no-store not cached, a day-old answer with no cache hint not cached, cross-origin refusal, wrong media type, port-agnostic loopback, transient failure retried and not remembered, invalid and 404 remembered, a per-request failure not poisoning the positive cache), cimd_fetches_are_coalesced_and_bounded_per_host (one fetch for three concurrent misses, one shared failure for three concurrent misses, a fifth document on one host refused with every slot released), cimd_flight_retirement_rules (published → retired at once however many hold it; unpublished → kept for the waiter, retired by the last holder; a newer flight never touched), cimd_cancelled_fetch_leaves_nothing_behind (a lone fetcher aborted mid-fetch: no flight entry, host slot or permit left behind, and the next request succeeds), cimd_cancelled_fetcher_hands_over_to_a_waiter (a fetcher aborted with a waiter in the flight: the flight survives, the waiter fetches once, a newcomer fetches nothing, the map is empty afterwards), cimd_state_is_shared_by_every_store, as_metadata_advertises_cimd_only_where_enabled, authorize_points_an_unvetted_cimd_origin_at_the_contact, authorize_tells_a_cimd_client_to_retry_when_its_document_is_unavailable (the retry response at the endpoint itself, for an unreachable origin and for a 425 answer: 503 temporarily_unavailable to a programmatic caller, the sign-in error page to a browser, never invalid_client, neither body reflecting the URL or the cause, and nothing remembered so the next request fetches again); public_fetch: guard refusals (loopback, private, link-local, site-local, unique-local, discard-only, unallocated IPv6 space, the IETF protocol-assignment block, documentation, SRv6, the 6to4 relay block, metadata, IPv4-mapped), an unresolvable host as Unreachable, the single deadline, refusal of redirects, 4xx, 5xx and every non-200 2xx as typed errors, exact cap, UTF-8, Age (including overflowing, non-numeric, signed, non-ASCII, and several lines) and the current age on its own, stale Date, Age versus apparent age either way round, clock skew, Expires with and without Date, past and invalid Expires, Cache-Control precedence over Expires, Vary: *, multi-line Cache-Control, an undecodable Cache-Control or Vary line, duplicate max-age, malformed and signed max-age, the delta_seconds grammar on its own, private, s-maxage, argued no-cache, commas and escapes inside quoted arguments, an unterminated quoted-string; discover.rs's classifier test refuses the special-purpose ranges, unassigned 2001::/23 space and everything outside 2000::/3, while keeping the registry's reachable exceptions global. The pre-existing LRU-stamp test is updated for the verdict type.
• README.md, docs/anthropic-directory-submission.md, docs/openai-directory-submission.md — CIMD documented as supported: trust policy, opt-in and rollback, same-origin rule, public-client rule, media-type rule, in-flight bounds, negative cache.

Testing

• cargo build --locked --workspace --all-targets
• cargo test --locked --workspace --all-targets — 297 tests, 0 failures
• cargo fmt --all -- --check — clean under the rustfmt.toml Check in rustfmt.toml and enforce formatting in CI #190 checked in
• cargo clippy --locked --workspace --all-targets — the 9 warnings are the pre-existing ones in imcp2-core (calls.rs, discover.rs, tools.rs, management.rs); none in auth.rs or public_fetch.rs, and this change adds none
• npm test --prefix monitoring/mcp-status — 71 tests, 0 failures
• .github/scripts/scan-internal-identifiers.sh origin/main...HEAD — clean, commit messages included

All six re-run on the merge commit (f2ed7b1, main at ab35deb), on the round-27 and round-28 fixes (fa5fd7f, 34453ca), on d8795de (the rate limiter out), on 76cf859 (the serde struct), on 28dcb9f (null refused), on 6f1e05a (the port test) and on dbce157 (the warn sampling; 297 tests now); same results — bar one run where the pre-existing live oisy.com test in discover.rs failed to resolve on this sandbox's egress and passed on both re-runs, the same kind of blip as the svault.tech one noted below.

Negative controls, each restored afterwards: removing the allow-list-before-fetch check flips the rogue-redirect case from Refused to MetadataUnavailable("must not be fetched"), proving the test observes whether a fetch happened; removing the metadata flag fails the metadata test; putting 421/425 back among the URL failures fails both the classification test and the endpoint test (the 425 client gets 403, not 503); disabling the round-trip rule fails the shape test at its first rewritten spelling (an internal space). The two vendor documents in the fixtures are byte-faithful to what chatgpt.com and claude.ai served on 2026-09-03.

Review rounds (Copilot): round 1 found same-origin redirects accepted, DNS outside the deadline, a cache floor overriding no-store, and a usize::MAX overflow; round 2 found Age ignored and only one Cache-Control line read, lossy UTF-8 decoding, no media-type check, no single-flight, one host able to take every permit, and redirect handling untested; round 3 found CIMD defaulting on with nothing in the deploy path to turn it off, and single-flight sharing only cache hits rather than outcomes; round 4 found a resolver outage classified as a URL refusal (and so negative-cached), the environment's proxy bypassing the address pin, the first of several max-age values winning, redirect_uris bounded in count but not length, and the in-flight limits being per store rather than per process; round 5 (suppressed comments, no threads) found private and s-maxage ignored by what is a shared cache, a non-string token_endpoint_auth_method read as absent, and HTTP 408 treated as a permanent failure; round 6 found the client_id unbounded in length before becoming a cache key, a trailing-dot host buying a second per-host quota, and the README narrowing the public-client rule; round 7 found grant_types / response_types ignored, so a document declaring only another flow was accepted; round 8 found a request cancelled mid-fetch leaving its single-flight entry behind; round 9 found the shared IPv6 classifier not refusing the deprecated site-local fec0::/10; round 10 found the discard-only 100::/64 likewise, a loopback redirect with a fragment retained (and matched fragment-free), no rate cap behind the concurrency cap (added then; taken out again in d8795de at the author's request — see §5), and a cancelled fetcher's waiters and newcomers landing on two flights; round 11 found the benchmarking and ORCHID IPv6 ranges likewise, freshness ignoring the apparent age a Date header implies, and rate tokens spent on requests refused for congestion (moot since d8795de); round 12 found the documentation and SRv6 IPv6 ranges likewise, and a published flight joinable until its last holder left, prolonging a no-store outcome; round 13 (suppressed, no thread) found the runbook implying that unsetting the variable alone rolls CIMD back, when a redeploy is needed; round 14 found unassigned 2001::/23 space still classified public (fixed by default-denying the block), a malformed max-age falling through to the default lifetime, and a gap between publishing an outcome and retiring its flight; round 15 found Cache-Control split at commas inside quoted-strings, the canonical-form requirement refusing client_id spellings the draft allows, and 2001:1::3 (DNS-SD anycast, reachable) wrongly refused; round 16 found an upper-case HTTPS:// scheme refused before parsing, a test doc comment describing the removed canonical-form contract, and Expires ignored where Cache-Control grants no freshness; round 17 found the deprecated 6to4 relay block 192.88.99.0/24 classified public, Vary: * responses cached, and a raw client_id the parser would silently alter (empty userinfo, tab/newline/CR) accepted; round 18 (suppressed, no threads) found an overflowing Age read as zero, and per-request warn logs on the unauthenticated path that a flood could make unbounded; rounds 19 and 20 (suppressed, no threads) found three doc comments still describing earlier contracts; round 21 found only the first Age line read (a non-ASCII one counting as zero), and a client_id with leading or trailing controls or spaces accepted though the parser trims them; round 22 (suppressed, no thread) found the IPv6 classifier still default-allow outside its named exclusions (fixed by refusing everything outside 2000::/3); round 23 found a 206 Partial Content accepted as the document, and an undecodable Cache-Control line silently dropped; round 24 found the default cache lifetime granted in full to an answer that was already a day old; round 25 (summary only, no comment) found a client_id with backslashes, which the parser reads as slashes, accepted; round 26 (summary only, no comment) found 421 Misdirected Request and 425 Too Early — both defined as answers the client may retry — classified as failures of the URL and so negative-cached, and the endpoint's retry response covered only through the verdict it maps rather than directly; round 27 (on the main merge) found a signed Age: +1 or max-age=+300 accepted, Rust's integer parser taking a leading + where HTTP's delta-seconds is digits only, so a malformed header could extend a document's reuse instead of ending it; round 28 found an internal space or control character in a client_id accepted, though the parser percent-encodes it and so fetches a URL other than the identifier given (fixed by the one rule that covers the whole class, present and future: the parsed URL must serialise back to the raw identifier, bar scheme and host case and an explicit :443, which replaces the four special cases); round 29, on 34453ca, found nothing; rounds 30 and 31 (on b46e0ea and d8795de) asked for the description and the limits to agree and for the rate limiter back — the former is done, the latter is the author's scoping decision (§5), answered and resolved; round 32 found an explicit null in an optional member read as an omission since the serde refactor (fixed: present); round 33 (summary only, no comment) found the parsing test's different-port case putting :8443 after the path, so it tested a path, not a port (fixed); round 34 (summary only, no comment) found the fetch-site warnings unbounded once the rate cap was gone — a fast 404 frees its slot at once (fixed: sampled once a minute per vendor); round 35, on dbce157, found nothing. Sixty-five fixed, two — the rate cap and its refusal accounting — deliberately taken out again, and rounds 30–31's three superseded or declined as above; see the commits and the threads. One earlier test failure was a racy deadline test of mine (two competing timeouts), fixed by making the outer deadline the only one; another was a transport blip in the pre-existing live svault.tech test, not this PR's (comment on the PR).

Human review (sea-snake, 2026-09-22; approved 2026-09-23 on dbce157): the fetch bounds were low enough to be a denial-of-service lever in themselves — resolved by taking the rate limiter out (d8795de; the per-minute budget was the lever) and raising the in-flight bounds to 16/4, which free within the fetch deadline. Whether CIMD should need an allow-list at all — answered on the thread: the allow-list here is #143's Phase 1 trust policy for the outbound fetch an unauthenticated endpoint triggers, not client registration (which CIMD does remove), and opening it is Phase 3, with the consent screen. And to parse the document into a typed struct rather than walk a serde_json::Value — done in 76cf859 (ClientMetadataDocument). All three threads are resolved.

Not tested here: a live authorization from Claude or ChatGPT against a deployed build — this needs the deploy, and see "Opt-in, and rollout" above.

Checklist

• I have read the Contributing guidelines.
• Docs (README / comments) updated for any user-visible change.
• No secrets, credentials, or internal-only information are included.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Xd7VT72Qt16qiAJynu9EKj

[aterga]

CI test failed on 6d6124a in discoverability::tests::authorizes_a_declared_canister_and_refuses_an_undeclared_one, and I'm treating it as not this PR's:

• The panic is a transport error fetching a live third-party site: could not read /.well-known/ic-architecture at https://svault.tech: error sending request for url.
• This PR does not touch discoverability.rs (or that code path at all), and the same job passed on the previous head c776135, whose only difference is one string in a public_fetch test.
• The test passes locally right now, and https://svault.tech/.well-known/ic-architecture answers 200 in under a second, so the site is up — the runner hit a blip.

The flake has a mechanism worth fixing separately: the test guards its first fetch_declared_manifest with an early return if svault.tech is unreachable ("skips if it stops publishing, rather than failing CI on someone else's deploy"), but authorize_call re-fetches the manifest at discoverability.rs:202, and that second call is .expected. A blip between the two fetches panics instead of skipping. Two ways to close the gap, either outside this PR's scope: have the test tolerate a transport error from authorize_call the way it tolerates one from the first fetch, or fetch the manifest once in the test and drive the decision (decide_*) on constructed input, as the unit tests above it already do.

Re-running the failed job once.

---

Generated by Claude Code

[Copilot]

🟡 Changes recommended

Redirect handling, DNS timeout coverage, and cache-control behavior undermine the promised strict fetch guarantees.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds Client ID Metadata Documents as a stateless OAuth registration mode alongside DCR.

Changes:

• Implements CIMD detection, validation, caching, authorization, and kill switch.
• Adds an SSRF-guarded public document fetcher.
• Updates monitoring, tests, and directory documentation.

File summaries

File	Description
src/auth.rs	Implements CIMD authorization and caching.
crates/imcp2-core/src/public_fetch.rs	Adds guarded document fetching.
crates/imcp2-core/src/discover.rs	Exposes shared SSRF helpers internally.
crates/imcp2-core/src/lib.rs	Exports the public-fetch module.
monitoring/mcp-status/checks.js	Reports CIMD status.
monitoring/mcp-status/checks.test.js	Tests CIMD status reporting.
README.md	Documents CIMD behavior.
docs/openai-directory-submission.md	Records OpenAI CIMD readiness.
docs/anthropic-directory-submission.md	Records Anthropic CIMD readiness.

Review details

Suppressed comments (2)

crates/imcp2-core/src/public_fetch.rs:53

• This policy follows same-host and public-IP redirects even though this strict fetch promises that any redirect target is rejected. The later check compares only origins, so https://host/client.json -> https://host/other is accepted and parsed as the original client's document. Disable redirects here; the existing non-success check will then reject the 3xx response.

        .redirect(ssrf_redirect_policy())

crates/imcp2-core/src/public_fetch.rs:81

• This public API accepts any usize, so max_bytes + 1 overflows for usize::MAX (panic in checked builds, wrap to zero otherwise), potentially returning an empty document as a successful fetch. Use saturating addition or reject that input explicitly.

    let body = match read_capped_inner(resp, max_bytes + 1).await {

• Files reviewed: 9/9 changed files
• Comments generated: 2
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

Metadata validation, HTTP caching, decoding, and fetch admission contain unresolved correctness and reliability issues.

Review details

Suppressed comments (5)

Previously missed (5) — in code that hasn't changed since the last review.

crates/imcp2-core/src/public_fetch.rs:88

• The cache hint ignores the response's current age. For a CDN response with Cache-Control: max-age=86400 and Age: 86399, client_metadata_for starts a fresh 24-hour TTL, so a withdrawn redirect can remain authorized almost a day beyond the origin's freshness lifetime. Return or compute the remaining freshness lifetime using the HTTP Age/Date semantics rather than forwarding the raw max-age value as a new TTL.
  crates/imcp2-core/src/public_fetch.rs:88
• HeaderMap::get observes only one Cache-Control field, but repeated Cache-Control fields are semantically combined. A response containing separate max-age=86400 and no-store fields can therefore be cached for a day depending on field order, violating the explicit no-store instruction. Combine all field values before parsing directives.
  crates/imcp2-core/src/public_fetch.rs:92
• This strict fetch delegates decoding to read_capped_inner, which uses String::from_utf8_lossy. Invalid UTF-8 bytes inside a JSON string are therefore replaced with U+FFFD and the resulting document can pass serde_json validation, even though CIMD JSON must be UTF-8. Preserve the existing fail-soft discovery behavior, but make this strict path reject decoding errors rather than normalizing them.
  src/auth.rs:1263
• The global fail-fast semaphore can be monopolized with very little unauthenticated traffic: eight slow distinct URLs occupy every permit for up to five seconds, failures are not cached, and subsequent legitimate cold/expired CIMD requests all receive 503. Concurrent misses for one popular client also issue duplicate fetches and can consume all eight slots. Add per-key single-flight coalescing and admission control that one source/origin cannot exhaust before enabling this on the public authorize endpoint.
  src/auth.rs:1271
• The fetched media type is discarded here, so a 200 response with a missing or text/html Content-Type is accepted whenever its body parses as the expected JSON. CIMD metadata documents are required to be served as application/json; validate the case-insensitive media-type essence (while allowing parameters) and classify a mismatch as an invalid document before parsing it.

• Files reviewed: 10/10 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

Content-type validation, HTTP cache freshness, and cold-cache request coordination remain incomplete.

Review details

Suppressed comments (4)

Previously missed (4) — in code that hasn't changed since the last review.

crates/imcp2-core/src/public_fetch.rs:92

• This does not fully honor HTTP freshness semantics: HeaderMap::get reads only one of potentially several legal Cache-Control field lines, and the returned max-age is reused from receipt without subtracting the response's Age. For example, a CDN response with max-age=86400, Age: 86399 is cached locally for another day, and a second Cache-Control: no-store line can be missed. Combine all Cache-Control values and reduce the lifetime by Age before exposing it to callers.
  src/auth.rs:1263
• Cold-cache requests for the same client_id are not coalesced: all eight can consume permits fetching the identical document, while the ninth legitimate authorization is immediately rejected. This creates a thundering herd for popular directory clients and also lets a very low request rate monopolize the process-wide pool with slow URLs. Add per-key single-flight coordination with a cache recheck so one fetch serves concurrent requests; retain a separate global bound for distinct URLs.
  src/auth.rs:1269
• The fetched response's Content-Type is recorded but never validated before accepting the metadata. The CIMD requirements call for application/json; as written, a document served as HTML or plain text is accepted whenever its bytes happen to parse as JSON. Validate the media type (allowing normal parameters such as charset) and classify a mismatch as CimdError::Invalid.
  crates/imcp2-core/src/public_fetch.rs:74
• No test exercises a 3xx response, so the redirect behavior that was corrected during the prior review is not protected against regression; the current deadline test never receives an HTTP response. Add a deterministic fetch test (or extract a response-policy seam) proving that same-origin and cross-origin redirects are returned as errors without requesting their targets.

• Files reviewed: 10/10 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The deployment enables a currently failing integration by default, and failed or uncacheable fetches are not correctly coalesced.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 10/11 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

SSRF proxy handling, transient DNS classification, cache freshness, redirect bounds, and process-wide concurrency limits need correction.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 14/15 changed files
• Comments generated: 5
• Review effort level: Balanced

[Copilot]

Copilot review overview

🟡 Changes recommended

CIMD validation still accepts internal control characters that the URL parser rewrites before fetching.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Reject internal spaces and control characters in client IDs · New

Resolved since last review (1)

• Reject plus-prefixed Age header values

[Copilot]

Copilot review overview

🔵 Needs a closer look

The security-sensitive OAuth and outbound-fetch changes require final human review and live vendor integration validation.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Reject internal spaces and control characters in client IDs

[Copilot]

Copilot review overview

🟡 Changes recommended

The implemented concurrency and rate limits conflict with the security and operational limits stated in the PR description.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2

Open (2)

• Concurrency limits exceed documented process-wide and per-host ceilings · New
• Fetch rate limits exceed documented process-wide and vendor ceilings · New

[Copilot]

Copilot review overview

🟡 Changes recommended

The documented rate limits and reserved capacity are absent, allowing unauthenticated traffic to starve legitimate CIMD refreshes.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 2

Open (3)

• Prevent outbound DoS with per-domain rate limits and reserved capacity · New
• Fetch rate limits exceed documented process-wide and vendor ceilings
• Concurrency limits exceed documented process-wide and per-host ceilings

[Copilot]

Copilot review overview

🟡 Changes recommended

Explicit null metadata values bypass required type validation and are treated as omitted fields.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Reject explicit null for documented non-nullable metadata fields · New

Resolved since last review (3)

• Prevent outbound DoS with per-domain rate limits and reserved capacity
• Fetch rate limits exceed documented process-wide and vendor ceilings
• Concurrency limits exceed documented process-wide and per-host ceilings

[Copilot]

Copilot review overview

🔵 Needs a closer look

The security-sensitive authorization and outbound-fetch changes require human review, and one intended different-port regression does not currently test a port change.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Reject explicit null for documented non-nullable metadata fields

Previously missed (1)

In code that hasn't changed since last review

Test redirect handling with a different-origin port

src/​auth.rs:3548

This does not test a different-origin port: appending :8443 to OWN places it after /api/mcp/auth_callback, so it becomes part of the path. Use a URL with the port in the authority; otherwise a regression that ignores redirect ports could still pass this test.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Unauthenticated fetch failures can generate unbounded warning-level logs despite the concurrency limits.

Review effort: Balanced
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Rate-limit caller-triggerable warnings to prevent log flooding

src/​auth.rs:1759

The in-flight cap does not bound log volume: a fast 404/invalid response immediately releases its slot, and an unauthenticated caller can rotate paths under a trusted host to emit one warning per fetch (the negative cache is keyed by the full URL). Since production enables warnings via RUST_LOG=info and this PR deliberately has no rate cap, this can flood journald and evict useful diagnostics. Please sample/rate-limit these messages or keep caller-triggerable failures at debug level.

[Copilot]

Copilot review overview

🔵 Needs a closer look

It introduces a security-sensitive unauthenticated outbound-fetch path and still requires live vendor authorization validation.

Review effort: Balanced
Findings: None