Add WebMCP registration, discovery, and execution - #1
MiguelsPizza wants to merge 10 commits into
Conversation
855027d to
843ec28
Compare
Match the published official types with a document-local implementation and real-browser, package-consumer, and pinned WPT checks. Resolve package dependencies with fresh registry metadata so clean CI runners do not depend on a warm offline cache. Allow only the downloaded Chrome executable to create user namespaces on Ubuntu, keeping its sandbox enabled, and expose WPT startup logs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
843ec28 to
74ccd8d
Compare
Implement executeTool() against the current draft: object input, JSON-serialized results, and abort handling that gives the callback its own signal and a default AbortError while the caller receives its own reason. Extend the WPT selection from 13 files to 18, and from 27 assertions to 56, now that the interface is complete enough for the IDL harness. Record as per-subtest expected failures in wpt-metadata the four upstream files that disagree with the draft on omitted input and on result serialization, rather than patching tests or relaxing the implementation. Carry a temporary ModelContext augmentation mirroring webmcp-types#3 until that ships. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
beaufortfrancois
left a comment
There was a problem hiding this comment.
Thank you @MiguelsPizza for starting this effort! See comments below
| "imperative/non-secure.html", | ||
| "imperative/register-tool-title.https.html", | ||
| "imperative/register_tool_with_empty_annotation.https.html", | ||
| "imperative/getTools-imperative-annotations.https.html", |
There was a problem hiding this comment.
Why is imperative/getTools.https.html not covered? I may have missed others as well. Can you explain which ones you picked?
There was a problem hiding this comment.
No reason other than things were (are) still in WIP. They should all be covered now
There was a problem hiding this comment.
Gotcha. Let me know when the PR is ready for review then.
Use released webmcp-types 0.1.8 and remove the temporary executeTool augmentation. Shorten comments and test setup while preserving Web IDL conversion, cancellation, and native-installation behavior. Run every pinned WebMCP testharness file with standard WPT expectations instead of a selected list. Record failures, timeouts, and blocked subtests separately so the supported scope is visible. Apply the review's fixture, header, and Chrome flag corrections. Validate blob URLs using their inherited origin; a browser regression demonstrates the former SecurityError for trustworthy blob origins. Validated with pnpm test (100 browser tests), pnpm test:package, and pnpm test:wpt (58 files; all results match their explicit expectations). References: #1 #2 https://github.com/webmachinelearning/webmcp-types/releases/tag/v0.1.8 https://url.spec.whatwg.org/#concept-url-origin
Browser tests and their type checks used separate filename allowlists, so a new test could be silently omitted. Discover test files by convention while keeping the native Chromium test in its own project. Share execution-test setup and use resolved promises instead of definitely-assigned test state. CI and the local WPT runner now read one unchanged revision from wpt-revision.txt. Preserve the full testharness selection and reviewed expectations established after Francois's review: #1 (comment) Use URL.parse for invalid-origin handling, keeping the same security errors without exception-driven parsing. Expand the installer's public JSDoc with its no-op cases, frame scope, example, and first-party references. Keep Web IDL boundary checks and document the native validation behind their casts. https://url.spec.whatwg.org/#dom-url-parse https://webmachinelearning.github.io/webmcp/#document-extension Validation: pnpm test (100 passes), pnpm test:package, formatting, and pnpm test:wpt on Canary 156.0.8062.0. All 58 WPT files match expectations: 72 PASS, 23 expected FAIL, 25 expected TIMEOUT, and 19 expected NOTRUN.
The implementation was compact but made readers unpack argument conversion, security checks, and asynchronous execution at the same time. Put installation and public operations first, move Web IDL conversion into named local helpers, and distinguish the caller's settlement from the callback's completion. Make discovery an explicit copy, sort, and return sequence. Preserve optional member omission and Web IDL property order, with an assertion in the existing metadata test. Expand guards and give stored schemas and signals names that explain their roles. This deliberately favors reading order over line count. Flatten browser tests so each action finishes before its outcome is asserted. Record the readability convention for future edits. No public API, dependency, WPT selection, or expectation changes. References: https://webidl.spec.whatwg.org/#es-dictionary https://webmachinelearning.github.io/webmcp/ Validation: pnpm test (100 passes), pnpm test:package, formatting, and pnpm test:wpt on Canary 156.0.8062.0. All 58 WPT files match expectations: 72 PASS, 23 expected FAIL, 25 expected TIMEOUT, and 19 expected NOTRUN.
Expand dense test phases, fixture code, and the WPT runner so the combined registration and execution work is easier to follow. Review every tracked file and comment; explain actual WPT blockers without changing expectations. Replace Reflect.get/apply in origin conversion while retaining a single iterator lookup and the original receiver. Cover both registration and discovery with an iterator whose function properties cannot be read. Run all 15 Rook anti-slop rules externally without adding the plugin here. Keep required Web IDL boundary types and browser checks. Document the schema JSON/type mismatch instead of claiming an unchecked cast is safe. Reference: https://webidl.spec.whatwg.org/#es-sequence Validation: 100 browser tests, lint, build, typecheck, packed consumer, formatting, and diff checks pass. WPT: 58 files / 139 subtests, no unexpected results (72 pass, 23 expected fail, 25 expected timeout, 19 expected notrun).
Use plain conditionals for schema serialization, title conversion, and the Permissions Policy fallback. Each step now reads in execution order without mixing an optional value, conversion, and fallback into one expression. Preserve the existing undefined/null checks and property access order. Validation: lint, build, typecheck, 100 browser tests, packed consumer, formatting, and WPT pass with no unexpected results. All 29 previously documented anti-slop diagnostics are unchanged.
Calling a toolchange handler through its own call property skips valid callbacks when authors shadow that property. Invoke it with Reflect.apply, and honor a false return value by cancelling cancelable events. Both bugs were reproduced against native Chromium and are covered by browser tests. Use the same direct invocation for origin iterators instead of the less readable Function.prototype.call.bind workaround. Keep concise conditional assignments and remove workflow labels from the remaining comments. References: https://html.spec.whatwg.org/multipage/webappapis.html#event-handler-processing-algorithm https://webidl.spec.whatwg.org/#es-sequence Validation: pnpm test (106 browser tests), pnpm test:package, formatting, and git diff --check passed. Pinned WPT ran 58 files and 139 subtests with no unexpected results: 72 PASS, 23 expected FAIL, 25 expected TIMEOUT, and 19 expected NOTRUN. All 22 IDL subtests passed.
Separate the implementation, local tests, and upstream WPT runner into src/, tests/, and wpt/ so contributors can find each part without scanning the repository root. Run the Node scripts directly as TypeScript on Node 24 and include them, plus the browser fixture JavaScript, in strict type checking. Update CI, documentation, and ignored cache paths while keeping the package entry points and WPT expectations unchanged. Use the caller signal and native promise settlement instead of duplicate execution state. Share USVString conversion and load the official global types through the test config. Preserve a reflexive sort comparator and cover cancellation during result serialization, before queued delivery. References: https://nodejs.org/docs/latest-v24.x/api/typescript.html#type-stripping https://webidl.spec.whatwg.org/#es-USVString https://tc39.es/ecma262/multipage/indexed-collections.html#sec-sortindexedproperties https://webmachinelearning.github.io/webmcp/#dom-modelcontext-executetool Validation: lint, type checking, formatting, 109 browser tests, and the packed-consumer checks passed. Pinned WPT ran 58 files and 139 subtests with no unexpected results: 72 PASS, 23 expected FAIL, 25 expected TIMEOUT, and 19 expected NOTRUN. All 22 IDL subtests passed.
Route discovery and execution to the tool's owning frame so exposedTo and fromOrigins work across the frame tree. Authenticate window handshakes, carry requests and cancellation over MessageChannel, and enforce exposure and inherited iframe permissions at the receiver. Use webmcp-types 0.1.9, default omitted execution input to a fresh object, and enable strict optional-property and indexed-access checks. Keep conversion at API boundaries and share normalized frame metadata. Expand real-browser and packed-consumer coverage, advance the WPT pin, and document remaining conformance gaps rather than hiding them behind test exclusions. Clarify startup timing and keep generated local files out of version control.
|
@beaufortfrancois this one's good to review. Would love your thoughts on the auto global install ( |
|
|
||
| const protocol = "webmcp-polyfill"; | ||
| // Bounds a handshake and a control reply; a running tool callback has no deadline. | ||
| const deadline = 500; |
There was a problem hiding this comment.
@beaufortfrancois curious to hear your thoughts on this
There was a problem hiding this comment.
This seems like a good compromise to me.
| await this.#started.promise; | ||
| const ownOrigin = this.#window.origin; | ||
| const origins = [ownOrigin, ...fromOrigins]; | ||
| const results = await Promise.all( |
There was a problem hiding this comment.
@beaufortfrancois all settled vs all? wdyt?
There was a problem hiding this comment.
I'd keep all here.
beaufortfrancois
left a comment
There was a problem hiding this comment.
What a great PR!
| throw new Error("The page-title tool is unavailable"); | ||
| } | ||
|
|
||
| const result = await context.executeTool(pageTitleTool, {}); |
There was a problem hiding this comment.
Nit: {} is optional.
| await this.#started.promise; | ||
| const ownOrigin = this.#window.origin; | ||
| const origins = [ownOrigin, ...fromOrigins]; | ||
| const results = await Promise.all( |
There was a problem hiding this comment.
I'd keep all here.
| return reply.value.map((value: unknown) => | ||
| toRegisteredTool(readToolMetadata(value), peer, reply.origin), | ||
| ); | ||
| } catch { |
There was a problem hiding this comment.
This catch swallows readToolMetadata validation failures, #request timeouts, and genuine programming errors identically, all as "this frame has no tools". Could we at least console.warn on non-executionError throws?
| testIgnore: [], | ||
| use: { | ||
| browserName: "chromium", | ||
| launchOptions: { args: ["--enable-experimental-web-platform-features"] }, |
There was a problem hiding this comment.
Do you need this option?
| File and subtest counts catch missing coverage and early harness exits. | ||
| Results and browser details are written to `wpt-results/report.json`. | ||
|
|
||
| ### Recorded results |
There was a problem hiding this comment.
Where are the results for webkit and firefox?
| } | ||
| peerOrigin = event.origin; | ||
| clearTimeout(timer); | ||
| if (request.kind !== "execute") { |
There was a problem hiding this comment.
Since dispatchEvent is synchronous, every page-author toolchange listener in that iframe runs inside the requester's 500 ms budget. We can opt-out then. WDYT?
| if (request.kind !== "execute") { | |
| // Both run page-author code of unbounded duration: tool callbacks and toolchange listeners. | |
| if (request.kind !== "execute" && request.kind !== "changed") { |
|
|
||
| const protocol = "webmcp-polyfill"; | ||
| // Bounds a handshake and a control reply; a running tool callback has no deadline. | ||
| const deadline = 500; |
There was a problem hiding this comment.
This seems like a good compromise to me.
| <script> | ||
| const context = document.modelContext; | ||
| Promise.all([ | ||
| context.registerTool({ name: "startup", description: "Startup", execute: () => "startup" }), |
There was a problem hiding this comment.
Shouldn't we await first for registerTool and call getTools after?
| return view; | ||
| } | ||
|
|
||
| // Message tasks approximate the WebMCP task source. Chained zero-delay timers are clamped to |
There was a problem hiding this comment.
That's a good approximation indeed!
This adds WebMCP registration, discovery, and execution on
document.modelContext, including tools in participating frames. It combines the original implementation with the execution work in #2 and uses the releasedwebmcp-types@0.1.9, following the upstream execution types work.Import
webmcp-polyfill/autofor installation, or callinstallWebMCP()explicitly. Existing native contexts are preserved. Registration signals unregister tools; execution accepts object input, defaults omitted input to a fresh empty object, and returns JSON-serialized results. Public declarations and schema inference come from the types package.Frames announce their presence with window messages, authenticate source and origin when transferring a
MessagePort, and use that channel for discovery, execution, and cancellation. The owner checks tool exposure and inherited iframe permissions. Each participating frame must load the polyfill. Initial discovery and channel setup have bounded waits; running callbacks have no execution deadline.The two design choices most useful to review are the automatic installation entry point and the MessageChannel protocol, including its discovery, permission checks, and navigation cleanup.
Validation
noUncheckedIndexedAccessandexactOptionalPropertyTypes.2699eaa, on Chrome Canary 156.0.8066.0, runs all 67 WebMCP testharness files and 161 subtests: 96 PASS, 23 expected FAIL, 24 expected TIMEOUT, and 18 expected NOTRUN. All 22 pinned IDL subtests pass; there are no unexpected results. Non-testharness tests are outside this lane.f5645e9.This does not claim full conformance. Declarative forms and lifecycle events remain unimplemented; native task ordering, Permissions Policy introspection, and some frame lifecycle behavior have documented limits. Some WPT assertions expect raw strings rather than the draft's JSON-serialized results. TESTING.md records the disagreements, expected failures, and browser limitations separately.
The test runner includes the requested discovery coverage, uses standard WPT expectations, and rejects unexpected passes as well as failures. The local fixture uses a Node server; Python is needed only by upstream WPT.
Extension integration remains separate in #3.