diff --git a/CHANGELOG.md b/CHANGELOG.md index f4776c98b2..60a58b82df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,11 @@ ## Unreleased - Fix `ensureDaemon()`, the CLI and MCP killing a daemon they had just asked to leave. After the daemon answers the quiesce request with `draining`, the client now waits up to 60 s for it to exit by itself and clean up its registration, so plugin disposal and lease finalization finish (on Windows a signal ends the process outright, and on POSIX a drain longer than 3 s was killed). A daemon still running at the deadline that fails its liveness probes is force-retired; one that still answers is left running and the call fails with `details.reason: 'daemon-drain-timeout'`, with a hint naming `agent-device daemon stop --state-dir `. +- Provider plugins have a new contract. A package exports a factory at `./agent-device` that returns `{ apiVersion: 2, provider, credentials, connection?, setup(host) }`, typed by `AgentDevicePluginFactory` from `agent-device/plugins`. The store calls the factory with the package's `config.json` options, and a JavaScript host can call it directly. Version 1 plugins (`agentDevicePlugin.apiVersion: 1`) keep loading. +- Bundled providers and installed plugins now register through one registry, in that order, with a third slot for embedder plugins (#3757). The first plugin to name a provider id claims it; a later duplicate is recorded as failed, and `cloud` and `proxy` stay reserved. +- A plugin that fails to load, validate, or set up no longer stops the daemon. It is listed as `failed` with a typed `error.reason`, and the other providers still register. The new `agent-device daemon status [--json]` shows the inventory, which the daemon also publishes in `daemon.json`; `plugins list --json` works without a daemon and reports a running daemon's inventory under `daemon`. +- The client reads provider credential variable names from that inventory, narrowed per lease backend, and falls back to version 1 manifests. A Limrun credential fingerprint now hashes the exact variable values, so padding whitespace around a value counts as a change, as it does for BrowserStack. +- A partial Limrun instance set now appears in `daemon status` as a failed provider and as `provider_plugin_failed` in `daemon.log`, replacing `provider_runtime_skipped`. - Behavior change: the CLI and MCP now call `ensureDaemon()` explicitly once their config is resolved, then attach to the daemon it leaves. A command that finds a daemon from another version or build replaces it only if the daemon reports no request, session, repair session or live lease. A resumable upload still streaming and a host human-control hold that names no lease are not counted, so a replacement can end them. A daemon that answers nothing on any transport is retired after repeated failed liveness probes, which cannot show it idle. A busy one fails the command with `details.reason: 'daemon-busy'` instead of being replaced with its sessions, and a daemon from a release that predates quiescing fails it with `'daemon-incompatible'`; stop either with `agent-device daemon stop --state-dir `. The takeover notice moves from `Replacing daemon … : ` to `Replaced idle daemon …` (an incompatible daemon) or `Replaced unreachable daemon …` (one that answered nothing). A request that times out no longer retires an unresponsive daemon from the CLI or MCP; the error names `agent-device daemon stop --state-dir `. A one-shot `replay` or `test` with an explicit `--state-dir` leaves the daemon it started running until its idle reap instead of stopping it. Remote `--daemon-base-url` connections and replay on its private ephemeral daemon are unchanged. Node.js clients keep `daemonLifecycle: 'managed'` until 0.22. - Add `ensureDaemon({ stateDir })` and the client config `daemonLifecycle: 'attach' | 'managed'`. An `attach` client connects to the daemon registered in the state directory and never starts, replaces or kills one, including after a request times out. It fails with `details.reason: 'daemon-not-running'` (the hint points at `ensureDaemon()`) or `'daemon-incompatible'` (both identities in `details`, the hint names `agent-device daemon stop`). `ensureDaemon()` starts a missing daemon and replaces an incompatible one that answers only through an authenticated quiesce request the daemon answers itself (a resumable upload still streaming and a human-control hold that names no lease are not counted, so a replacement can end them): it refuses with `daemon-busy` while any request is in flight or any session, repair session or live lease exists, and a daemon from a release before this one is never replaced. A registered daemon that answers nothing is retired, as is one that times out on the quiesce request and then fails repeated liveness probes; neither can be asked. A remote `daemonBaseUrl` connection never starts or retires a local daemon in either mode. `managed` stays the default and behaves as before until 0.22. - `lease_allocate` (and `client.leases.allocate()`) with a `leaseProvider` that no registered runtime claims now refuses before recording a lease, with `UNSUPPORTED_OPERATION`, `error.details.reason: provider-not-registered`, `details.provider`, `details.registeredProviders`, and `details.failedPlugins` naming a plugin that failed to register. Previously a plugin provider that the daemon never loaded passed admission and recorded a lease with no device behind it. Leases with no provider (a `leaseBackend` alone included), `proxy` and `cloud` need no runtime and still allocate. Restart the daemon with the provider configured, then retry. diff --git a/docs/adr/0034-provider-plugin-registry.md b/docs/adr/0034-provider-plugin-registry.md new file mode 100644 index 0000000000..5e1f971ce4 --- /dev/null +++ b/docs/adr/0034-provider-plugin-registry.md @@ -0,0 +1,72 @@ +# 0034. Provider plugin registry + +Status: accepted. Umbrella #3747; contract and registry #3755. + +## Rules at a glance + +- One contract. A provider plugin is a factory that returns `{ apiVersion: 2, provider, credentials, + connection?, setup(host) }`. `setup` returns `{ runtime, platformModule, dispose? }` or + `{ webDriver, dispose? }`. A `{ webDriver }` result stays a data descriptor the host interprets. +- One registry. Sources compose in the order built-in, store, embedder; the embedder slot arrives with #3757. Built-in providers use the + same shape as an installed plugin, and one function validates what every factory returns. +- Plugins import only types from agent-device. The type-only contract is `agent-device/plugins`; + there is no runtime helper, and the contract does not depend on a plugin's module URL. +- The first registration of a provider id claims it, even when that registration fails. Later ones + are recorded as failed. `cloud` and `proxy` are routing identities no plugin can claim. +- A plugin that fails to enumerate, load, validate or set up is recorded and skipped. The daemon + starts with the rest. The inventory lists every plugin as `active` or `failed`, with a typed + `error.reason`, and is published in `daemon.json` and by `agent-device daemon status`. +- `registry.close()` is the single, idempotent cleanup owner. It shuts down each runtime, then calls + its `dispose`, and returns an outcome per plugin and step. Runtimes are never wrapped. + Composition is transactional: when a later step fails, it closes the registry before it throws. +- The registry receives candidates. Composition chooses the sources. +- `daemon.json` has one reader. Its states are `absent`, `unreadable`, and `invalid`. A missing or + stale record reads as a stopped daemon; an unreadable or invalid one never does. +- The plugin set is fixed for a daemon's lifetime. A plugin change takes effect on the next daemon. +- v1 plugins (`agentDevicePlugin.apiVersion: 1`) without a `./agent-device` export are dispatched by + their manifest before any code runs and load through an adapter until 0.22 removes them (#3759). + A package that keeps the manifest and also exports `./agent-device` loads as v2; hosts without + v2 still read the manifest. + +## Context + +Three things loaded providers: a hand-kept `RESERVED_PLUGIN_PROVIDERS` list, a second list of +providers the daemon must have a runtime for, and a store that treated one bad plugin as a daemon +startup failure. Each provider kind also needed its own path for credential fingerprints and for +`connect` capabilities, which read manifests the bundled providers do not have. An embedding host +had to fake the store to load a plugin from its own dependencies (#3747). + +## Decisions + +- **A factory takes options.** The store calls it with `config.json` `plugins[name].options`; a JS + host calls it directly with its own, such as credentials. A plugin that takes credentials as + options declares no environment variables for them, so they stay out of the environment + fingerprint. +- **A package exports its entry as `./agent-device`.** The host resolves it with import + conditions and falls back to the package root only when the subpath is absent. A subpath whose + import or setup throws never falls back. A v2 package needs no manifest. +- **The registry owns identity, not the retired lists.** Duplicate ids and bundled ids are one + question answered by claim order. Credential variables, per-backend narrowing and connect + capabilities are read from the registry's plugins, not from manifests. A v1 manifest still + supplies them for a v1 plugin. +- **Failure is data.** A failed plugin is an inventory entry, not an exception. Behavior keys on + `error.reason`, never on the message. +- **A client reads a daemon's inventory from `daemon.json`.** The client credential fingerprint + takes variable names from it and falls back to v1 manifests. A v2 plugin declares its connect + capabilities in code, so a client loads the store, without setting up a runtime, before it reads + them. + +## Daemon ownership + +The target is that starting or replacing a daemon is an explicit act, not a side effect of a client +call. Today a client call can still start a daemon; #3754 makes that explicit for the client +lifecycle, and #3757 adds `agent-device/host`, where an embedder owns its daemon and passes its +plugins as the third source. + +## Alternatives refused + +- **Keep the manifest as the contract.** It cannot describe options-dependent credentials, and a + bundled provider has no package to hold one. +- **A runtime helper such as `definePlugin`.** It would make plugins depend on a runtime export of + agent-device and its version, which is what a type-only contract avoids. +- **Fail the daemon on a bad plugin.** One broken optional package then takes every provider down. diff --git a/docs/adr/README.md b/docs/adr/README.md index de4f5fa894..1c56e3cb49 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -35,6 +35,7 @@ | [0031 macOS Native App Backend](0031-macos-native-app-backend.md) | `AGENT_DEVICE_MACOS_APP_BACKEND`, driving macOS app sessions without XCTest Automation Mode, why pointer actions are accessibility actions only, and the ghost cursor | | [0032 Layering Graph Engine](0032-layering-graph-engine.md) | evaluating replacement of `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, why graph-based layering checks share one replaceable import model, the measured edge-set and parity gaps, and the revisit triggers | | [0033 Collocation decision table](0033-collocation-decision-table.md) | moving or adding a file listed in umbrella #3276 §5, re-proposing a move that table refused, reading a zone's collocation decisions | +| [0034 Provider plugin registry](0034-provider-plugin-registry.md) | provider plugins, the `agent-device/plugins` contract, plugin sources and order, duplicate provider ids, plugin failure isolation, the daemon plugin inventory and `daemon status` | ADRs record *why*; the registries and gates they describe are the living source of truth — when prose and a registry disagree, the registry wins and the ADR needs a follow-up. diff --git a/website/docs/docs/build-an-integration.md b/website/docs/docs/build-an-integration.md index 0d4c814463..b19b124860 100644 --- a/website/docs/docs/build-an-integration.md +++ b/website/docs/docs/build-an-integration.md @@ -85,7 +85,7 @@ Entry points for integrations: - `agent-device/plugins/webdriver` - experimental `WebDriverPluginOptions` for providers using the shared engine. - `agent-device/plugins` - - experimental factory context: `ProviderPluginHost`; see [provider plugins](./plugins.md). + - experimental plugin contract: `AgentDevicePluginFactory` and `ProviderPluginHost`; see [provider plugins](./plugins.md). ## Android ADB providers diff --git a/website/docs/docs/limrun.md b/website/docs/docs/limrun.md index d95345efa1..ebf13c8f16 100644 --- a/website/docs/docs/limrun.md +++ b/website/docs/docs/limrun.md @@ -29,7 +29,7 @@ An agent in a sandbox can drive an instance that someone else created, such as t | iOS | `LIM_IOS_INSTANCE_URL` (`status.apiUrl`), `LIM_IOS_INSTANCE_TOKEN` (`status.token`) | | Android | `LIM_ANDROID_INSTANCE_URL` (`status.apiUrl`), `LIM_ANDROID_INSTANCE_TOKEN` (`status.token`), `LIM_ANDROID_INSTANCE_ADB_URL` (`status.adbWebSocketUrl`) | -The `lim` CLI reads the same variables, so one set works for both tools. Set all of a platform's variables or none. With a partial set, `connect` fails, and the daemon disables Limrun and records `provider_runtime_skipped` in `daemon.log`. +The `lim` CLI reads the same variables, so one set works for both tools. Set all of a platform's variables or none. With a partial set, `connect` fails, and the daemon registers without Limrun, lists it as `failed` in `agent-device daemon status`, and records `provider_plugin_failed` in `daemon.log`. ```bash export LIM_IOS_INSTANCE_URL=... diff --git a/website/docs/docs/plugins.md b/website/docs/docs/plugins.md index 499661d2c1..4ceddbf591 100644 --- a/website/docs/docs/plugins.md +++ b/website/docs/docs/plugins.md @@ -17,49 +17,116 @@ Install only packages you trust: plugin factories run with the daemon's host per A failed install keeps the previous selection. A running daemon keeps the plugin set and files it started with, including removed installations. To apply changes, close sessions and run `agent-device daemon stop` with the matching `--state-dir`; the next device command uses the new set. Use separate state directories for different plugin homes. -Compatibility checks run offline against local metadata; provider operations may still need network access. An incompatible plugin stops the daemon from starting. Run `plugins list --json` to see the errors, then update or remove the affected package. Upgrading agent-device does not download compatible plugin versions for you. +## See which providers are registered -## Create a plugin - -The interface is experimental. Publish this declaration in your package manifest: +`agent-device daemon status` lists every provider the running daemon registered: the bundled ones (BrowserStack, AWS Device Farm, and Limrun when it is configured), then installed plugins. With `--json`, each entry has: ```json -{"agentDevicePlugin": {"apiVersion": 1, "provider": "example", "entry": "./dist/plugin.mjs"}} +{ + "provider": "testmu", + "source": "store", + "package": "@agent-device/testmu", + "version": "0.21.29", + "state": "active", + "credentials": { "variables": ["LT_USERNAME", "LT_ACCESS_KEY"], "byBackend": {} } +} ``` -Use `agent-device` as a development dependency. The default factory accepts `ProviderPluginHost` from `agent-device/plugins`: `env`, package-specific `options`, `clientVersion`, and `createError` for host-recognized errors. Return `{ runtime, platformModule }` implementing the [provider runtime](https://github.com/callstack/agent-device/blob/main/packages/contracts/src/provider-device-runtime.ts) and [platform module](https://github.com/callstack/agent-device/blob/main/packages/contracts/src/platform-runtime-operations.ts) contracts. Both provider IDs must match the manifest; the module owner must declare `kind: 'provider-runtime'` and a nonempty `instance`. Core checks required runtime methods and owner metadata at startup; operation contracts are checked when used. Provider packages own implementation types; the SDK exposes only factory context. Set options through `plugins[""].options` in user config; leave `installation` unchanged. +`source` is `builtin` or `store`; `embedder` is the third slot, reserved for plugins an embedding host passes itself (#3757). A plugin that cannot register has `state: "failed"` and an `error` with `code`, `message`, and a `reason` such as `invalid_plugin`, `incompatible_plugin`, or `duplicate_provider`. The daemon starts without it and keeps every other provider. Fix or remove the package, then stop the daemon so the next command starts a new one. -Keep the factory fast, with no network I/O or device allocation; a stalled factory blocks daemon startup. Load platform mechanics through `platformModule.loadRuntime` and perform remote work in request-bound operations. A failing factory cleans up its own resources; core shuts down previously returned runtimes if another plugin fails. +When a daemon's `daemon.json` cannot be used, `daemon status` says so instead of reporting a stopped daemon: `unreadable` means the file exists but could not be read, and `invalid` means it was read but is not a daemon record. A missing or stale record means `stopped`; an unreadable or invalid file never does. -Incompatible contract changes require a new API version. Plugins cannot replace bundled providers or register arbitrary commands. Plugins can add `connect ` through the connection callbacks described below. Limrun, BrowserStack, and AWS Device Farm remain bundled. +When the daemon stops, it closes every plugin once: each runtime shuts down, then its `dispose` runs. A step that fails is logged as `provider_plugin_cleanup_failed` with the provider and step, and the other plugins still close. + +`plugins list --json` works without a daemon. It lists your installed plugins with their provider and whether their factory loads and validates; whether `setup` succeeds shows only in a running daemon's inventory. When a daemon is running for the same `--state-dir`, it also reports that daemon's inventory under `daemon`. + +When two plugins claim the same provider, the first one wins: bundled providers, then installed plugins in package-name order, then, once #3757 lets an embedding host pass plugins, those. The others are `failed` with `duplicate_provider`. The first claim holds even when that plugin then fails to set up, so a later plugin for the same provider doesn't take over. `cloud` and `proxy` are reserved and can't be claimed. + +A lease that names a provider this daemon has no runtime for refuses with `UNSUPPORTED_OPERATION` and reason `provider-not-registered`, before any lease is recorded. The error lists the registered providers and, as `failedPlugins`, any plugin that failed to register for that provider (or failed before its provider was known); restart the daemon with the provider's plugin installed. Leases with no provider, `proxy`, and `cloud` need no provider runtime. -For an Appium or WebDriver service, return `{ webDriver: options }` instead of building an engine. `WebDriverPluginOptions` is available through the type-only `agent-device/plugins/webdriver` import. Core supplies the client version and creates the shared runtime. Provider callbacks prepare sessions, upload apps, and retrieve artifacts. +## Create a plugin -To support `agent-device connect example`, declare `agentDevicePlugin.connection` in the package manifest: +The interface is experimental. A plugin package exports a factory at `./agent-device`. Use `agent-device` as a development dependency; plugins import only types from it. ```json { - "agentDevicePlugin": { - "apiVersion": 1, - "provider": "example", - "entry": "./dist/plugin.mjs", - "connection": { - "leaseKind": "direct-device-provider", - "requiresAppAttachment": false, - "requiresRemoteDaemon": false, - "supportsArtifacts": false, - "supportsDeferredAppSelection": true, - "supportsDirectPortReverse": false, - "usesCloudWebDriverLease": false - } + "name": "@example/agent-device-provider", + "type": "module", + "exports": { + "./agent-device": { "import": "./dist/plugin.mjs" } } } ``` -If the provider reads credentials from the environment, list the variables in `agentDevicePlugin.credentialVariables`, for example `["EXAMPLE_USERNAME", "EXAMPLE_ACCESS_KEY"]`. A local daemon keeps the values it started with; when the shell holds different ones, the first command that allocates a lease refuses with reason `provider-credentials-changed` until the daemon is stopped. Core reads this list from the manifest without loading the plugin, and treats a whitespace-only value as unset. +```ts +import type { AgentDevicePluginFactory } from 'agent-device/plugins'; -A lease that names a provider this daemon has no runtime for refuses with `UNSUPPORTED_OPERATION` and reason `provider-not-registered`, before any lease is recorded. The error lists the registered providers and, as `failedPlugins`, any plugin that failed to register for that provider (or failed before its provider was known); restart the daemon with the provider's plugin installed. Leases with no provider, `proxy`, and `cloud` need no provider runtime. +type ExampleOptions = { credentials?: { username: string; accessKey: string } }; + +const example: AgentDevicePluginFactory = (options = {}) => ({ + apiVersion: 2, + provider: 'example', + credentials: { + // Variables the daemon reads credentials from; none when the caller passes them as options. + variables: options.credentials ? [] : ['EXAMPLE_USERNAME', 'EXAMPLE_ACCESS_KEY'], + }, + setup: async (host) => ({ runtime, platformModule }), +}); + +export default example; +``` + +The factory is called with the package's `options` from `config.json` (`plugins[""].options`; leave `installation` unchanged). A JavaScript host that imports the package calls the same factory directly with its own options. + +The host resolves the `./agent-device` export with import conditions. It falls back to the package's root export only when that subpath does not exist. If the subpath exists but its import or `setup` throws, the plugin is reported as failed and the root export is not tried. + +`setup(host)` receives `ProviderPluginHost`: `env`, `options`, `clientVersion`, and `createError` for host-recognized errors. Return one of: + +- `{ runtime, platformModule, dispose? }` implementing the [provider runtime](https://github.com/callstack/agent-device/blob/main/packages/contracts/src/provider-device-runtime.ts) and [platform module](https://github.com/callstack/agent-device/blob/main/packages/contracts/src/platform-runtime-operations.ts) contracts. The runtime's `provider` and the module owner's `provider` must equal the plugin's `provider`, and the module owner must declare `kind: 'provider-runtime'` and a nonempty `instance`. +- `{ webDriver, dispose? }` for an Appium or WebDriver service. `WebDriverPluginOptions` is available through the type-only `agent-device/plugins/webdriver` import. The host supplies the client version and creates the shared runtime. Provider callbacks prepare sessions, upload apps, and retrieve artifacts. -Return `connection` alongside the runtime or WebDriver options. Its `resolve({ flags, env, cwd, stateDir })` callback validates provider flags and returns `{ profile, extraFlags? }`; `profile.leaseProvider` must match the manifest. Core supplies connection identity, session defaults, Metro settings, and persists the profile. Its async `verify({ flags, env })` callback returns the provider verification result. These callbacks run without allocating a device and their temporary runtimes are shut down afterwards. +The host checks required runtime methods and owner metadata when it sets the plugin up; operation contracts are checked when used. `dispose` runs once after the runtime shuts down. A failing `setup` cleans up its own resources; the host shuts down a runtime it already received if validation then refuses it. + +Keep the factory and `setup` fast, with no network I/O or device allocation; a stalled plugin delays daemon startup. Load platform mechanics through `platformModule.loadRuntime` and perform remote work in request-bound operations. + +Incompatible contract changes require a new `apiVersion`. Plugins can't replace bundled providers or register arbitrary commands. + +### Credentials + +List the environment variables the provider reads in `credentials.variables`. A local daemon keeps the values it started with; when the shell holds different ones, the first command that allocates a lease refuses with reason `provider-credentials-changed` until the daemon is stopped. The check treats a whitespace-only value as unset. If a lease reads fewer variables on one backend, narrow them with `credentials.byBackend`, for example `{ "android-instance": ["EXAMPLE_API_KEY"] }`. Credentials the caller passes as factory options are not in the environment, so list no variable for them. + +### Connect + +To support `agent-device connect example`, add a `connection` to the plugin: + +```ts +connection: { + capabilities: { + leaseKind: 'direct-device-provider', + requiresAppAttachment: false, + requiresRemoteDaemon: false, + supportsArtifacts: false, + supportsDeferredAppSelection: true, + supportsDirectPortReverse: false, + usesCloudWebDriverLease: false, + }, + resolve: async ({ flags, env, cwd, stateDir, host }) => ({ profile }), + verify: async ({ flags, env, host }) => verification, +}, +``` + +`resolve` validates provider flags and returns `{ profile, extraFlags? }`; `profile.leaseProvider` must match the plugin's `provider`. The host supplies connection identity, session defaults, Metro settings, and persists the profile. `verify` returns the provider verification result. Both also receive `host`, the same `ProviderPluginHost` that `setup` gets, built from the `env` of that call. Both run without allocating a device or setting up a runtime. `agent-device` loads installed plugins' factories, not their `setup`, to read these capabilities. + +### Package for installation + +Bundle the plugin implementation and ship ready-to-run ESM: installation disables lifecycle scripts. Bundle shared helpers with the plugin; `AppError` carries a shared brand so the host preserves its code and details across package copies. Import types with `import type` to keep them out of the runtime dependency graph. + +## Version 1 plugins + +Packages that declare `agentDevicePlugin` with `apiVersion: 1` in `package.json` keep working. The host reads the manifest's `provider`, `entry`, `connection`, and `credentialVariables` before running any plugin code, then calls the default export with `ProviderPluginHost` and registers the `{ runtime, platformModule }` or `{ webDriver, connection? }` it returns. Support for this form ends in a later release; move to the `./agent-device` factory. + +```json +{"agentDevicePlugin": {"apiVersion": 1, "provider": "example", "entry": "./dist/plugin.mjs"}} +``` -Bundle the plugin implementation and ship ready-to-run ESM: installation disables lifecycle scripts. Bundle shared implementation helpers with the plugin; `AppError` carries a shared brand so core preserves its code and details across package copies. Import types with `import type` to keep them out of the runtime dependency graph. +A package that exports `./agent-device` is loaded as a version 2 plugin even if it also keeps the manifest, so one package can serve both old and new hosts.