diff --git a/contracts/agents-api/environments.md b/contracts/agents-api/environments.md index a1fcd2752..b984dd8ac 100644 --- a/contracts/agents-api/environments.md +++ b/contracts/agents-api/environments.md @@ -75,8 +75,6 @@ The application owns the machine. It creates the Session with a clean absolute ` - Compute, workspace and files stay the application's. Deleting the Session or revoking the credential denies further access but does not stop native processes; the machine owner stops and cleans up. - The workspace must survive a daemon restart. Losing it never authorizes silent replacement or replay. -**Application-managed E2B.** An application can run the Runtime in an E2B sandbox it creates, renews and destroys with the E2B SDK, then enroll that Runtime as a `self_hosted` Environment ([E2B Runtime guide](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/deploy/e2b/README.md)). Core keeps no E2B allocation for it and never renews or kills it. - ### Ownership rules - Keep Environment identity, ownership, configuration and lifecycle in Core, separate from Provider compute, device identity, daemon sockets and native sessions. Keep mutable connection state out of immutable configuration; a replacement owner fences stale observations. diff --git a/contracts/agents-api/machine-api.md b/contracts/agents-api/machine-api.md index 65ef4200c..2eecb098c 100644 --- a/contracts/agents-api/machine-api.md +++ b/contracts/agents-api/machine-api.md @@ -33,7 +33,6 @@ The generated [`runtime.openapi.yaml`](./runtime.openapi.yaml) describes only th | Node credential | The node itself: it generates a secret of 32 to 256 characters without whitespace and registers it at enrollment | `sandbox-node/configuration` with `X-OAC-Node-ID`, `sandbox-node/identity`, `sandbox-node/connect` | | Installation grant | The `x_agents_core.installation` command of a `self_hosted` Session; short-lived | `agent-daemon/installation` and its `claim` | | Executor credential | The installation claim, or the Core-key [executor credential routes](./environment-executor-credentials.md) | `agent-daemon/enroll` and `agent-daemon/connection`; after enrollment it is also the Serve credential of the Environment's enrollment on `sandbox-link` | -| Daemon credential of a hosted sandbox | Core, for each managed allocation, delivered in the [bootstrap file](../../docs/runtime-bootstrap.md) | `agent-daemon/bootstrap`, `device-status` and `ws` | | Operator device profile | `oac-core-device`, run by an operator with database access | `agent-daemon/bootstrap`, `device-status` and `ws` | Core keeps only a SHA-256 digest of each token and credential it stores; installation grants are signed and not stored. Credentials are not interchangeable: each works only on its own routes. diff --git a/contracts/agents-api/node-generation-protocol.md b/contracts/agents-api/node-generation-protocol.md index b5bfe951a..2d6969653 100644 --- a/contracts/agents-api/node-generation-protocol.md +++ b/contracts/agents-api/node-generation-protocol.md @@ -6,7 +6,7 @@ A sandbox node runs the Docker or microsandbox Provider on its host and connects ## Frames and version -Every frame is one JSON text message whose `version` equals `node.ProtocolVersion`; both peers reject any other version, and there is no fallback decoder. Member names are exact and unique: unknown members, case aliases, duplicates and unexpected nulls are rejected. Control frames (`hello`, `welcome`, `heartbeat`, `heartbeat_ack`, `retention`, `retention_ack`) are at most 32 KiB; `request` and `response` frames at most 72 MiB. An invalid frame closes the connection. +Every frame is one JSON text message whose `version` equals `node.ProtocolVersion`; both peers reject any other version, and there is no fallback decoder. Member names are exact and unique: unknown members, case aliases, duplicates and unexpected nulls are rejected. Control frames (`hello`, `welcome`, `heartbeat`, `heartbeat_ack`, `retention`, `retention_ack`) are at most 32 KiB; `request` and `response` frames at most 1 MiB. An invalid frame closes the connection. ## Connection @@ -41,19 +41,17 @@ Each operation carries its own arguments and returns the following result on suc | `info` | `GetInfo` | None | `info` | | `renew` | `Renew` | None | `info` | | `kill` | `Kill` | None | None | -| `command` | `RunCommand` | `command` | `command` | | `observe` | `Observe` | `observation` | `sample` | | `initial` | `Initial` | None | `compute` | | `new_compute` | `NewCompute` | Positive compute `generation` and optional `snapshot` | `compute` | | `compute` | `GetCompute` | `compute` | `state` | | `kill_compute` | `KillCompute` | `compute` | None | | `resume_compute` | `ResumeCompute` | `compute` | `state` | -| `command_compute` | `RunCommandCompute` | `compute` and `command` | `command` | | `suspend` | `Suspend` | `suspend` | `state` | | `resume` | `Resume` | `resume` | `state` | | `delete_snapshot` | `DeleteSnapshot` | `snapshot` | None | -`bootstrap` is the Provider's `sandbox.Bootstrap`, including the [Sandbox bootstrap](../../docs/sandbox-bootstrap.md) input in `SandboxIO`; Core validates it before it sends `create`. +`bootstrap` is the Provider's `sandbox.Bootstrap`: the reference and the [Sandbox bootstrap](../../docs/sandbox-bootstrap.md) input in `SandboxIO`; Core validates it before it sends `create`. A request whose `connection_id`, `owner_epoch` or `sequence` does not match closes the connection. A malformed request gets an `invalid` response. A node without generation management accepts only its enrolled `deployment_generation`; a generation-managing node runs the request on that generation's provider and answers `unconfirmed` when it cannot. Core sends `create` and a `resume` that is not observe-only only to a generation that is ready on that node, and keeps at most 32 requests pending per connection. @@ -64,14 +62,13 @@ The `response` frame carries `id` and `connection_id`. A successful response car | `error_code` | Meaning | | --- | --- | | `invalid`, `ownership`, `exists`, `not_found` | `ErrInvalid`, `ErrOwnership`, `ErrExists`, `ErrNotFound` | -| `command_unconfirmed` | `ErrCommandUnconfirmed` | | `observation_unavailable`, `runtime_not_running` | The observation outcomes | | `unsupported` | The operation is declared unsupported; see below | | `unconfirmed`, or any other value | The outcome is unknown | A failed response carries no result, except an `info` that is an exact-reference `CreateSettled` receipt: a confirmed native Create that failed a later check can still prove that the attempt settled. A timeout, a lost response or a disconnect is unavailable or uncertain, never evidence of absence, and Core never replays a mutation after one; it observes the original operation instead. The [Sandbox Provider guide](../../docs/sandbox-provider.md#operation-outcomes-and-retries) defines each outcome. -Node startup and generation loading validate complete Provider operation declarations before accepting work, and the Core proxy uses the same registered declaration, so an unsupported operation rejects before node resolution or native I/O. The [operation contract](../../docs/sandbox-provider.md#explicit-operation-contracts) owns the inventory. An `unsupported` response carries an `unsupported` object with the exact method `operation` and an authored safe `reason`; the proxy checks both against the request. Missing, malformed or mismatched evidence is an unconfirmed result, never proof that a mutation was rejected. Unsupported stays distinct from observation unavailability and unknown compute or command results, and it neither settles resource ownership nor authorizes a replay. +Node startup and generation loading validate complete Provider operation declarations before accepting work, and the Core proxy uses the same registered declaration, so an unsupported operation rejects before node resolution or native I/O. The [operation contract](../../docs/sandbox-provider.md#explicit-operation-contracts) owns the inventory. An `unsupported` response carries an `unsupported` object with the exact method `operation` and an authored safe `reason`; the proxy checks both against the request. Missing, malformed or mismatched evidence is an unconfirmed result, never proof that a mutation was rejected. Unsupported stays distinct from observation unavailability and unknown compute results, and it neither settles resource ownership nor authorizes a replay. ## Generation control diff --git a/contracts/agents-api/zh/environments.md b/contracts/agents-api/zh/environments.md index 98a676688..46aa26767 100644 --- a/contracts/agents-api/zh/environments.md +++ b/contracts/agents-api/zh/environments.md @@ -1,7 +1,7 @@ --- title: "环境与模板" source: contracts/agents-api/environments.md -source_hash: 91a4b01924b65e48003f3a66a76a840272b5f672d0e0b667e6afd3626b8e4564 +source_hash: 2862ddcde357d25ab32c400dca1d7fa3812ea8967c2b40092941075034eb726d --- Environment 是 Session 的执行资源，包括 Harness 运行所在的机器、工作区以及已完成准备的能力。Session 通过其 `environment` 配置创建 Environment；不存在独立的 create 调用。Environment Template 是 Session 创建时解析的可复用准备配置。本契约涵盖这两类资源、两种放置方式、输入接纳、能力准备、Skills、Plugins 和 MCP 连接来源。 @@ -77,8 +77,6 @@ Core 在 Session 创建事务中创建 Environment 记录；Session upsert 会 - 计算资源、工作区和文件仍归应用程序所有。删除 Session 或撤销凭据会拒绝后续访问，但不会停止原生进程；机器所有者负责停止和清理。 - 工作区必须能在 daemon 重启后继续存在。丢失它绝不授权进行静默替换或重播。 -**应用管理的 E2B。** 应用程序可以在由其使用 E2B SDK 创建、续期和销毁的 E2B sandbox 中运行 Runtime，然后将该 Runtime 注册为 `self_hosted` Environment（[E2B Runtime guide](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/deploy/e2b/README.md)）。Core 不为其保留 E2B 分配，也绝不续期或终止它。 - ### 所有权规则 {#ownership-rules} - 将 Environment 身份、所有权、配置和生命周期保留在 Core 中，并使其与 Provider 计算资源、设备身份、daemon 套接字和原生会话相分离。将可变连接状态排除在不可变配置之外；替换后的所有者会使过期观察值失效。 diff --git a/contracts/agents-api/zh/machine-api.md b/contracts/agents-api/zh/machine-api.md index 70ce5314b..ea68d0c94 100644 --- a/contracts/agents-api/zh/machine-api.md +++ b/contracts/agents-api/zh/machine-api.md @@ -1,7 +1,7 @@ --- title: "机器连接 API" source: contracts/agents-api/machine-api.md -source_hash: 1540a3b84eb1cfa977654b46d407e1a4de7f502bab9c5264e21a8cc57e96d238 +source_hash: 873d25773a865039ae1f1f8983d7b26e3bc772fad7301b36fc03a3ffd7084186 --- 机器通过 `/api/v1` 调用 Core：包括沙箱节点、Runtime daemon、Sandbox I/O 服务和自托管安装器。各路由仅接受所列凭据，不接受 Core 密钥或 Project API 密钥；控制台登录也不授予此处权限。反向代理将 `/api/v1` 直接发送给 Core；Web 不提供这些路由。 @@ -35,7 +35,6 @@ source_hash: 1540a3b84eb1cfa977654b46d407e1a4de7f502bab9c5264e21a8cc57e96d238 | 节点凭据 | 节点自身：生成 32 至 256 个无空白字符的密钥，在登记时注册 | 带 `X-OAC-Node-ID` 的 `sandbox-node/configuration`、`sandbox-node/identity`、`sandbox-node/connect` | | 安装授权 | `self_hosted` Session 的 `x_agents_core.installation` 命令；短期有效 | `agent-daemon/installation` 及其 `claim` | | 执行器凭据 | 安装领取，或 Core 密钥[执行器凭据路由](environment-executor-credentials.md) | `agent-daemon/enroll` 和 `agent-daemon/connection`；登记后也作为该 Environment 的 enrollment 在 `sandbox-link` 上的 Serve 凭据 | -| 托管沙箱 daemon 凭据 | Core 为每个受管分配签发，通过[引导文件](../../../docs/zh/runtime-bootstrap.md)交付 | `agent-daemon/bootstrap`、`device-status` 和 `ws` | | 操作者设备配置 | 具有数据库访问权限的操作者运行 `oac-core-device` | `agent-daemon/bootstrap`、`device-status` 和 `ws` | Core 对存储的每个 token 和凭据仅保留 SHA-256 摘要；安装授权经签名但不存储。凭据不可互换：各自仅适用于自身路由。 diff --git a/contracts/agents-api/zh/node-generation-protocol.md b/contracts/agents-api/zh/node-generation-protocol.md index fb74ac11c..5c0fb651f 100644 --- a/contracts/agents-api/zh/node-generation-protocol.md +++ b/contracts/agents-api/zh/node-generation-protocol.md @@ -1,14 +1,14 @@ --- title: "沙箱节点协议" source: contracts/agents-api/node-generation-protocol.md -source_hash: e349ba887f9788e8f39990d33638182afcda553593e934788746fa0423cb4ee7 +source_hash: 4c74de8583bc8f0b85b91a338357899a1bcc03d8b2bd7063e6c71dd0430e717b --- 沙箱节点在其主机上运行 Docker 或 microsandbox Provider，并通过一个 WebSocket 与 Core 相连。Core 通过该连接发送 Provider 操作；节点针对本地 Provider 执行这些操作，并报告就绪状态、主机测量值及其持有的部署代次。Core 始终是唯一的生命周期所有者：节点绝不重试变更操作或调度工作。帧和校验器位于 [`services/core/internal/sandbox/node`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/node)（`wire.go`、`generation_wire.go`）；节点用于注册和读取配置的 HTTP 路由位于[机器连接 API](machine-api.md#node-routes)。 ## 帧与版本 {#frames-and-version} -每个帧都是一个 JSON 文本消息，其 `version` 等于 `node.ProtocolVersion`；两端都会拒绝任何其他版本，并且没有回退解码器。成员名必须精确且唯一：未知成员、大小写别名、重复项和意外的空值都会被拒绝。控制帧（`hello`、`welcome`、`heartbeat`、`heartbeat_ack`、`retention`、`retention_ack`）最多为 32 KiB；`request` 和 `response` 帧最多为 72 MiB。无效帧会关闭连接。 +每个帧都是一个 JSON 文本消息，其 `version` 等于 `node.ProtocolVersion`；两端都会拒绝任何其他版本，并且没有回退解码器。成员名必须精确且唯一：未知成员、大小写别名、重复项和意外的空值都会被拒绝。控制帧（`hello`、`welcome`、`heartbeat`、`heartbeat_ack`、`retention`、`retention_ack`）最多为 32 KiB；`request` 和 `response` 帧最多为 1 MiB。无效帧会关闭连接。 ## 连接 {#connection} @@ -43,19 +43,17 @@ Core 发送包含以下内容的 `request` 帧： | `info` | `GetInfo` | 无 | `info` | | `renew` | `Renew` | 无 | `info` | | `kill` | `Kill` | 无 | 无 | -| `command` | `RunCommand` | `command` | `command` | | `observe` | `Observe` | `observation` | `sample` | | `initial` | `Initial` | 无 | `compute` | | `new_compute` | `NewCompute` | 大于零的计算 `generation` 和可选的 `snapshot` | `compute` | | `compute` | `GetCompute` | `compute` | `state` | | `kill_compute` | `KillCompute` | `compute` | 无 | | `resume_compute` | `ResumeCompute` | `compute` | `state` | -| `command_compute` | `RunCommandCompute` | `compute` 和 `command` | `command` | | `suspend` | `Suspend` | `suspend` | `state` | | `resume` | `Resume` | `resume` | `state` | | `delete_snapshot` | `DeleteSnapshot` | `snapshot` | 无 | -`bootstrap` 是 Provider 的 `sandbox.Bootstrap`，其 `SandboxIO` 包含[沙箱引导](../../../docs/zh/sandbox-bootstrap.md)输入；Core 在发送 `create` 前完成校验。 +`bootstrap` 是 Provider 的 `sandbox.Bootstrap`：reference 以及 `SandboxIO` 中的[沙箱引导](../../../docs/zh/sandbox-bootstrap.md)输入；Core 在发送 `create` 前完成校验。 只要 `connection_id`、`owner_epoch` 或 `sequence` 中任一值不匹配，请求就会关闭连接。格式错误的请求会得到 `invalid` 响应。未启用代次管理的节点仅接受其登记的 `deployment_generation`；支持代次管理的节点在对应代次的 Provider 上运行请求，无法运行时回复 `unconfirmed`。Core 仅向节点上已就绪的代次发送 `create` 和非 observe-only 的 `resume`，并且每条连接最多保留 32 个待处理请求。 @@ -66,14 +64,13 @@ Core 发送包含以下内容的 `request` 帧： | `error_code` | 含义 | | --- | --- | | `invalid`、`ownership`、`exists`、`not_found` | `ErrInvalid`、`ErrOwnership`、`ErrExists`、`ErrNotFound` | -| `command_unconfirmed` | `ErrCommandUnconfirmed` | | `observation_unavailable`、`runtime_not_running` | 对应的观察结果 | | `unsupported` | 该操作被声明为不支持；见下文 | | `unconfirmed` 或任何其他值 | 结果未知 | 失败响应不携带结果，唯一的例外是作为精确引用 `CreateSettled` 回执的 `info` 结果：即便已确认的原生 Create 在后续检查中失败，仍可证明该尝试已有确定结果。超时、响应丢失或断连属于不可用或不确定情况，绝不能证明资源不存在；发生这些情况后，Core 绝不重放变更操作，而是改为观察原始操作。[Sandbox Provider 指南](../../../docs/zh/sandbox-provider.md#operation-outcomes-and-retries) 定义了每种结果。 -节点启动和代次加载会在接受工作前验证完整的 Provider 操作声明，Core 代理使用同一份已注册声明，因此不支持的操作会在节点解析或原生 I/O 之前被拒绝。操作清单由[操作契约](../../../docs/zh/sandbox-provider.md#explicit-operation-contracts)维护。`unsupported` 响应包含一个 `unsupported` 对象，其中有精确的方法 `operation` 和经作者编写且安全的 `reason`；代理会将两者与请求进行核对。证据缺失、格式错误或不匹配会得到 `unconfirmed` 结果，而绝不会证明变更操作被拒绝。`unsupported` 始终不同于观察不可用，也不同于计算或命令结果未知；它既不确定资源所有权，也不授权重放。 +节点启动和代次加载会在接受工作前验证完整的 Provider 操作声明，Core 代理使用同一份已注册声明，因此不支持的操作会在节点解析或原生 I/O 之前被拒绝。操作清单由[操作契约](../../../docs/zh/sandbox-provider.md#explicit-operation-contracts)维护。`unsupported` 响应包含一个 `unsupported` 对象，其中有精确的方法 `operation` 和经作者编写且安全的 `reason`；代理会将两者与请求进行核对。证据缺失、格式错误或不匹配会得到 `unconfirmed` 结果，而绝不会证明变更操作被拒绝。`unsupported` 始终不同于观察不可用，也不同于计算结果未知；它既不确定资源所有权，也不授权重放。 ## 代次控制 {#generation-control} diff --git a/deploy/node/node_install.py b/deploy/node/node_install.py index d563a29cf..68d08fce1 100644 --- a/deploy/node/node_install.py +++ b/deploy/node/node_install.py @@ -285,7 +285,7 @@ def provider_config(root, args, runtime_image): if args.provider == "docker": result["native"] = {"host": "unix:///var/run/docker.sock", "image": runtime_image, "network": "oac-node-" + args.installation_id, - "seccomp_file": str(root / "runtime/seccomp.json"), "nested_sandbox": True} + "seccomp_file": str(root / "runtime/seccomp.json")} else: endpoint = urlsplit(args.core_url) port = endpoint.port or (443 if endpoint.scheme == "https" else 80) diff --git a/docs/architecture.md b/docs/architecture.md index 27245f187..05404fea5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -31,7 +31,7 @@ Dashed arrows show provisioning and installation. Solid arrows show component in | Component | Responsibility | Reference | | --- | --- | --- | | Core | Authenticate callers, resolve and freeze configuration, schedule Turns, handle cancellation and pending interactions, persist resources and execution facts in PostgreSQL | [Core service](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/README.md) | -| Sandbox Provider | Create, observe, renew and reclaim compute; supply Runtime startup input | [Sandbox Provider](./sandbox-provider.md), [Runtime bootstrap](./runtime-bootstrap.md) | +| Sandbox Provider | Create, observe, renew and reclaim compute; start the Sandbox I/O service in it | [Sandbox Provider](./sandbox-provider.md), [Sandbox bootstrap](./sandbox-bootstrap.md) | | Sandbox node | Operate a Docker or microsandbox host and reconcile its assigned generation and allocations | [Sandbox node protocol](../contracts/agents-api/node-generation-protocol.md) | | Runtime | Prepare the workspace and capabilities, manage Session Executors, execute Turns and report events and receipts | [Core–Runtime protocol](./runtime-protocol.md) | | Harness adapter | Validate native configuration, invoke the upstream SDK or protocol, translate events and confirm native cleanup | [Harness onboarding](../contracts/agents-api/harness-onboarding.md) | diff --git a/docs/configuration.md b/docs/configuration.md index 2d609a372..72c2b90ce 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -130,7 +130,6 @@ The node installer writes Docker’s host settings into the `native` object of t | `image` | The Runtime image’s local ID after loading | The release’s `image_id` or `image_manifest_digest`. The host’s image store decides which digest names the loaded image, so the value is node-local; the adapter accepts only these two | | `network` | `oac-node-` | Runtime container network | | `seccomp_file` | `/runtime/seccomp.json` | Matched distribution’s seccomp profile | -| `nested_sandbox` | `true` | Enables the Docker adapter’s init process and proc-mask configuration | The [Docker adapter](./sandbox-provider.md#docker-adapter) owns container isolation, volume layout and lifecycle behavior. diff --git a/docs/sandbox-provider.md b/docs/sandbox-provider.md index 072a78839..6df4efab4 100644 --- a/docs/sandbox-provider.md +++ b/docs/sandbox-provider.md @@ -2,16 +2,16 @@ title: "Add a Sandbox Provider" --- -A **Sandbox Provider** supplies the outer compute that a Runtime daemon runs in for a Core-managed Environment, and the bounded bootstrap that starts that daemon. This guide is the path for adding one and the reference for how Core drives it. The interface is [`SandboxProvider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/sandbox_provider.go). +A **Sandbox Provider** supplies the compute of a Core-managed Environment and the bounded bootstrap that starts the [Sandbox I/O service](#oac-sandbox-io) in it, the only process a Provider starts. This guide is the path for adding one and the reference for how Core drives it. The interface is [`SandboxProvider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/sandbox_provider.go). | Term | Meaning | | --- | --- | | Environment | Durable execution place owned by Core; see [Environments](../contracts/agents-api/environments.md) | | Allocation | One Core-owned compute lease for an Environment, identified by `Reference` | -| Runtime | The daemon inside the Environment; it prepares capabilities and executes Turns | +| Runtime | The daemon on the [agent host](./configuration.md#agent-host-container); it prepares capabilities and executes Turns | | Deployment | The single deployment-wide provider selection; see [Sandbox deployment](../contracts/agents-api/sandbox-deployment.md) | -Core owns durable Environment, allocation, placement and cleanup state; the Provider owns compute and bootstrap only. The Runtime prepares capabilities and runs Turns over the [Core–Runtime protocol](./runtime-protocol.md), and the provider hands it its identity through the [Runtime bootstrap](./runtime-bootstrap.md) file. A provider never runs Environment initialization, Skills, Plugins, MCP setup, initial files, execution or Files; those use the Runtime. Isolation belongs to the provider's infrastructure, not the daemon; see [Runtime and outer isolation](./concepts.md#runtime-and-outer-isolation). Use the vendor's maintained SDK behind a thin adapter. +Core owns durable Environment, allocation, placement and cleanup state; the Provider owns compute and bootstrap only. The Runtime prepares capabilities and runs Turns over the [Core–Runtime protocol](./runtime-protocol.md), and reaches the sandbox's files, processes and network only through the [Sandbox link](./sandbox-link-protocol.md), which the sandbox's Sandbox I/O service Serves. A provider never runs Environment initialization, Skills, Plugins, MCP setup, initial files, execution or Files; those use the Runtime. Isolation belongs to the provider's infrastructure, not the daemon or Sandbox I/O; see [Runtime and outer isolation](./concepts.md#runtime-and-outer-isolation). Use the vendor's maintained SDK behind a thin adapter. ## Steps @@ -25,17 +25,16 @@ Core owns durable Environment, allocation, placement and cleanup state; the Prov ## Implement the interface -`sandbox_provider.go` holds the Core–Sandbox Provider protocol: the `SandboxProvider` interface for allocation, checkpoint and observation, its request and result types, and the setup-time `ConfigurationAdapter` with its typed errors. Value types that Core also uses beyond this boundary, such as `DeploymentSpec` and `CallFence`, live in their own files of the same package. Every method is required at compile time, and `ProviderOperations()` declares which ones the Provider supports. Five operations are always supported: +`sandbox_provider.go` holds the Core–Sandbox Provider protocol: the `SandboxProvider` interface for allocation, checkpoint and observation, its request and result types, and the setup-time `ConfigurationAdapter` with its typed errors. Value types that Core also uses beyond this boundary, such as `DeploymentSpec` and `CallFence`, live in their own files of the same package. Every method is required at compile time, and `ProviderOperations()` declares which ones the Provider supports. Four operations are always supported: | Operation | Purpose | | --- | --- | -| `Create` | Create compute for a `Reference` and run the bounded daemon bootstrap | +| `Create` | Create compute for a `Reference` and run the bounded bootstrap that starts Sandbox I/O | | `GetInfo` | Observe current compute without changing it | | `Renew` | Extend a native lease, or only observe when the backend has none | | `Kill` | Reclaim the allocation's compute and retained resources | -| `RunCommand` | Run a bounded command in the allocation's compute | -A backend without a native renewable lease, such as Docker, still keeps Core's hosted expiry and cleanup requirements. Every adapter implements `RunCommand` and its tests exercise it, but Core's orchestration does not call it; only the node transport forwards it. Confidential command input travels in `Command.Stdin`, never in arguments or logs, and the result keeps byte order, bounded output and the actual exit status. +A backend without a native renewable lease, such as Docker, still keeps Core's hosted expiry and cleanup requirements. ### Explicit operation contracts @@ -43,15 +42,15 @@ Every provider returns a complete `ProviderOperations()` declaration with one en | Operations | Requirement | Responsibility | | --- | --- | --- | -| `Create`, `GetInfo`, `Renew`, `Kill`, `RunCommand` | Supported | Allocation lifecycle and bounded commands | +| `Create`, `GetInfo`, `Renew`, `Kill` | Supported | Allocation lifecycle | | `Observe` | Explicit decision | Ownership-checked read-only observation of one allocation | -| `Initial`, `NewCompute`, `GetCompute`, `Suspend`, `Resume`, `ResumeCompute`, `KillCompute`, `DeleteSnapshot`, `RunCommandCompute` | The same decision for every checkpoint method; supported only by a `nodes` registration | Exact compute incarnations, capture and restore, retained-source resume and cleanup | +| `Initial`, `NewCompute`, `GetCompute`, `Suspend`, `Resume`, `ResumeCompute`, `KillCompute`, `DeleteSnapshot` | The same decision for every checkpoint method; supported only by a `nodes` registration | Exact compute incarnations, capture and restore, retained-source resume and cleanup | -`Initial` and `NewCompute` construct compute references without allocating, `ResumeCompute` thaws only the same resident instance after an aborted pause, and `RunCommandCompute` runs a bounded command in one exact compute incarnation. Core uses `RunCommandCompute` to wake a parked daemon after a restore ([`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)). +`Initial` and `NewCompute` construct compute references without allocating, and `ResumeCompute` thaws only the same resident instance after an aborted pause. Each declaration entry is `state: supported` with no reason, or `state: unsupported` with an authored reason code. Missing, zero, unknown or unsafe entries fail validation. Adding a method to `SandboxProvider` requires an explicit decision and implementation in every adapter; never supply a base type or generate blanket unsupported implementations. -An unsupported method returns `providercontract.UnsupportedError` before any native I/O. The error names the exact operation and a safe code, never a native message, resource identity, endpoint or credential. An empty result, a nil error, `Unavailable` or an unknown mutation outcome never stands in for unsupported, and the five required methods can never return it. +An unsupported method returns `providercontract.UnsupportedError` before any native I/O. The error names the exact operation and a safe code, never a native message, resource identity, endpoint or credential. An empty result, a nil error, `Unavailable` or an unknown mutation outcome never stands in for unsupported, and the four required methods can never return it. Each adapter owns one `Operations()` function, shared by its instance and its registration. `providers.ValidateBinding` checks both against the interface and each other, and Runtime admission and node generation loading also reject incomplete providers. Callers check the declaration with `providercontract.Require` before they call an operation, never a type assertion. @@ -75,7 +74,7 @@ Core serializes lifecycle operations and keeps the allocation after any uncertai ### Operation outcomes and retries -Every call receives a bounded context. Expiry or cancellation ends the caller's wait; it proves no rollback, stop, cleanup or absence. An adapter or transport never detaches untracked mutations or replays a timed-out command. +Every call receives a bounded context. Expiry or cancellation ends the caller's wait; it proves no rollback, stop, cleanup or absence. An adapter or transport never detaches untracked mutations or replays a timed-out mutation. | Operation | Confirmed result | Failure or unknown result | Recovery | | --- | --- | --- | --- | @@ -83,9 +82,8 @@ Every call receives a bounded context. Expiry or cancellation ends the caller's | `GetInfo` | Current compute observation without change | `ErrNotFound` is only a missing observation; an error is not proof of absence | Repeat a bounded read; never turn it into create, start or renew | | `Renew` | The native lease extended, or an observation for a provider without leases | A timeout may hide an extension; stopped or missing compute stays so | Observe, then let the reconciler renew the same allocation. Never revive compute or fabricate a lease expiry | | `Kill` | Owned compute and retained storage removed; repeated confirmed absence succeeds | An error keeps ownership and cleanup intent; an ownership mismatch never deletes foreign resources | Retry cleanup of the same `Reference` after outstanding creation or mutation is fenced; never release the owner early | -| `RunCommand`, `RunCommandCompute` | Collected output and actual exit code; a nonzero exit is a settled command failure | Missing native completion is `ErrCommandUnconfirmed`; partial output is not success | Never replay. Keep the owner and reclaim before reuse when completion cannot be proved | -`ErrInvalid`, `ErrOwnership`, `ErrExists`, `ErrNotFound`, `ErrComputeUnconfirmed` and `ErrCommandUnconfirmed` keep their defined meanings. An unclassified native or transport error is unknown, never permission to retry a mutation. Core never reads provider diagnostics as lifecycle truth or exposes native error text or credentials; the node transport maps errors to fixed codes, and direct SDK details stay private. +`ErrInvalid`, `ErrOwnership`, `ErrExists`, `ErrNotFound` and `ErrComputeUnconfirmed` keep their defined meanings. An unclassified native or transport error is unknown, never permission to retry a mutation. Core never reads provider diagnostics as lifecycle truth or exposes native error text or credentials; the node transport maps errors to fixed codes, and direct SDK details stay private. Checkpoint support adds `Compute` generation, name and ID and `SnapshotIdentity`; persist operation IDs and the provider's snapshot provenance unchanged. `ObserveOnly` on suspend or resume observes the previous attempt and never starts another capture or restore. `ResumeCompute` only thaws the retained source and never cold-starts a stopped one. Cleanup targets the exact compute incarnation and snapshot, not whatever instance now has the same name. Read [`runtime_compute.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute.go) and its failure tests before declaring checkpoint support. @@ -152,7 +150,7 @@ This is what Core does around every provider. Adapters implement none of it, but At startup Core claims the stable installation identity and a new owner epoch before it selects a provider. The runtime manager keeps generation-aware provider facades. Initial setup and replacement prepare and validate candidates before any database write, and a rejected candidate leaves the active configuration and workers unchanged. A backend replacement uses the deployment mutation gate: it pauses manager admission, drains old calls and loops, then repeats the resource and generation guards in the commit transaction, where the new selection, its generation and the retirement of old nodes and unused enrollment tokens commit together. After commit, Core publishes the prevalidated configuration and the shared observation and bootstrap cache under the manager mutex, with no further external work or fallible step, so a request cancelled after commit cannot discard it. An interrupted drain stays a barrier for retries. Provider I/O and draining never hold a database transaction or the manager's map mutex. -A locally unavailable provider dependency keeps hosted admission closed while the existing scan waits for repair; administrator recovery stays available, also after a restart. Database and ownership errors stay failures. An unconfigured deployment refuses hosted admission with 503 `execution_unavailable` and creates no Session state. Core derives the Runtime bootstrap and daemon WebSocket addresses from the installation public URL, never from request headers, and reads the current selection from the database, never from a startup file. +A locally unavailable provider dependency keeps hosted admission closed while the existing scan waits for repair; administrator recovery stays available, also after a restart. Database and ownership errors stay failures. An unconfigured deployment refuses hosted admission with 503 `execution_unavailable` and creates no Session state. Core derives the sandbox Link address from the installation public URL, never from request headers, and reads the current selection from the database, never from a startup file. Node readiness binds to the exact generation, the current connection and the owner epoch. A durable serving pin is promoted only for readiness of the then-current target, under deployment serialization, so a late report for a superseded target never acquires a pin. @@ -168,7 +166,7 @@ Terminal cleanup atomically revokes the device's authority, records the Environm ### `oac-sandbox-io` -Every hosted sandbox also runs `oac-sandbox-io`, which Serves the allocation over the [Sandbox link](./sandbox-link-protocol.md). `Bootstrap.SandboxIO` is its [Sandbox bootstrap](./sandbox-bootstrap.md) input: the Link URL derived from the [public URL](./configuration.md#changing-the-public-url), the allocation's Serve credential, and the allocation with its Serve generation as resource. Core validates the whole `Bootstrap` once, with `Bootstrap.Validate`, before `Create`, and adapters deliver it as given. `Create` writes `SandboxIO` to a private file, `/home/runtime/sandbox-io-bootstrap.json` (mode 0600, UID 1000) in the reference adapters, and starts `oac-sandbox-io --bootstrap-file` with that path as the daemon's account, beside the daemon. `BootstrapComplete` implies that both processes were started. The input never travels in an argument or environment variable. Every Runtime image ships `/usr/local/bin/oac-sandbox-io`. Allocation cleanup revokes the resource at the relay before it calls `Kill`. +`oac-sandbox-io` is the only process a Provider starts in a hosted sandbox. It Serves the allocation over the [Sandbox link](./sandbox-link-protocol.md). `Bootstrap` is the allocation's `Reference` and `SandboxIO`, the service's [Sandbox bootstrap](./sandbox-bootstrap.md) input: the Link URL derived from the [public URL](./configuration.md#changing-the-public-url), the allocation's Serve credential, and the allocation with its Serve generation as resource. Core validates the whole `Bootstrap` once, with `Bootstrap.Validate`, which also checks that `SandboxIO` serves the `Reference`, before `Create`, and adapters deliver it as given. `Create` writes `SandboxIO` to a private file, `/home/runtime/sandbox-io-bootstrap.json` (mode 0600, UID 1000) in the reference adapters, and starts `oac-sandbox-io --bootstrap-file` with that path as UID 1000. `BootstrapComplete` implies that it was started. The input never travels in an argument or environment variable. Every Runtime image ships `/usr/local/bin/oac-sandbox-io`. Allocation cleanup revokes the resource at the relay before it calls `Kill`. ### Per-node lifecycle workers @@ -190,7 +188,7 @@ Core suspends the idle work of every provider that declares checkpoint support, The Worker lease, the Session lock and the per-node gates own suspension for every provider. New Turn claims, file-write intents and capture admission serialize under the Session lock and share one compute-phase check; new pending work cancels a capture and wakes the same source. Normal preparation waits for the compute phase to be running, after the authenticated resume handshake, and pending input stays pending when its promotion conflicts with a lifecycle transition. Compute phases and revision-checked receipts live on the allocation. Core persists quiesce, capture and restore intent before the effect, only a fresh receipt performs a capture or restore, and recovery observes the exact attempt without retrying an unknown creation, capture or restore. A consumed snapshot never rolls a running generation back. Deletion, revocation and retention expiry win over wake, up to the final database compare-and-swap, and unknown cleanup identities are kept until owned resources are confirmed absent. Consumed artifacts and old compute are deleted, so suspension cycles never build a chain of writable disks. -Queued work and live Environment file access wake a suspended Environment; history and published Artifact reads do not. Planned suspension uses an Environment and suspension token on the daemon connection. A PID and start-time fenced local control signal (`RunCommandCompute`) wakes the parked daemon, which authenticates again before admitting work. A transient disconnect before confirmation retries the same armed suspension with bounded attempts and backoff; a permanent authentication or protocol rejection closes it. Core owns the snapshot's retention deadline, and the daemon has no timer for it. A lost quiesce acknowledgement may thaw the same source through explicit rollback but never authorizes capturing it. +Queued work and live Environment file access wake a suspended Environment; history and published Artifact reads do not. Planned suspension uses an Environment and suspension token on the daemon connection. After a restore, Core waits until the sandbox's Sandbox I/O Serves again, then resumes the Environment on the daemon connection with the same suspension token ([`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)). A transient disconnect before confirmation retries the same armed suspension with bounded attempts and backoff; a permanent authentication or protocol rejection closes it. Core owns the snapshot's retention deadline, and the daemon has no timer for it. A lost quiesce acknowledgement may thaw the same source through explicit rollback but never authorizes capturing it. ### Reset and archive @@ -208,7 +206,7 @@ Run `make check-sandbox-provider-contract` while developing. It runs the shared Node tests separately cover disconnect and reconnect fencing and cleanup after a lost Create response. Helper protocols and the [sandbox node protocol](../contracts/agents-api/node-generation-protocol.md) require an exact version match; direct in-process interfaces have no separate wire version. -Native acceptance proves what fixtures cannot: creation, lease behavior, owned partial cleanup, declared isolation and limits, and snapshots where supported. The opt-in Docker lifecycle, recovery, Serve and node transport tests use `AGENTS_RUNTIME_DOCKER_TEST_IMAGE`, an image digest with `/bin/sh` and a `/usr/local/bin/oac-daemon` that keeps running; the Serve test adds this tree's `oac-sandbox-io` to an image it derives; the SDK helpers use `make check-e2b-provider` and `make check-microsandbox-provider`. Mocked compute proves neither reclamation nor isolation. +Native acceptance proves what fixtures cannot: creation, lease behavior, owned partial cleanup, declared isolation and limits, and snapshots where supported. The opt-in Docker lifecycle, recovery, Serve and node transport tests use `AGENTS_RUNTIME_DOCKER_TEST_IMAGE`, an image digest with `/bin/sh` and `/usr/local/bin/oac-sandbox-io`, such as the `sandbox` image of `scripts/build-agent-host-images.sh`; the Serve test adds this tree's `oac-sandbox-io` to an image it derives; the SDK helpers use `make check-e2b-provider` and `make check-microsandbox-provider`. Mocked compute proves neither reclamation nor isolation. ## Reference adapters @@ -216,19 +214,18 @@ Native acceptance proves what fixtures cannot: creation, lease behavior, owned p | --- | --- | --- | --- | | Docker (node) | [`sandbox/docker`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/docker) | Node proxy in [`sandbox/node`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/node) | [Docker adapter](#docker-adapter) | | microsandbox (node) | [`sandbox/microsandbox`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/microsandbox) | [`tools/microsandbox-provider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/microsandbox-provider/README.md) | [Nodes](./getting-started/nodes.md) | -| E2B (direct) | [`sandbox/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/e2b) | [`tools/e2b-provider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) | [Sandbox deployment](../contracts/agents-api/sandbox-deployment.md#e2b-configuration); application-managed templates in [`deploy/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/deploy/e2b/README.md) | +| E2B (direct) | [`sandbox/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/e2b) | [`tools/e2b-provider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) | [Sandbox deployment](../contracts/agents-api/sandbox-deployment.md#e2b-configuration); templates in [`deploy/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/deploy/e2b/README.md) | ## Docker adapter -The Docker Sandbox Provider ([`sandbox/docker`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/docker)) runs every Runtime image, whichever Harness it serves, with the same container settings ([`container_options.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/docker/container_options.go)): +The Docker Sandbox Provider ([`sandbox/docker`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/docker)) runs every allocation's sandbox as a container whose only process is [`oac-sandbox-io`](#oac-sandbox-io), with the same container settings ([`container_options.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/docker/container_options.go)): - user 1000:1000, read-only root filesystem, all capabilities dropped, `no-new-privileges`, the [seccomp profile](#seccomp-profile) and AppArmor `unconfined`; - the node’s configured network and extra hosts ([node configuration](./configuration.md#docker-node-configuration)); - CPU and memory from the deployment specification, a 128-process limit and a 128 MiB `/tmp` tmpfs; -- two named volumes labelled with the installation, tenant, Environment and allocation: `-home` at `/home` and `-environment` at `/environment`, whose `workspace` subdirectory is also mounted at `/workspace`. The Docker Engine must support volume subpath mounts; -- with the configured `nested_sandbox` option, Docker's `/proc` masks are lifted (`/sys/firmware` and `/sys/devices/virtual/powercap` stay masked) and the container runs an init process. +- two named volumes labelled with the installation, tenant, Environment and allocation: `-home` at `/home` and `-environment` at `/environment`, whose `workspace` subdirectory is also mounted at `/workspace`. The Docker Engine must support volume subpath mounts. -Create refuses to reuse retained volumes that have no container. It copies the [Runtime bootstrap](./runtime-bootstrap.md) file to `/home/runtime/runtime-bootstrap.json` (mode 0600, UID 1000) and the `/environment` workspace, staging, initialization and package directories into the container, then starts `oac-daemon connect --profile default --bootstrap-file /home/runtime/runtime-bootstrap.json` and the [`oac-sandbox-io`](#oac-sandbox-io). When the created container does not have the configured CPU, memory and exact image, Create returns the error with `CreateSettled`. Docker has no lease, so Renew only reads the container state. Kill checks the ownership labels of the container and both volumes before removing any of them, then confirms that all three are gone. +Create refuses to reuse retained volumes that have no container. It copies the Sandbox bootstrap file to `/home/runtime/sandbox-io-bootstrap.json` (mode 0600, UID 1000) and the `/environment` workspace, initialization and package directories into the container, which then runs `oac-sandbox-io --bootstrap-file /home/runtime/sandbox-io-bootstrap.json` as its entry point. The service is the container's first process and reaps its orphaned descendants, so the container needs no init process. When the created container does not have the configured CPU, memory and exact image, Create returns the error with `CreateSettled`. Docker has no lease, so Renew only reads the container state. Kill checks the ownership labels of the container and both volumes before removing any of them, then confirms that all three are gone. The node uses the explicit Unix socket in its [provider configuration](./configuration.md#docker-node-configuration) and ignores `DOCKER_HOST`. No Docker socket, host home or Core credential is mounted into a Runtime. diff --git a/docs/zh/architecture.md b/docs/zh/architecture.md index f2d222e5b..c43b15a1e 100644 --- a/docs/zh/architecture.md +++ b/docs/zh/architecture.md @@ -1,7 +1,7 @@ --- title: "架构" source: docs/architecture.md -source_hash: 84e503ee46a293d0fa283aaa674dd3f307eb29560ea1b791fee263f2d0f26efc +source_hash: bceec7f192812cd3d1895bb1b89c361c771dbb008ef50a44f2a153f8c555c925 --- OpenAgentCore 将编排、计算资源和原生执行分开。Core 负责 API 和持久状态。Sandbox Provider 管理计算资源。Runtime daemon 准备 Environment 并运行选定的 Harness；Harness 的原生 SDK 或协议负责模型与工具循环。 @@ -33,7 +33,7 @@ flowchart TB | 组件 | 职责 | 参考 | | --- | --- | --- | | Core | 认证调用方，解析并冻结配置，调度 Turn，处理取消和待处理交互，将资源与执行事实持久化到 PostgreSQL | [Core 服务](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/README.md) | -| Sandbox Provider | 创建、观察、续期和回收计算资源；提供 Runtime 启动输入 | [Sandbox Provider](sandbox-provider.md)、[Runtime 引导](runtime-bootstrap.md) | +| Sandbox Provider | 创建、观察、续期和回收计算资源；在其中启动 Sandbox I/O 服务 | [Sandbox Provider](sandbox-provider.md)、[沙箱引导](sandbox-bootstrap.md) | | Sandbox node | 运行 Docker 或 microsandbox 主机，协调分配给它的 generation 与 allocation | [Sandbox node 协议](../../contracts/agents-api/zh/node-generation-protocol.md) | | Runtime | 准备工作区和能力，管理 Session Executor，执行 Turn 并报告事件与回执 | [Core–Runtime 协议](runtime-protocol.md) | | Harness adapter | 验证原生配置，调用上游 SDK 或协议，转换事件并确认原生清理完成 | [Harness 接入](../../contracts/agents-api/zh/harness-onboarding.md) | diff --git a/docs/zh/configuration.md b/docs/zh/configuration.md index 820d83477..8b77e8ca3 100644 --- a/docs/zh/configuration.md +++ b/docs/zh/configuration.md @@ -1,7 +1,7 @@ --- title: "配置参考" source: docs/configuration.md -source_hash: 054bd156ff66980a8bf82c9beb9aad77c2f724f3c314a2ad0a851704fcb23902 +source_hash: 9356569940dd30051693b6f16e0a80a843451f80185b8c618660c656163fcf87 --- Core 安装的每项设置都恰好只有一个归属位置，分属以下三类： @@ -134,7 +134,6 @@ Web 的 **System** 页面显示该安装的地址、默认模型和沙箱配置 | `image` | 加载后 Runtime 镜像的本地 ID | 发行版本的 `image_id` 或 `image_manifest_digest`。主机的镜像存储决定由哪个 digest 指代已加载的镜像，因此该值属于节点本地；适配器只接受这两个值 | | `network` | `oac-node-` | Runtime 容器网络 | | `seccomp_file` | `/runtime/seccomp.json` | 所匹配发行版的 seccomp 配置文件 | -| `nested_sandbox` | `true` | 启用 Docker 适配器的 init 进程和 proc-mask 配置 | [Docker 适配器](sandbox-provider.md#docker-adapter)负责容器隔离、卷布局和生命周期行为。 diff --git a/docs/zh/sandbox-provider.md b/docs/zh/sandbox-provider.md index c97b615dc..6bb27e917 100644 --- a/docs/zh/sandbox-provider.md +++ b/docs/zh/sandbox-provider.md @@ -1,19 +1,19 @@ --- title: "添加 Sandbox Provider" source: docs/sandbox-provider.md -source_hash: 04791a64fc66e846e420383221b7e87aed47c06fe9b46dae5e990547ce9a4f1a +source_hash: 2dd362d329e39d15c3b6ddfa727429ab282c037015dabcece938d6daaeffaa99 --- -**Sandbox Provider** 为 Core 管理的 Environment 提供 Runtime daemon 运行所需的外层计算资源，以及启动 daemon 的有界引导流程。本指南说明如何添加 Provider，并作为 Core 驱动 Provider 的参考。接口为 [`SandboxProvider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/sandbox_provider.go)。 +**Sandbox Provider** 为 Core 管理的 Environment 提供计算资源，以及在其中启动 [Sandbox I/O 服务](#oac-sandbox-io)的有界引导流程；该服务是 Provider 启动的唯一进程。本指南说明如何添加 Provider，并作为 Core 驱动 Provider 的参考。接口为 [`SandboxProvider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/sandbox_provider.go)。 | 术语 | 含义 | | --- | --- | | Environment | Core 拥有的持久执行场所；参见 [Environment](../../contracts/agents-api/zh/environments.md) | | Allocation | Core 为 Environment 拥有的一份计算资源租约，由 `Reference` 标识 | -| Runtime | Environment 内的 daemon；准备能力并执行 Turn | +| Runtime | [agent host](configuration.md#agent-host-container) 上的 daemon；准备能力并执行 Turn | | Deployment | 整个部署唯一的 provider 选择；参见[沙箱部署](../../contracts/agents-api/zh/sandbox-deployment.md) | -Core 拥有持久 Environment、allocation、placement 和 cleanup 状态；Provider 仅负责计算资源与引导。Runtime 通过 [Core–Runtime 协议](runtime-protocol.md)准备能力并运行 Turn，provider 通过 [Runtime 引导](runtime-bootstrap.md)文件交付身份。provider 不运行 Environment 初始化、Skill、Plugin、MCP 设置、初始文件、execution 或 Files；这些操作使用 Runtime。隔离属于 provider 基础设施，不属于 daemon；参见 [Runtime 与外层隔离](concepts.md#runtime-and-outer-isolation)。使用薄 adapter 封装厂商维护的 SDK。 +Core 拥有持久 Environment、allocation、placement 和 cleanup 状态；Provider 仅负责计算资源与引导。Runtime 通过 [Core–Runtime 协议](runtime-protocol.md)准备能力并运行 Turn，并且只通过 sandbox 中 Sandbox I/O 服务 Serve 的[沙箱 Link](sandbox-link-protocol.md) 访问 sandbox 的文件、进程和网络。provider 不运行 Environment 初始化、Skill、Plugin、MCP 设置、初始文件、execution 或 Files；这些操作使用 Runtime。隔离属于 provider 基础设施，不属于 daemon 或 Sandbox I/O；参见 [Runtime 与外层隔离](concepts.md#runtime-and-outer-isolation)。使用薄 adapter 封装厂商维护的 SDK。 ## 步骤 {#steps} @@ -27,17 +27,16 @@ Core 拥有持久 Environment、allocation、placement 和 cleanup 状态；Prov ## 实现接口 {#implement-the-interface} -`sandbox_provider.go` 包含 Core–Sandbox Provider 协议：负责 allocation、checkpoint 和观测的 `SandboxProvider` 接口及其请求与结果类型，以及 setup 阶段的 `ConfigurationAdapter` 及其类型化错误。Core 在此边界之外也使用的值类型，例如 `DeploymentSpec` 和 `CallFence`，位于同一 package 的独立文件中。每个方法在编译期都必须实现，`ProviderOperations()` 声明 Provider 支持哪些方法。以下五项操作始终支持： +`sandbox_provider.go` 包含 Core–Sandbox Provider 协议：负责 allocation、checkpoint 和观测的 `SandboxProvider` 接口及其请求与结果类型，以及 setup 阶段的 `ConfigurationAdapter` 及其类型化错误。Core 在此边界之外也使用的值类型，例如 `DeploymentSpec` 和 `CallFence`，位于同一 package 的独立文件中。每个方法在编译期都必须实现，`ProviderOperations()` 声明 Provider 支持哪些方法。以下四项操作始终支持： | 操作 | 用途 | | --- | --- | -| `Create` | 为 `Reference` 创建计算资源，并运行有界 daemon 引导 | +| `Create` | 为 `Reference` 创建计算资源，并运行启动 Sandbox I/O 的有界引导 | | `GetInfo` | 观察当前计算资源，不修改它 | | `Renew` | 延长原生租约；backend 没有租约时仅观察 | | `Kill` | 回收 allocation 的计算资源与保留资源 | -| `RunCommand` | 在 allocation 的计算资源中运行有界命令 | -Docker 等没有原生可续期租约的 backend 仍遵守 Core 的 hosted expiry 和 cleanup 要求。每个 adapter 都实现 `RunCommand`，测试也执行它，但 Core 编排不调用它；只有 node transport 转发。机密命令输入通过 `Command.Stdin` 传输，不放在参数或日志中；结果保留字节顺序、有界输出和实际退出状态。 +Docker 等没有原生可续期租约的 backend 仍遵守 Core 的 hosted expiry 和 cleanup 要求。 ### 明确的操作契约 {#explicit-operation-contracts} @@ -45,15 +44,15 @@ Docker 等没有原生可续期租约的 backend 仍遵守 Core 的 hosted expir | 操作 | 要求 | 职责 | | --- | --- | --- | -| `Create`、`GetInfo`、`Renew`、`Kill`、`RunCommand` | 支持 | Allocation 生命周期与有界命令 | +| `Create`、`GetInfo`、`Renew`、`Kill` | 支持 | Allocation 生命周期 | | `Observe` | 明确决定 | 检查所有权、只读地观测一个 allocation | -| `Initial`、`NewCompute`、`GetCompute`、`Suspend`、`Resume`、`ResumeCompute`、`KillCompute`、`DeleteSnapshot`、`RunCommandCompute` | 所有 checkpoint 方法决定一致；仅 `nodes` 注册可以支持 | 精确计算实例、捕获与恢复、保留源恢复和清理 | +| `Initial`、`NewCompute`、`GetCompute`、`Suspend`、`Resume`、`ResumeCompute`、`KillCompute`、`DeleteSnapshot` | 所有 checkpoint 方法决定一致；仅 `nodes` 注册可以支持 | 精确计算实例、捕获与恢复、保留源恢复和清理 | -`Initial` 和 `NewCompute` 构造 compute reference，不分配资源；`ResumeCompute` 在暂停中止后仅解冻同一驻留实例；`RunCommandCompute` 在一个精确 compute incarnation 中运行有界命令。Core 使用 `RunCommandCompute` 在恢复后唤醒 parked daemon（[`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)）。 +`Initial` 和 `NewCompute` 构造 compute reference，不分配资源；`ResumeCompute` 在暂停中止后仅解冻同一驻留实例。 每个声明项为不带 reason 的 `state: supported`，或带 authored reason code 的 `state: unsupported`。缺失、零值、未知或不安全项都会验证失败。给 `SandboxProvider` 添加方法时，必须在每个 adapter 中明确决定并实现；不提供 base type，也不生成笼统的不支持实现。 -不支持的方法在任何原生 I/O 前返回 `providercontract.UnsupportedError`。错误指明精确操作和安全 code，不包含原生消息、资源身份、endpoint 或凭据。空结果、nil error、`Unavailable` 或未知 mutation 结果都不能代替 unsupported，五项必需方法不能返回 unsupported。 +不支持的方法在任何原生 I/O 前返回 `providercontract.UnsupportedError`。错误指明精确操作和安全 code，不包含原生消息、资源身份、endpoint 或凭据。空结果、nil error、`Unavailable` 或未知 mutation 结果都不能代替 unsupported，四项必需方法不能返回 unsupported。 每个 adapter 拥有一个 `Operations()` 函数，由实例和注册共享。`providers.ValidateBinding` 根据接口并相互对照检查两者，Runtime admission 与 node generation 加载也拒绝不完整 provider。调用方在调用操作前用 `providercontract.Require` 检查声明，从不使用 type assertion。 @@ -77,7 +76,7 @@ Core 串行化生命周期操作，并在任何不确定 mutation 后保留 allo ### 操作结果与重试 {#operation-outcomes-and-retries} -每次调用接收有界 context。到期或取消结束调用方等待，不证明回滚、停止、清理或不存在。adapter 或 transport 不得脱离跟踪执行 mutation，也不重放超时命令。 +每次调用接收有界 context。到期或取消结束调用方等待，不证明回滚、停止、清理或不存在。adapter 或 transport 不得脱离跟踪执行 mutation，也不重放超时 mutation。 | 操作 | 已确认结果 | 失败或未知结果 | 恢复 | | --- | --- | --- | --- | @@ -85,9 +84,8 @@ Core 串行化生命周期操作，并在任何不确定 mutation 后保留 allo | `GetInfo` | 不修改的当前计算资源观测 | `ErrNotFound` 仅表示缺少观测；error 不是不存在的证明 | 重复有界读取；不将其变成 create、start 或 renew | | `Renew` | 原生租约已延长，或无租约 provider 的观测 | 超时可能隐藏延期；停止或缺失计算资源仍保持原样 | 先观察，再由 reconciler 续期同一 allocation。不复活计算资源或虚构 lease expiry | | `Kill` | 所属计算资源与保留存储已移除；重复已确认不存在时成功 | error 保留所有权与清理意图；所有权不匹配不删除外来资源 | 未决创建或 mutation 已隔离后重试同一 `Reference` 的清理；不提前释放 owner | -| `RunCommand`, `RunCommandCompute` | 收集到输出和实际 exit code；非零退出是已结算命令失败 | 缺失原生完成为 `ErrCommandUnconfirmed`；部分输出不是成功 | 不重放。无法证明完成时保留 owner，并在复用前回收 | -`ErrInvalid`、`ErrOwnership`、`ErrExists`、`ErrNotFound`、`ErrComputeUnconfirmed` 和 `ErrCommandUnconfirmed` 保持其定义含义。未分类原生或 transport error 表示未知，不授权重试 mutation。Core 不将 provider diagnostics 读作生命周期事实，也不暴露原生错误文本或凭据；node transport 将错误映射为固定 code，直接 SDK 细节保持私有。 +`ErrInvalid`、`ErrOwnership`、`ErrExists`、`ErrNotFound` 和 `ErrComputeUnconfirmed` 保持其定义含义。未分类原生或 transport error 表示未知，不授权重试 mutation。Core 不将 provider diagnostics 读作生命周期事实，也不暴露原生错误文本或凭据；node transport 将错误映射为固定 code，直接 SDK 细节保持私有。 Checkpoint 支持增加 `Compute` generation、name、ID 和 `SnapshotIdentity`；原样持久化 operation ID 与 provider snapshot provenance。suspend 或 resume 的 `ObserveOnly` 仅观察上次尝试，不启动另一 capture 或 restore。`ResumeCompute` 仅解冻保留源，不冷启动已停止源。清理针对精确 compute incarnation 和 snapshot，不针对当前同名实例。声明 checkpoint 支持前阅读 [`runtime_compute.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute.go) 及其失败测试。 @@ -154,7 +152,7 @@ launcher 从[派生进程环境](configuration.md)提供 `sandbox.ProcessPaths` 启动时 Core 在选择 provider 前领取稳定 installation identity 和新的 owner epoch。runtime manager 保留了解 generation 的 provider facade。初始 setup 与 replacement 在任何数据库写入前准备并验证 candidate；candidate 被拒绝时不改变活动配置或 worker。backend replacement 使用 deployment mutation gate：暂停 manager admission，排空旧调用与循环，再在 commit transaction 中重复 resource 与 generation guard；新选择、其 generation、旧 node 和未使用 enrollment token 的退役一起提交。提交后，Core 在 manager mutex 下发布预验证配置和共享 observation、bootstrap cache，没有进一步外部工作或可失败步骤，因此提交后取消的请求不能丢弃配置。中断的 drain 保留为重试 barrier。Provider I/O 和 drain 不持有数据库事务或 manager map mutex。 -本地 provider 依赖不可用时，现有 scan 等待修复，hosted admission 保持关闭；管理员恢复仍可使用，重启后也如此。数据库和所有权错误仍是失败。未配置的部署以 503 `execution_unavailable` 拒绝 hosted admission，且不创建 Session 状态。Core 从 installation public URL 派生 Runtime bootstrap 和 daemon WebSocket 地址，不使用请求 header；从数据库读取当前选择，不使用 startup file。 +本地 provider 依赖不可用时，现有 scan 等待修复，hosted admission 保持关闭；管理员恢复仍可使用，重启后也如此。数据库和所有权错误仍是失败。未配置的部署以 503 `execution_unavailable` 拒绝 hosted admission，且不创建 Session 状态。Core 从 installation public URL 派生沙箱 Link 地址，不使用请求 header；从数据库读取当前选择，不使用 startup file。 Node readiness 绑定到精确 generation、当前连接和 owner epoch。持久 serving pin 仅在部署串行化下为当时目标的 readiness 提升，因此已被替代目标的延迟报告不获得 pin。 @@ -170,7 +168,7 @@ Core 在 Turn 之间检查已连接且已观察的计算资源仍是其 Session ### `oac-sandbox-io` {#oac-sandbox-io} -每个托管 sandbox 还会运行 `oac-sandbox-io`，它通过[沙箱 Link](./sandbox-link-protocol.md) Serve 该 allocation。`Bootstrap.SandboxIO` 是它的[沙箱引导](./sandbox-bootstrap.md)输入：从[公开 URL](./configuration.md#changing-the-public-url) 派生的 Link URL、allocation 的 Serve credential，以及作为 resource 的 allocation 及其 Serve generation。Core 在 `Create` 前用 `Bootstrap.Validate` 对整个 `Bootstrap` 校验一次，adapter 原样交付。`Create` 将 `SandboxIO` 写入私有文件（参考 adapter 中为 `/home/runtime/sandbox-io-bootstrap.json`，mode 0600、UID 1000），并以 daemon 的账户在 daemon 旁边启动 `oac-sandbox-io --bootstrap-file`，参数为该路径。`BootstrapComplete` 意味着两个进程都已启动。该输入从不通过命令参数或环境变量传递。每个 Runtime 镜像都包含 `/usr/local/bin/oac-sandbox-io`。allocation cleanup 在调用 `Kill` 前先在 relay 撤销该 resource。 +`oac-sandbox-io` 是 Provider 在托管 sandbox 中启动的唯一进程，它通过[沙箱 Link](./sandbox-link-protocol.md) Serve 该 allocation。`Bootstrap` 是 allocation 的 `Reference` 和 `SandboxIO`，后者是该服务的[沙箱引导](./sandbox-bootstrap.md)输入：从[公开 URL](./configuration.md#changing-the-public-url) 派生的 Link URL、allocation 的 Serve credential，以及作为 resource 的 allocation 及其 Serve generation。Core 在 `Create` 前用 `Bootstrap.Validate` 对整个 `Bootstrap` 校验一次，同时检查 `SandboxIO` Serve 的正是该 `Reference`；adapter 原样交付。`Create` 将 `SandboxIO` 写入私有文件（参考 adapter 中为 `/home/runtime/sandbox-io-bootstrap.json`，mode 0600、UID 1000），并以 UID 1000 启动 `oac-sandbox-io --bootstrap-file`，参数为该路径。`BootstrapComplete` 意味着它已启动。该输入从不通过命令参数或环境变量传递。每个 Runtime 镜像都包含 `/usr/local/bin/oac-sandbox-io`。allocation cleanup 在调用 `Kill` 前先在 relay 撤销该 resource。 ### 每节点生命周期 worker {#per-node-lifecycle-workers} @@ -192,7 +190,7 @@ Core 用同一个固定 policy 暂停每个声明 checkpoint 支持的 provider Worker lease、Session lock 与 per-node gate 对每个 provider 负责 suspension。新 Turn claim、file-write intent 和 capture admission 在 Session lock 下串行化，共享一个 compute-phase 检查；新 pending work 取消 capture 并唤醒同一 source。正常 preparation 在经过认证的 resume handshake 后等待 compute phase 为 running；pending input 的 promotion 与 lifecycle transition 冲突时保持 pending。compute phase 和 revision-checked receipt 位于 allocation。Core 在 effect 前持久化 quiesce、capture 和 restore intent，仅新 receipt 执行 capture 或 restore，恢复观察精确 attempt，不重试未知 creation、capture 或 restore。已消费 snapshot 不让 running generation 回滚。删除、撤销和 retention expiry 优先于 wake，一直持续到最终数据库 compare-and-swap；未知 cleanup identity 保留，直到确认所属资源不存在。已消费 artifact 和旧 compute 被删除，因此暂停循环不累积可写磁盘链。 -排队工作和实时 Environment file access 唤醒 suspended Environment；history 和已发布 Artifact read 不唤醒。计划暂停在 daemon 连接上使用 Environment 和 suspension token。受 PID 与 start-time fencing 的本地 control signal（`RunCommandCompute`）唤醒 parked daemon，daemon 在准入工作前重新认证。确认前临时断连通过有界 attempt 和 backoff 重试同一已 armed suspension；永久认证或协议拒绝则关闭。Core 负责 snapshot retention deadline，daemon 没有相应 timer。quiesce 确认丢失时可以通过明确 rollback 解冻同一 source，但不授权 capture。 +排队工作和实时 Environment file access 唤醒 suspended Environment；history 和已发布 Artifact read 不唤醒。计划暂停在 daemon 连接上使用 Environment 和 suspension token。恢复后，Core 等待 sandbox 的 Sandbox I/O 重新 Serve，然后在 daemon 连接上用同一 suspension token 恢复该 Environment（[`runtime_compute_wake.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/execution/runtime_compute_wake.go)）。确认前临时断连通过有界 attempt 和 backoff 重试同一已 armed suspension；永久认证或协议拒绝则关闭。Core 负责 snapshot retention deadline，daemon 没有相应 timer。quiesce 确认丢失时可以通过明确 rollback 解冻同一 source，但不授权 capture。 ### 重置与归档 {#reset-and-archive} @@ -210,7 +208,7 @@ installer 与 Go adapter 的 E2B template 和 endpoint validator 消费共享 [s node 测试单独覆盖 disconnect、reconnect fencing，以及 Create response 丢失后的 cleanup。helper protocol 和 [sandbox node 协议](../../contracts/agents-api/zh/node-generation-protocol.md)要求精确版本匹配；直接进程内接口没有独立 wire version。 -原生验收证明 fixture 无法证明的事实：creation、lease 行为、所属 partial cleanup、声明的 isolation 与 limit，以及支持时的 snapshot。显式启用的 Docker lifecycle、recovery、Serve 和 node transport 测试使用 `AGENTS_RUNTIME_DOCKER_TEST_IMAGE`，即一个包含 `/bin/sh` 和持续运行的 `/usr/local/bin/oac-daemon` 的镜像 digest；Serve 测试会把本源码树的 `oac-sandbox-io` 加入它派生的镜像；SDK helper 使用 `make check-e2b-provider` 和 `make check-microsandbox-provider`。mock compute 不能证明 reclamation 或 isolation。 +原生验收证明 fixture 无法证明的事实：creation、lease 行为、所属 partial cleanup、声明的 isolation 与 limit，以及支持时的 snapshot。显式启用的 Docker lifecycle、recovery、Serve 和 node transport 测试使用 `AGENTS_RUNTIME_DOCKER_TEST_IMAGE`，即一个包含 `/bin/sh` 和 `/usr/local/bin/oac-sandbox-io` 的镜像 digest，例如 `scripts/build-agent-host-images.sh` 构建的 `sandbox` 镜像；Serve 测试会把本源码树的 `oac-sandbox-io` 加入它派生的镜像；SDK helper 使用 `make check-e2b-provider` 和 `make check-microsandbox-provider`。mock compute 不能证明 reclamation 或 isolation。 ## 参考 adapter {#reference-adapters} @@ -218,19 +216,18 @@ node 测试单独覆盖 disconnect、reconnect fencing，以及 Create response | --- | --- | --- | --- | | Docker (node) | [`sandbox/docker`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/docker) | [`sandbox/node`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/node) 中的 node proxy | [Docker adapter](#docker-adapter) | | microsandbox (node) | [`sandbox/microsandbox`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/microsandbox) | [`tools/microsandbox-provider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/microsandbox-provider/README.md) | [Node](getting-started/nodes.md) | -| E2B (direct) | [`sandbox/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/e2b) | [`tools/e2b-provider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) | [沙箱部署](../../contracts/agents-api/zh/sandbox-deployment.md#e2b-configuration)；应用管理的模板见 [`deploy/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/deploy/e2b/README.md) | +| E2B (direct) | [`sandbox/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/e2b) | [`tools/e2b-provider`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/tools/e2b-provider/README.md) | [沙箱部署](../../contracts/agents-api/zh/sandbox-deployment.md#e2b-configuration)；模板见 [`deploy/e2b`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/deploy/e2b/README.md) | ## Docker adapter {#docker-adapter} -Docker Sandbox Provider（[`sandbox/docker`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/docker)）对所有 Runtime image 使用相同 container setting，无论服务哪个 Harness（[`container_options.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/docker/container_options.go)）： +Docker Sandbox Provider（[`sandbox/docker`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/docker)）将每个 allocation 的 sandbox 作为唯一进程为 [`oac-sandbox-io`](#oac-sandbox-io) 的 container 运行，并使用相同 container setting（[`container_options.go`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/services/core/internal/sandbox/docker/container_options.go)）： - user 1000:1000、只读 root filesystem、移除全部 capability、`no-new-privileges`、[seccomp profile](#seccomp-profile) 和 AppArmor `unconfined`； - node 配置的 network 和 extra host（[node 配置](configuration.md#docker-node-configuration)）； - deployment specification 中的 CPU 和 memory，128-process limit 和 128 MiB `/tmp` tmpfs； -- 两个 named volume，label 包含 installation、tenant、Environment 和 allocation：`-home` 挂载到 `/home`，`-environment` 挂载到 `/environment`，后者的 `workspace` 子目录也挂载到 `/workspace`。Docker Engine 必须支持 volume subpath mount； -- 配置 `nested_sandbox` option 时，解除 Docker `/proc` mask（`/sys/firmware` 和 `/sys/devices/virtual/powercap` 保持 mask），container 运行 init process。 +- 两个 named volume，label 包含 installation、tenant、Environment 和 allocation：`-home` 挂载到 `/home`，`-environment` 挂载到 `/environment`，后者的 `workspace` 子目录也挂载到 `/workspace`。Docker Engine 必须支持 volume subpath mount。 -Create 拒绝复用没有 container 的保留 volume。它将 [Runtime 引导](runtime-bootstrap.md)文件复制到 `/home/runtime/runtime-bootstrap.json`（mode 0600、UID 1000），并将 `/environment` workspace、staging、initialization 和 package directory 放入 container，然后启动 `oac-daemon connect --profile default --bootstrap-file /home/runtime/runtime-bootstrap.json` 和 [`oac-sandbox-io`](#oac-sandbox-io)。创建的 container 不具备配置的 CPU、memory 和精确 image 时，Create 返回 error 和 `CreateSettled`。Docker 没有 lease，因此 Renew 仅读取 container state。Kill 在删除前检查 container 和两个 volume 的 ownership label，再确认三者都已不存在。 +Create 拒绝复用没有 container 的保留 volume。它将沙箱引导文件复制到 `/home/runtime/sandbox-io-bootstrap.json`（mode 0600、UID 1000），并将 `/environment` workspace、initialization 和 package directory 放入 container；container 以 `oac-sandbox-io --bootstrap-file /home/runtime/sandbox-io-bootstrap.json` 作为 entry point 运行。该服务是 container 的第一个进程，并回收其孤儿后代进程，因此 container 不需要 init process。创建的 container 不具备配置的 CPU、memory 和精确 image 时，Create 返回 error 和 `CreateSettled`。Docker 没有 lease，因此 Renew 仅读取 container state。Kill 在删除前检查 container 和两个 volume 的 ownership label，再确认三者都已不存在。 node 使用 [provider 配置](configuration.md#docker-node-configuration)中的明确 Unix socket，忽略 `DOCKER_HOST`。不将 Docker socket、host home 或 Core credential 挂载进 Runtime。 diff --git a/services/core/cmd/server/managed_generation_operations.go b/services/core/cmd/server/managed_generation_operations.go index 2f7c99b65..04019cc22 100644 --- a/services/core/cmd/server/managed_generation_operations.go +++ b/services/core/cmd/server/managed_generation_operations.go @@ -83,17 +83,6 @@ func (p *generationRouter) DeleteSnapshot(ctx context.Context, r sandbox.Referen defer done() return v.DeleteSnapshot(ctx, r, snapshot) } -func (p *generationRouter) RunCommandCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute, command sandbox.Command) (sandbox.CommandResult, error) { - if err := providercontract.Require(p, "RunCommandCompute"); err != nil { - return sandbox.CommandResult{}, err - } - v, done, err := p.route(ctx, r) - if err != nil { - return sandbox.CommandResult{}, err - } - defer done() - return v.RunCommandCompute(ctx, r, c, command) -} func (p *generationRouter) ResumeCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute) (sandbox.ComputeState, error) { if err := providercontract.Require(p, "ResumeCompute"); err != nil { return sandbox.ComputeState{}, err diff --git a/services/core/cmd/server/managed_generations.go b/services/core/cmd/server/managed_generations.go index e509ec494..21c8ecd0d 100644 --- a/services/core/cmd/server/managed_generations.go +++ b/services/core/cmd/server/managed_generations.go @@ -71,14 +71,6 @@ func (p *generationRouter) Kill(ctx context.Context, r sandbox.Reference) error defer done() return v.Kill(ctx, r) } -func (p *generationRouter) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - v, done, err := p.route(ctx, r) - if err != nil { - return sandbox.CommandResult{}, err - } - defer done() - return v.RunCommand(ctx, r, c) -} func (p *generationRouter) ProviderOperations() providercontract.Operations { return maps.Clone(p.operations) diff --git a/services/core/cmd/server/managed_nodes.go b/services/core/cmd/server/managed_nodes.go index f5d2974d1..8383abb35 100644 --- a/services/core/cmd/server/managed_nodes.go +++ b/services/core/cmd/server/managed_nodes.go @@ -72,7 +72,7 @@ func configureManagedNodes(nodes *deployment.Service, reader deployment.Reader, }) // An origin without a sandbox Link admits no hosted sandbox. link, _ := config.PublicOrigin.SandboxLink() - result.setup = &managedSetup{processPaths: config.ProviderPaths, registry: registry, deployment: nodes, allocations: reader, hub: result.hub, installationID: config.InstallationID, runtimeAPI: config.PublicOrigin.RuntimeAPI(), sandboxLink: link} + result.setup = &managedSetup{processPaths: config.ProviderPaths, registry: registry, deployment: nodes, allocations: reader, hub: result.hub, installationID: config.InstallationID, sandboxLink: link} result.runtime = execution.NewDeferredRuntimeProvider(config.InstallationID, result.setup.load, result.setup.prepare) result.runtime.PublishUnconfigured = result.setup.publishUnconfigured return result diff --git a/services/core/cmd/server/managed_setup.go b/services/core/cmd/server/managed_setup.go index dba227b56..cf8cfaa80 100644 --- a/services/core/cmd/server/managed_setup.go +++ b/services/core/cmd/server/managed_setup.go @@ -29,12 +29,11 @@ type managedSetup struct { allocations generationAllocations hub *node.Hub installationID string - // runtimeAPI is the /api/v1 base of OAC_PUBLIC_URL; every sandbox reaches - // Core through it and Serves on sandboxLink, empty when the origin has no - // Link. - runtimeAPI, sandboxLink string - selected atomic.Pointer[managedSelection] - providerCalls sandbox.CallFence + // sandboxLink is the Link every sandbox Serves on, empty when the origin + // has no Link. + sandboxLink string + selected atomic.Pointer[managedSelection] + providerCalls sandbox.CallFence } // deploymentSetups reads the committed deployment setup and its retained @@ -152,7 +151,7 @@ func (s *managedSetup) configuration(setup deployment.Setup) (execution.Prepared return execution.PreparedRuntimeDeployment{}, fmt.Errorf("%w: %v", execution.ErrExecutionUnavailable, err) } selected := &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, - CoreURL: s.runtimeAPI, SandboxLink: s.sandboxLink, BackendFingerprint: setup.BackendFingerprint, Provider: provider} + SandboxLink: s.sandboxLink, BackendFingerprint: setup.BackendFingerprint, Provider: provider} if setup.Suspension != nil { selected.Suspension = &execution.RuntimeSuspensionPolicy{IdleTimeout: time.Duration(setup.Suspension.IdleSeconds) * time.Second, Retention: time.Duration(setup.Suspension.RetentionSeconds) * time.Second} diff --git a/services/core/cmd/server/managed_setup_test.go b/services/core/cmd/server/managed_setup_test.go index aaaf9cd02..3967268f9 100644 --- a/services/core/cmd/server/managed_setup_test.go +++ b/services/core/cmd/server/managed_setup_test.go @@ -31,7 +31,7 @@ func TestWebSetupCreatesManagerWithoutLocalProvider(t *testing.T) { } m := configureManagedNodes(nil, nil, providers.Builtin(), processconfig.Config{InstallationID: uuid.NewString(), PublicOrigin: origin}, func(context.Context) error { return nil }) defer m.hub.Close() - if m.setup == nil || m.hub == nil || m.runtime == nil || m.runtime.Provider != nil || m.setup.runtimeAPI != "https://core.example/api/v1" { + if m.setup == nil || m.hub == nil || m.runtime == nil || m.runtime.Provider != nil || m.setup.sandboxLink != "wss://core.example/api/v1/sandbox-link" { t.Fatal("zero-node setup unexpectedly instantiated local compute or omitted management") } } @@ -167,14 +167,14 @@ func TestManagedSetupPreparesWithoutPublishing(t *testing.T) { id := uuid.NewString() hub := node.NewHub(node.HubOptions{}) defer hub.Close() - s := &managedSetup{registry: providers.Builtin(), installationID: id, hub: hub, deployment: &fakeDeploymentSetups{t: t}, allocations: &fakeGenerationAllocations{t: t}, runtimeAPI: "https://core.example/api/v1"} + s := &managedSetup{registry: providers.Builtin(), installationID: id, hub: hub, deployment: &fakeDeploymentSetups{t: t}, allocations: &fakeGenerationAllocations{t: t}, sandboxLink: "wss://core.example/api/v1/sandbox-link"} previous := &execution.RuntimeProvider{InstallationID: id, Generation: 1, ProviderKind: "docker"} s.publish(previous) candidate, err := s.prepare(t.Context(), deployment.Setup{InstallationID: id, Provider: "microsandbox", Mode: "nodes", Operations: microsandbox.Operations(), Suspension: &deployment.Suspension{IdleSeconds: 300, RetentionSeconds: 86400}}) if err != nil { t.Fatal(err) } - if s.selected.Load().Config != previous || candidate.Config.ProviderKind != "microsandbox" || candidate.Config.Suspension == nil || candidate.Config.CoreURL != "https://core.example/api/v1" { + if s.selected.Load().Config != previous || candidate.Config.ProviderKind != "microsandbox" || candidate.Config.Suspension == nil || candidate.Config.SandboxLink != "wss://core.example/api/v1/sandbox-link" { t.Fatal("preparation published or lost candidate configuration") } committed := *candidate.Config diff --git a/services/core/deploy/e2b/README.md b/services/core/deploy/e2b/README.md index 0fc17bf7f..75675ca23 100644 --- a/services/core/deploy/e2b/README.md +++ b/services/core/deploy/e2b/README.md @@ -1,11 +1,6 @@ -# E2B Runtime template and application-managed launch +# E2B sandbox template -An E2B template packages a qualified Runtime image so that its daemon, native Harnesses and workspace run together in one E2B sandbox. The template serves two ownership models: - -- **Core-managed** (`openai_hosted`): the deployment selects E2B as its Sandbox Provider and Core creates, renews and destroys sandboxes from the template. [Sandbox deployment](../../../../contracts/agents-api/sandbox-deployment.md) owns the selection and the [E2B helper](../../tools/e2b-provider/README.md) owns the adapter. -- **Application-managed** (`self_hosted`): the application creates, renews and destroys its own sandbox and enrolls the daemon into a `self_hosted` Environment. Core receives neither the E2B account key nor an allocation request, and deleting the Session leaves the sandbox running. - -This page covers building the template and the application-managed launch. Execution and Files always go through Core and the daemon; E2B commands and files are used only to start and inspect the sandbox. +An E2B template packages a qualified Runtime image for Core-managed E2B sandboxes: the deployment selects E2B as its Sandbox Provider, and Core creates, renews and destroys sandboxes from the template. [Sandbox deployment](../../../../contracts/agents-api/sandbox-deployment.md) owns the selection, and the [E2B helper](../../tools/e2b-provider/README.md) owns the adapter and the startup script, [`managed_init.py`](managed_init.py), that starts the Sandbox I/O service in each sandbox. ## Build a template @@ -21,68 +16,9 @@ python -m venv "$HOME/.oac/build/e2b-sdk" --output "$HOME/.oac/build/e2b-template.json" ``` -[`build-template.py`](build-template.py) requires a `sha256:` image ID, a Linux amd64 image and the Runtime layout (`OAC_RUNTIME_WORKSPACE=/environment/workspace`). It copies the image's `/usr/local/bin`, `/opt` and, when present, `/usr/local/codex-resources` and `/etc/codex`, together with its `HOME` and `OAC_*` environment, onto a digest-pinned `node:22.23.1-bookworm-slim` base with `ca-certificates`, `bash`, `git`, `python3`, `python3-pip`, `ripgrep` and `util-linux`. It installs [`init.py`](init.py) and [`managed_init.py`](managed_init.py) read-only under `/opt/oac-e2b`, runs the daemon as UID/GID 1000 (`runtime`, home `/home/runtime`) and builds with 2 vCPUs and 2048 MiB. Only the `usr`, `usr/local` and `etc` archive ancestors it creates get traversable modes; Runtime file modes, private build contexts, key inputs and the output's umask stay unchanged. - -The output file records `template` (the immutable `templateID:build_UUID`), the source `image`, the packaged `runtime_sha256` and the `base`. Use that exact `template` value. The template must qualify every Harness its image advertises. Install system dependencies into the image; the daemon runs as UID/GID 1000. No E2B account key, executor key or model credential belongs in a build, template environment, metadata, command argument or log. - -## Launch an application-managed Runtime - -1. Create a `self_hosted` Environment through Core and keep its ID and the exact returned `remote_url` (`wss://…/api/v1/agent-daemon/ws`). The VM must reach it. Set `workspace_directory` to `/workspace`, which the template binds to `/environment/workspace`. -2. Obtain a connect-only executor key for that Environment; [Environment executor credentials](../../../../contracts/agents-api/environment-executor-credentials.md) owns issuance. The key file is `{"key_id":"UUID","executor_token":"SECRET"}`, with an optional `environment_id` restriction. Store it and the E2B API key in private files with mode `0600`. -3. Generate an application launch UUID once and keep it. Run [`launch.py`](launch.py), a thin SDK example rather than a service: - -```sh -"$HOME/.oac/build/e2b-sdk/bin/python" services/core/deploy/e2b/launch.py \ - --template 'TEMPLATE_ID:BUILD_UUID' \ - --remote-url 'RETURNED_REMOTE_URL' \ - --environment-id 'RETURNED_ENVIRONMENT_UUID' \ - --launch-id 'YOUR_APPLICATION_LAUNCH_UUID' \ - --executor-key-file "$HOME/.oac/secrets/executor-key.json" \ - --api-key-file "$HOME/.oac/secrets/e2b.key" \ - --record "$HOME/.oac/runtimes/YOUR_APPLICATION_LAUNCH_UUID.json" \ - --timeout 7200 -``` - -The example creates the private launch record exclusively before Create, so reusing a record path fails before any cloud call. It saves the sandbox ID before startup and creates the sandbox with `on_timeout=kill` and auto-resume disabled; its metadata holds only `oac_launch_id` and `oac_environment_id`. It never repeats Create or startup, replaces a sandbox or deletes failure evidence. After an uncertain result, inspect the record and the sandbox; do not start again with a new record path. Choose a lease your E2B account supports and renew it before it expires. - -## Inspect, renew and destroy - -The application owns the lease and cleanup, including after Session deletion or daemon failure. These SDK calls use the recorded sandbox ID and never connect to, resume or recreate a sandbox: - -```python -import json -from pathlib import Path -from e2b import Sandbox - -record = json.loads(Path('/private/launch.json').read_text()) -api_key = Path('/private/e2b.key').read_text().strip() -sandbox_id = record['sandbox_id'] -info = Sandbox.get_info(sandbox_id, api_key=api_key) -assert info.metadata['oac_launch_id'] == record['launch_id'] -assert info.metadata['oac_environment_id'] == record['environment_id'] -Sandbox.set_timeout(sandbox_id, 7200, api_key=api_key) # When renewing the live VM. -# When the application is finished, or explicitly abandons this allocation: -Sandbox.kill(sandbox_id, api_key=api_key) -``` - -If Create's response was lost before its ID was saved, list candidates with `Sandbox.list(query=SandboxQuery(metadata={'oac_launch_id': launch_id}), api_key=api_key)` (`SandboxQuery` comes from `e2b`) and read every page while `paginator.has_next`, using `paginator.next_items()`. Check both metadata fields against the record, keep every matching sandbox ID and inspect or destroy each of those sandboxes. An empty listing does not permit another Create. Do not pick one candidate arbitrarily. - -After an uncertain startup, inspect the same VM or destroy it. The SDK's `Sandbox.connect` can resume a paused sandbox, so do not use it to inspect. The VM keeps these records: - -| Path | Content | -| --- | --- | -| `/root/.oac/e2b/launch.json` | The startup claim, written before any other startup step | -| `/root/.oac/e2b/ready.json` | Process handoff only (`daemon_started`, with the daemon PID), even if the daemon exits later | -| `/home/runtime/.oac/daemon/default/daemon.log` | The daemon log | -| `/home/runtime/.oac/daemon/environment.json` | The daemon's verified Environment and Session binding | - -Neither record proves enrollment, native readiness or a successful Turn; check the Environment status in Core. Keep the records and native history after a failure. `init.py` refuses to run again once any launch record exists. VM expiry destroys the history in it; a replacement VM never recovers the original Session. - -## Startup and security boundary - -`init.py` runs once as root. It restores the ownership and modes that E2B finalization changes under `/usr/local` and on `envd`, `/etc/inittab` and `/etc/init.d/rcS`, locks E2B's passwordless `user` account, checks that the image environment is not already bound to an Environment or Session and bind-mounts `/environment/workspace` at `/workspace`. It writes the executor key to `/home/runtime/.oac/daemon/executor-key.json` (mode 0600, owned by UID 1000), deletes the startup input and starts `oac-daemon connect --profile default --remote … --environment-id … --credential-file …` as UID/GID 1000. No credential enters the daemon's arguments or inherited environment. The daemon owns enrollment and the local binding. E2B clears `/run` at boot, so the records live under `/root/.oac/e2b`. +[`build-template.py`](build-template.py) requires a `sha256:` image ID, a Linux amd64 image and the Runtime layout (`OAC_RUNTIME_WORKSPACE=/environment/workspace`). It copies the image's `/usr/local/bin`, `/opt` and, when present, `/usr/local/codex-resources` and `/etc/codex`, together with its `HOME` and `OAC_*` environment, onto a digest-pinned `node:22.23.1-bookworm-slim` base with `ca-certificates`, `bash`, `git`, `python3`, `python3-pip`, `ripgrep` and `util-linux`. It installs [`managed_init.py`](managed_init.py) read-only under `/opt/oac-e2b`, makes UID/GID 1000 (`runtime`, home `/home/runtime`) the template user and builds with 2 vCPUs and 2048 MiB. Only the `usr`, `usr/local` and `etc` archive ancestors it creates get traversable modes; Runtime file modes, private build contexts, key inputs and the output's umask stay unchanged. -The E2B VM is the isolation boundary; tools have UID 1000's access to Runtime state ([Runtime and outer isolation](../../../../docs/concepts.md#runtime-and-outer-isolation)). +The output file records `template` (the immutable `templateID:build_UUID`), the source `image`, the packaged `runtime_sha256` and the `base`. Use that exact `template` value. The template must qualify every Harness its image advertises. Install system dependencies into the image; sandbox processes run as UID/GID 1000. No E2B account key or model credential belongs in a build, template environment, metadata, command argument or log. ## Tests @@ -90,4 +26,4 @@ The E2B VM is the isolation boundary; tools have UID 1000's access to Runtime st make check-e2b-provider ``` -With `OAC_TEST_E2B_SDK_PYTHON` pointing at the pinned SDK environment, this runs this directory's tests and the helper's. They cover startup input binding, the protected key file, the unchanged URL, credentials kept out of arguments, environment and records, the one-shot claim, and retained sandbox IDs after unknown outcomes. They create no billable resources. `make check-core` runs `managed_init_test.py` with only the standard library. +With `OAC_TEST_E2B_SDK_PYTHON` pointing at the pinned SDK environment, this runs this directory's tests and the helper's. They cover the startup input binding, the private Sandbox bootstrap file, credentials kept out of arguments and records, the one-shot claim and the template's archive modes. They create no billable resources. `make check-core` runs `managed_init_test.py` with only the standard library. diff --git a/services/core/deploy/e2b/build-template.py b/services/core/deploy/e2b/build-template.py index d78926213..c2243aa02 100644 --- a/services/core/deploy/e2b/build-template.py +++ b/services/core/deploy/e2b/build-template.py @@ -64,7 +64,7 @@ for entry in tree.iterdir(): archive.add(entry, arcname=entry.name) (context / 'runtime-env.json').write_text(json.dumps(environment)) - for name in ['init.py', 'managed_init.py', 'helper_contract_generated.py']: + for name in ['managed_init.py', 'helper_contract_generated.py']: (context / name).write_bytes(Path(__file__).with_name(name).read_bytes()) template = (Template(file_context_path=context).from_image(BASE) .run_cmd('apt-get update && apt-get install -y --no-install-recommends ' @@ -72,7 +72,6 @@ '&& rm -rf /var/lib/apt/lists/*', user='root') .copy('runtime.tar.gz', '/root/runtime.tar.gz', user='root') .copy('runtime-env.json', '/etc/oac-runtime-env.json', user='root') - .copy('init.py', '/opt/oac-e2b/init.py', user='root') .copy('managed_init.py', '/opt/oac-e2b/managed_init.py', user='root') .copy('helper_contract_generated.py', '/opt/oac-e2b/helper_contract_generated.py', user='root') .run_cmd('tar --no-same-owner -xzf /root/runtime.tar.gz -C / && rm /root/runtime.tar.gz ' @@ -81,7 +80,7 @@ '&& chown -R 1000:1000 /home/runtime /environment ' '&& chmod 0700 /home/runtime/.oac /environment/staging ' '&& chmod 0444 /etc/oac-runtime-env.json ' - '&& chmod 0555 /opt/oac-e2b /opt/oac-e2b/init.py /opt/oac-e2b/managed_init.py /opt/oac-e2b/helper_contract_generated.py', user='root') + '&& chmod 0555 /opt/oac-e2b /opt/oac-e2b/managed_init.py /opt/oac-e2b/helper_contract_generated.py', user='root') .set_user('runtime').set_workdir('/environment/workspace')) result = Template.build(template, name=args.name, cpu_count=2, memory_mb=2048, on_build_logs=lambda entry: print(entry.message, flush=True), diff --git a/services/core/deploy/e2b/build_template_test.py b/services/core/deploy/e2b/build_template_test.py index 9a3b3034f..6c85a585e 100644 --- a/services/core/deploy/e2b/build_template_test.py +++ b/services/core/deploy/e2b/build_template_test.py @@ -75,7 +75,7 @@ def build(instance, **kwargs): self.assertEqual(stat.S_IMODE(key.stat().st_mode), 0o600) projection = 'helper_contract_generated.py' self.assertEqual((context / projection).read_bytes(), Path(__file__).with_name(projection).read_bytes()) - for source in ('init.py', 'managed_init.py', projection): + for source in ('managed_init.py', projection): template.copy.assert_any_call(source, '/opt/oac-e2b/' + source, user='root') self.assertTrue(any('chmod 0555' in call.args[0] and '/opt/oac-e2b/' + projection in call.args[0] for call in template.run_cmd.call_args_list)) diff --git a/services/core/deploy/e2b/helper_contract_generated.py b/services/core/deploy/e2b/helper_contract_generated.py index fa90c0762..87139c0e7 100644 --- a/services/core/deploy/e2b/helper_contract_generated.py +++ b/services/core/deploy/e2b/helper_contract_generated.py @@ -1,17 +1,15 @@ # Code generated by contractgen; DO NOT EDIT. """Adapter-private wire declarations. No SDK or repository dependency.""" -ERROR_CODES = ["","invalid","ownership","exists","not_found","command_unconfirmed","unconfirmed","template_invalid","team_mismatch","unauthorized"] -MANAGED_BOOTSTRAP_FIELDS = ["TenantID","EnvironmentID","AllocationID","SessionID","DeviceID","NetworkAccess","AllowedDomains","SandboxIO","InstallationID","RuntimeBootstrap"] -MANAGED_IDENTITY_FIELDS = ["TenantID","EnvironmentID","AllocationID","SessionID","DeviceID","InstallationID"] -MAX_COMMAND_INPUT = 52428832 +ERROR_CODES = ["","invalid","ownership","exists","not_found","unconfirmed","template_invalid","team_mismatch","unauthorized"] +MANAGED_BOOTSTRAP_FIELDS = ["TenantID","EnvironmentID","AllocationID","SandboxIO","InstallationID"] +MANAGED_IDENTITY_FIELDS = ["TenantID","EnvironmentID","AllocationID","InstallationID"] MAX_CREDENTIAL_REFERENCES = 32 MAX_OUTPUT = 1048576 -MAX_REQUEST = 75497472 +MAX_REQUEST = 1048576 MAX_RESPONSE = 16777216 -NETWORK_ACCESS = ["enabled","disabled","restricted"] -OPERATIONS = ["create","inspect","renew","kill","command","validate_deployment","observe","list_templates","list_builds","verify_credential"] +OPERATIONS = ["create","inspect","renew","kill","validate_deployment","observe","list_templates","list_builds","verify_credential"] PROTOCOL_VERSION = 1 REFERENCE_FIELDS = ["TenantID","EnvironmentID","AllocationID"] -REQUEST_FIELDS = ["Version","Operation","Config","Reference","References","Bootstrap","RuntimeBootstrap","Command","Deadline"] -RESPONSE_FIELDS = ["Version","Info","Command","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observation"] +REQUEST_FIELDS = ["Version","Operation","Config","Reference","References","Bootstrap","Deadline"] +RESPONSE_FIELDS = ["Version","Info","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observation"] SDK_VERSION = "2.51.0" diff --git a/services/core/deploy/e2b/init.py b/services/core/deploy/e2b/init.py deleted file mode 100644 index 75d8e1f07..000000000 --- a/services/core/deploy/e2b/init.py +++ /dev/null @@ -1,132 +0,0 @@ -#!/usr/bin/env python3 -"""One-shot user-owned Runtime startup; never an enrollment or execution service.""" -import json -import os -from pathlib import Path -import subprocess -from uuid import UUID -from urllib.parse import urlsplit - -ROOT = Path('/root/.oac/e2b') -PROFILE = Path('/home/runtime/.oac/daemon/default') -IMAGE_ENV = Path('/etc/oac-runtime-env.json') - - -def launch_identity(payload): - """Reject malformed startup input without including confidential values in errors.""" - if not isinstance(payload, dict) or set(payload) != { - 'launch_id', 'environment_id', 'remote_url', 'executor_key'}: - raise ValueError('Invalid Runtime startup fields') - try: - for field in ['launch_id', 'environment_id']: - value = payload[field] - if not isinstance(value, str) or str(UUID(value)) != value or UUID(value).int == 0: - raise ValueError() - key = payload['executor_key'] - if (not isinstance(key, dict) or set(key) - {'key_id', 'executor_token', 'environment_id'} - or str(UUID(key['key_id'])) != key['key_id'] or UUID(key['key_id']).int == 0 - or not isinstance(key['executor_token'], str) or not key['executor_token'] - or any(character.isspace() or character == '\x00' for character in key['executor_token']) - or key.get('environment_id') not in (None, '', payload['environment_id']) - or len(json.dumps(key).encode()) > 16384): - raise ValueError() - # Keep credentials out of argv/receipts; the daemon owns enrollment validation. - remote = payload['remote_url'] - address = urlsplit(remote) - if (not isinstance(remote, str) or address.scheme not in ('ws', 'wss') - or not address.hostname or address.username is not None - or address.query or address.fragment or '?' in remote or '#' in remote - or address.path != '/api/v1/agent-daemon/ws' or remote.strip() != remote - or len(json.dumps(payload).encode()) > 32768): - raise ValueError() - except (KeyError, ValueError, TypeError, AttributeError): - raise ValueError('Invalid Runtime startup identity or credential') from None - return {field: payload[field] for field in ['launch_id', 'environment_id', 'remote_url']} - - -def sync_directory(path): - descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY) - try: - os.fsync(descriptor) - finally: - os.close(descriptor) - - -def write_private(path, value, owner=None): - with path.open('x') as stream: - os.fchmod(stream.fileno(), 0o600) - if owner is not None: - os.fchown(stream.fileno(), owner, owner) - json.dump(value, stream) - stream.flush() - os.fsync(stream.fileno()) - sync_directory(path.parent) - - -def prepare_runtime(): - """Restore the protected image and shared Runtime layout before any daemon starts.""" - # E2B finalization makes /usr/local world-writable after template commands. - subprocess.run(['chown', '-R', 'root:root', '/usr/local'], check=True) - subprocess.run(['chmod', '-R', 'go-w', '/usr/local'], check=True) - for protected in ['/usr/bin/envd', '/etc/inittab', '/etc/init.d/rcS']: - if protected == '/etc/init.d/rcS' and not Path(protected).exists(): - continue - os.chown(protected, 0, 0) - os.chmod(protected, 0o755) - # Disable E2B's unused passwordless sudo account before unprivileged startup. - subprocess.run(['usermod', '--lock', '--shell', '/usr/sbin/nologin', 'user'], check=True) - environment = json.loads(IMAGE_ENV.read_text()) - if (environment.get('OAC_RUNTIME_HOME') != '/home/runtime/.oac' - or environment.get('OAC_RUNTIME_WORKSPACE') != '/environment/workspace' - or any(key in environment for key in ['OAC_RUNTIME_ENVIRONMENT_ID', - 'OAC_RUNTIME_SESSION_ID'])): - raise ValueError('Image must contain an unbound packaged Runtime profile') - environment['PATH'] = '/usr/local/bin:/usr/bin:/bin' - subprocess.run(['mount', '--bind', '/environment/workspace', '/workspace'], check=True) - PROFILE.mkdir(mode=0o700, parents=True, exist_ok=True) - for directory in [Path('/home/runtime'), Path('/home/runtime/.oac'), PROFILE.parent, PROFILE, - Path('/environment/workspace'), Path('/environment/staging'), - Path('/environment/initialization'), Path('/environment/packages')]: - os.chown(directory, 1000, 1000) - directory.chmod(0o700) - return environment - - -def initialize(): - ROOT.mkdir(mode=0o700, parents=True, exist_ok=True) - ROOT.chmod(0o700) - source = ROOT / 'bootstrap.json' - receipt = ROOT / 'ready.json' - if any((ROOT / name).exists() for name in ['ready.json', 'launch.json', 'managed-launch.json', 'managed-ready.json']): - raise RuntimeError('Runtime startup cannot be replayed; inspect or destroy this sandbox') - if source.stat().st_size > 32768: - raise ValueError('Runtime startup input too large') - payload = json.loads(source.read_text()) - identity = launch_identity(payload) - # Claim before any side effect. An interrupted attempt must never start twice. - write_private(ROOT / 'launch.json', identity) - environment = prepare_runtime() - credential = PROFILE.parent / 'executor-key.json' - write_private(credential, payload['executor_key'], owner=1000) - source.unlink() - with (PROFILE / 'daemon.log').open('xb') as stream: - os.fchmod(stream.fileno(), 0o600) - os.fchown(stream.fileno(), 1000, 1000) - child = subprocess.Popen( - ['/usr/local/bin/oac-daemon', 'connect', '--profile', 'default', - '--remote', payload['remote_url'], '--environment-id', payload['environment_id'], - '--credential-file', str(credential)], - cwd='/environment/workspace', env=environment, user=1000, group=1000, - extra_groups=[], start_new_session=True, stdin=subprocess.DEVNULL, - stdout=stream, stderr=subprocess.STDOUT, umask=0o077) - # This acknowledges process handoff only. Core owns enrollment and readiness. - write_private(ROOT / 'ready.tmp', dict(identity, status='daemon_started', daemon_pid=child.pid)) - os.replace(ROOT / 'ready.tmp', receipt) - sync_directory(ROOT) - - -if __name__ == '__main__': - try: - initialize() - except Exception: - raise SystemExit('Runtime startup failed; inspect the retained launch record and sandbox') from None diff --git a/services/core/deploy/e2b/init_test.py b/services/core/deploy/e2b/init_test.py deleted file mode 100644 index 3c416c13f..000000000 --- a/services/core/deploy/e2b/init_test.py +++ /dev/null @@ -1,122 +0,0 @@ -"""Controlled startup contract tests; these do not qualify an E2B Runtime.""" -import copy -import json -from pathlib import Path -import tempfile -import unittest -from unittest.mock import Mock, patch - -import init -import launch - - -PAYLOAD = { - 'launch_id': 'bfe27bc4-dcd7-4aa3-9196-94f7617c9d6b', - 'environment_id': 'b03296db-a32d-49e3-afc3-0b2a3c00f73b', - 'remote_url': 'wss://core.example.com/api/v1/agent-daemon/ws', - 'executor_key': {'key_id': '96e3ba3e-6eb7-4f1b-b65b-1c836c419297', - 'executor_token': 'test-private-key'}, -} -TEMPLATE = 'qualified:' + PAYLOAD['launch_id'] - - -class StartupTest(unittest.TestCase): - def test_identity_rejects_confused_binding_and_secret_urls(self): - for field, value in [('environment_id', 'not-a-uuid'), - ('remote_url', PAYLOAD['remote_url'] + '?token=private'), - ('remote_url', 'wss://secret@core.example.com/api/v1/agent-daemon/ws'), - ('executor_key', dict(PAYLOAD['executor_key'], environment_id=PAYLOAD['launch_id']))]: - with self.subTest(field=field), self.assertRaises(ValueError): - init.launch_identity(dict(PAYLOAD, **{field: value})) - self.assertEqual(init.launch_identity(PAYLOAD)['remote_url'], PAYLOAD['remote_url']) - - def test_launch_handoff_is_private_and_cannot_replay(self): - with tempfile.TemporaryDirectory() as temporary: - root = Path(temporary) / 'root' - root.mkdir() - profile = Path(temporary) / 'private/default' - environment_file = Path(temporary) / 'image.json' - environment_file.write_text(json.dumps({'HOME': '/home/runtime', - 'OAC_RUNTIME_HOME': '/home/runtime/.oac', 'OAC_RUNTIME_WORKSPACE': '/environment/workspace'})) - (root / 'bootstrap.json').write_text(json.dumps(PAYLOAD)) - real_chmod = Path.chmod - - def chmod(path, mode): - if str(path).startswith(temporary): - real_chmod(path, mode) - - with patch.object(init, 'ROOT', root), patch.object(init, 'PROFILE', profile), \ - patch.object(init, 'IMAGE_ENV', environment_file), \ - patch.object(init.os, 'chown'), patch.object(init.os, 'fchown'), \ - patch.object(init.os, 'chmod'), patch.object(Path, 'chmod', chmod), \ - patch.object(init.subprocess, 'run') as run, \ - patch.object(init.subprocess, 'Popen', return_value=Mock(pid=321)) as popen: - init.initialize() - argv = popen.call_args.args[0] - options = popen.call_args.kwargs - self.assertEqual(argv[argv.index('--remote') + 1], PAYLOAD['remote_url']) - self.assertNotIn('test-private-key', repr(popen.call_args)) - self.assertNotIn('OAC_RUNTIME_SESSION_ID', options['env']) - self.assertEqual((options['user'], options['group'], options['extra_groups']), (1000, 1000, [])) - self.assertEqual(options['umask'], 0o077) - key = profile.parent / 'executor-key.json' - self.assertEqual(json.loads(key.read_text()), PAYLOAD['executor_key']) - self.assertEqual(key.stat().st_mode & 0o777, 0o600) - self.assertFalse((profile / 'auth.json').exists()) - self.assertFalse((root / 'bootstrap.json').exists()) - receipt = json.loads((root / 'ready.json').read_text()) - self.assertEqual(receipt['status'], 'daemon_started') - self.assertNotIn('session_id', receipt) - self.assertNotIn('test-private-key', (root / 'launch.json').read_text()) - run.reset_mock() - with self.assertRaises(RuntimeError): - init.initialize() - run.assert_not_called() - popen.assert_called_once() - - def test_failed_initialization_retains_claim_without_ready(self): - with tempfile.TemporaryDirectory() as temporary: - root = Path(temporary) - (root / 'bootstrap.json').write_text(json.dumps(PAYLOAD)) - with patch.object(init, 'ROOT', root), patch.object(init.subprocess, 'run', side_effect=RuntimeError): - with self.assertRaises(RuntimeError): - init.initialize() - self.assertTrue((root / 'launch.json').exists()) - self.assertFalse((root / 'ready.json').exists()) - with self.assertRaisesRegex(RuntimeError, 'cannot be replayed'): - init.initialize() - - def test_sdk_startup_retains_id_and_never_retries_unknown_outcomes(self): - for fail in ['create', 'start', None]: - with self.subTest(fail=fail), tempfile.TemporaryDirectory() as temporary: - record = Path(temporary) / 'launch.json' - sandbox = Mock(sandbox_id='owned-id') - sandbox.files.read.return_value = json.dumps(dict(init.launch_identity(PAYLOAD), - status='daemon_started', daemon_pid=123)) - if fail == 'start': - sandbox.commands.run.side_effect = RuntimeError('test-private-key') - with patch.object(launch.Sandbox, 'create', return_value=sandbox) as create: - if fail == 'create': - create.side_effect = RuntimeError('test-private-key') - if fail: - with self.assertRaisesRegex(RuntimeError, 'outcome uncertain') as raised: - launch.launch(copy.deepcopy(PAYLOAD), TEMPLATE, 'e2b-private-key', record) - self.assertNotIn('test-private-key', str(raised.exception)) - else: - result = launch.launch(PAYLOAD, TEMPLATE, 'e2b-private-key', record) - self.assertEqual(result['status'], 'daemon_started') - persisted = json.loads(record.read_text()) - self.assertNotIn('private-key', record.read_text()) - self.assertEqual(record.stat().st_mode & 0o777, 0o600) - self.assertEqual(persisted.get('sandbox_id'), None if fail == 'create' else 'owned-id') - self.assertEqual(create.call_args.kwargs['lifecycle'], - {'on_timeout': 'kill', 'auto_resume': False}) - self.assertNotIn('private-key', repr(create.call_args.kwargs['metadata'])) - with self.assertRaises(FileExistsError): - launch.launch(PAYLOAD, TEMPLATE, 'e2b-private-key', record) - create.assert_called_once() - sandbox.kill.assert_not_called() - - -if __name__ == '__main__': - unittest.main() diff --git a/services/core/deploy/e2b/launch.py b/services/core/deploy/e2b/launch.py deleted file mode 100644 index 72f3ef6a1..000000000 --- a/services/core/deploy/e2b/launch.py +++ /dev/null @@ -1,85 +0,0 @@ -#!/usr/bin/env python3 -"""Application-owned E2B startup example using the maintained SDK.""" -import argparse -import json -import os -from pathlib import Path -from uuid import UUID - -from e2b import Sandbox -from init import launch_identity, sync_directory, write_private - - -def launch(payload, template, api_key, record_path, timeout=7200): - identity = launch_identity(payload) - try: - template_id, build_id = template.split(':') - if not template_id or str(UUID(build_id)) != build_id or UUID(build_id).int == 0 or timeout <= 0: - raise ValueError() - except (ValueError, AttributeError): - raise ValueError('Use a pinned templateID:build_UUID and a positive lease') from None - record_path = Path(record_path) - if not record_path.is_absolute(): - raise ValueError('Use an absolute private launch record path') - record_path.parent.mkdir(mode=0o700, parents=True, exist_ok=True) - record = dict(identity, template=template, status='create_pending') - # Exclusive creation prevents accidental retries, including unknown Create outcomes. - write_private(record_path, record) - - def save(status, **fields): - record.update(status=status, **fields) - temporary = record_path.with_name(record_path.name + '.tmp') - write_private(temporary, record) - os.replace(temporary, record_path) - sync_directory(record_path.parent) - - try: - sandbox = Sandbox.create( - template=template, timeout=timeout, api_key=api_key, - metadata={'oac_launch_id': payload['launch_id'], - 'oac_environment_id': payload['environment_id']}, - lifecycle={'on_timeout': 'kill', 'auto_resume': False}) - # Retain the actual provider ID before uploading credentials or starting anything. - save('created', sandbox_id=sandbox.sandbox_id) - save('startup_pending') - sandbox.files.write('/root/.oac/e2b/bootstrap.json', json.dumps(payload), - user='root', request_timeout=30) - sandbox.commands.run('/usr/bin/python3 /opt/oac-e2b/init.py', user='root', timeout=60) - receipt = json.loads(sandbox.files.read('/root/.oac/e2b/ready.json', - user='root', request_timeout=30)) - if (any(receipt.get(key) != value for key, value in identity.items()) - or receipt.get('status') != 'daemon_started' - or type(receipt.get('daemon_pid')) is not int or receipt['daemon_pid'] <= 0): - raise ValueError('Unexpected startup receipt') - save('daemon_started', receipt=receipt) - return record - except Exception: - # SDK exceptions can contain request/command details. Keep the durable record, - # do not log credentials, repeat Create/start, or implicitly delete evidence. - raise RuntimeError('Launch outcome uncertain; inspect the private record and owned sandbox') from None - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('--remote-url', required=True) - parser.add_argument('--environment-id', required=True) - parser.add_argument('--launch-id', required=True, help='Application-generated UUID, retained before Create') - parser.add_argument('--template', required=True, help='Immutable templateID:build_UUID') - parser.add_argument('--executor-key-file', required=True, type=Path) - parser.add_argument('--api-key-file', required=True, type=Path) - parser.add_argument('--record', required=True, type=Path) - parser.add_argument('--timeout', type=int, default=7200, help='User-owned lease in seconds') - args = parser.parse_args() - try: - result = launch( - {'launch_id': args.launch_id, 'environment_id': args.environment_id, - 'remote_url': args.remote_url, - 'executor_key': json.loads(args.executor_key_file.read_text())}, - args.template, args.api_key_file.read_text().strip(), args.record, args.timeout) - except Exception: - parser.exit(1, 'Startup not confirmed. Inspect your launch record; do not rerun or replace the sandbox.\n') - print(json.dumps(result)) - - -if __name__ == '__main__': - main() diff --git a/services/core/deploy/e2b/managed_init.py b/services/core/deploy/e2b/managed_init.py index e56619663..8646929e5 100644 --- a/services/core/deploy/e2b/managed_init.py +++ b/services/core/deploy/e2b/managed_init.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""One-shot managed startup through the Runtime bootstrap contract.""" +"""One-shot managed startup: prepare the sandbox and start only Sandbox I/O.""" import importlib.util import json import os @@ -8,75 +8,99 @@ from uuid import UUID # -I excludes the script directory from sys.path; load only its protected sibling. -_spec = importlib.util.spec_from_file_location('runtime_init', Path(__file__).with_name('init.py')) -shared = importlib.util.module_from_spec(_spec) -_spec.loader.exec_module(shared) - - _contract_spec = importlib.util.spec_from_file_location('helper_contract', Path(__file__).with_name('helper_contract_generated.py')) contract = importlib.util.module_from_spec(_contract_spec) _contract_spec.loader.exec_module(contract) +ROOT = Path('/root/.oac/e2b') +HOME = Path('/home/runtime') + def identity(payload): - fields = contract.MANAGED_IDENTITY_FIELDS if not isinstance(payload, dict) or set(payload) != set(contract.MANAGED_BOOTSTRAP_FIELDS): raise ValueError('Invalid managed bootstrap fields') - for field in fields: + for field in contract.MANAGED_IDENTITY_FIELDS: value = payload[field] if not isinstance(value, str) or str(UUID(value)) != value or UUID(value).int == 0: raise ValueError('Invalid managed bootstrap identity') - if (payload['NetworkAccess'] not in contract.NETWORK_ACCESS or - payload['AllowedDomains'] is not None and - (not isinstance(payload['AllowedDomains'], list) or - any(not isinstance(domain, str) for domain in payload['AllowedDomains']))): + # Sandbox I/O validates its own input; this only keeps it a JSON object. + if not isinstance(payload['SandboxIO'], dict): raise ValueError('Invalid managed bootstrap configuration') - return {field: payload[field] for field in fields} + return {field: payload[field] for field in contract.MANAGED_IDENTITY_FIELDS} + + +def sync_directory(path): + descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(descriptor) + finally: + os.close(descriptor) + + +def write_private(path, value, owner=None): + with path.open('x') as stream: + os.fchmod(stream.fileno(), 0o600) + if owner is not None: + os.fchown(stream.fileno(), owner, owner) + json.dump(value, stream) + stream.flush() + os.fsync(stream.fileno()) + sync_directory(path.parent) + + +def prepare_sandbox(): + """Restore the protected image and the Environment layout before Sandbox I/O starts.""" + # E2B finalization makes /usr/local world-writable after template commands. + subprocess.run(['chown', '-R', 'root:root', '/usr/local'], check=True) + subprocess.run(['chmod', '-R', 'go-w', '/usr/local'], check=True) + for protected in ['/usr/bin/envd', '/etc/inittab', '/etc/init.d/rcS']: + if protected == '/etc/init.d/rcS' and not Path(protected).exists(): + continue + os.chown(protected, 0, 0) + os.chmod(protected, 0o755) + # Disable E2B's unused passwordless sudo account before unprivileged startup. + subprocess.run(['usermod', '--lock', '--shell', '/usr/sbin/nologin', 'user'], check=True) + subprocess.run(['mount', '--bind', '/environment/workspace', '/workspace'], check=True) + for directory in [HOME, Path('/environment/workspace'), Path('/environment/initialization'), + Path('/environment/packages')]: + os.chown(directory, 1000, 1000) + directory.chmod(0o700) def initialize(): - root = shared.ROOT - root.mkdir(mode=0o700, parents=True, exist_ok=True) - root.chmod(0o700) - source = root / 'managed-bootstrap.json' - if any((root / name).exists() for name in ['launch.json', 'ready.json', 'managed-launch.json', 'managed-ready.json']): - raise RuntimeError('Runtime bootstrap cannot be replayed') + ROOT.mkdir(mode=0o700, parents=True, exist_ok=True) + ROOT.chmod(0o700) + source = ROOT / 'managed-bootstrap.json' + if any((ROOT / name).exists() for name in ['managed-launch.json', 'managed-ready.json']): + raise RuntimeError('Sandbox startup cannot be replayed') if source.stat().st_size > 65536: raise ValueError('Managed bootstrap input too large') payload = json.loads(source.read_text()) binding = identity(payload) - shared.write_private(root / 'managed-launch.json', binding) - environment = shared.prepare_runtime() - environment.update(OAC_RUNTIME_ENVIRONMENT_ID=payload['EnvironmentID'], - OAC_RUNTIME_SESSION_ID=payload['SessionID'], - OAC_RUNTIME_NETWORK_ACCESS=payload['NetworkAccess'], - OAC_RUNTIME_ALLOWED_DOMAINS=json.dumps(payload['AllowedDomains'] or [])) - connection = Path(environment['OAC_RUNTIME_HOME']).parent / 'runtime-bootstrap.json' - shared.write_private(connection, payload['RuntimeBootstrap'], owner=1000) - serve = connection.with_name('sandbox-io-bootstrap.json') - shared.write_private(serve, payload['SandboxIO'], owner=1000) + # Claim before any side effect. An interrupted attempt must never start twice. + write_private(ROOT / 'managed-launch.json', binding) + prepare_sandbox() + bootstrap = HOME / 'sandbox-io-bootstrap.json' + write_private(bootstrap, payload['SandboxIO'], owner=1000) source.unlink() - # The Sandbox I/O service runs beside the daemon, as the same account; its - # file is its only input. - pids = [] - for log, command, env in [ - (shared.PROFILE / 'daemon.log', ['/usr/local/bin/oac-daemon', 'connect', '--profile', 'default', - '--bootstrap-file', str(connection)], environment), - (connection.with_name('sandbox-io.log'), ['/usr/local/bin/oac-sandbox-io', '--bootstrap-file', str(serve)], {})]: - with log.open('xb') as stream: - os.fchmod(stream.fileno(), 0o600) - os.fchown(stream.fileno(), 1000, 1000) - pids.append(subprocess.Popen(command, cwd='/environment/workspace', env=env, user=1000, group=1000, - extra_groups=[], start_new_session=True, stdin=subprocess.DEVNULL, - stdout=stream, stderr=subprocess.STDOUT, umask=0o077).pid) - shared.write_private(root / 'managed-ready.tmp', - {'identity': binding, 'status': 'daemon_started', 'daemon_pid': pids[0]}) - os.replace(root / 'managed-ready.tmp', root / 'managed-ready.json') - shared.sync_directory(root) + # Sandbox I/O is the only process started in the sandbox; its file is its + # only input. + with (HOME / 'sandbox-io.log').open('xb') as stream: + os.fchmod(stream.fileno(), 0o600) + os.fchown(stream.fileno(), 1000, 1000) + child = subprocess.Popen(['/usr/local/bin/oac-sandbox-io', '--bootstrap-file', str(bootstrap)], + cwd='/environment/workspace', env={}, user=1000, group=1000, + extra_groups=[], start_new_session=True, stdin=subprocess.DEVNULL, + stdout=stream, stderr=subprocess.STDOUT, umask=0o077) + # This acknowledges process handoff only. Serving is Core's to observe. + write_private(ROOT / 'managed-ready.tmp', + {'identity': binding, 'status': 'sandbox_io_started', 'sandbox_io_pid': child.pid}) + os.replace(ROOT / 'managed-ready.tmp', ROOT / 'managed-ready.json') + sync_directory(ROOT) if __name__ == '__main__': try: initialize() except Exception: - raise SystemExit('Managed Runtime startup failed; retain and reclaim the owned sandbox') from None + raise SystemExit('Managed sandbox startup failed; retain and reclaim the owned sandbox') from None diff --git a/services/core/deploy/e2b/managed_init_test.py b/services/core/deploy/e2b/managed_init_test.py index 0af13ac36..0884d427d 100644 --- a/services/core/deploy/e2b/managed_init_test.py +++ b/services/core/deploy/e2b/managed_init_test.py @@ -1,5 +1,4 @@ -"""Managed bootstrap reuses image protection without changing self-hosted enrollment.""" -import copy +"""Managed startup starts only Sandbox I/O, once, with its input in a private file.""" import json from pathlib import Path import tempfile @@ -14,20 +13,16 @@ def payload(): - value = {key: str(uuid4()) for key in ['InstallationID', 'TenantID', 'EnvironmentID', - 'AllocationID', 'SessionID', 'DeviceID']} - return dict(value, RuntimeBootstrap={'version': 1, 'core_url': 'https://core.example/api/v1', - 'device_id': value['DeviceID'], 'credential': 'private-managed-token'}, - SandboxIO={'version': 1, 'link_url': 'wss://core.example/api/v1/sandbox-link', 'credential': 'private-serve-token', - 'resource': {'tenant_id': value['TenantID'], 'environment_id': value['EnvironmentID'], - 'kind': 'allocation', 'id': value['AllocationID'], 'generation': 1}}, - NetworkAccess='restricted', AllowedDomains=['example.com']) + value = {key: str(uuid4()) for key in ['InstallationID', 'TenantID', 'EnvironmentID', 'AllocationID']} + return dict(value, SandboxIO={'version': 1, 'link_url': 'wss://core.example/api/v1/sandbox-link', 'credential': 'private-serve-token', + 'resource': {'tenant_id': value['TenantID'], 'environment_id': value['EnvironmentID'], + 'kind': 'allocation', 'id': value['AllocationID'], 'generation': 1}}) class ManagedStartupTest(unittest.TestCase): def test_isolated_packaged_import(self): with tempfile.TemporaryDirectory() as temporary: - for name in ('init.py', 'managed_init.py', 'helper_contract_generated.py'): + for name in ('managed_init.py', 'helper_contract_generated.py'): shutil.copy2(Path(__file__).with_name(name), Path(temporary, name)) subprocess.run([sys.executable, '-I', '-c', "import runpy,sys; runpy.run_path(sys.argv[1], run_name='fixture')", @@ -47,28 +42,20 @@ def test_shared_bootstrap_exchanges(self): valid = False self.assertEqual(valid, case['valid']) - def test_invalid_binding_rejected(self): - source = payload() - for key, value in [('DeviceID', 'other'), ('NetworkAccess', 'unknown')]: - with self.subTest(key=key), self.assertRaises(ValueError): - managed_init.identity(dict(source, **{key: value})) - def exercise(self, failed=False): with tempfile.TemporaryDirectory() as temporary: root = Path(temporary) / 'receipt' root.mkdir() - profile = Path(temporary) / 'profile' - profile.mkdir() + home = Path(temporary) / 'home' + home.mkdir() source = root / 'managed-bootstrap.json' data = payload() source.write_text(json.dumps(data)) process = Mock(return_value=Mock(pid=456)) if failed: process.side_effect = RuntimeError('private process diagnostic') - image_env = {'PATH': '/usr/local/bin:/usr/bin:/bin', 'OAC_RUNTIME_HOME': str(Path(temporary) / '.oac'), - 'OAC_RUNTIME_WORKSPACE': '/environment/workspace'} - with patch.object(managed_init.shared, 'ROOT', root), patch.object(managed_init.shared, 'PROFILE', profile), \ - patch.object(managed_init.shared, 'prepare_runtime', return_value=image_env), \ + with patch.object(managed_init, 'ROOT', root), patch.object(managed_init, 'HOME', home), \ + patch.object(managed_init, 'prepare_sandbox'), \ patch.object(managed_init.os, 'fchown'), patch.object(managed_init.subprocess, 'Popen', process): if failed: with self.assertRaises(RuntimeError): @@ -76,51 +63,33 @@ def exercise(self, failed=False): else: managed_init.initialize() self.assertTrue((root / 'managed-launch.json').exists()) - connection_file = Path(temporary) / 'runtime-bootstrap.json' - self.assertEqual(json.loads(connection_file.read_text()), data['RuntimeBootstrap']) - self.assertEqual(connection_file.stat().st_mode & 0o777, 0o600) - serve_file = Path(temporary) / 'sandbox-io-bootstrap.json' + serve_file = home / 'sandbox-io-bootstrap.json' self.assertEqual(json.loads(serve_file.read_text()), data['SandboxIO']) self.assertEqual(serve_file.stat().st_mode & 0o777, 0o600) - self.assertFalse((profile / 'auth.json').exists()) + self.assertEqual(sorted(p.name for p in home.iterdir()), ['sandbox-io-bootstrap.json', 'sandbox-io.log']) self.assertFalse(source.exists()) - daemon = process.call_args_list[0] - self.assertEqual(daemon.args[0][-2:], ['--bootstrap-file', str(connection_file)]) - self.assertEqual(daemon.kwargs['env']['OAC_RUNTIME_ENVIRONMENT_ID'], data['EnvironmentID']) - if not failed: - serve = process.call_args_list[1] - self.assertEqual(serve.args[0], ['/usr/local/bin/oac-sandbox-io', '--bootstrap-file', str(serve_file)]) - self.assertEqual(serve.kwargs['env'], {}) - for call in process.call_args_list: - self.assertEqual(call.kwargs['user'], 1000) - for credential in (data['RuntimeBootstrap']['credential'], data['SandboxIO']['credential']): - self.assertNotIn(credential, json.dumps(call.args)) - self.assertNotIn(credential, json.dumps(call.kwargs['env'])) + self.assertEqual(process.call_count, 1) + serve = process.call_args + self.assertEqual(serve.args[0], ['/usr/local/bin/oac-sandbox-io', '--bootstrap-file', str(serve_file)]) + self.assertEqual((serve.kwargs['env'], serve.kwargs['user'], serve.kwargs['group']), ({}, 1000, 1000)) + self.assertNotIn(data['SandboxIO']['credential'], json.dumps(serve.args)) if failed: self.assertFalse((root / 'managed-ready.json').exists()) else: receipt = json.loads((root / 'managed-ready.json').read_text()) - self.assertEqual(receipt['identity'], managed_init.identity(data)) - self.assertNotIn(data['RuntimeBootstrap']['credential'], json.dumps(receipt)) - self.assertEqual(receipt['daemon_pid'], 456) - calls = process.call_count + self.assertEqual(receipt, {'identity': managed_init.identity(data), 'status': 'sandbox_io_started', + 'sandbox_io_pid': 456}) source.write_text(json.dumps(data)) with self.assertRaises(RuntimeError): managed_init.initialize() - self.assertEqual(process.call_count, calls) + self.assertEqual(process.call_count, 1) - def test_managed_credentials_and_environment(self): + def test_starts_only_sandbox_io(self): self.exercise() def test_unknown_start_preserves_claim_and_never_replays(self): self.exercise(failed=True) - def test_existing_self_hosted_claim_prevents_managed_start(self): - with tempfile.TemporaryDirectory() as temporary, patch.object(managed_init.shared, 'ROOT', Path(temporary)): - Path(temporary, 'launch.json').write_text('{}') - with self.assertRaises(RuntimeError): - managed_init.initialize() - if __name__ == '__main__': unittest.main() diff --git a/services/core/internal/deployment/public_url.go b/services/core/internal/deployment/public_url.go index 842a52db3..f3e9e3693 100644 --- a/services/core/internal/deployment/public_url.go +++ b/services/core/internal/deployment/public_url.go @@ -59,9 +59,6 @@ func (o PublicOrigin) String() string { return o.origin } // API is the base URL of the Agents API. func (o PublicOrigin) API() string { return o.origin + "/v1" } -// RuntimeAPI is the base URL sandboxes and nodes call. -func (o PublicOrigin) RuntimeAPI() string { return o.origin + "/api/v1" } - // DaemonWebSocket is the daemon transport: ws on an http origin, wss on https. func (o PublicOrigin) DaemonWebSocket() string { return "ws" + strings.TrimPrefix(o.origin, "http") + "/api/v1/agent-daemon/ws" diff --git a/services/core/internal/deployment/rules_test.go b/services/core/internal/deployment/rules_test.go index c7a505126..896c6eeed 100644 --- a/services/core/internal/deployment/rules_test.go +++ b/services/core/internal/deployment/rules_test.go @@ -22,7 +22,7 @@ func TestPublicOrigin(t *testing.T) { } for value, socket := range map[string]string{"https://core.example": "wss://core.example/api/v1/agent-daemon/ws", "http://[::1]:8091": "ws://[::1]:8091/api/v1/agent-daemon/ws"} { origin, err := NewPublicOrigin(value) - if err != nil || origin.DaemonWebSocket() != socket || origin.API() != value+"/v1" || origin.RuntimeAPI() != value+"/api/v1" || origin.InstallerBase() != value+"/api/v1/agent-daemon/install/" { + if err != nil || origin.DaemonWebSocket() != socket || origin.API() != value+"/v1" || origin.InstallerBase() != value+"/api/v1/agent-daemon/install/" { t.Errorf("addresses derived from %q: %+v %v", value, origin, err) } } diff --git a/services/core/internal/execution/provider_operations_fixture_test.go b/services/core/internal/execution/provider_operations_fixture_test.go index 98e951c04..8b9a57161 100644 --- a/services/core/internal/execution/provider_operations_fixture_test.go +++ b/services/core/internal/execution/provider_operations_fixture_test.go @@ -10,21 +10,19 @@ import ( func (*lifecycleOnlySandbox) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } func (*lifecycleOnlySandbox) Initial(context.Context, sandbox.Reference) (sandbox.Compute, error) { @@ -48,9 +46,6 @@ func (*lifecycleOnlySandbox) KillCompute(context.Context, sandbox.Reference, san func (*lifecycleOnlySandbox) DeleteSnapshot(context.Context, sandbox.Reference, sandbox.SnapshotIdentity) error { return &providercontract.UnsupportedError{Operation: "DeleteSnapshot", Reason: "fixture_operation_not_supported"} } -func (*lifecycleOnlySandbox) RunCommandCompute(context.Context, sandbox.Reference, sandbox.Compute, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, &providercontract.UnsupportedError{Operation: "RunCommandCompute", Reason: "fixture_operation_not_supported"} -} func (*lifecycleOnlySandbox) ResumeCompute(context.Context, sandbox.Reference, sandbox.Compute) (sandbox.ComputeState, error) { return sandbox.ComputeState{}, &providercontract.UnsupportedError{Operation: "ResumeCompute", Reason: "fixture_operation_not_supported"} } diff --git a/services/core/internal/execution/runtime_lifecycle.go b/services/core/internal/execution/runtime_lifecycle.go index 1920a1edd..0d9cb602c 100644 --- a/services/core/internal/execution/runtime_lifecycle.go +++ b/services/core/internal/execution/runtime_lifecycle.go @@ -5,7 +5,6 @@ import ( "crypto/rand" "encoding/hex" "errors" - "net/url" "sync" "time" @@ -31,7 +30,6 @@ type RuntimeProvider struct { loadDeployment func(context.Context) (*RuntimeProvider, error) prepareDeployment RuntimeDeploymentPreparer ProviderKind string - CoreURL string SandboxLink string InstallationID string BackendFingerprint string @@ -76,8 +74,7 @@ func newRuntimeManager(owner Owner, deployments *deployment.Service, deploymentR } func validatedRuntimeProvider(config *RuntimeProvider, registry *runtimegateway.Registry) (RuntimeProvider, error) { - u, err := url.Parse(config.CoreURL) - if err != nil || u.Host == "" || (u.Scheme != "http" && u.Scheme != "https") || u.User != nil || u.RawQuery != "" || u.Fragment != "" || config.Provider == nil || registry == nil { + if config.Provider == nil || registry == nil { return RuntimeProvider{}, sandbox.ErrInvalid } id, err := uuid.Parse(config.InstallationID) @@ -194,9 +191,7 @@ func (r *runtimeLifecycle) provision(ctx context.Context, tenant, environment, p if err := r.lease.CheckOwnership(ctx); err != nil { return owner, err } - bootstrap := sandbox.Bootstrap{ - Reference: runtimeReference(owner), SessionID: owner.SessionID, DeviceID: owner.DeviceID, - CoreURL: r.config.CoreURL, Credential: token, NetworkAccess: placement.NetworkAccess, AllowedDomains: placement.AllowedDomains, + bootstrap := sandbox.Bootstrap{Reference: runtimeReference(owner), SandboxIO: sandboxbootstrap.Input{Version: sandboxbootstrap.Version, LinkURL: r.config.SandboxLink, Credential: serve, Resource: serveResource(owner)}, } // An invalid input creates nothing: release the allocation as settled absent. diff --git a/services/core/internal/execution/runtime_observation.go b/services/core/internal/execution/runtime_observation.go index e4d93e3d0..7d8b0953d 100644 --- a/services/core/internal/execution/runtime_observation.go +++ b/services/core/internal/execution/runtime_observation.go @@ -22,7 +22,7 @@ func (r *runtimeLifecycle) recordObservation(ctx context.Context, owner deployme if owner.CreateSettled { diagnostic = "resource_missing" } - case errors.Is(observed, sandbox.ErrComputeUnconfirmed), errors.Is(observed, sandbox.ErrCommandUnconfirmed): + case errors.Is(observed, sandbox.ErrComputeUnconfirmed): diagnostic = "compute_unconfirmed" default: diagnostic = "provider_unavailable" diff --git a/services/core/internal/execution/sandbox_deployment_drain_test.go b/services/core/internal/execution/sandbox_deployment_drain_test.go index ed9723ce0..4b1920ec3 100644 --- a/services/core/internal/execution/sandbox_deployment_drain_test.go +++ b/services/core/internal/execution/sandbox_deployment_drain_test.go @@ -93,7 +93,7 @@ func testLifecycleCancellationPreservesLease(t *testing.T, mode string) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return configuration, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -231,7 +231,7 @@ func TestSandboxDeploymentDrainFailureCannotReactivate(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return configuration, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) diff --git a/services/core/internal/execution/sandbox_deployment_setup.go b/services/core/internal/execution/sandbox_deployment_setup.go index 165198028..6ccf10692 100644 --- a/services/core/internal/execution/sandbox_deployment_setup.go +++ b/services/core/internal/execution/sandbox_deployment_setup.go @@ -122,7 +122,7 @@ func (m *runtimeManager) prepareCandidate(ctx context.Context, input sandbox.Sel return PreparedRuntimeDeployment{}, err } config := candidate.Config - if config == nil || config.InstallationID != setup.InstallationID || config.ProviderKind != setup.Provider || config.Mode != setup.Mode || config.CoreURL == "" || config.BackendFingerprint != setup.BackendFingerprint || config.loadDeployment != nil || config.prepareDeployment != nil { + if config == nil || config.InstallationID != setup.InstallationID || config.ProviderKind != setup.Provider || config.Mode != setup.Mode || config.BackendFingerprint != setup.BackendFingerprint || config.loadDeployment != nil || config.prepareDeployment != nil { return PreparedRuntimeDeployment{}, sandbox.ErrInvalid } copied, err := validatedRuntimeProvider(config, m.registry) diff --git a/services/core/internal/execution/sandbox_deployment_setup_test.go b/services/core/internal/execution/sandbox_deployment_setup_test.go index f0a0c08ea..39f107eb8 100644 --- a/services/core/internal/execution/sandbox_deployment_setup_test.go +++ b/services/core/internal/execution/sandbox_deployment_setup_test.go @@ -25,7 +25,7 @@ func TestDeferredSandboxDeploymentLoadsOnceBeforeNodeCreation(t *testing.T) { id := uuid.NewString() var selected atomic.Bool var loads atomic.Int32 - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { loads.Add(1) if !selected.Load() { @@ -58,9 +58,9 @@ func TestDeferredSandboxDeploymentLoadsOnceBeforeNodeCreation(t *testing.T) { if loads.Load() != 2 { t.Fatal("configuration loaded again after selection", loads.Load()) } - configuration.CoreURL = "https://changed.example/api/v1" + configuration.SandboxLink = "wss://changed.example/api/v1/sandbox-link" a, err := m.node("first") - if err != nil || a.lifecycle.config.CoreURL != "https://core.example/api/v1" { + if err != nil || a.lifecycle.config.SandboxLink != "wss://core.example/api/v1/sandbox-link" { t.Fatal("mutable config reached worker", err) } m.stop() @@ -95,7 +95,7 @@ func TestDeferredSandboxProviderFailureKeepsRecoveryAvailable(t *testing.T) { id := uuid.NewString() available := false loadErr := ErrExecutionUnavailable - configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + configuration := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { if !available { return nil, loadErr @@ -126,7 +126,7 @@ func TestRejectedSandboxCandidatePreservesActiveGeneration(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} rejected := errors.New("candidate provider unavailable") m, err := newRuntimeManager(Owner{Lease: heldLease{}}, unitDeploymentService(t), nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, func(context.Context, deployment.Setup) (PreparedRuntimeDeployment, error) { @@ -200,7 +200,7 @@ func TestCommittedSandboxCandidatePublishesAfterShutdown(t *testing.T) { if err := m.pauseDeployment(t.Context()); err != nil { t.Fatal(err) } - config := &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} var published *RuntimeProvider candidate := PreparedRuntimeDeployment{Config: config, Publish: func(value *RuntimeProvider) { published = value }} m.stop() diff --git a/services/core/internal/execution/sandbox_deployment_switch_test.go b/services/core/internal/execution/sandbox_deployment_switch_test.go index 767463d81..aa3fb0930 100644 --- a/services/core/internal/execution/sandbox_deployment_switch_test.go +++ b/services/core/internal/execution/sandbox_deployment_switch_test.go @@ -19,7 +19,7 @@ func TestSandboxManagerSwitchDrainsBeforeDirectActivation(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -63,7 +63,7 @@ func TestSandboxManagerSwitchDrainsBeforeDirectActivation(t *testing.T) { case <-time.After(time.Second): t.Fatal("switch drain blocked") } - config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} if err := m.activateDeployment(t.Context(), deployment.View{InstallationID: id, Generation: 2, Mode: "direct", Provider: "e2b"}); err != nil { t.Fatal(err) } @@ -101,7 +101,7 @@ func TestSandboxManagerCancelledSwitchCannotResumeBeforeDrain(t *testing.T) { hub := node.NewHub(node.HubOptions{}) defer hub.Close() id := uuid.NewString() - config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + config := &RuntimeProvider{InstallationID: id, ProviderKind: "docker", Mode: "nodes", Generation: 1, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -130,7 +130,7 @@ func TestSandboxManagerCancelledSwitchCannotResumeBeforeDrain(t *testing.T) { m.mu.Lock() originalDrain := m.switchDrained m.mu.Unlock() - config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} + config = &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: 2, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("b", 64), Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)} expected := deployment.View{InstallationID: id, Generation: 2, Mode: "direct", Provider: "e2b"} ctx, cancel = context.WithTimeout(t.Context(), 10*time.Millisecond) if err := m.activateDeployment(ctx, expected); !errors.Is(err, context.DeadlineExceeded) { @@ -159,7 +159,7 @@ func TestSandboxActivationCannotBypassOutstandingDrain(t *testing.T) { m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return nil, nil }, func(_ context.Context, setup deployment.Setup) (PreparedRuntimeDeployment, error) { - return PreparedRuntimeDeployment{Config: &RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}}, nil + return PreparedRuntimeDeployment{Config: &RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}}, nil })) if err != nil { t.Fatal(err) diff --git a/services/core/internal/execution/sandbox_generations_test.go b/services/core/internal/execution/sandbox_generations_test.go index bdfae1e0b..9b80618b8 100644 --- a/services/core/internal/execution/sandbox_generations_test.go +++ b/services/core/internal/execution/sandbox_generations_test.go @@ -39,9 +39,9 @@ func TestE2BReplacementVerifiesTwiceAndNeverPublishesFailedCommit(t *testing.T) if err != nil { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: setup.Generation, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", Generation: setup.Generation, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider}, nil }, func(ctx context.Context, setup deployment.Setup) (PreparedRuntimeDeployment, error) { - return PreparedRuntimeDeployment{Config: &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider}, + return PreparedRuntimeDeployment{Config: &RuntimeProvider{InstallationID: id, ProviderKind: "e2b", Mode: "direct", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider}, VerifyCredential: func(context.Context) error { verifyCalls++ if verifyCalls == rejectAt { diff --git a/services/core/internal/execution/sandbox_provider_contract_test.go b/services/core/internal/execution/sandbox_provider_contract_test.go index f86a7b4bd..c2c4dfe8b 100644 --- a/services/core/internal/execution/sandbox_provider_contract_test.go +++ b/services/core/internal/execution/sandbox_provider_contract_test.go @@ -18,7 +18,7 @@ func TestSandboxProviderRegistrationDoesNotRequireAnExecutionVendorBranch(t *tes t.Run(mode, func(t *testing.T) { id := uuid.NewString() config := &RuntimeProvider{InstallationID: id, ProviderKind: "contract-fixture", Mode: mode, - CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: &lifecycleOnlySandbox{}} + SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: strings.Repeat("a", 64), Provider: &lifecycleOnlySandbox{}} m, err := newRuntimeManager(Owner{Lease: heldLease{}}, nil, nil, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, NewDeferredRuntimeProvider(id, func(context.Context) (*RuntimeProvider, error) { return config, nil }, unusedPreparation(t))) if err != nil { t.Fatal(err) @@ -50,6 +50,3 @@ func (*lifecycleOnlySandbox) Renew(context.Context, sandbox.Reference) (sandbox. func (*lifecycleOnlySandbox) Kill(context.Context, sandbox.Reference) error { panic("registration killed compute") } -func (*lifecycleOnlySandbox) RunCommand(context.Context, sandbox.Reference, sandbox.Command) (sandbox.CommandResult, error) { - panic("registration executed command") -} diff --git a/services/core/internal/execution/sandbox_reset_test.go b/services/core/internal/execution/sandbox_reset_test.go index 7deeee337..c2113fea1 100644 --- a/services/core/internal/execution/sandbox_reset_test.go +++ b/services/core/internal/execution/sandbox_reset_test.go @@ -89,7 +89,7 @@ func TestSandboxResetPageTimeoutRecoversCommittedOwner(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil }, unusedPreparation(t)) m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, config) if err != nil { @@ -203,7 +203,7 @@ func TestSandboxResetPublishesCommittedGenerationWithoutReading(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil }, unusedPreparation(t)) var published []uint64 config.PublishUnconfigured = func(generation uint64) { diff --git a/services/core/internal/execution/sandbox_snapshot_budget_test.go b/services/core/internal/execution/sandbox_snapshot_budget_test.go index 065622a0d..b0f0d71f3 100644 --- a/services/core/internal/execution/sandbox_snapshot_budget_test.go +++ b/services/core/internal/execution/sandbox_snapshot_budget_test.go @@ -73,7 +73,7 @@ func TestSandboxResetSnapshotFitsPageBudget(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil + return &RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: hub.Proxy(uuid.NewString(), docker.Operations(), 1)}, nil }, unusedPreparation(t)) m, err := newRuntimeManager(owner, deployments, reader, nil, runtimegateway.NewRegistry(), relay.New(nil), nil, configuration) if err != nil { diff --git a/services/core/internal/sandbox/contracttest/provider.go b/services/core/internal/sandbox/contracttest/provider.go index e94eb63ec..d2a77b334 100644 --- a/services/core/internal/sandbox/contracttest/provider.go +++ b/services/core/internal/sandbox/contracttest/provider.go @@ -9,8 +9,6 @@ import ( "testing" "time" - "github.com/google/uuid" - "github.com/MiniMax-AI/OpenAgentCore/internal/sandboxbootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" @@ -42,7 +40,7 @@ type Fixture struct { // Bootstrap returns a valid Create input for r. func Bootstrap(r sandbox.Reference) sandbox.Bootstrap { - return sandbox.Bootstrap{Reference: r, SessionID: uuid.NewString(), DeviceID: uuid.NewString(), CoreURL: "https://core.example/api/v1", Credential: "synthetic", NetworkAccess: "enabled", + return sandbox.Bootstrap{Reference: r, SandboxIO: sandboxbootstrap.Input{Version: sandboxbootstrap.Version, LinkURL: "wss://core.example/api/v1/sandbox-link", Credential: "synthetic-serve", Resource: sandboxbootstrap.Resource{TenantID: r.TenantID, EnvironmentID: r.EnvironmentID, Kind: "allocation", ID: r.AllocationID, Generation: 1}}} } diff --git a/services/core/internal/sandbox/docker/bootstrap.go b/services/core/internal/sandbox/docker/bootstrap.go index eb2cfde95..ec2d8d9af 100644 --- a/services/core/internal/sandbox/docker/bootstrap.go +++ b/services/core/internal/sandbox/docker/bootstrap.go @@ -17,24 +17,17 @@ type entry struct { } func (p *Provider) bootstrap(ctx context.Context, id string, b sandbox.Bootstrap) error { - // Deliver the daemon's and the Sandbox I/O service's launch inputs before - // native work can start. - auth, e := b.RuntimeConnection().Marshal() - if e != nil { - return e - } + // Deliver the Sandbox I/O service's launch input before it can start. serve, e := b.SandboxIO.Marshal() if e != nil { return e } if e = copyRuntimeFiles(ctx, p.client, id, "/home", []entry{ - {name: "runtime", directory: true}, {name: "runtime/.oac", directory: true}, - {name: "runtime/runtime-bootstrap.json", content: auth}, - {name: "runtime/sandbox-io-bootstrap.json", content: serve}, + {name: "runtime", directory: true}, {name: "runtime/sandbox-io-bootstrap.json", content: serve}, }); e != nil { return e } - return copyRuntimeFiles(ctx, p.client, id, "/environment", []entry{{name: "workspace", directory: true}, {name: "staging", directory: true}, {name: "initialization", directory: true}, {name: "packages", directory: true}}) + return copyRuntimeFiles(ctx, p.client, id, "/environment", []entry{{name: "workspace", directory: true}, {name: "initialization", directory: true}, {name: "packages", directory: true}}) } func copyRuntimeFiles(ctx context.Context, c *client.Client, id, path string, entries []entry) error { var content bytes.Buffer diff --git a/services/core/internal/sandbox/docker/bootstrap_test.go b/services/core/internal/sandbox/docker/bootstrap_test.go index 56744ed08..3faa30bcc 100644 --- a/services/core/internal/sandbox/docker/bootstrap_test.go +++ b/services/core/internal/sandbox/docker/bootstrap_test.go @@ -8,7 +8,6 @@ import ( "strings" "testing" - "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" "github.com/MiniMax-AI/OpenAgentCore/internal/sandboxbootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/contracttest" @@ -16,9 +15,9 @@ import ( "github.com/moby/moby/client" ) -func TestBootstrapDeliversOnlyPublicLaunchInputs(t *testing.T) { +func TestBootstrapDeliversOnlySandboxIOInput(t *testing.T) { b := contracttest.Bootstrap(sandbox.Reference{TenantID: uuid.NewString(), EnvironmentID: uuid.NewString(), AllocationID: uuid.NewString()}) - found := map[string]bool{} + var files []string server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if r.Method != "PUT" || !strings.HasSuffix(r.URL.Path, "/containers/test/archive") { t.Errorf("unexpected Docker operation %s", r.URL.Path) @@ -33,13 +32,10 @@ func TestBootstrapDeliversOnlyPublicLaunchInputs(t *testing.T) { t.Error(err) break } - if strings.Contains(h.Name, "auth.json") { - t.Error("provider wrote Runtime private storage") - } - if h.Name != "runtime/runtime-bootstrap.json" && h.Name != "runtime/sandbox-io-bootstrap.json" { + if h.Typeflag != tar.TypeReg { continue } - found[h.Name] = true + files = append(files, h.Name) raw, err := io.ReadAll(tr) if err != nil { t.Error(err) @@ -47,11 +43,7 @@ func TestBootstrapDeliversOnlyPublicLaunchInputs(t *testing.T) { if h.Mode != 0600 || h.Uid != 1000 || h.Gid != 1000 { t.Error("launch input permissions", h.Name) } - if h.Name == "runtime/runtime-bootstrap.json" { - if c, err := runtimebootstrap.Decode(raw); err != nil || c != b.RuntimeConnection() { - t.Error("invalid Runtime launch input") - } - } else if in, err := sandboxbootstrap.Decode(raw); err != nil || in != b.SandboxIO { + if in, err := sandboxbootstrap.Decode(raw); err != nil || in != b.SandboxIO { t.Error("invalid Sandbox I/O launch input") } } @@ -66,7 +58,7 @@ func TestBootstrapDeliversOnlyPublicLaunchInputs(t *testing.T) { if err := (&Provider{client: c}).bootstrap(t.Context(), "test", b); err != nil { t.Fatal(err) } - if len(found) != 2 { - t.Fatal("missing launch input", found) + if len(files) != 1 || files[0] != "runtime/sandbox-io-bootstrap.json" { + t.Fatal("bootstrap files", files) } } diff --git a/services/core/internal/sandbox/docker/command.go b/services/core/internal/sandbox/docker/command.go deleted file mode 100644 index 546109160..000000000 --- a/services/core/internal/sandbox/docker/command.go +++ /dev/null @@ -1,91 +0,0 @@ -package docker - -import ( - "bytes" - "context" - "errors" - "io" - "path" - - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" - "github.com/moby/moby/api/pkg/stdcopy" - "github.com/moby/moby/client" -) - -// RunCommand is for trusted initialization only. Closing an exec attachment does -// not kill its process. On any uncertain outcome, the caller must reclaim the -// allocation with Kill instead of admitting native work or replaying the command. -func (p *Provider) RunCommand(ctx context.Context, r sandbox.Reference, command sandbox.Command) (sandbox.CommandResult, error) { - var result sandbox.CommandResult - if len(command.Args) == 0 || len(command.Stdin) > sandbox.MaxCommandInputBytes || (command.Directory != "" && !path.IsAbs(command.Directory)) { - return result, sandbox.ErrInvalid - } - c, e := p.inspect(ctx, r) - if e != nil { - return result, e - } - if _, ok := ctx.Deadline(); !ok { - return result, sandbox.ErrInvalid - } - exec, e := p.client.ExecCreate(ctx, c.Container.ID, client.ExecCreateOptions{User: "1000:1000", Cmd: command.Args, WorkingDir: command.Directory, AttachStdin: command.Stdin != nil, AttachStdout: true, AttachStderr: true}) - if e != nil { - return result, e - } - attached, e := p.client.ExecAttach(ctx, exec.ID, client.ExecAttachOptions{}) - if e != nil { - return result, errors.Join(sandbox.ErrCommandUnconfirmed, e) - } - defer attached.Close() - written := make(chan error, 1) - if command.Stdin != nil { - go func() { - _, err := io.Copy(attached.Conn, bytes.NewReader(command.Stdin)) - if err == nil { - err = attached.CloseWrite() - } - written <- err - }() - } else { - written <- nil - } - stdout, stderr := &boundedBuffer{}, &boundedBuffer{} - done := make(chan error, 1) - go func() { _, e := stdcopy.StdCopy(stdout, stderr, attached.Reader); done <- e }() - select { - case e = <-done: - case <-ctx.Done(): - attached.Close() - <-done - e = ctx.Err() - } - if e != nil { - attached.Close() - <-written - return result, errors.Join(sandbox.ErrCommandUnconfirmed, e) - } - select { - case e = <-written: - case <-ctx.Done(): - attached.Close() - <-written - e = ctx.Err() - } - if e != nil { - return result, errors.Join(sandbox.ErrCommandUnconfirmed, e) - } - status, e := p.client.ExecInspect(ctx, exec.ID, client.ExecInspectOptions{}) - if e != nil || status.Running { - return result, errors.Join(sandbox.ErrCommandUnconfirmed, e) - } - return sandbox.CommandResult{Stdout: stdout.String(), Stderr: stderr.String(), ExitCode: status.ExitCode}, nil -} - -type boundedBuffer struct{ bytes.Buffer } - -func (b *boundedBuffer) Write(data []byte) (int, error) { - const max = 1024 * 1024 - if b.Len()+len(data) > max { - return 0, errors.New("initialization output exceeds 1 MiB") - } - return b.Buffer.Write(data) -} diff --git a/services/core/internal/sandbox/docker/container_options.go b/services/core/internal/sandbox/docker/container_options.go index f8bee5840..7a7a23327 100644 --- a/services/core/internal/sandbox/docker/container_options.go +++ b/services/core/internal/sandbox/docker/container_options.go @@ -6,36 +6,24 @@ import ( "github.com/moby/moby/client" ) -// runtimeContainerOptions is shared by managed and user-owned V1 Runtime launch. -// Keep isolation and volume layout identical; only bootstrap authority differs. -func runtimeContainerOptions(config Config, name string, labels map[string]string, environment []string) client.ContainerCreateOptions { +// runtimeContainerOptions is the allocation's container: the Sandbox I/O +// service as its only process, reading the input that bootstrap writes. +func runtimeContainerOptions(config Config, name string, labels map[string]string) client.ContainerCreateOptions { limit := int64(128) memory, cpus := int64(2*1024*1024*1024), int64(2*1000000000) if config.Resources != nil { memory = int64(config.Resources.MemoryMiB) * 1024 * 1024 cpus = int64(config.Resources.CPUs) * 1000000000 } - // A nested native sandbox must mount its own procfs. Keep sysfs secrets - // masked; only this qualified image profile opts out of Docker's proc masks. - var masked, readonly []string - var init *bool - if config.NestedSandbox { - masked = []string{"/sys/firmware", "/sys/devices/virtual/powercap"} - readonly = []string{} - enabled := true - init = &enabled - } return client.ContainerCreateOptions{Name: name, Image: config.Image, - Config: &container.Config{User: "1000:1000", WorkingDir: "/environment/workspace", Labels: labels, Env: environment}, + Config: &container.Config{User: "1000:1000", WorkingDir: "/environment/workspace", Labels: labels, + Entrypoint: []string{"/usr/local/bin/oac-sandbox-io", "--bootstrap-file", "/home/runtime/sandbox-io-bootstrap.json"}}, HostConfig: &container.HostConfig{ReadonlyRootfs: true, CapDrop: []string{"ALL"}, SecurityOpt: []string{"no-new-privileges", "seccomp=" + config.Seccomp, "apparmor=unconfined"}, NetworkMode: container.NetworkMode(config.Network), - MaskedPaths: masked, ReadonlyPaths: readonly, Init: init, Resources: container.Resources{PidsLimit: &limit, Memory: memory, NanoCPUs: cpus}, Tmpfs: map[string]string{"/tmp": "rw,nosuid,nodev,size=128m"}, Mounts: []mount.Mount{ {Type: mount.TypeVolume, Source: name + "-home", Target: "/home"}, {Type: mount.TypeVolume, Source: name + "-environment", Target: "/environment"}, - // The native sandbox mounts canonical roots, omitting symlink aliases. - // Expose the same workspace at its public path; trusted atomic staging - // remains entirely on the original /environment mount. + // The workspace is also at its public path. {Type: mount.TypeVolume, Source: name + "-environment", Target: "/workspace", VolumeOptions: &mount.VolumeOptions{Subpath: "workspace", NoCopy: true}}, }, }, diff --git a/services/core/internal/sandbox/docker/deployment_test.go b/services/core/internal/sandbox/docker/deployment_test.go index 9df0727d2..710d75065 100644 --- a/services/core/internal/sandbox/docker/deployment_test.go +++ b/services/core/internal/sandbox/docker/deployment_test.go @@ -87,7 +87,7 @@ func TestManagedDriftRejectsBeforeBootstrapAndRetainsCleanup(t *testing.T) { t.Fatal(err) } resources.CPUs = 9 - b := sandbox.Bootstrap{Reference: sandbox.Reference{TenantID: uuid.NewString(), EnvironmentID: uuid.NewString(), AllocationID: uuid.NewString()}, SessionID: uuid.NewString(), DeviceID: uuid.NewString(), CoreURL: "https://core.example/api/v1", Credential: "secret", NetworkAccess: "enabled"} + b := sandbox.Bootstrap{Reference: sandbox.Reference{TenantID: uuid.NewString(), EnvironmentID: uuid.NewString(), AllocationID: uuid.NewString()}} info, err := p.Create(t.Context(), b) if !errors.Is(err, sandbox.ErrInvalid) { t.Fatal("drift accepted", err) @@ -109,7 +109,7 @@ func TestManagedDriftRejectsBeforeBootstrapAndRetainsCleanup(t *testing.T) { } func TestNilManagedLimitsKeepCallerManagedDefaults(t *testing.T) { - options := runtimeContainerOptions(Config{}, "fixture", nil, nil) + options := runtimeContainerOptions(Config{}, "fixture", nil) if options.HostConfig.NanoCPUs != 2e9 || options.HostConfig.Memory != 2*1024*1024*1024 { t.Fatal("caller-managed defaults changed") } diff --git a/services/core/internal/sandbox/docker/node.go b/services/core/internal/sandbox/docker/node.go index 115d6bf0c..1b51ef530 100644 --- a/services/core/internal/sandbox/docker/node.go +++ b/services/core/internal/sandbox/docker/node.go @@ -20,16 +20,15 @@ import ( // (image_manifest_digest) names the loaded image, so the installer records the // one this host resolves. type Native struct { - Host string `json:"host"` - Image string `json:"image"` - Network string `json:"network"` - SeccompFile string `json:"seccomp_file"` - NestedSandbox bool `json:"nested_sandbox"` + Host string `json:"host"` + Image string `json:"image"` + Network string `json:"network"` + SeccompFile string `json:"seccomp_file"` } func decodeNative(config sandbox.NodeConfig) (Native, error) { var entry Native - if sandbox.DecodeConfigurationObject(config.Native, &entry, "host", "image", "network", "seccomp_file", "nested_sandbox") != nil { + if sandbox.DecodeConfigurationObject(config.Native, &entry, "host", "image", "network", "seccomp_file") != nil { return entry, errors.New("invalid managed Docker node configuration") } release := config.Specification.Runtime @@ -62,7 +61,7 @@ func BuildNode(config sandbox.NodeConfig, _ sandbox.LocalOptions, result *sandbo return closeProvider, errors.New("invalid managed Docker endpoint") } closeProvider = func() { _ = c.Close() } - provider, err := New(c, Config{InstallationID: config.InstallationID, Image: entry.Image, Network: entry.Network, Seccomp: string(seccomp), NestedSandbox: entry.NestedSandbox, Resources: &config.Specification.Resources}) + provider, err := New(c, Config{InstallationID: config.InstallationID, Image: entry.Image, Network: entry.Network, Seccomp: string(seccomp), Resources: &config.Specification.Resources}) if err != nil { closeProvider() return func() {}, errors.New("invalid managed Docker provider configuration") diff --git a/services/core/internal/sandbox/docker/operations.go b/services/core/internal/sandbox/docker/operations.go index 2722da0dd..998470a49 100644 --- a/services/core/internal/sandbox/docker/operations.go +++ b/services/core/internal/sandbox/docker/operations.go @@ -9,21 +9,19 @@ import ( // Operations is this adapter's complete authored resource contract. func Operations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "Suspend": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "Resume": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, - "Observe": {State: providercontract.Supported}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "Suspend": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "Resume": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "docker_does_not_support_checkpoints"}, + "Observe": {State: providercontract.Supported}, } } func (*Provider) ProviderOperations() providercontract.Operations { return Operations() } @@ -48,9 +46,6 @@ func (p *Provider) KillCompute(context.Context, sandbox.Reference, sandbox.Compu func (p *Provider) DeleteSnapshot(context.Context, sandbox.Reference, sandbox.SnapshotIdentity) error { return &providercontract.UnsupportedError{Operation: "DeleteSnapshot", Reason: Operations()["DeleteSnapshot"].Reason} } -func (p *Provider) RunCommandCompute(context.Context, sandbox.Reference, sandbox.Compute, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, &providercontract.UnsupportedError{Operation: "RunCommandCompute", Reason: Operations()["RunCommandCompute"].Reason} -} func (p *Provider) ResumeCompute(context.Context, sandbox.Reference, sandbox.Compute) (sandbox.ComputeState, error) { return sandbox.ComputeState{}, &providercontract.UnsupportedError{Operation: "ResumeCompute", Reason: Operations()["ResumeCompute"].Reason} } diff --git a/services/core/internal/sandbox/docker/provider.go b/services/core/internal/sandbox/docker/provider.go index f48799ab8..56feb96e4 100644 --- a/services/core/internal/sandbox/docker/provider.go +++ b/services/core/internal/sandbox/docker/provider.go @@ -1,16 +1,15 @@ -// Package docker is a thin SDK adapter for the dedicated, colocated Runtime. +// Package docker is a thin SDK adapter that runs each allocation's sandbox as +// a Docker container whose only process is the Sandbox I/O service. package docker import ( "context" "crypto/sha256" "encoding/hex" - "encoding/json" "errors" "fmt" "strings" - "github.com/MiniMax-AI/OpenAgentCore/internal/agentnetwork" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/containerd/errdefs" "github.com/google/uuid" @@ -20,12 +19,11 @@ import ( const labelPrefix = "io.oac." // Config is trusted operator configuration, never public Session input. The -// immutable image contains the qualified native profile and all Runtime binaries. -// Seccomp is JSON content, not a path on the Docker host. Network must provide -// trusted daemon/model connectivity; native tool network policy is in the image. +// immutable image contains oac-sandbox-io and the tools sandbox processes run. +// Seccomp is JSON content, not a path on the Docker host. Network must reach +// Core's Sandbox link. type Config struct { InstallationID, Image, Network, Seccomp string - NestedSandbox bool Resources *sandbox.Resources } type Provider struct { @@ -111,16 +109,11 @@ func (p *Provider) Renew(ctx context.Context, r sandbox.Reference) (sandbox.Info func (p *Provider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Info, error) { info := sandbox.Info{Reference: b.Reference} - policy := agentnetwork.Policy{Access: b.NetworkAccess, AllowedDomains: b.AllowedDomains} if existing, e := p.GetInfo(ctx, b.Reference); e == nil { return existing, sandbox.ErrExists } else if !errors.Is(e, sandbox.ErrNotFound) { return info, e } - domains, err := json.Marshal(policy.Hosts()) - if err != nil { - return info, sandbox.ErrInvalid - } name := p.name(b.Reference) // Retained volumes without a container are partial or lost state, not an // invitation to overwrite native history with a new bootstrap identity. @@ -145,12 +138,10 @@ func (p *Provider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Inf return info, sandbox.ErrOwnership } } - options := runtimeContainerOptions(p.config, name, p.labels(b.Reference), []string{"OAC_RUNTIME_ENVIRONMENT_ID=" + b.EnvironmentID, "OAC_RUNTIME_SESSION_ID=" + b.SessionID, "OAC_RUNTIME_NETWORK_ACCESS=" + policy.Access, "OAC_RUNTIME_ALLOWED_DOMAINS=" + string(domains)}) - // The container's own command starts both processes from the files + // The container's command is the Sandbox I/O service, reading the file // bootstrap writes before start, so ContainerStart is the last mutating - // step and a running container has started the Sandbox I/O service. - options.Config.Entrypoint = []string{"/bin/sh", "-c", launch} - v, e := p.client.ContainerCreate(ctx, options) + // step and a running container has started the service. + v, e := p.client.ContainerCreate(ctx, runtimeContainerOptions(p.config, name, p.labels(b.Reference))) if errdefs.IsConflict(e) { return info, sandbox.ErrExists } @@ -174,7 +165,7 @@ func (p *Provider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Inf // Any failure returns the retained allocation reference. The caller must Kill // it, including on a lost acknowledgement. Never erase uncertain owner state. if e = p.bootstrap(ctx, v.ID, b); e != nil { - return info, fmt.Errorf("runtime bootstrap: %w", e) + return info, fmt.Errorf("sandbox bootstrap: %w", e) } if _, e = p.client.ContainerStart(ctx, v.ID, client.ContainerStartOptions{}); e != nil { return info, e @@ -182,11 +173,6 @@ func (p *Provider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Inf return p.GetInfo(ctx, b.Reference) } -// launch starts the Sandbox I/O service in the background, then replaces the -// shell with the daemon, which stays the container's main process. -const launch = "/usr/local/bin/oac-sandbox-io --bootstrap-file /home/runtime/sandbox-io-bootstrap.json & " + - "exec /usr/local/bin/oac-daemon connect --profile default --bootstrap-file /home/runtime/runtime-bootstrap.json" - // Kill is idempotent only for absence, not for errors or foreign ownership. It // checks all resources before removing any and confirms removal of named volumes. func (p *Provider) Kill(ctx context.Context, r sandbox.Reference) error { diff --git a/services/core/internal/sandbox/docker/provider_test.go b/services/core/internal/sandbox/docker/provider_test.go index bdfac0e94..5d87d1acc 100644 --- a/services/core/internal/sandbox/docker/provider_test.go +++ b/services/core/internal/sandbox/docker/provider_test.go @@ -3,21 +3,20 @@ package docker import ( "bytes" "context" - "crypto/sha256" - "encoding/hex" "errors" + "io" "os" "strings" "testing" "time" - "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" "github.com/MiniMax-AI/OpenAgentCore/internal/sandboxbootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/contracttest" "github.com/containerd/errdefs" "github.com/google/uuid" + "github.com/moby/moby/api/pkg/stdcopy" "github.com/moby/moby/client" ) @@ -37,9 +36,10 @@ func TestProviderRejectsUnsafeOperatorConfiguration(t *testing.T) { } } -// This optional Docker mechanism test uses a pinned fixture image whose -// oac-daemon only sleeps. It is not native/model acceptance; the real Runtime -// has separate checks. +// This optional Docker mechanism test runs AGENTS_RUNTIME_DOCKER_TEST_IMAGE, +// an image with oac-sandbox-io, such as the sandbox image. Its Link is +// unreachable, so the service keeps retrying and the container keeps running. +// It is not native or model acceptance. func TestDockerProviderLifecycle(t *testing.T) { image := os.Getenv("AGENTS_RUNTIME_DOCKER_TEST_IMAGE") if image == "" { @@ -63,11 +63,10 @@ func TestDockerProviderLifecycle(t *testing.T) { defer cancel() bootstrap := func() sandbox.Bootstrap { b := contracttest.Bootstrap(sandbox.Reference{TenantID: uuid.NewString(), EnvironmentID: uuid.NewString(), AllocationID: uuid.NewString()}) - b.Credential, b.SandboxIO.Credential = "synthetic-test-credential", "synthetic-serve-credential" + b.SandboxIO.Credential = "synthetic-serve-credential" return b } b := bootstrap() - b.NetworkAccess, b.AllowedDomains = "restricted", []string{"Example.com", "api.example.com"} t.Cleanup(func() { ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) defer cancel() @@ -75,31 +74,11 @@ func TestDockerProviderLifecycle(t *testing.T) { t.Error(e) } }) - t.Run("stdin concurrent output and EOF", func(t *testing.T) { - inputOwner := bootstrap() - if _, err := p.Create(ctx, inputOwner); err != nil { - t.Fatal(err) - } - defer func() { - cleanup, stop := context.WithTimeout(context.Background(), 20*time.Second) - defer stop() - if err := p.Kill(cleanup, inputOwner.Reference); err != nil { - t.Error(err) - } - }() - for _, data := range [][]byte{{}, bytes.Repeat([]byte{0, 255, 10, 1, 42}, 900000)} { - result, err := p.RunCommand(ctx, inputOwner.Reference, sandbox.Command{Args: []string{"/bin/sh", "-c", "head -c 131072 /dev/zero; sha256sum"}, Stdin: data}) - digest := sha256.Sum256(data) - if err != nil || result.ExitCode != 0 || !strings.HasSuffix(result.Stdout, hex.EncodeToString(digest[:])+" -\n") || len(result.Stdout) != 131072+68 { - t.Fatalf("stdin/EOF failure: input=%d stdout=%d exit=%d error=%v", len(data), len(result.Stdout), result.ExitCode, err) - } - } - }) info, e := p.Create(ctx, b) contracttest.AssertObservation(t, info, e, b.Reference, "", "running") resources, e := p.Observe(ctx, runtimeobs.Target{ - TenantID: b.TenantID, SessionID: b.SessionID, EnvironmentID: b.EnvironmentID, Mode: runtimeobs.ModeManaged, - Instance: runtimeobs.Instance{AllocationID: b.AllocationID, ProviderKey: installationID, DeviceID: b.DeviceID}, + TenantID: b.TenantID, EnvironmentID: b.EnvironmentID, Mode: runtimeobs.ModeManaged, + Instance: runtimeobs.Instance{AllocationID: b.AllocationID, ProviderKey: installationID}, }) if e != nil || resources.StartedAt == nil || resources.CPUUsageSecondsTotal == nil || resources.MemoryUsageBytes == nil || resources.CPUCapacityCores == nil || resources.MemoryLimitBytes == nil { t.Fatalf("bad resource observation: %+v %v", resources, e) @@ -108,42 +87,27 @@ func TestDockerProviderLifecycle(t *testing.T) { if e != nil { t.Fatal(e) } - if strings.Contains(string(inspected.Raw), b.Credential) || strings.Contains(string(inspected.Raw), b.SandboxIO.Credential) || inspected.Container.Config.User != "1000:1000" || !inspected.Container.HostConfig.ReadonlyRootfs || inspected.Container.HostConfig.Privileged { + if strings.Contains(string(inspected.Raw), b.SandboxIO.Credential) || inspected.Container.Config.User != "1000:1000" || !inspected.Container.HostConfig.ReadonlyRootfs || inspected.Container.HostConfig.Privileged { t.Fatal("unsafe Docker configuration") } - for _, value := range []string{"OAC_RUNTIME_NETWORK_ACCESS=restricted", `OAC_RUNTIME_ALLOWED_DOMAINS=["api.example.com","example.com"]`} { - found := false - for _, entry := range inspected.Container.Config.Env { - found = found || entry == value - } - if !found { - t.Fatalf("bootstrap lost network policy: %s", value) - } + if got := strings.Join(inspected.Container.Config.Entrypoint, " ") + "|" + strings.Join(inspected.Container.Config.Cmd, " "); got != "/usr/local/bin/oac-sandbox-io --bootstrap-file /home/runtime/sandbox-io-bootstrap.json|" { + t.Fatalf("container command %q", got) } changed := b - changed.Credential = "must-not-replace-existing" + changed.SandboxIO.Credential = "must-not-replace-existing" if _, e = p.Create(ctx, changed); !errors.Is(e, sandbox.ErrExists) { t.Fatalf("duplicate not rejected: %v", e) } - r, e := p.RunCommand(ctx, b.Reference, sandbox.Command{Args: []string{"cat", "/home/runtime/runtime-bootstrap.json"}}) - if e != nil { - t.Fatal(e) - } - auth, decodeErr := runtimebootstrap.Decode([]byte(r.Stdout)) - if decodeErr != nil || auth.Credential != b.Credential || auth.DeviceID != b.DeviceID { - t.Fatal("bootstrap changed or malformed") + run := func(script string) (string, int) { + t.Helper() + return execInContainer(t, ctx, c, info.ProviderID, script) } - r, e = p.RunCommand(ctx, b.Reference, sandbox.Command{Args: []string{"cat", "/home/runtime/sandbox-io-bootstrap.json"}}) - if serve, decodeErr := sandboxbootstrap.Decode([]byte(r.Stdout)); e != nil || decodeErr != nil || serve != b.SandboxIO { - t.Fatal("Sandbox I/O bootstrap changed or malformed", e) + out, code := run("cat /home/runtime/sandbox-io-bootstrap.json") + if serve, err := sandboxbootstrap.Decode([]byte(out)); code != 0 || err != nil || serve != b.SandboxIO { + t.Fatal("Sandbox I/O bootstrap changed or malformed") } - r, e = p.RunCommand(ctx, b.Reference, sandbox.Command{Args: []string{"sh", "-c", "printf retained > /environment/workspace/history; printf failed >&2; exit 7"}}) - if e != nil || r.ExitCode != 7 || r.Stderr != "failed" { - t.Fatalf("lost command status: %+v %v", r, e) - } - r, e = p.RunCommand(ctx, b.Reference, sandbox.Command{Args: []string{"sh", "-c", "set -eu; test \"$(cat /workspace/history)\" = retained; printf replaced > /environment/staging/replacement; mv /environment/staging/replacement /environment/workspace/history; cat /workspace/history"}}) - if e != nil || r.ExitCode != 0 || r.Stdout != "replaced" { - t.Fatal("public workspace view or atomic staging failed", e) + if out, code = run("ls -A /home/runtime; printf retained > /environment/workspace/history; cat /workspace/history"); code != 0 || out != "sandbox-io-bootstrap.json\nretained" { + t.Fatalf("home or public workspace view: %q %d", out, code) } wrong := b.Reference wrong.TenantID = uuid.NewString() @@ -160,12 +124,10 @@ func TestDockerProviderLifecycle(t *testing.T) { if _, e = c.ContainerRestart(ctx, info.ProviderID, client.ContainerRestartOptions{Timeout: &timeout}); e != nil { t.Fatal(e) } - r, e = p.RunCommand(ctx, b.Reference, sandbox.Command{Args: []string{"cat", "/environment/workspace/history"}}) - if e != nil || r.Stdout != "replaced" { + if out, code = run("cat /environment/workspace/history"); code != 0 || out != "retained" { t.Fatal("restart lost workspace") } - r, e = p.RunCommand(ctx, b.Reference, sandbox.Command{Args: []string{"sh", "-c", "touch /cannot-write-root"}}) - if e != nil || r.ExitCode == 0 { + if _, code = run("touch /cannot-write-root"); code == 0 { t.Fatal("root filesystem writable") } // Container loss must not trigger credential overwrite or state replacement. @@ -202,20 +164,28 @@ func TestDockerProviderLifecycle(t *testing.T) { if _, e = p.Create(ctx, foreign); !errors.Is(e, sandbox.ErrOwnership) { t.Fatal("foreign volume bootstrap accepted") } - // Closing initialization output is not process termination. Require explicit - // reclamation, without returning partial output as a successful command. - next := bootstrap() - defer p.Kill(context.Background(), next.Reference) - if _, e = p.Create(ctx, next); e != nil { - t.Fatal(e) - } - short, stop := context.WithTimeout(ctx, 100*time.Millisecond) - _, e = p.RunCommand(short, next.Reference, sandbox.Command{Args: []string{"sleep", "30"}}) - stop() - if !errors.Is(e, sandbox.ErrCommandUnconfirmed) { - t.Fatalf("timeout classified as certain: %v", e) - } - if e = p.Kill(ctx, next.Reference); e != nil { - t.Fatal(e) - } +} + +// execInContainer runs a shell script in the container as its user and +// returns its standard output and exit code. +func execInContainer(t *testing.T, ctx context.Context, c *client.Client, id, script string) (string, int) { + t.Helper() + created, err := c.ExecCreate(ctx, id, client.ExecCreateOptions{Cmd: []string{"/bin/sh", "-c", script}, AttachStdout: true, AttachStderr: true}) + if err != nil { + t.Fatal(err) + } + attached, err := c.ExecAttach(ctx, created.ID, client.ExecAttachOptions{}) + if err != nil { + t.Fatal(err) + } + defer attached.Close() + var stdout bytes.Buffer + if _, err = stdcopy.StdCopy(&stdout, io.Discard, attached.Reader); err != nil { + t.Fatal(err) + } + status, err := c.ExecInspect(ctx, created.ID, client.ExecInspectOptions{}) + if err != nil || status.Running { + t.Fatal("exec status unknown", err) + } + return stdout.String(), status.ExitCode } diff --git a/services/core/internal/sandbox/e2b/credential.go b/services/core/internal/sandbox/e2b/credential.go index 42452ff62..c560a98c9 100644 --- a/services/core/internal/sandbox/e2b/credential.go +++ b/services/core/internal/sandbox/e2b/credential.go @@ -35,7 +35,7 @@ func (ConfigurationAdapter) VerifyCredential(ctx context.Context, c sandbox.Dire case "team_mismatch", "invalid": return sandbox.ErrCredentialOwnership case "": - if out.DeploymentValid && out.Info == nil && out.Command == nil && out.Observation == nil && out.TemplateBuild == nil { + if out.DeploymentValid && out.Info == nil && out.Observation == nil && out.TemplateBuild == nil { return nil } } diff --git a/services/core/internal/sandbox/e2b/helper_contract.go b/services/core/internal/sandbox/e2b/helper_contract.go index 7d701cf51..1f477abf4 100644 --- a/services/core/internal/sandbox/e2b/helper_contract.go +++ b/services/core/internal/sandbox/e2b/helper_contract.go @@ -4,7 +4,6 @@ import ( "slices" "time" - "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) @@ -13,19 +12,18 @@ import ( // This adapter-private boundary is documented in tools/e2b-provider/README.md. const ProtocolVersion = 1 const MaxOutputBytes = 1024 * 1024 -const MaxRequestBytes = 72 * 1024 * 1024 +const MaxRequestBytes = 1024 * 1024 const MaxResponseBytes = 16 * 1024 * 1024 const MaxCredentialReferences = 32 -const MaxCommandInputBytes = sandbox.MaxCommandInputBytes // HelperOperations declares the complete set of one-shot helper operations. func HelperOperations() []string { - return []string{"create", "inspect", "renew", "kill", "command", "validate_deployment", "observe", "list_templates", "list_builds", "verify_credential"} + return []string{"create", "inspect", "renew", "kill", "validate_deployment", "observe", "list_templates", "list_builds", "verify_credential"} } // HelperErrors are sanitized wire outcomes; an empty code denotes success. func HelperErrors() []string { - return []string{"", "invalid", "ownership", "exists", "not_found", "command_unconfirmed", "unconfirmed", "template_invalid", "team_mismatch", "unauthorized"} + return []string{"", "invalid", "ownership", "exists", "not_found", "unconfirmed", "template_invalid", "team_mismatch", "unauthorized"} } // Validate checks the operation envelope before the helper can start. Native @@ -62,16 +60,13 @@ type Request struct { Config Config Reference sandbox.Reference // References lists the allocations of one verify_credential request. - References []sandbox.Reference `json:",omitempty"` - Bootstrap *sandbox.Bootstrap `json:",omitempty"` - RuntimeBootstrap *runtimebootstrap.Connection `json:",omitempty"` - Command *sandbox.Command `json:",omitempty"` - Deadline time.Time + References []sandbox.Reference `json:",omitempty"` + Bootstrap *sandbox.Bootstrap `json:",omitempty"` + Deadline time.Time } type Response struct { Version int - Info *sandbox.Info `json:",omitempty"` - Command *sandbox.CommandResult `json:",omitempty"` + Info *sandbox.Info `json:",omitempty"` ErrorCode string DeploymentValid bool `json:",omitempty"` TemplateBuild *TemplateBuild `json:",omitempty"` diff --git a/services/core/internal/sandbox/e2b/helper_contract_test.go b/services/core/internal/sandbox/e2b/helper_contract_test.go index 0e13b0119..672f59433 100644 --- a/services/core/internal/sandbox/e2b/helper_contract_test.go +++ b/services/core/internal/sandbox/e2b/helper_contract_test.go @@ -7,7 +7,6 @@ import ( "os/exec" "testing" - "github.com/MiniMax-AI/OpenAgentCore/internal/agentnetwork" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" ) @@ -56,7 +55,7 @@ func TestSharedHelperExchanges(t *testing.T) { } // Reconstruct the Go bootstrap input and check the Python-managed projection -// against its owning types. Credentials travel only in RuntimeBootstrap. +// against its owning types: the Bootstrap with the installation. func validManagedExchange(data []byte) bool { var fields map[string]json.RawMessage if json.Unmarshal(data, &fields) != nil { @@ -65,10 +64,7 @@ func validManagedExchange(data []byte) bool { bootstrapBytes, _ := json.Marshal(sandbox.Bootstrap{}) var expected map[string]json.RawMessage _ = json.Unmarshal(bootstrapBytes, &expected) - delete(expected, "CoreURL") - delete(expected, "Credential") expected["InstallationID"] = nil - expected["RuntimeBootstrap"] = nil if len(fields) != len(expected) { return false } @@ -82,11 +78,7 @@ func validManagedExchange(data []byte) bool { return false } delete(fields, "InstallationID") - delete(fields, "RuntimeBootstrap") data, _ = json.Marshal(fields) var bootstrap sandbox.Bootstrap - if json.Unmarshal(data, &bootstrap) != nil || !validReference(bootstrap.Reference) || !validID(bootstrap.DeviceID) || !validID(bootstrap.SessionID) { - return false - } - return (agentnetwork.Policy{Access: bootstrap.NetworkAccess, AllowedDomains: bootstrap.AllowedDomains}).Validate() == nil + return json.Unmarshal(data, &bootstrap) == nil && bootstrap.Validate() == nil } diff --git a/services/core/internal/sandbox/e2b/internal/contractgen/main.go b/services/core/internal/sandbox/e2b/internal/contractgen/main.go index 7f40f447e..5cb279d01 100644 --- a/services/core/internal/sandbox/e2b/internal/contractgen/main.go +++ b/services/core/internal/sandbox/e2b/internal/contractgen/main.go @@ -6,10 +6,7 @@ import ( "encoding/json" "flag" "fmt" - "go/ast" "go/format" - "go/parser" - "go/token" "os" "path/filepath" "reflect" @@ -19,8 +16,6 @@ import ( "strings" "time" - "github.com/MiniMax-AI/OpenAgentCore/internal/agentnetwork" - "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" "github.com/MiniMax-AI/OpenAgentCore/internal/sandboxbootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/e2b" @@ -70,26 +65,18 @@ func main() { goCode, err := format.Source([]byte("// Code generated by contractgen; DO NOT EDIT.\npackage e2b\nconst SDKVersion = " + strconv.Quote(sdk) + "\n")) must(err) write("services/core/internal/sandbox/e2b/helper_sdk_generated.go", goCode) - bootstrap := fields(reflect.TypeFor[sandbox.Bootstrap]()) - bootstrap = slices.DeleteFunc(bootstrap, func(s string) bool { return s == "CoreURL" || s == "Credential" }) - bootstrap = append(bootstrap, "InstallationID", "RuntimeBootstrap") - identity := fields(reflect.TypeFor[sandbox.Reference]()) - for _, name := range []string{"SessionID", "DeviceID"} { - if !slices.Contains(bootstrap, name) { - panic("bootstrap identity field missing: " + name) - } - identity = append(identity, name) - } - identity = append(identity, "InstallationID") + // The managed bootstrap is the Bootstrap with the installation; its + // identity is the allocation's Reference with the installation. + bootstrap := append(fields(reflect.TypeFor[sandbox.Bootstrap]()), "InstallationID") + identity := append(fields(reflect.TypeFor[sandbox.Reference]()), "InstallationID") values := map[string]any{ "PROTOCOL_VERSION": e2b.ProtocolVersion, "SDK_VERSION": sdk, "MAX_REQUEST": e2b.MaxRequestBytes, "MAX_RESPONSE": e2b.MaxResponseBytes, - "MAX_OUTPUT": e2b.MaxOutputBytes, "MAX_COMMAND_INPUT": e2b.MaxCommandInputBytes, "MAX_CREDENTIAL_REFERENCES": e2b.MaxCredentialReferences, + "MAX_OUTPUT": e2b.MaxOutputBytes, "MAX_CREDENTIAL_REFERENCES": e2b.MaxCredentialReferences, "OPERATIONS": e2b.HelperOperations(), "ERROR_CODES": e2b.HelperErrors(), "REQUEST_FIELDS": fields(reflect.TypeFor[e2b.Request]()), "RESPONSE_FIELDS": fields(reflect.TypeFor[e2b.Response]()), "REFERENCE_FIELDS": fields(reflect.TypeFor[sandbox.Reference]()), "MANAGED_BOOTSTRAP_FIELDS": bootstrap, "MANAGED_IDENTITY_FIELDS": identity, - "NETWORK_ACCESS": networkValues(filepath.Join(root, "internal/agentnetwork/policy.go")), } var python bytes.Buffer python.WriteString("# Code generated by contractgen; DO NOT EDIT.\n\"\"\"Adapter-private wire declarations. No SDK or repository dependency.\"\"\"\n") @@ -113,12 +100,11 @@ func main() { // envelopes and managed bootstrap inputs in both implementations. func fixtures() []byte { r := sandbox.Reference{TenantID: "11111111-1111-4111-8111-111111111111", EnvironmentID: "22222222-2222-4222-8222-222222222222", AllocationID: "33333333-3333-4333-8333-333333333333"} - b := sandbox.Bootstrap{Reference: r, SessionID: "44444444-4444-4444-8444-444444444444", DeviceID: "55555555-5555-4555-8555-555555555555", CoreURL: "https://core.example/api/v1", Credential: "fixture-only", NetworkAccess: "enabled", + b := sandbox.Bootstrap{Reference: r, SandboxIO: sandboxbootstrap.Input{Version: sandboxbootstrap.Version, LinkURL: "wss://core.example/api/v1/sandbox-link", Credential: "fixture-serve-only", Resource: sandboxbootstrap.Resource{TenantID: r.TenantID, EnvironmentID: r.EnvironmentID, Kind: "allocation", ID: r.AllocationID, Generation: 1}}} installation := "66666666-6666-4666-8666-666666666666" - connection := b.RuntimeConnection() - q := e2b.Request{Version: e2b.ProtocolVersion, Operation: "create", Config: e2b.Config{InstallationID: installation}, Reference: r, Bootstrap: &b, RuntimeBootstrap: &connection, Deadline: time.Date(2099, 1, 1, 0, 0, 0, 0, time.UTC)} + q := e2b.Request{Version: e2b.ProtocolVersion, Operation: "create", Config: e2b.Config{InstallationID: installation}, Reference: r, Bootstrap: &b, Deadline: time.Date(2099, 1, 1, 0, 0, 0, 0, time.UTC)} var cases []map[string]any add := func(kind, name string, valid bool, payload any) { cases = append(cases, map[string]any{"kind": kind, "name": name, "valid": valid, "payload": payload}) @@ -178,20 +164,12 @@ func fixtures() []byte { add("response", "invalid-"+item.field, false, value) } managed := object(b) - delete(managed, "CoreURL") - delete(managed, "Credential") managed["InstallationID"] = installation - managed["RuntimeBootstrap"] = runtimebootstrap.Connection(connection) - for _, policy := range []agentnetwork.Policy{{Access: "enabled"}, {Access: "disabled"}, {Access: "restricted", AllowedDomains: []string{"example.com"}}} { - value := object(managed) - value["NetworkAccess"] = policy.Access - value["AllowedDomains"] = policy.AllowedDomains - add("managed", policy.Access, true, value) - } + add("managed", "valid", true, managed) for _, item := range []struct { field string value any - }{{"Extra", true}, {"DeviceID", nil}, {"DeviceID", "invalid"}, {"NetworkAccess", nil}, {"NetworkAccess", "unknown"}, {"NetworkAccess", true}, {"AllowedDomains", "example.com"}, {"AllowedDomains", []any{1}}} { + }{{"Extra", true}, {"AllocationID", nil}, {"AllocationID", "invalid"}, {"InstallationID", "00000000-0000-0000-0000-000000000000"}, {"SandboxIO", nil}, {"SandboxIO", "invalid"}} { value := object(managed) value[item.field] = item.value add("managed", "invalid-"+item.field, false, value) @@ -240,52 +218,6 @@ func fields(t reflect.Type) []string { return result } -// Read only the access switch of Policy.Validate. Unexpected shapes fail rather -// than silently emitting an incomplete enum or interpreting a general validator. -func networkValues(path string) []string { - f, err := parser.ParseFile(token.NewFileSet(), path, nil, 0) - must(err) - var result []string - for _, d := range f.Decls { - fn, ok := d.(*ast.FuncDecl) - if !ok || fn.Name.Name != "Validate" || fn.Recv == nil { - continue - } - for _, statement := range fn.Body.List { - s, ok := statement.(*ast.SwitchStmt) - if !ok { - continue - } - tag, ok := s.Tag.(*ast.SelectorExpr) - if !ok || tag.Sel.Name != "Access" { - continue - } - for _, clause := range s.Body.List { - for _, value := range clause.(*ast.CaseClause).List { - literal, ok := value.(*ast.BasicLit) - if !ok || literal.Kind != token.STRING { - panic("network access cases must be string literals") - } - access, err := strconv.Unquote(literal.Value) - must(err) - p := agentnetwork.Policy{Access: access} - if p.Validate() != nil { - p.AllowedDomains = []string{"example.com"} - } - if p.Validate() != nil || slices.Contains(result, access) { - panic("invalid network access projection") - } - result = append(result, access) - } - } - } - } - if len(result) == 0 { - panic("network access switch not found") - } - return result -} - func must(err error) { if err != nil { panic(err) diff --git a/services/core/internal/sandbox/e2b/observations.go b/services/core/internal/sandbox/e2b/observations.go index 063ba1307..4ea46e064 100644 --- a/services/core/internal/sandbox/e2b/observations.go +++ b/services/core/internal/sandbox/e2b/observations.go @@ -49,7 +49,7 @@ func (p *Provider) Observe(ctx context.Context, target runtimeobs.Target) (runti if ctxErr := ctx.Err(); ctxErr != nil { return runtimeobs.Sample{}, ctxErr } - if err != nil || out.Version != ProtocolVersion || out.Info != nil || out.Command != nil || out.DeploymentValid || out.TemplateBuild != nil { + if err != nil || out.Version != ProtocolVersion || out.Info != nil || out.DeploymentValid || out.TemplateBuild != nil { return runtimeobs.Sample{}, runtimeobs.ErrUnavailable } switch out.ErrorCode { diff --git a/services/core/internal/sandbox/e2b/operations.go b/services/core/internal/sandbox/e2b/operations.go index a3da4688e..42e211a83 100644 --- a/services/core/internal/sandbox/e2b/operations.go +++ b/services/core/internal/sandbox/e2b/operations.go @@ -9,21 +9,19 @@ import ( // Operations is this adapter's complete authored resource contract. func Operations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "Suspend": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "Resume": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, - "Observe": {State: providercontract.Supported}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "Suspend": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "Resume": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "e2b_does_not_support_checkpoints"}, + "Observe": {State: providercontract.Supported}, } } func (*Provider) ProviderOperations() providercontract.Operations { return Operations() } @@ -48,9 +46,6 @@ func (p *Provider) KillCompute(context.Context, sandbox.Reference, sandbox.Compu func (p *Provider) DeleteSnapshot(context.Context, sandbox.Reference, sandbox.SnapshotIdentity) error { return &providercontract.UnsupportedError{Operation: "DeleteSnapshot", Reason: Operations()["DeleteSnapshot"].Reason} } -func (p *Provider) RunCommandCompute(context.Context, sandbox.Reference, sandbox.Compute, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, &providercontract.UnsupportedError{Operation: "RunCommandCompute", Reason: Operations()["RunCommandCompute"].Reason} -} func (p *Provider) ResumeCompute(context.Context, sandbox.Reference, sandbox.Compute) (sandbox.ComputeState, error) { return sandbox.ComputeState{}, &providercontract.UnsupportedError{Operation: "ResumeCompute", Reason: Operations()["ResumeCompute"].Reason} } diff --git a/services/core/internal/sandbox/e2b/provider.go b/services/core/internal/sandbox/e2b/provider.go index 29fd159bd..1ac5231d7 100644 --- a/services/core/internal/sandbox/e2b/provider.go +++ b/services/core/internal/sandbox/e2b/provider.go @@ -11,8 +11,6 @@ import ( "time" "unicode" - "github.com/MiniMax-AI/OpenAgentCore/internal/agentnetwork" - "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/google/uuid" ) @@ -56,7 +54,7 @@ func Discover(ctx context.Context, caller Caller, binary, apiKey, apiURL, domain if out.ErrorCode == "invalid" { return Response{}, sandbox.ErrInvalid } - if out.ErrorCode != "" || out.Info != nil || out.Command != nil || out.TemplateBuild != nil || out.Observation != nil { + if out.ErrorCode != "" || out.Info != nil || out.TemplateBuild != nil || out.Observation != nil { return Response{}, sandbox.ErrComputeUnconfirmed } if operation == "list_templates" { @@ -178,7 +176,7 @@ func newDirect(c sandbox.DirectConfig) (*Provider, error) { } return provider, nil } -func (p *Provider) call(ctx context.Context, operation string, r sandbox.Reference, b *sandbox.Bootstrap, command *sandbox.Command) (Response, error) { +func (p *Provider) call(ctx context.Context, operation string, r sandbox.Reference, b *sandbox.Bootstrap) (Response, error) { deadline, ok := ctx.Deadline() if !ok || (operation != "validate_deployment" && !validReference(r)) { return unstarted(operation, r), sandbox.ErrInvalid @@ -186,19 +184,11 @@ func (p *Provider) call(ctx context.Context, operation string, r sandbox.Referen if err := ctx.Err(); err != nil { return unstarted(operation, r), err } - var connection *runtimebootstrap.Connection - if b != nil { - value := b.RuntimeConnection() - connection = &value - } - out, err := p.caller.Call(ctx, Request{Version: ProtocolVersion, Operation: operation, Config: p.config, Reference: r, Bootstrap: b, RuntimeBootstrap: connection, Command: command, Deadline: deadline}) + out, err := p.caller.Call(ctx, Request{Version: ProtocolVersion, Operation: operation, Config: p.config, Reference: r, Bootstrap: b, Deadline: deadline}) if errors.Is(err, errHelperNotStarted) { return unstarted(operation, r), sandbox.ErrComputeUnconfirmed } if err != nil || out.Version != ProtocolVersion { - if operation == "command" { - return Response{}, sandbox.ErrCommandUnconfirmed - } return Response{}, sandbox.ErrComputeUnconfirmed } if out.Info != nil && (out.Info.Reference != r || len(out.Info.ProviderID) > 256 || len(out.Info.State) > 64 || out.Info.BootstrapComplete && !out.Info.CreateSettled) { @@ -208,7 +198,7 @@ func (p *Provider) call(ctx context.Context, operation string, r sandbox.Referen case "": return out, nil case "template_invalid": - return out, fmt.Errorf("%w: This E2B template lacks the current Runtime startup entry point. Build a template with this release's build-template.py and select it in the sandbox deployment.", sandbox.ErrInvalid) + return out, fmt.Errorf("%w: This E2B template lacks the current sandbox startup entry point. Build a template with this release's build-template.py and select it in the sandbox deployment.", sandbox.ErrInvalid) case "team_mismatch": return out, sandbox.ErrCredentialOwnership case "unauthorized": @@ -221,8 +211,6 @@ func (p *Provider) call(ctx context.Context, operation string, r sandbox.Referen return out, sandbox.ErrExists case "not_found": return out, sandbox.ErrNotFound - case "command_unconfirmed": - return out, sandbox.ErrCommandUnconfirmed default: return out, sandbox.ErrComputeUnconfirmed } @@ -235,12 +223,12 @@ func (p *Provider) call(ctx context.Context, operation string, r sandbox.Referen func (p *Provider) ValidateDeployment(ctx context.Context) (TemplateBuild, error) { ctx, cancel := context.WithTimeout(ctx, 30*time.Second) defer cancel() - out, err := p.call(ctx, "validate_deployment", sandbox.Reference{}, nil, nil) + out, err := p.call(ctx, "validate_deployment", sandbox.Reference{}, nil) if err != nil { return TemplateBuild{}, err } build := out.TemplateBuild - if !out.DeploymentValid || out.Info != nil || out.Command != nil || out.Observation != nil || build == nil || build.Status != "ready" || + if !out.DeploymentValid || out.Info != nil || out.Observation != nil || build == nil || build.Status != "ready" || build.CPUs == 0 || build.MemoryMiB == 0 || build.RootDiskMiB != nil && *build.RootDiskMiB == 0 || p.config.Resources != nil && (build.CPUs != p.config.Resources.CPUs || build.MemoryMiB != p.config.Resources.MemoryMiB) { return TemplateBuild{}, sandbox.ErrComputeUnconfirmed @@ -248,7 +236,7 @@ func (p *Provider) ValidateDeployment(ctx context.Context) (TemplateBuild, error return *build, nil } func (p *Provider) info(ctx context.Context, operation string, r sandbox.Reference, b *sandbox.Bootstrap) (sandbox.Info, error) { - out, err := p.call(ctx, operation, r, b, nil) + out, err := p.call(ctx, operation, r, b) if out.Info != nil { return *out.Info, err } @@ -258,7 +246,6 @@ func (p *Provider) info(ctx context.Context, operation string, r sandbox.Referen return sandbox.Info{Reference: r}, err } func (p *Provider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Info, error) { - b.AllowedDomains = agentnetwork.Policy{Access: b.NetworkAccess, AllowedDomains: b.AllowedDomains}.Hosts() return p.info(ctx, "create", b.Reference, &b) } func (p *Provider) GetInfo(ctx context.Context, r sandbox.Reference) (sandbox.Info, error) { @@ -268,30 +255,12 @@ func (p *Provider) Renew(ctx context.Context, r sandbox.Reference) (sandbox.Info return p.info(ctx, "renew", r, nil) } func (p *Provider) Kill(ctx context.Context, r sandbox.Reference) error { - out, err := p.call(ctx, "kill", r, nil, nil) + out, err := p.call(ctx, "kill", r, nil) if err == nil && (out.Info == nil || !out.Info.CreateSettled || out.Info.State != "absent") { return sandbox.ErrComputeUnconfirmed } return err } -func (p *Provider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - if len(c.Args) == 0 || len(c.Stdin) > sandbox.MaxCommandInputBytes || c.Directory != "" && !filepath.IsAbs(c.Directory) { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - for _, arg := range c.Args { - if strings.ContainsRune(arg, 0) { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - } - out, err := p.call(ctx, "command", r, nil, &c) - if err != nil { - return sandbox.CommandResult{}, err - } - if out.Command == nil || len(out.Command.Stdout) > MaxOutputBytes || len(out.Command.Stderr) > MaxOutputBytes { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - return *out.Command, nil -} // A fresh allocation Create rejected before process startup has no cloud effects. // The common Provider contract forbids replaying an earlier unknown Create. diff --git a/services/core/internal/sandbox/e2b/provider_test.go b/services/core/internal/sandbox/e2b/provider_test.go index b5c199ee9..305b780c3 100644 --- a/services/core/internal/sandbox/e2b/provider_test.go +++ b/services/core/internal/sandbox/e2b/provider_test.go @@ -102,7 +102,7 @@ func TestKillRequiresTerminalProof(t *testing.T) { t.Fatal(err) } } -func TestCreateAndCommandUseOnlyPrivateRequest(t *testing.T) { +func TestCreateUsesOnlyPrivateRequest(t *testing.T) { p, f, r := fixture(t) b := contracttest.Bootstrap(r) f.response.Info = &sandbox.Info{Reference: r, State: "running", ProviderID: "native-id", CreateSettled: true, BootstrapComplete: true} @@ -110,33 +110,15 @@ func TestCreateAndCommandUseOnlyPrivateRequest(t *testing.T) { t.Fatal(err) } q := f.requests[0] - if q.Operation != "create" || q.Bootstrap == nil || q.Bootstrap.Credential != b.Credential || q.Bootstrap.SandboxIO != b.SandboxIO || q.Config.APIKey != p.config.APIKey { + if q.Operation != "create" || q.Bootstrap == nil || *q.Bootstrap != b || q.Config.APIKey != p.config.APIKey { t.Fatal("private request lost") } - f.response.Info = nil - f.response.Command = &sandbox.CommandResult{ExitCode: 7, Stdout: "output"} - got, err := p.RunCommand(bounded(t), r, sandbox.Command{Args: []string{"cat"}, Stdin: []byte("private input")}) - if err != nil || got.ExitCode != 7 { - t.Fatal(got, err) - } - f.err = context.DeadlineExceeded - if _, err = p.RunCommand(bounded(t), r, sandbox.Command{Args: []string{"true"}}); !errors.Is(err, sandbox.ErrCommandUnconfirmed) { - t.Fatal(err) - } - if len(f.requests) != 3 { - t.Fatal("operation was replayed") - } } -func TestDeadlineAndCommandAdmission(t *testing.T) { +func TestDeadlineAdmission(t *testing.T) { p, f, r := fixture(t) if _, err := p.GetInfo(context.Background(), r); !errors.Is(err, sandbox.ErrInvalid) { t.Fatal(err) } - for _, cmd := range []sandbox.Command{{}, {Args: []string{"x\x00"}}, {Args: []string{"cat"}, Directory: "relative"}} { - if _, err := p.RunCommand(bounded(t), r, cmd); !errors.Is(err, sandbox.ErrInvalid) { - t.Fatal(err) - } - } if len(f.requests) != 0 { t.Fatal("invalid operation reached helper") } diff --git a/services/core/internal/sandbox/microsandbox/creation_settlement_test.go b/services/core/internal/sandbox/microsandbox/creation_settlement_test.go index 0ac26e6f9..1863f30a9 100644 --- a/services/core/internal/sandbox/microsandbox/creation_settlement_test.go +++ b/services/core/internal/sandbox/microsandbox/creation_settlement_test.go @@ -10,8 +10,7 @@ import ( func TestCreateConfigurationRejectionSettlement(t *testing.T) { config, ref := testConfig(), testRef() - bootstrap := sandbox.Bootstrap{Reference: ref, SessionID: ref.TenantID, DeviceID: ref.EnvironmentID, - CoreURL: "https://core.example/api/v1", Credential: "fixture", NetworkAccess: "disabled"} + bootstrap := sandbox.Bootstrap{Reference: ref} for _, test := range []struct { name string change func(*Response) diff --git a/services/core/internal/sandbox/microsandbox/identity.go b/services/core/internal/sandbox/microsandbox/identity.go index 5464f66ad..7e51143f1 100644 --- a/services/core/internal/sandbox/microsandbox/identity.go +++ b/services/core/internal/sandbox/microsandbox/identity.go @@ -74,17 +74,6 @@ func ValidateSnapshot(c Config, r sandbox.Reference, s SnapshotIdentity) error { } return nil } -func ValidateCommand(c sandbox.Command) error { - if len(c.Args) == 0 || c.Args[0] == "" || len(c.Stdin) > sandbox.MaxCommandInputBytes || (c.Directory != "" && !filepath.IsAbs(c.Directory)) { - return sandbox.ErrInvalid - } - for _, a := range c.Args { - if strings.IndexByte(a, 0) >= 0 { - return sandbox.ErrInvalid - } - } - return nil -} func ValidateRequest(q Request) error { if q.Version != ProtocolVersion || q.Config.Validate() != nil || !ValidReference(q.Reference) || q.Deadline.IsZero() { return sandbox.ErrInvalid @@ -99,11 +88,6 @@ func ValidateRequest(q Request) error { return sandbox.ErrInvalid } return ValidateCompute(q.Config, q.Reference, q.Compute) - case "command": - if q.Command == nil || ValidateCommand(*q.Command) != nil { - return sandbox.ErrInvalid - } - return ValidateCompute(q.Config, q.Reference, q.Compute) case "suspend": s := q.Suspend if s == nil || s.Reference != q.Reference || !validID(s.OperationID) || s.Source.ID == "" || ValidateCompute(q.Config, q.Reference, s.Source) != nil { diff --git a/services/core/internal/sandbox/microsandbox/operations.go b/services/core/internal/sandbox/microsandbox/operations.go index 13e81ff96..56e5a3fd0 100644 --- a/services/core/internal/sandbox/microsandbox/operations.go +++ b/services/core/internal/sandbox/microsandbox/operations.go @@ -5,21 +5,19 @@ import "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontr // Operations is this adapter's complete authored resource contract. func Operations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Supported}, - "NewCompute": {State: providercontract.Supported}, - "GetCompute": {State: providercontract.Supported}, - "Suspend": {State: providercontract.Supported}, - "Resume": {State: providercontract.Supported}, - "KillCompute": {State: providercontract.Supported}, - "DeleteSnapshot": {State: providercontract.Supported}, - "RunCommandCompute": {State: providercontract.Supported}, - "ResumeCompute": {State: providercontract.Supported}, - "Observe": {State: providercontract.Supported}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Supported}, + "NewCompute": {State: providercontract.Supported}, + "GetCompute": {State: providercontract.Supported}, + "Suspend": {State: providercontract.Supported}, + "Resume": {State: providercontract.Supported}, + "KillCompute": {State: providercontract.Supported}, + "DeleteSnapshot": {State: providercontract.Supported}, + "ResumeCompute": {State: providercontract.Supported}, + "Observe": {State: providercontract.Supported}, } } func (*Provider) ProviderOperations() providercontract.Operations { return Operations() } diff --git a/services/core/internal/sandbox/microsandbox/provider.go b/services/core/internal/sandbox/microsandbox/provider.go index d876702be..4ba63ae0a 100644 --- a/services/core/internal/sandbox/microsandbox/provider.go +++ b/services/core/internal/sandbox/microsandbox/provider.go @@ -44,9 +44,6 @@ func (p *Provider) call(ctx context.Context, q Request) (Response, error) { } out, err := p.caller.Call(ctx, q) if err != nil { - if q.Operation == "command" { - return out, errors.Join(sandbox.ErrCommandUnconfirmed, err) - } return out, errors.Join(ErrUnconfirmed, err) } if out.Version != ProtocolVersion { @@ -63,8 +60,6 @@ func (p *Provider) call(ctx context.Context, q Request) (Response, error) { return out, sandbox.ErrExists case "not_found": return out, sandbox.ErrNotFound - case "command_unconfirmed": - return out, sandbox.ErrCommandUnconfirmed case "metrics_unavailable": return out, runtimeobs.ErrUnavailable default: @@ -157,13 +152,6 @@ func (p *Provider) Kill(ctx context.Context, r sandbox.Reference) error { } return p.KillCompute(ctx, r, c) } -func (p *Provider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - compute, e := p.Initial(ctx, r) - if e != nil { - return sandbox.CommandResult{}, e - } - return p.RunCommandCompute(ctx, r, compute, c) -} func (p *Provider) GetCompute(ctx context.Context, r sandbox.Reference, c Compute) (State, error) { return p.state(ctx, Request{Operation: "inspect", Reference: r, Compute: c}) } @@ -181,16 +169,6 @@ func (p *Provider) Suspend(ctx context.Context, q SuspendRequest) (State, error) func (p *Provider) Resume(ctx context.Context, q ResumeRequest) (State, error) { return p.state(ctx, Request{Operation: "resume", Reference: q.Reference, Resume: &q}) } -func (p *Provider) RunCommandCompute(ctx context.Context, r sandbox.Reference, c Compute, command sandbox.Command) (sandbox.CommandResult, error) { - out, e := p.call(ctx, Request{Operation: "command", Reference: r, Compute: c, Command: &command}) - if e != nil { - return sandbox.CommandResult{}, e - } - if out.Command == nil || len(out.Command.Stdout) > MaxOutputBytes || len(out.Command.Stderr) > MaxOutputBytes { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - return *out.Command, nil -} // ResumeCompute thaws the exact resident source after an aborted suspension. // It never starts stopped compute or restores a checkpoint. diff --git a/services/core/internal/sandbox/microsandbox/provider_test.go b/services/core/internal/sandbox/microsandbox/provider_test.go index 94bfa9881..bb7dc0e36 100644 --- a/services/core/internal/sandbox/microsandbox/provider_test.go +++ b/services/core/internal/sandbox/microsandbox/provider_test.go @@ -88,25 +88,6 @@ func TestObserveOnlyPreservesCapturedButResidentState(t *testing.T) { t.Fatalf("state=%+v error=%v", got, e) } } -func TestCommandUncertaintyAndResultLimits(t *testing.T) { - for _, tc := range []struct { - name string - response Response - failure error - }{ - {"transport", Response{}, errors.New("lost result")}, - {"missing", Response{Version: ProtocolVersion}, nil}, - {"too_large", Response{Version: ProtocolVersion, Command: &sandbox.CommandResult{Stdout: strings.Repeat("x", MaxOutputBytes+1)}}, nil}, - } { - t.Run(tc.name, func(t *testing.T) { - p, _ := NewWithCaller(testConfig(), callerFunc(func(context.Context, Request) (Response, error) { return tc.response, tc.failure })) - _, e := p.RunCommand(deadline(t), testRef(), sandbox.Command{Args: []string{"/bin/true"}}) - if !errors.Is(e, sandbox.ErrCommandUnconfirmed) { - t.Fatalf("uncertainty lost: %v", e) - } - }) - } -} func TestNoDeadlineOrForeignAllocationNeverCallsHelper(t *testing.T) { calls := 0 p, _ := NewWithCaller(testConfig(), callerFunc(func(context.Context, Request) (Response, error) { calls++; return Response{}, nil })) diff --git a/services/core/internal/sandbox/microsandbox/types.go b/services/core/internal/sandbox/microsandbox/types.go index 12ddc1f69..453b5e6c9 100644 --- a/services/core/internal/sandbox/microsandbox/types.go +++ b/services/core/internal/sandbox/microsandbox/types.go @@ -12,7 +12,7 @@ import ( const ProtocolVersion = 2 const SDKVersion = "v0.7.2" const MaxOutputBytes = 1024 * 1024 -const MaxRequestBytes = 72 * 1024 * 1024 +const MaxRequestBytes = 1024 * 1024 const MaxResponseBytes = 16 * 1024 * 1024 // Config is trusted deployment configuration. Paths and hashes refer to one @@ -47,8 +47,8 @@ type State = sandbox.ComputeState type SuspendRequest = sandbox.SuspendRequest type ResumeRequest = sandbox.ResumeRequest -// Request and Response are the finite, private helper boundary. Confidential -// Bootstrap and Command bytes travel only through stdin and are never logged. +// Request and Response are the finite, private helper boundary. The +// confidential Bootstrap travels only through stdin and is never logged. type Request struct { Version int Operation string @@ -56,7 +56,6 @@ type Request struct { Reference sandbox.Reference Compute Compute Bootstrap *sandbox.Bootstrap - Command *sandbox.Command Suspend *SuspendRequest Resume *ResumeRequest Snapshot *SnapshotIdentity @@ -68,7 +67,6 @@ type Response struct { CreateSettled bool `json:",omitempty"` Version int State *State - Command *sandbox.CommandResult Metrics *Metrics ErrorCode string } diff --git a/services/core/internal/sandbox/node/docker_live_test.go b/services/core/internal/sandbox/node/docker_live_test.go index 6b728b3eb..27cd8e587 100644 --- a/services/core/internal/sandbox/node/docker_live_test.go +++ b/services/core/internal/sandbox/node/docker_live_test.go @@ -5,7 +5,6 @@ import ( "errors" "net/http/httptest" "os" - "strings" "testing" "time" @@ -16,10 +15,10 @@ import ( "github.com/moby/moby/client" ) -// This uses the same pinned fixture image, whose oac-daemon only sleeps, as -// the Docker mechanism tests. It exercises real Docker resources through the -// node transport, not a native harness/model workflow. No provider -// credentials are required. +// This uses the same image as the Docker mechanism tests, whose Sandbox I/O +// service keeps retrying an unreachable Link. It exercises real Docker +// resources through the node transport, not a native harness/model workflow. +// No provider credentials are required. func TestDockerNodeTransportLifecycle(t *testing.T) { image := os.Getenv("AGENTS_RUNTIME_DOCKER_TEST_IMAGE") if image == "" { @@ -98,17 +97,6 @@ func TestDockerNodeTransportLifecycle(t *testing.T) { if !info.BootstrapComplete || info.State != "running" { t.Fatalf("create compute state: %+v", info) } - command := func(args ...string) sandbox.CommandResult { - t.Helper() - commandCtx, stop := context.WithTimeout(ctx, 15*time.Second) - defer stop() - out, err := proxy.RunCommand(commandCtx, r, sandbox.Command{Args: args, Directory: "/workspace"}) - if err != nil || out.ExitCode != 0 { - t.Fatalf("command failed: err=%v exit=%d", err, out.ExitCode) - } - return out - } - command("/bin/sh", "-c", "printf node-transport-persisted > /workspace/node-transport-proof") hub.mu.Lock() previous := hub.peers[id.NodeID] hub.mu.Unlock() @@ -119,8 +107,8 @@ func TestDockerNodeTransportLifecycle(t *testing.T) { if !retained { t.Fatal("duplicate connection replaced real Docker node") } - if command("/bin/cat", "/workspace/node-transport-proof").Stdout != "node-transport-persisted" { - t.Fatal("duplicate disrupted live node") + if observed, err := proxy.GetInfo(ctx, r); err != nil || observed.ProviderID != info.ProviderID || observed.State != "running" { + t.Fatal("duplicate disrupted live node", err) } hub.Disconnect(id.NodeID) wait(t, func() bool { @@ -129,12 +117,9 @@ func TestDockerNodeTransportLifecycle(t *testing.T) { return hub.peers[id.NodeID] != nil && hub.peers[id.NodeID] != previous }) observed, err := proxy.GetInfo(ctx, r) - if err != nil || observed.ProviderID != info.ProviderID { + if err != nil || observed.ProviderID != info.ProviderID || observed.State != "running" { t.Fatal("connection loss changed compute", err) } - if command("/bin/cat", "/workspace/node-transport-proof").Stdout != "node-transport-persisted" { - t.Fatal("workspace changed after disconnect") - } stop() if err = <-done; err != nil { t.Fatal(err) @@ -149,12 +134,9 @@ func TestDockerNodeTransportLifecycle(t *testing.T) { running = true wait(t, func() bool { return hub.Online(id.NodeID) }) observed, err = proxy.GetInfo(ctx, r) - if err != nil || observed.ProviderID != info.ProviderID { + if err != nil || observed.ProviderID != info.ProviderID || observed.State != "running" { t.Fatal("node restart changed compute", err) } - if strings.TrimSpace(command("/bin/cat", "/workspace/node-transport-proof").Stdout) != "node-transport-persisted" { - t.Fatal("workspace changed after node restart") - } if err = proxy.Kill(ctx, r); err != nil { t.Fatal("node cleanup", err) } @@ -162,5 +144,5 @@ func TestDockerNodeTransportLifecycle(t *testing.T) { t.Fatal("container retained after cleanup", err) } // Kill verifies removal of both named Runtime volumes before returning. - t.Logf("real Docker node transport passed: installation=%s allocation=%s compute=%s; duplicate connection rejected; reconnect and node restart retained identity/file; owned container and volumes removed", id.InstallationID, r.AllocationID, info.ProviderID) + t.Logf("real Docker node transport passed: installation=%s allocation=%s compute=%s; duplicate connection rejected; reconnect and node restart retained identity and compute; owned container and volumes removed", id.InstallationID, r.AllocationID, info.ProviderID) } diff --git a/services/core/internal/sandbox/node/hub.go b/services/core/internal/sandbox/node/hub.go index bda1da1c4..a3360905d 100644 --- a/services/core/internal/sandbox/node/hub.go +++ b/services/core/internal/sandbox/node/hub.go @@ -313,11 +313,11 @@ func (h *Hub) call(ctx context.Context, id string, q request) (response, error) ctx, cancel := h.lifetime(ctx) defer cancel() if err := ctx.Err(); err != nil { - return response{}, uncertain(q.Operation, err) + return response{}, uncertain(err) } epoch, err := callbackValue(ctx, h.options.OwnerEpoch) if err != nil { - return response{}, uncertain(q.Operation, err) + return response{}, uncertain(err) } h.mu.Lock() p := h.peers[id] @@ -326,7 +326,7 @@ func (h *Hub) call(ctx context.Context, id string, q request) (response, error) } h.mu.Unlock() if p == nil || p.epoch != epoch { - return response{}, uncertain(q.Operation, ErrUnavailable) + return response{}, uncertain(ErrUnavailable) } p.mu.Lock() ready := p.ready @@ -335,11 +335,11 @@ func (h *Hub) call(ctx context.Context, id string, q request) (response, error) ready = ok && status.State == "ready" } else if q.DeploymentGeneration != p.identity.DeploymentGeneration { p.mu.Unlock() - return response{}, uncertain(q.Operation, ErrUnavailable) + return response{}, uncertain(ErrUnavailable) } p.mu.Unlock() if requiresReady(q) && !ready { - return response{}, uncertain(q.Operation, ErrUnavailable) + return response{}, uncertain(ErrUnavailable) } deadline, _ := ctx.Deadline() @@ -351,13 +351,13 @@ func (h *Hub) call(ctx context.Context, id string, q request) (response, error) p.mu.Lock() if len(p.pending) >= maxPending { p.mu.Unlock() - return response{}, uncertain(q.Operation, ErrUnavailable) + return response{}, uncertain(ErrUnavailable) } p.pending[q.ID] = ch p.mu.Unlock() defer func() { p.mu.Lock(); delete(p.pending, q.ID); p.mu.Unlock() }() if err = p.lockSend(ctx); err != nil { - return response{}, uncertain(q.Operation, err) + return response{}, uncertain(err) } select { case <-p.done: @@ -374,7 +374,7 @@ func (h *Hub) call(ctx context.Context, id string, q request) (response, error) p.unlockSend() if err != nil { p.close() - return response{}, uncertain(q.Operation, err) + return response{}, uncertain(err) } timer := time.NewTimer(time.Until(deadline)) defer timer.Stop() @@ -382,11 +382,11 @@ func (h *Hub) call(ctx context.Context, id string, q request) (response, error) case result := <-ch: return result, responseError(result) case <-p.done: - return response{}, uncertain(q.Operation, ErrUnavailable) + return response{}, uncertain(ErrUnavailable) case <-ctx.Done(): - return response{}, uncertain(q.Operation, ctx.Err()) + return response{}, uncertain(ctx.Err()) case <-timer.C: - return response{}, uncertain(q.Operation, context.DeadlineExceeded) + return response{}, uncertain(context.DeadlineExceeded) } } diff --git a/services/core/internal/sandbox/node/node_test.go b/services/core/internal/sandbox/node/node_test.go index 946470389..11027651f 100644 --- a/services/core/internal/sandbox/node/node_test.go +++ b/services/core/internal/sandbox/node/node_test.go @@ -55,9 +55,6 @@ func (p *fakeProvider) Kill(context.Context, sandbox.Reference) error { p.mu.Unlock() return nil } -func (p *fakeProvider) RunCommand(context.Context, sandbox.Reference, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, nil -} func reference() sandbox.Reference { return sandbox.Reference{TenantID: uuid.NewString(), EnvironmentID: uuid.NewString(), AllocationID: uuid.NewString()} } diff --git a/services/core/internal/sandbox/node/operations.go b/services/core/internal/sandbox/node/operations.go index 8f016664b..9740acd03 100644 --- a/services/core/internal/sandbox/node/operations.go +++ b/services/core/internal/sandbox/node/operations.go @@ -6,7 +6,6 @@ var operationMethods = map[string]string{ "info": "GetInfo", "renew": "Renew", "kill": "Kill", - "command": "RunCommand", "initial": "Initial", "new_compute": "NewCompute", "compute": "GetCompute", @@ -14,7 +13,6 @@ var operationMethods = map[string]string{ "resume": "Resume", "kill_compute": "KillCompute", "delete_snapshot": "DeleteSnapshot", - "command_compute": "RunCommandCompute", "resume_compute": "ResumeCompute", "observe": "Observe", } diff --git a/services/core/internal/sandbox/node/operations_test.go b/services/core/internal/sandbox/node/operations_test.go index e508988b5..c273e64e9 100644 --- a/services/core/internal/sandbox/node/operations_test.go +++ b/services/core/internal/sandbox/node/operations_test.go @@ -49,7 +49,7 @@ func TestUnsupportedProxyRejectsBeforeNodeResolution(t *testing.T) { } } func TestNodeOperationMappingCoversForwardedMethods(t *testing.T) { - for _, method := range []string{"Create", "GetInfo", "Renew", "Kill", "RunCommand", "Initial", "NewCompute", "GetCompute", "Suspend", "Resume", "KillCompute", "DeleteSnapshot", "RunCommandCompute", "ResumeCompute", "Observe"} { + for _, method := range []string{"Create", "GetInfo", "Renew", "Kill", "Initial", "NewCompute", "GetCompute", "Suspend", "Resume", "KillCompute", "DeleteSnapshot", "ResumeCompute", "Observe"} { if wire := operationWire(method); wire == "" || operationMethod(wire) != method { t.Fatal(method) } diff --git a/services/core/internal/sandbox/node/provider_operations_fixture_test.go b/services/core/internal/sandbox/node/provider_operations_fixture_test.go index 3f892e9bb..43da1c527 100644 --- a/services/core/internal/sandbox/node/provider_operations_fixture_test.go +++ b/services/core/internal/sandbox/node/provider_operations_fixture_test.go @@ -10,21 +10,19 @@ import ( func (*fakeProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } func (*fakeProvider) Initial(context.Context, sandbox.Reference) (sandbox.Compute, error) { @@ -48,9 +46,6 @@ func (*fakeProvider) KillCompute(context.Context, sandbox.Reference, sandbox.Com func (*fakeProvider) DeleteSnapshot(context.Context, sandbox.Reference, sandbox.SnapshotIdentity) error { return &providercontract.UnsupportedError{Operation: "DeleteSnapshot", Reason: "fixture_operation_not_supported"} } -func (*fakeProvider) RunCommandCompute(context.Context, sandbox.Reference, sandbox.Compute, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, &providercontract.UnsupportedError{Operation: "RunCommandCompute", Reason: "fixture_operation_not_supported"} -} func (*fakeProvider) ResumeCompute(context.Context, sandbox.Reference, sandbox.Compute) (sandbox.ComputeState, error) { return sandbox.ComputeState{}, &providercontract.UnsupportedError{Operation: "ResumeCompute", Reason: "fixture_operation_not_supported"} } @@ -59,20 +54,18 @@ func (*fakeProvider) Observe(context.Context, runtimeobs.Target) (runtimeobs.Sam } func (*observationProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Observe": {State: providercontract.Supported}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Supported}, } } diff --git a/services/core/internal/sandbox/node/proxy.go b/services/core/internal/sandbox/node/proxy.go index fc126ebe8..a0f6b29cf 100644 --- a/services/core/internal/sandbox/node/proxy.go +++ b/services/core/internal/sandbox/node/proxy.go @@ -82,19 +82,6 @@ func (p *provider) Kill(ctx context.Context, r sandbox.Reference) error { _, e := p.call(ctx, request{Operation: "kill", Reference: r}) return e } -func (p *provider) command(ctx context.Context, q request) (sandbox.CommandResult, error) { - r, e := p.call(ctx, q) - if e != nil { - return sandbox.CommandResult{}, e - } - if r.Command == nil || len(r.Command.Stdout) > 1024*1024 || len(r.Command.Stderr) > 1024*1024 { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - return *r.Command, nil -} -func (p *provider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - return p.command(ctx, request{Operation: "command", Reference: r, Command: &c}) -} func (p *provider) Initial(ctx context.Context, r sandbox.Reference) (sandbox.Compute, error) { out, e := p.call(ctx, request{Operation: "initial", Reference: r}) if e != nil { @@ -142,9 +129,6 @@ func (p *provider) DeleteSnapshot(ctx context.Context, r sandbox.Reference, s sa _, e := p.call(ctx, request{Operation: "delete_snapshot", Reference: r, Snapshot: &s}) return e } -func (p *provider) RunCommandCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute, v sandbox.Command) (sandbox.CommandResult, error) { - return p.command(ctx, request{Operation: "command_compute", Reference: r, Compute: &c, Command: &v}) -} func (p *provider) ResumeCompute(ctx context.Context, r sandbox.Reference, c sandbox.Compute) (sandbox.ComputeState, error) { return p.state(ctx, request{Operation: "resume_compute", Reference: r, Compute: &c}) } diff --git a/services/core/internal/sandbox/node/wire.go b/services/core/internal/sandbox/node/wire.go index a3d60ae58..5540edd77 100644 --- a/services/core/internal/sandbox/node/wire.go +++ b/services/core/internal/sandbox/node/wire.go @@ -19,7 +19,7 @@ import ( const ProtocolVersion = 5 const MaxControlFrameBytes = 32 * 1024 -const MaxFrameBytes = 72 * 1024 * 1024 +const MaxFrameBytes = 1024 * 1024 const maxPending = 32 const maxRequestTimeoutMillis int64 = 120000 @@ -89,7 +89,6 @@ type request struct { Bootstrap *sandbox.Bootstrap `json:"bootstrap,omitempty"` Compute *sandbox.Compute `json:"compute,omitempty"` Generation uint64 `json:"generation,omitempty"` - Command *sandbox.Command `json:"command,omitempty"` Suspend *sandbox.SuspendRequest `json:"suspend,omitempty"` Resume *sandbox.ResumeRequest `json:"resume,omitempty"` Snapshot *sandbox.SnapshotIdentity `json:"snapshot,omitempty"` @@ -104,7 +103,6 @@ type response struct { Info *sandbox.Info `json:"info,omitempty"` Compute *sandbox.Compute `json:"compute,omitempty"` State *sandbox.ComputeState `json:"state,omitempty"` - Command *sandbox.CommandResult `json:"command,omitempty"` Sample *runtimeobs.Sample `json:"sample,omitempty"` } @@ -187,8 +185,6 @@ func errorCode(err error) string { return "exists" case errors.Is(err, sandbox.ErrNotFound): return "not_found" - case errors.Is(err, sandbox.ErrCommandUnconfirmed): - return "command_unconfirmed" default: return "unconfirmed" } @@ -221,16 +217,11 @@ func responseError(out response) error { return sandbox.ErrExists case "not_found": return sandbox.ErrNotFound - case "command_unconfirmed": - return sandbox.ErrCommandUnconfirmed default: return sandbox.ErrComputeUnconfirmed } } -func uncertain(operation string, err error) error { - if operation == "command" || operation == "command_compute" { - return errors.Join(sandbox.ErrCommandUnconfirmed, err) - } +func uncertain(err error) error { return errors.Join(sandbox.ErrComputeUnconfirmed, err) } func (q request) validate() error { @@ -238,7 +229,7 @@ func (q request) validate() error { return sandbox.ErrInvalid } count := 0 - for _, ok := range []bool{q.Bootstrap != nil, q.Compute != nil, q.Command != nil, q.Suspend != nil, q.Resume != nil, q.Snapshot != nil, q.Observation != nil} { + for _, ok := range []bool{q.Bootstrap != nil, q.Compute != nil, q.Suspend != nil, q.Resume != nil, q.Snapshot != nil, q.Observation != nil} { if ok { count++ } @@ -264,14 +255,6 @@ func (q request) validate() error { if count == 1 && q.Compute != nil { return nil } - case "command": - if count == 1 && q.Command != nil && len(q.Command.Stdin) <= sandbox.MaxCommandInputBytes { - return nil - } - case "command_compute": - if count == 2 && q.Command != nil && q.Compute != nil && len(q.Command.Stdin) <= sandbox.MaxCommandInputBytes { - return nil - } case "suspend": if count == 1 && q.Suspend != nil && q.Suspend.Reference == q.Reference { return nil @@ -303,7 +286,6 @@ func execute(ctx context.Context, p sandbox.SandboxProvider, q request) response return out } var info sandbox.Info - var command sandbox.CommandResult switch q.Operation { case "observe": var sample runtimeobs.Sample @@ -320,9 +302,6 @@ func execute(ctx context.Context, p sandbox.SandboxProvider, q request) response out.Info = &info case "kill": err = p.Kill(ctx, q.Reference) - case "command": - command, err = p.RunCommand(ctx, q.Reference, *q.Command) - out.Command = &command default: var state sandbox.ComputeState var compute sandbox.Compute @@ -349,9 +328,6 @@ func execute(ctx context.Context, p sandbox.SandboxProvider, q request) response case "resume_compute": state, err = p.ResumeCompute(ctx, q.Reference, *q.Compute) out.State = &state - case "command_compute": - command, err = p.RunCommandCompute(ctx, q.Reference, *q.Compute, *q.Command) - out.Command = &command default: err = sandbox.ErrInvalid } @@ -367,7 +343,6 @@ func execute(ctx context.Context, p sandbox.SandboxProvider, q request) response out.Info = nil } out.State = nil - out.Command = nil out.Compute = nil } return out diff --git a/services/core/internal/sandbox/operations_test.go b/services/core/internal/sandbox/operations_test.go index de41670bd..d0d96d366 100644 --- a/services/core/internal/sandbox/operations_test.go +++ b/services/core/internal/sandbox/operations_test.go @@ -101,11 +101,6 @@ func TestBootstrapValidateRejections(t *testing.T) { } for name, change := range map[string]func(*sandbox.Bootstrap){ "noncanonical tenant": func(b *sandbox.Bootstrap) { b.TenantID = strings.ToUpper(b.TenantID) }, - "nil session": func(b *sandbox.Bootstrap) { b.SessionID = uuid.Nil.String() }, - "missing device": func(b *sandbox.Bootstrap) { b.DeviceID = "" }, - "Core URL off the API base": func(b *sandbox.Bootstrap) { b.CoreURL = "https://core.example" }, - "empty Runtime credential": func(b *sandbox.Bootstrap) { b.Credential = "" }, - "unknown network access": func(b *sandbox.Bootstrap) { b.NetworkAccess = "sometimes" }, "plain ws Link off loopback": func(b *sandbox.Bootstrap) { b.SandboxIO.LinkURL = "ws://core.example/api/v1/sandbox-link" }, "empty Serve credential": func(b *sandbox.Bootstrap) { b.SandboxIO.Credential = "" }, "zero generation": func(b *sandbox.Bootstrap) { b.SandboxIO.Resource.Generation = 0 }, diff --git a/services/core/internal/sandbox/runtime_bootstrap.go b/services/core/internal/sandbox/runtime_bootstrap.go deleted file mode 100644 index a4a05ecad..000000000 --- a/services/core/internal/sandbox/runtime_bootstrap.go +++ /dev/null @@ -1,12 +0,0 @@ -package sandbox - -import "github.com/MiniMax-AI/OpenAgentCore/internal/runtimebootstrap" - -// RuntimeConnection projects provider allocation input into Runtime's public -// startup contract. Providers must never construct a private auth profile. -func (b Bootstrap) RuntimeConnection() runtimebootstrap.Connection { - return runtimebootstrap.Connection{ - Version: runtimebootstrap.Version, CoreURL: b.CoreURL, - DeviceID: b.DeviceID, Credential: b.Credential, - } -} diff --git a/services/core/internal/sandbox/sandbox_provider.go b/services/core/internal/sandbox/sandbox_provider.go index bb7487ed7..b39185146 100644 --- a/services/core/internal/sandbox/sandbox_provider.go +++ b/services/core/internal/sandbox/sandbox_provider.go @@ -29,20 +29,16 @@ import ( "fmt" "reflect" - "github.com/google/uuid" - - "github.com/MiniMax-AI/OpenAgentCore/internal/agentnetwork" "github.com/MiniMax-AI/OpenAgentCore/internal/sandboxbootstrap" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/providercontract" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimeobs" ) var ( - ErrInvalid = errors.New("invalid sandbox configuration") - ErrOwnership = errors.New("sandbox ownership mismatch") - ErrExists = errors.New("sandbox allocation already exists") - ErrNotFound = errors.New("sandbox allocation not found") - ErrCommandUnconfirmed = errors.New("initialization command outcome unconfirmed; reclaim allocation before reuse") + ErrInvalid = errors.New("invalid sandbox configuration") + ErrOwnership = errors.New("sandbox ownership mismatch") + ErrExists = errors.New("sandbox allocation already exists") + ErrNotFound = errors.New("sandbox allocation not found") // ErrComputeUnconfirmed requires observation of the retained operation identity; // it does not authorize another Create, capture, restore, or cold start. ErrComputeUnconfirmed = errors.New("sandbox lifecycle outcome unconfirmed") @@ -53,63 +49,43 @@ var ( // an allocation; a lost Create response is resolved with GetInfo, never by replay. type Reference struct{ TenantID, EnvironmentID, AllocationID string } +// Bootstrap is Create's input. The Provider starts one process in the +// sandbox, the Sandbox I/O service, and SandboxIO is its launch input, which +// the Provider delivers as a private file (docs/sandbox-bootstrap.md). type Bootstrap struct { Reference - SessionID, DeviceID, CoreURL, Credential string - NetworkAccess string - AllowedDomains []string - // SandboxIO is the Sandbox I/O service's launch input, which the Provider - // delivers as a private file (docs/sandbox-bootstrap.md). SandboxIO sandboxbootstrap.Input } // Validate checks a Create input once, before Create: Providers deliver it -// as given. SandboxIO serves the Reference's allocation. +// as given. SandboxIO is valid and serves the Reference's allocation, so the +// Reference is valid too. func (b Bootstrap) Validate() error { - for _, id := range []string{b.TenantID, b.EnvironmentID, b.AllocationID, b.SessionID, b.DeviceID} { - if u, err := uuid.Parse(id); err != nil || u == uuid.Nil || u.String() != id { - return ErrInvalid - } - } resource := sandboxbootstrap.Resource{TenantID: b.TenantID, EnvironmentID: b.EnvironmentID, Kind: "allocation", ID: b.AllocationID, Generation: b.SandboxIO.Resource.Generation} - if b.RuntimeConnection().Validate() != nil || (agentnetwork.Policy{Access: b.NetworkAccess, AllowedDomains: b.AllowedDomains}).Validate() != nil || - b.SandboxIO.Validate() != nil || b.SandboxIO.Resource != resource { + if b.SandboxIO.Validate() != nil || b.SandboxIO.Resource != resource { return ErrInvalid } return nil } -// Info describes compute only. Running does not establish daemon authentication, -// native preparation, Environment readiness or a renewable provider lease. +// Info describes compute only. Running does not establish a Serving Link +// resource, Environment readiness or a renewable provider lease. type Info struct { Reference ProviderID, State string - // BootstrapComplete is provider evidence that initialization has reached its - // last mutating step. It does not establish daemon or native readiness. + // BootstrapComplete is provider evidence that the bootstrap has reached its + // last mutating step, starting Sandbox I/O. It does not establish Serving. BootstrapComplete bool // CreateSettled proves that the original create and initialization attempt can // no longer mutate resources. An absent observation needs this explicit proof; // an ordinary missing resource or empty provider listing is not sufficient. CreateSettled bool } -type Command struct { - Args []string - Directory string - // Stdin carries confidential initialization bytes without exposing them in argv. - Stdin []byte -} - -const MaxCommandInputBytes = 50*1024*1024 + 32 - -type CommandResult struct { - Stdout, Stderr string - ExitCode int -} // SandboxProvider manages one persisted Reference at a time. Every call has a // bounded context; cancellation ends the caller's wait, not proof of native stop. // Non-nil errors retain ownership, including partial results. Do not retry Create -// or RunCommand after unknown delivery; observe/reclaim the original Reference. +// after unknown delivery; observe/reclaim the original Reference. // Implementations verify installation plus Reference ownership before mutation. // See docs/sandbox-provider.md for settlement, cleanup and retry requirements. type SandboxProvider interface { @@ -124,7 +100,6 @@ type SandboxProvider interface { // Kill confirms removal of owned compute and retained resources. Absence is // idempotent, but nil alone cannot settle an outstanding Create. Kill(context.Context, Reference) error - RunCommand(context.Context, Reference, Command) (CommandResult, error) // The checkpoint lifecycle supplies exact-incarnation operations; Worker and // Store remain the lifecycle owner. Its operations share one declaration. @@ -135,7 +110,6 @@ type SandboxProvider interface { Resume(context.Context, ResumeRequest) (ComputeState, error) KillCompute(context.Context, Reference, Compute) error DeleteSnapshot(context.Context, Reference, SnapshotIdentity) error - RunCommandCompute(context.Context, Reference, Compute, Command) (CommandResult, error) // ResumeCompute thaws only the same resident instance after an aborted pause. ResumeCompute(context.Context, Reference, Compute) (ComputeState, error) @@ -145,11 +119,11 @@ type SandboxProvider interface { } // requiredOperations are supported by every Provider. -var requiredOperations = []string{"Create", "GetInfo", "Renew", "Kill", "RunCommand"} +var requiredOperations = []string{"Create", "GetInfo", "Renew", "Kill"} // checkpointOperations are all supported or all unsupported: partial cleanup or // restore support cannot safely own a compute incarnation. -var checkpointOperations = []string{"Initial", "NewCompute", "GetCompute", "Suspend", "Resume", "KillCompute", "DeleteSnapshot", "RunCommandCompute", "ResumeCompute"} +var checkpointOperations = []string{"Initial", "NewCompute", "GetCompute", "Suspend", "Resume", "KillCompute", "DeleteSnapshot", "ResumeCompute"} // Compute identifies one incarnation of an allocation. Name is provider-derived. // ID is empty only until the original create or restore result is observed. diff --git a/services/core/tests/e2b_native_isolation.py b/services/core/tests/e2b_native_isolation.py deleted file mode 100644 index 9b21c54c6..000000000 --- a/services/core/tests/e2b_native_isolation.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Run through a real native tool, with non-secret canaries supplied by acceptance.""" -import glob -import hashlib -import json -import os -import subprocess -import urllib.error -import urllib.request -from pathlib import Path - -fixture = json.loads(Path('/workspace/isolation-fixture.json').read_text()) -paths = ['/environment/staging/canary', - '/home/runtime/.oac/daemon/executor-key.json', - fixture['history_path'], - '/proc/1/root/home/runtime/.oac/daemon/executor-key.json', - '/root/.oac/e2b/ready.json', '/var/run/docker.sock'] - - -def read(path): - try: - with open(path, 'rb') as stream: - return stream.read(65536) - except OSError: - return None - - -result = {'private_readable': {path: read(path) is not None for path in paths}} -assert not any(result['private_readable'].values()), result -result['pid_namespace'] = os.readlink('/proc/self/ns/pid') -result['pid_namespace_isolated'] = result['pid_namespace'] != fixture['outer_pid_namespace'] -assert result['pid_namespace_isolated'], 'native tools share outer daemon PID namespace' -result['pid1_comm'] = (read('/proc/1/comm') or b'').decode(errors='replace').strip() -result['pid1_env_readable'] = read('/proc/1/environ') is not None -leaks = [] -for path in glob.glob('/proc/[0-9]*/environ') + glob.glob('/proc/[0-9]*/cmdline'): - data = read(path) - if data is None: - continue - for field in data.split(b'\x00'): - value = field.split(b'=', 1)[-1] - if hashlib.sha256(value).hexdigest() in fixture['secret_hashes']: - leaks.append(path) -result['sensitive_process_leaks'] = leaks -assert not leaks, 'protected outer process credential accessible' -for name, command in [('sudo', ['sudo', '-n', 'id', '-u']), - ('privileged_account', ['su', 'user', '-c', 'id -u'])]: - try: - process = subprocess.run(command, input='', capture_output=True, text=True, timeout=8) - result[name + '_denied'] = process.returncode != 0 - except FileNotFoundError: - result[name + '_unavailable'] = True - result[name + '_denied'] = True - assert result[name + '_denied'], 'native shell gained privileged account' -try: - urllib.request.urlopen('http://127.0.0.1:49983/envs', timeout=5) -except urllib.error.HTTPError as error: - assert error.code in [401, 403], error.code - result['envd_denied'] = True -except (urllib.error.URLError, PermissionError, TimeoutError): - result['envd_denied'] = True -else: - raise AssertionError('unauthenticated envd authority accessible') -result['passed'] = True -Path('/workspace/isolation-result.json').write_text(json.dumps(result)) -print('ISOLATION-PASSED') diff --git a/services/core/tests/integration/admin_session_archive_worker_http_test.go b/services/core/tests/integration/admin_session_archive_worker_http_test.go index 1c6bc829f..1005a35b5 100644 --- a/services/core/tests/integration/admin_session_archive_worker_http_test.go +++ b/services/core/tests/integration/admin_session_archive_worker_http_test.go @@ -34,7 +34,7 @@ func TestAdminSessionArchiveWorkerHTTPPostgres(t *testing.T) { provider := &lifecycleProvider{resources: map[string]sandbox.Info{}} deployments := deploymentService(t, s) providerConfig := func(setup deployment.Setup) *execution.RuntimeProvider { - return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider} + return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider} } configuration := execution.NewDeferredRuntimeProvider(installation, func(ctx context.Context) (*execution.RuntimeProvider, error) { setup, err := deployments.Setup(ctx) diff --git a/services/core/tests/integration/credential_matrix_http_test.go b/services/core/tests/integration/credential_matrix_http_test.go index c07b09f40..ad5ce68bd 100644 --- a/services/core/tests/integration/credential_matrix_http_test.go +++ b/services/core/tests/integration/credential_matrix_http_test.go @@ -100,9 +100,9 @@ func TestCredentialNamespaceMatrix(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, BackendFingerprint: setup.BackendFingerprint, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", Provider: provider}, nil + return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, BackendFingerprint: setup.BackendFingerprint, SandboxLink: "wss://core.example/api/v1/sandbox-link", Provider: provider}, nil }, func(_ context.Context, setup deployment.Setup) (execution.PreparedRuntimeDeployment, error) { - return execution.PreparedRuntimeDeployment{Config: &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider}}, nil + return execution.PreparedRuntimeDeployment{Config: &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: provider}}, nil }) worker := startWorker(t, ctx, s, &execution.Dispatcher{Registry: runtimegateway.NewRegistry(), ManagedRuntimes: runtimes}) var stop sync.Once diff --git a/services/core/tests/integration/hosted_initialization_failure_public_test.go b/services/core/tests/integration/hosted_initialization_failure_public_test.go index 481e723a9..c5b29a0b4 100644 --- a/services/core/tests/integration/hosted_initialization_failure_public_test.go +++ b/services/core/tests/integration/hosted_initialization_failure_public_test.go @@ -57,11 +57,11 @@ func hostedFailureSkill(t *testing.T) environmentconfig.Skill { type hostedFailureProvider struct { lifecycleProvider initializationPeer - fail string // runtime-initialize action, or "file" for the initial file writer - skip int // matching steps that succeed before the failure - result proto.RuntimePrepareResultPayload - err error - steps []string + fail string // runtime-initialize action, or "file" for the initial file writer + skip int // matching steps that succeed before the failure + result proto.RuntimePrepareResultPayload + unknown bool // the step's outcome is unconfirmed + steps []string } func (p *hostedFailureProvider) setRuntimeGateway(t *testing.T, s *Store, endpoint string, registry *runtimegateway.Registry, link *sandboxlinktest.Server) { @@ -75,9 +75,6 @@ func (p *hostedFailureProvider) Create(ctx context.Context, b sandbox.Bootstrap) } return info, err } -func (p *hostedFailureProvider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - return p.initializationPeer.RunCommand(ctx, r, c) -} func (p *hostedFailureProvider) prepare(request proto.RuntimePreparePayload, _ []byte) proto.RuntimePrepareResultPayload { action := request.Action if request.Initialization != nil { @@ -88,7 +85,7 @@ func (p *hostedFailureProvider) prepare(request proto.RuntimePreparePayload, _ [ p.steps = append(p.steps, action) if action == p.fail { if p.skip == 0 { - if p.err != nil { + if p.unknown { return proto.RuntimePrepareResultPayload{Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"} } return p.result @@ -131,10 +128,10 @@ func failHostedInitialization(t *testing.T, s *Store, key, tenant string, enviro func TestHostedInitializationFailureRecordsSafeSessionFailure(t *testing.T) { commands := []environmentconfig.SetupCommand{{Command: "echo " + hostedFailureCanary + "; exit 0"}, {Command: "echo " + hostedFailureCanary + "; exit 3"}, {Command: "touch never"}} type failure struct { - fail string - skip int - result proto.RuntimePrepareResultPayload - err error + fail string + skip int + result proto.RuntimePrepareResultPayload + unknown bool } for _, test := range []struct { name string @@ -156,7 +153,7 @@ func TestHostedInitializationFailureRecordsSafeSessionFailure(t *testing.T) { failure{fail: "setup", result: failedInitialization(0)}, "Failed to provision environment: initialization did not complete", []string{"configure", "setup"}}, {"unknown effect", sessions.CreateSession{Initialization: environmentconfig.Setup{Commands: commands[1:]}}, - failure{fail: "setup", err: sandbox.ErrCommandUnconfirmed}, + failure{fail: "setup", unknown: true}, "Failed to provision environment: initialization did not complete", []string{"configure", "setup"}}, {"invalid failure code", sessions.CreateSession{Initialization: environmentconfig.Setup{Commands: commands[1:]}}, failure{fail: "setup", result: proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: hostedFailureCanary}}, @@ -173,9 +170,9 @@ func TestHostedInitializationFailureRecordsSafeSessionFailure(t *testing.T) { tenant := uuid.NewString() session, environment := hostedFailureSession(t, s, tenant, test.input) p := &hostedFailureProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, - fail: test.p.fail, skip: test.p.skip, result: test.p.result, err: test.p.err} + fail: test.p.fail, skip: test.p.skip, result: test.p.result, unknown: test.p.unknown} failHostedInitialization(t, s, key, tenant, environment, p) - if !reflect.DeepEqual(p.steps, test.steps) || p.kills != 0 || p.commandCalls.Load() != 0 { + if !reflect.DeepEqual(p.steps, test.steps) || p.kills != 0 { t.Fatal("failed initialization continued or reclaimed compute", p.steps, p.kills) } diff --git a/services/core/tests/integration/provider_operations_fixture_test.go b/services/core/tests/integration/provider_operations_fixture_test.go index f528d5ae0..901b8b601 100644 --- a/services/core/tests/integration/provider_operations_fixture_test.go +++ b/services/core/tests/integration/provider_operations_fixture_test.go @@ -10,21 +10,19 @@ import ( func (*lifecycleProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "RunCommandCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "NewCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "GetCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Suspend": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Resume": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "KillCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "DeleteSnapshot": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "ResumeCompute": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } func (*lifecycleProvider) Initial(context.Context, sandbox.Reference) (sandbox.Compute, error) { @@ -48,9 +46,6 @@ func (*lifecycleProvider) KillCompute(context.Context, sandbox.Reference, sandbo func (*lifecycleProvider) DeleteSnapshot(context.Context, sandbox.Reference, sandbox.SnapshotIdentity) error { return &providercontract.UnsupportedError{Operation: "DeleteSnapshot", Reason: "fixture_operation_not_supported"} } -func (*lifecycleProvider) RunCommandCompute(context.Context, sandbox.Reference, sandbox.Compute, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, &providercontract.UnsupportedError{Operation: "RunCommandCompute", Reason: "fixture_operation_not_supported"} -} func (*lifecycleProvider) ResumeCompute(context.Context, sandbox.Reference, sandbox.Compute) (sandbox.ComputeState, error) { return sandbox.ComputeState{}, &providercontract.UnsupportedError{Operation: "ResumeCompute", Reason: "fixture_operation_not_supported"} } @@ -59,20 +54,18 @@ func (*lifecycleProvider) Observe(context.Context, runtimeobs.Target) (runtimeob } func (*fakeCheckpointProvider) ProviderOperations() providercontract.Operations { return providercontract.Operations{ - "Create": {State: providercontract.Supported}, - "GetInfo": {State: providercontract.Supported}, - "Renew": {State: providercontract.Supported}, - "Kill": {State: providercontract.Supported}, - "RunCommand": {State: providercontract.Supported}, - "Initial": {State: providercontract.Supported}, - "NewCompute": {State: providercontract.Supported}, - "GetCompute": {State: providercontract.Supported}, - "Suspend": {State: providercontract.Supported}, - "Resume": {State: providercontract.Supported}, - "KillCompute": {State: providercontract.Supported}, - "DeleteSnapshot": {State: providercontract.Supported}, - "RunCommandCompute": {State: providercontract.Supported}, - "ResumeCompute": {State: providercontract.Supported}, - "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, + "Create": {State: providercontract.Supported}, + "GetInfo": {State: providercontract.Supported}, + "Renew": {State: providercontract.Supported}, + "Kill": {State: providercontract.Supported}, + "Initial": {State: providercontract.Supported}, + "NewCompute": {State: providercontract.Supported}, + "GetCompute": {State: providercontract.Supported}, + "Suspend": {State: providercontract.Supported}, + "Resume": {State: providercontract.Supported}, + "KillCompute": {State: providercontract.Supported}, + "DeleteSnapshot": {State: providercontract.Supported}, + "ResumeCompute": {State: providercontract.Supported}, + "Observe": {State: providercontract.Unsupported, Reason: "fixture_operation_not_supported"}, } } diff --git a/services/core/tests/integration/runtime_compute_lifecycle_test.go b/services/core/tests/integration/runtime_compute_lifecycle_test.go index d595b9360..2cd74267b 100644 --- a/services/core/tests/integration/runtime_compute_lifecycle_test.go +++ b/services/core/tests/integration/runtime_compute_lifecycle_test.go @@ -248,9 +248,6 @@ func (p *fakeCheckpointProvider) ResumeCompute(ctx context.Context, r sandbox.Re p.serve(r.AllocationID) return state, nil } -func (p *fakeCheckpointProvider) RunCommandCompute(context.Context, sandbox.Reference, sandbox.Compute, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, errors.New("unexpected compute command") -} // connectHost connects the agent host unless it is connected; restart drops // its connection first, as a restarted agent host does. diff --git a/services/core/tests/integration/runtime_initialization_peer_test.go b/services/core/tests/integration/runtime_initialization_peer_test.go index 4579d0d6b..c2749d8f0 100644 --- a/services/core/tests/integration/runtime_initialization_peer_test.go +++ b/services/core/tests/integration/runtime_initialization_peer_test.go @@ -24,21 +24,20 @@ import ( // initialization runs, and has a fake sandbox Serve each bootstrap's Link // resource at link. type initializationPeer struct { - t *testing.T - endpoint string - registry *runtimegateway.Registry - link *sandboxlinktest.Server - host agentHost - tenant string // scopes the agent host setRuntimeGateway registers; see registerAgentHost - serving map[string]*linkServe // by Link resource ID - apply func(proto.RuntimePreparePayload, []byte) proto.RuntimePrepareResultPayload - writes atomic.Int32 - commandCalls atomic.Int32 - deferred bool - unavailable bool - bootstrap sandbox.Bootstrap - binds chan proto.AssignmentBindPayload // when not nil, receives each bind's payload - closeOnBind bool // close the socket at a bind instead of replying + t *testing.T + endpoint string + registry *runtimegateway.Registry + link *sandboxlinktest.Server + host agentHost + tenant string // scopes the agent host setRuntimeGateway registers; see registerAgentHost + serving map[string]*linkServe // by Link resource ID + apply func(proto.RuntimePreparePayload, []byte) proto.RuntimePrepareResultPayload + writes atomic.Int32 + deferred bool + unavailable bool + bootstrap sandbox.Bootstrap + binds chan proto.AssignmentBindPayload // when not nil, receives each bind's payload + closeOnBind bool // close the socket at a bind instead of replying } // setRuntimeGateway points the peer at the gateway and the relay, and @@ -118,10 +117,6 @@ func (p *initializationPeer) connect(b sandbox.Bootstrap) error { } return context.DeadlineExceeded } -func (p *initializationPeer) RunCommand(context.Context, sandbox.Reference, sandbox.Command) (sandbox.CommandResult, error) { - p.commandCalls.Add(1) - return sandbox.CommandResult{}, sandbox.ErrInvalid -} func completedInitialization(proto.RuntimePreparePayload, []byte) proto.RuntimePrepareResultPayload { return proto.RuntimePrepareResultPayload{Outcome: "completed"} } diff --git a/services/core/tests/integration/runtime_initialization_test.go b/services/core/tests/integration/runtime_initialization_test.go index dded2b3cf..ca5a908af 100644 --- a/services/core/tests/integration/runtime_initialization_test.go +++ b/services/core/tests/integration/runtime_initialization_test.go @@ -35,9 +35,6 @@ func (p *initializingProvider) Create(ctx context.Context, b sandbox.Bootstrap) } return info, err } -func (p *initializingProvider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - return p.initializationPeer.RunCommand(ctx, r, c) -} func TestEnvironmentInitializationCompletionUnknownAndRestart(t *testing.T) { for _, mode := range []string{"complete", "restart", "uncertain", "unavailable", "setup-complete", "setup-restart", "setup-uncertain"} { @@ -131,8 +128,8 @@ func TestEnvironmentInitializationCompletionUnknownAndRestart(t *testing.T) { p.mu.Lock() kills := p.kills p.mu.Unlock() - if kills != 0 || p.commandCalls.Load() != 0 { - t.Fatal("preparation changed compute lifecycle", kills, p.commandCalls.Load()) + if kills != 0 { + t.Fatal("preparation changed compute lifecycle", kills) } }) } @@ -187,8 +184,8 @@ func TestManagedRuntimePreparationAllOperationsUsePeer(t *testing.T) { actionsMu.Lock() defer actionsMu.Unlock() expected := []string{"file", "configure", "skill", "plugin", "npm", "python", "setup", "finalize"} - if !reflect.DeepEqual(actions, expected) || provider.commandCalls.Load() != 0 { - t.Fatal("typed ordering or provider isolation", actions, provider.commandCalls.Load()) + if !reflect.DeepEqual(actions, expected) { + t.Fatal("typed ordering", actions) } allocation, err := deploymentStore(s).EnvironmentAllocation(t.Context(), deployment.AllocationKey{TenantID: tenant, EnvironmentID: environment.ID}) if err != nil || initializationState(t, s, allocation.TenantID, allocation.EnvironmentID) != "complete" { diff --git a/services/core/tests/integration/runtime_lifecycle_test.go b/services/core/tests/integration/runtime_lifecycle_test.go index d4cfc41e6..614511b9c 100644 --- a/services/core/tests/integration/runtime_lifecycle_test.go +++ b/services/core/tests/integration/runtime_lifecycle_test.go @@ -16,7 +16,6 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/internal/sandboxlink/sandboxlinktest" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/deployment" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/execution" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimedevice" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/runtimegateway" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sessions" @@ -28,8 +27,6 @@ type lifecycleProvider struct { resources map[string]sandbox.Info creates, kills, gets int loseCreate, absent, unavailable bool - credentialHash string - credential string serve sandboxbootstrap.Input } @@ -37,8 +34,6 @@ func (p *lifecycleProvider) Create(_ context.Context, b sandbox.Bootstrap) (sand p.mu.Lock() defer p.mu.Unlock() p.creates++ - p.credentialHash = runtimedevice.HashCredential(b.Credential) - p.credential = b.Credential p.serve = b.SandboxIO i := sandbox.Info{Reference: b.Reference, ProviderID: b.AllocationID, State: "running", BootstrapComplete: true} if !p.absent { @@ -72,9 +67,6 @@ func (p *lifecycleProvider) Kill(_ context.Context, r sandbox.Reference) error { delete(p.resources, r.AllocationID) return nil } -func (p *lifecycleProvider) RunCommand(context.Context, sandbox.Reference, sandbox.Command) (sandbox.CommandResult, error) { - return sandbox.CommandResult{}, errors.New("not used") -} // managedWorker starts a Worker that runs the Web setup webDeployment // committed for installation key on p. @@ -165,10 +157,6 @@ func TestManagedRuntimeLostCreateRestartAndDeletion(t *testing.T) { if err == nil || owner.ID == "" { t.Fatal("fault did not retain allocation") } - credential, ok, err := sessionAdapter(s).GetDeviceCredential(t.Context(), owner.DeviceID) - if err != nil || !ok || credential.CredentialHash != p.credentialHash { - t.Fatal("provider received unbound credential") - } stop() next, _ := managedWorker(t, s, key, p) reconcileManagedState(t, next, s, tenant, env.ID, "running") diff --git a/services/core/tests/integration/runtime_node_lifecycle_fixture_test.go b/services/core/tests/integration/runtime_node_lifecycle_fixture_test.go index f20421499..6f7f07df4 100644 --- a/services/core/tests/integration/runtime_node_lifecycle_fixture_test.go +++ b/services/core/tests/integration/runtime_node_lifecycle_fixture_test.go @@ -56,9 +56,6 @@ func (p *nodeIsolationProvider) GetCompute(ctx context.Context, r sandbox.Refere } return p.fakeCheckpointProvider.GetCompute(ctx, r, c) } -func (p *nodeIsolationProvider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - return p.preparation.RunCommand(ctx, r, c) -} type nodeIsolationFixture struct { t *testing.T diff --git a/services/core/tests/integration/runtime_node_lifecycle_test.go b/services/core/tests/integration/runtime_node_lifecycle_test.go index 14eb9c336..950e72104 100644 --- a/services/core/tests/integration/runtime_node_lifecycle_test.go +++ b/services/core/tests/integration/runtime_node_lifecycle_test.go @@ -124,9 +124,6 @@ func TestManagedNodesIsolateBlockedProviderAndInitialization(t *testing.T) { f.provider.mu.Lock() restores := f.provider.restores f.provider.mu.Unlock() - if f.provider.preparation.commandCalls.Load() != 0 { - t.Fatal("initialization invoked Provider.RunCommand") - } if restores != 1 || f.provider.promptFrames.Load() != 0 { t.Fatal("restore replayed or lifecycle sent model work") } diff --git a/services/core/tests/integration/sandbox_deployment_switch_worker_test.go b/services/core/tests/integration/sandbox_deployment_switch_worker_test.go index caf7f5bb1..7656e50ff 100644 --- a/services/core/tests/integration/sandbox_deployment_switch_worker_test.go +++ b/services/core/tests/integration/sandbox_deployment_switch_worker_test.go @@ -31,13 +31,13 @@ func TestSandboxWorkerSwitchesAndRecoversFailedActivation(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &execution.RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p}, nil + return &execution.RuntimeProvider{InstallationID: id, ProviderKind: setup.Provider, Generation: setup.Generation, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p}, nil }, func(ctx context.Context, setup deployment.Setup) (execution.PreparedRuntimeDeployment, error) { preparations.Add(1) if fail.Load() { return execution.PreparedRuntimeDeployment{}, errors.New("fixture provider unavailable") } - return execution.PreparedRuntimeDeployment{Config: &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p}}, nil + return execution.PreparedRuntimeDeployment{Config: &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p}}, nil }) w := startWorker(t, t.Context(), s, &execution.Dispatcher{Registry: runtimegateway.NewRegistry(), ManagedRuntimes: configuration}) ctx, cancel := context.WithCancel(t.Context()) diff --git a/services/core/tests/integration/sandbox_deployment_worker_test.go b/services/core/tests/integration/sandbox_deployment_worker_test.go index bd09da501..ff052e1c0 100644 --- a/services/core/tests/integration/sandbox_deployment_worker_test.go +++ b/services/core/tests/integration/sandbox_deployment_worker_test.go @@ -27,10 +27,10 @@ func TestSandboxDeploymentWorkerActivatesWithoutRestart(t *testing.T) { if err != nil || setup.Provider == "" { return nil, err } - return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, BackendFingerprint: setup.BackendFingerprint, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", Provider: p}, nil + return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, BackendFingerprint: setup.BackendFingerprint, SandboxLink: "wss://core.example/api/v1/sandbox-link", Provider: p}, nil }, func(ctx context.Context, setup deployment.Setup) (execution.PreparedRuntimeDeployment, error) { - return execution.PreparedRuntimeDeployment{Config: &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, CoreURL: "https://core.example/api/v1", SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p}}, nil + return execution.PreparedRuntimeDeployment{Config: &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, SandboxLink: "wss://core.example/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p}}, nil }) start := func() (*execution.Worker, func()) { t.Helper() diff --git a/services/core/tests/integration/worker_fixture_test.go b/services/core/tests/integration/worker_fixture_test.go index 0a071b07b..ba1134072 100644 --- a/services/core/tests/integration/worker_fixture_test.go +++ b/services/core/tests/integration/worker_fixture_test.go @@ -193,7 +193,7 @@ func webRuntimes(t testing.TB, s *Store, installation string, p sandbox.SandboxP return nil, err } return &execution.RuntimeProvider{InstallationID: setup.InstallationID, ProviderKind: setup.Provider, Mode: setup.Mode, Generation: setup.Generation, - CoreURL: "http://core.invalid/api/v1", SandboxLink: "wss://core.invalid/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p, Suspension: suspension}, nil + SandboxLink: "wss://core.invalid/api/v1/sandbox-link", BackendFingerprint: setup.BackendFingerprint, Provider: p, Suspension: suspension}, nil }, unusedPreparation(t)) } diff --git a/services/core/tests/official_environment_skills.py b/services/core/tests/official_environment_skills.py index 579bbcb43..48af5137b 100644 --- a/services/core/tests/official_environment_skills.py +++ b/services/core/tests/official_environment_skills.py @@ -19,8 +19,6 @@ def inline_skill(): from pathlib import Path for name in ['ANTHROPIC_API_KEY', 'ANTHROPIC_AUTH_TOKEN', 'OPENAI_API_KEY', 'MINIMAX_API_KEY']: assert name not in os.environ, 'native credential reached a Skill helper' -for path in ['/environment/staging/initial-files-private-canary', '/home/runtime/.oac/daemon/default/auth.json']: - assert not os.access(path, os.R_OK), 'private Runtime state reached a Skill helper' manifest = Path('/environment/initialization/capabilities/skills/proof-skill/SKILL.md') try: manifest.write_text('tampered') diff --git a/services/core/tests/official_user_runtime.py b/services/core/tests/official_user_runtime.py index 9101634b0..9be79a09f 100644 --- a/services/core/tests/official_user_runtime.py +++ b/services/core/tests/official_user_runtime.py @@ -2,8 +2,8 @@ run_acceptance must run on the main thread of the Linux operator process. The supplied Session is unused and its /workspace/outputs directory is empty. -The operator creates nonempty .user-runtime-isolation-canary files under -/home/runtime/.oac/daemon and /environment/staging before calling. +The operator creates a nonempty .user-runtime-isolation-canary file under +/home/runtime/.oac/daemon before calling. The actual executor-key.json must remain under that protected daemon root. runtime.read(path) returns bytes within a bounded timeout, raising FileNotFoundError only for absence. runtime.restart() preserves workspace and native state and waits for reconnect. @@ -74,8 +74,7 @@ def run_acceptance(client, foreign, http, session, runtime, evidence_path, secre "/workspace/outputs/b.txt": ("native-user-runtime-" + nonce + "\n").encode()} artifacts = {} private_paths = [protected_credential_path, - "/home/runtime/.oac/daemon/.user-runtime-isolation-canary", - "/environment/staging/.user-runtime-isolation-canary"] + "/home/runtime/.oac/daemon/.user-runtime-isolation-canary"] def private_hashes(): values = {} diff --git a/services/core/tools/e2b-provider/README.md b/services/core/tools/e2b-provider/README.md index 98f746b5c..7cbdc888a 100644 --- a/services/core/tools/e2b-provider/README.md +++ b/services/core/tools/e2b-provider/README.md @@ -1,6 +1,6 @@ # E2B Sandbox Provider helper -Core's E2B Sandbox Provider ([`sandbox/e2b`](../../internal/sandbox/e2b)) is a pure-Go adapter that runs this one-shot Python helper for each lifecycle and read operation. The helper uses the official E2B Python SDK 2.51.0 ([`requirements.lock`](requirements.lock)); it implements no provider HTTP, envd RPC, scheduler or network service. E2B uses direct placement: there is no node, and each sandbox's daemon connects to Core over the public URL. Runtime execution and Files use that daemon connection. [Add a Sandbox Provider](../../../../docs/sandbox-provider.md) owns the provider contract this adapter implements. +Core's E2B Sandbox Provider ([`sandbox/e2b`](../../internal/sandbox/e2b)) is a pure-Go adapter that runs this one-shot Python helper for each lifecycle and read operation. The helper uses the official E2B Python SDK 2.51.0 ([`requirements.lock`](requirements.lock)); it implements no provider HTTP, envd RPC, scheduler or network service. E2B uses direct placement: there is no node, and each sandbox's Sandbox I/O service connects to Core's [Sandbox link](../../../../docs/sandbox-link-protocol.md) over the public URL. [Add a Sandbox Provider](../../../../docs/sandbox-provider.md) owns the provider contract this adapter implements. ## Deployment @@ -16,7 +16,6 @@ The account key is stored encrypted in Core's database and is write-only. It rea | `inspect` | `GetInfo` | Reads the sandbox by recorded ID, or by ownership metadata when no ID is recorded, and checks ownership, domain, template and resources | | `renew` | `Renew` | Extends the lease of the running sandbox to the configured timeout, then rereads it | | `kill` | `Kill` | Destroys every matching sandbox and confirms that none remains | -| `command` | `RunCommand` | Runs one bounded command as the Runtime user on a running sandbox whose bootstrap completed; output is limited to 1 MiB per stream | | `validate_deployment` | Deployment setup | Reads the template's builds and requires the exact build to be ready with the configured CPU and memory. Without configured resources the selection adopts the build's CPU and memory. Returns the build's status, CPU, memory and reported disk for Core to record; bounded to 30 seconds | | `list_templates`, `list_builds` | [Configuration discovery](../../../../contracts/agents-api/sandbox-deployment.md#configuration-discovery) | Pages the key's visible templates (`GET /v2/templates`) or one template's ready builds, with a transient key. Results are capped at 200 and write no receipt | | `observe` | Runtime observations | One allocation; see [Observations](#observations) | @@ -26,7 +25,7 @@ Compatible endpoints must return the SDK 2.51.0 template-list and template-build ## Private JSON boundary -[`helper_contract.go`](../../internal/sandbox/e2b/helper_contract.go) owns the adapter-private wire types, version, operation and error vocabulary, and bounds. Its generator projects Python declarations into the helper and template sources, deriving managed-bootstrap fields from the Sandbox Provider types, network access values from `agentnetwork.Policy.Validate`, and the SDK version from the hashed dependency lock. The command-input limit comes from its shared Go contract. The generated modules have no SDK or repository dependency and ship with the frozen helper and protected template startup scripts. +[`helper_contract.go`](../../internal/sandbox/e2b/helper_contract.go) owns the adapter-private wire types, version, operation and error vocabulary, and bounds. Its generator projects Python declarations into the helper and template sources, deriving managed-bootstrap fields from the Sandbox Provider types and the SDK version from the hashed dependency lock. The generated modules have no SDK or repository dependency and ship with the frozen helper and protected template startup scripts. Run `go generate ./services/core/internal/sandbox/e2b` from the repository root after changing these declarations. `make check-e2b-provider` and the Go adapter tests reject stale projections; both languages consume generated valid and invalid exchanges covering wire types, extra fields, operation/reference bounds and managed-bootstrap fields. The helper build copies those fixtures with its source before running the pinned-SDK suite. @@ -44,7 +43,7 @@ A helper holds its allocation's lock until the SDK operation returns, even after 2. Record the sandbox ID and connection material, check the sandbox domain, then read the sandbox by ID and check its ownership metadata, template and resources before writing any credential. A mismatch records a settled rejection and returns `CreateSettled` with the error. 3. Check that `/opt/oac-e2b/managed_init.py` is readable, write the managed bootstrap input to `/root/.oac/e2b/managed-bootstrap.json` and run `managed_init.py` as root. -`managed_init.py` prepares the image as the [application-managed startup](../../deploy/e2b/README.md#startup-and-security-boundary) does, writes the [Runtime bootstrap](../../../../docs/runtime-bootstrap.md) file to `/home/runtime/runtime-bootstrap.json` (mode 0600, owned by UID 1000), sets the Environment, Session and network variables and starts `oac-daemon connect --profile default --bootstrap-file /home/runtime/runtime-bootstrap.json` as UID/GID 1000. It writes the [Sandbox bootstrap](../../../../docs/sandbox-bootstrap.md) file to `/home/runtime/sandbox-io-bootstrap.json` the same way and starts `oac-sandbox-io --bootstrap-file /home/runtime/sandbox-io-bootstrap.json` as UID/GID 1000 with an empty environment. It records process handoff in `/root/.oac/e2b/managed-ready.json` and refuses to run again once any launch record exists. `BootstrapComplete` becomes true when a later inspection reads that record with the expected identity; it does not prove enrollment or native readiness. +`managed_init.py` runs once as root. It restores the ownership and modes that E2B finalization changes under `/usr/local` and on `envd`, `/etc/inittab` and `/etc/init.d/rcS`, locks E2B's passwordless `user` account and bind-mounts `/environment/workspace` at `/workspace`. It writes the [Sandbox bootstrap](../../../../docs/sandbox-bootstrap.md) file to `/home/runtime/sandbox-io-bootstrap.json` (mode 0600, owned by UID 1000), deletes the startup input and starts `oac-sandbox-io --bootstrap-file /home/runtime/sandbox-io-bootstrap.json` as UID/GID 1000 with an empty environment; it starts no other process. It records the process handoff in `/root/.oac/e2b/managed-ready.json` (`status` `sandbox_io_started`, with the service's PID) and refuses to run again once a launch record exists. E2B clears `/run` at boot, so the records live under `/root/.oac/e2b`. `BootstrapComplete` becomes true when a later inspection reads that record with the expected identity; it does not prove that the sandbox Serves. The E2B VM is the isolation boundary ([Runtime and outer isolation](../../../../docs/concepts.md#runtime-and-outer-isolation)). An unknown Create is never repeated. A Create whose connection material was lost can be discovered and destroyed but cannot resume bootstrap, and an unconfirmed startup requires reclaiming the whole allocation. diff --git a/services/core/tools/e2b-provider/contract_test.py b/services/core/tools/e2b-provider/contract_test.py index a8d07ba40..cfbd98de2 100644 --- a/services/core/tools/e2b-provider/contract_test.py +++ b/services/core/tools/e2b-provider/contract_test.py @@ -48,7 +48,7 @@ def test_request_and_response_byte_bounds(self): output = io.StringIO() source = io.BytesIO(b' ' * (contract.MAX_REQUEST + 1) if oversized_request else b'{}') result = {'Version': contract.PROTOCOL_VERSION, 'ErrorCode': '', - 'Command': {'Stdout': 'x' * contract.MAX_RESPONSE}} + 'Templates': [{'Name': 'x' * contract.MAX_RESPONSE}]} provider = Mock(return_value=Mock(execute=Mock(return_value=result))) with self.subTest(request=oversized_request), patch.object(main.sys, 'argv', ['helper']), \ patch.object(main.sys, 'stdin', SimpleNamespace(buffer=source)), \ @@ -61,13 +61,8 @@ def test_request_and_response_byte_bounds(self): if oversized_request: provider.assert_not_called() - def test_command_limits_count_bytes(self): + def test_startup_output_limit_counts_bytes(self): client = Mock() - with patch.object(sdk.base64, 'b64decode', return_value=b'x' * (contract.MAX_COMMAND_INPUT + 1)): - with self.assertRaises(Failure) as raised: - sdk.run(client, {'Args': ['true'], 'Stdin': 'encoded'}, lambda: 3) - self.assertEqual(raised.exception.code, 'invalid') - client.commands.run.assert_not_called() for extra in ('', 'é'): text = 'x' * contract.MAX_OUTPUT + extra def wait(**callbacks): @@ -76,10 +71,10 @@ def wait(**callbacks): client.commands.run.return_value.wait.side_effect = wait if extra: with self.assertRaises(Failure) as raised: - sdk.run(client, {'Args': ['true']}, lambda: 3) - self.assertEqual(raised.exception.code, 'command_unconfirmed') + sdk.run(client, ['true'], lambda: 3) + self.assertEqual(raised.exception.code, 'unconfirmed') else: - self.assertEqual(sdk.run(client, {'Args': ['true']}, lambda: 3)['Stdout'], text) + self.assertEqual(sdk.run(client, ['true'], lambda: 3), 0) if __name__ == '__main__': diff --git a/services/core/tools/e2b-provider/helper_contract_generated.py b/services/core/tools/e2b-provider/helper_contract_generated.py index fa90c0762..87139c0e7 100644 --- a/services/core/tools/e2b-provider/helper_contract_generated.py +++ b/services/core/tools/e2b-provider/helper_contract_generated.py @@ -1,17 +1,15 @@ # Code generated by contractgen; DO NOT EDIT. """Adapter-private wire declarations. No SDK or repository dependency.""" -ERROR_CODES = ["","invalid","ownership","exists","not_found","command_unconfirmed","unconfirmed","template_invalid","team_mismatch","unauthorized"] -MANAGED_BOOTSTRAP_FIELDS = ["TenantID","EnvironmentID","AllocationID","SessionID","DeviceID","NetworkAccess","AllowedDomains","SandboxIO","InstallationID","RuntimeBootstrap"] -MANAGED_IDENTITY_FIELDS = ["TenantID","EnvironmentID","AllocationID","SessionID","DeviceID","InstallationID"] -MAX_COMMAND_INPUT = 52428832 +ERROR_CODES = ["","invalid","ownership","exists","not_found","unconfirmed","template_invalid","team_mismatch","unauthorized"] +MANAGED_BOOTSTRAP_FIELDS = ["TenantID","EnvironmentID","AllocationID","SandboxIO","InstallationID"] +MANAGED_IDENTITY_FIELDS = ["TenantID","EnvironmentID","AllocationID","InstallationID"] MAX_CREDENTIAL_REFERENCES = 32 MAX_OUTPUT = 1048576 -MAX_REQUEST = 75497472 +MAX_REQUEST = 1048576 MAX_RESPONSE = 16777216 -NETWORK_ACCESS = ["enabled","disabled","restricted"] -OPERATIONS = ["create","inspect","renew","kill","command","validate_deployment","observe","list_templates","list_builds","verify_credential"] +OPERATIONS = ["create","inspect","renew","kill","validate_deployment","observe","list_templates","list_builds","verify_credential"] PROTOCOL_VERSION = 1 REFERENCE_FIELDS = ["TenantID","EnvironmentID","AllocationID"] -REQUEST_FIELDS = ["Version","Operation","Config","Reference","References","Bootstrap","RuntimeBootstrap","Command","Deadline"] -RESPONSE_FIELDS = ["Version","Info","Command","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observation"] +REQUEST_FIELDS = ["Version","Operation","Config","Reference","References","Bootstrap","Deadline"] +RESPONSE_FIELDS = ["Version","Info","ErrorCode","DeploymentValid","TemplateBuild","Templates","Builds","Observation"] SDK_VERSION = "2.51.0" diff --git a/services/core/tools/e2b-provider/provider.py b/services/core/tools/e2b-provider/provider.py index 9b2fcafe7..1bacb6c53 100644 --- a/services/core/tools/e2b-provider/provider.py +++ b/services/core/tools/e2b-provider/provider.py @@ -1,4 +1,4 @@ -"""Five bounded SDK operations for an already authorized Core allocation, plus +"""Four bounded SDK operations for an already authorized Core allocation, plus read-only deployment validation and observation.""" from concurrent.futures import ThreadPoolExecutor import json @@ -194,8 +194,8 @@ def inspect(self): receipt = None if receipt is not None: expected = record.get('bootstrap_identity') - if (receipt.get('identity') != expected or receipt.get('status') != 'daemon_started' or - type(receipt.get('daemon_pid')) is not int or receipt['daemon_pid'] <= 0): + if (receipt.get('identity') != expected or receipt.get('status') != 'sandbox_io_started' or + type(receipt.get('sandbox_io_pid')) is not int or receipt['sandbox_io_pid'] <= 0): raise Failure('ownership') self.receipt.save(settled=True, bootstrap_complete=True) return cloud @@ -206,8 +206,7 @@ def create(self): bootstrap = self.q['Bootstrap'] if any(bootstrap.get(field) != value for field, value in self.reference.items()): raise Failure('invalid') - identity = dict(self.reference, InstallationID=self.config['InstallationID'], - SessionID=bootstrap['SessionID'], DeviceID=bootstrap['DeviceID']) + identity = dict(self.reference, InstallationID=self.config['InstallationID']) self.receipt.save(status='create_pending', bootstrap_identity=identity) try: cloud = Sandbox.create(template=self.config['Template'], timeout=self.config['TimeoutSeconds'], @@ -230,23 +229,19 @@ def create(self): self.receipt.save(status='configuration_rejected', settled=True) raise # Validate the current template entry point before writing any credential. - check = run(cloud, {'Args': ['/usr/bin/python3', '-I', '-c', - "import os,sys; sys.exit(78 if not os.path.isfile('/opt/oac-e2b/managed_init.py') or not os.access('/opt/oac-e2b/managed_init.py', os.R_OK) else 0)"]}, - self.remaining, user='root') - if check['ExitCode'] != 0: + check = run(cloud, ['/usr/bin/python3', '-I', '-c', + "import os,sys; sys.exit(78 if not os.path.isfile('/opt/oac-e2b/managed_init.py') or not os.access('/opt/oac-e2b/managed_init.py', os.R_OK) else 0)"], + self.remaining) + if check != 0: self.receipt.save(status='bootstrap_failed', settled=True) - raise Failure('template_invalid' if check['ExitCode'] == 78 else 'unconfirmed') - payload = dict(bootstrap, InstallationID=self.config['InstallationID'], - RuntimeBootstrap=self.q['RuntimeBootstrap']) - del payload['CoreURL'], payload['Credential'] + raise Failure('template_invalid' if check == 78 else 'unconfirmed') + payload = dict(bootstrap, InstallationID=self.config['InstallationID']) if set(payload) != set(MANAGED_BOOTSTRAP_FIELDS): raise Failure('invalid') cloud.files.write('/root/.oac/e2b/managed-bootstrap.json', json.dumps(payload), user='root', request_timeout=self.remaining()) self.receipt.save(status='bootstrap_pending') - result = run(cloud, {'Args': ['/usr/bin/python3', '-I', '/opt/oac-e2b/managed_init.py']}, - self.remaining, user='root') - if result['ExitCode'] != 0: + if run(cloud, ['/usr/bin/python3', '-I', '/opt/oac-e2b/managed_init.py'], self.remaining) != 0: self.receipt.save(status='bootstrap_failed', settled=True) raise Failure('unconfirmed') self.receipt.save(status='bootstrap_exited', settled=True) @@ -381,12 +376,6 @@ def execute(self): if operation == 'kill': self.kill() return {'Version': PROTOCOL_VERSION, 'Info': self.info(absent=True), 'ErrorCode': ''} - if operation == 'command': - cloud = self.inspect() - if cloud.state != 'running' or not self.receipt.data.get('bootstrap_complete'): - raise Failure('unconfirmed') - result = run(self.client(cloud), self.q['Command'], self.remaining) - return {'Version': PROTOCOL_VERSION, 'Command': result, 'ErrorCode': ''} cloud = {'create': self.create, 'inspect': self.inspect, 'renew': self.renew}[operation]() return {'Version': PROTOCOL_VERSION, 'Info': self.info(cloud, absent=cloud is None), 'ErrorCode': ''} except Failure as error: diff --git a/services/core/tools/e2b-provider/provider_test.py b/services/core/tools/e2b-provider/provider_test.py index 316af0e70..beac7f681 100644 --- a/services/core/tools/e2b-provider/provider_test.py +++ b/services/core/tools/e2b-provider/provider_test.py @@ -26,25 +26,18 @@ def setUp(self): 'APIKey': 'private-account-secret', 'Template': 'test:' + str(uuid4()), 'TimeoutSeconds': 120} self.request = {'Version': 1, 'Operation': 'create', 'Config': self.config, 'Reference': self.reference, 'Deadline': (datetime.now(timezone.utc) + timedelta(seconds=30)).isoformat(), - 'Bootstrap': dict(self.reference, SessionID=str(uuid4()), DeviceID=str(uuid4()), - CoreURL='https://core.example/api/v1', Credential='private-runtime-secret', - NetworkAccess='enabled', AllowedDomains=[], + 'Bootstrap': dict(self.reference, SandboxIO={'version': 1, 'link_url': 'wss://core.example/api/v1/sandbox-link', 'credential': 'private-serve-secret', 'resource': {'tenant_id': self.reference['TenantID'], 'environment_id': self.reference['EnvironmentID'], 'kind': 'allocation', 'id': self.reference['AllocationID'], 'generation': 1}})} - self.request['RuntimeBootstrap'] = { - 'version': 1, 'core_url': self.request['Bootstrap']['CoreURL'], - 'device_id': self.request['Bootstrap']['DeviceID'], - 'credential': self.request['Bootstrap']['Credential']} self.cloud = Mock(sandbox_id='owned-id', sandbox_domain='e2b.app', _envd_version='0.5.0', _envd_access_token='private-envd-secret', traffic_access_token=None, state='running') self.cloud.metadata = Provider(self.request).metadata self.cloud.template_id = self.config['Template'] - self.identity = dict(self.reference, InstallationID=self.config['InstallationID'], - SessionID=self.request['Bootstrap']['SessionID'], DeviceID=self.request['Bootstrap']['DeviceID']) - self.ready = json.dumps({'identity': self.identity, 'status': 'daemon_started', 'daemon_pid': 123}) + self.identity = dict(self.reference, InstallationID=self.config['InstallationID']) + self.ready = json.dumps({'identity': self.identity, 'status': 'sandbox_io_started', 'sandbox_io_pid': 123}) self.cloud.files.read.return_value = self.ready self.api = Mock() self.api.create.return_value = self.cloud @@ -56,7 +49,7 @@ def setUp(self): self.runtime = patch('provider.restore', return_value=self.cloud) self.runtime.start() self.addCleanup(self.runtime.stop) - self.command = patch('provider.run', return_value={'Stdout': '', 'Stderr': '', 'ExitCode': 0}) + self.command = patch('provider.run', return_value=0) self.command.start() self.addCleanup(self.command.stop) @@ -72,10 +65,7 @@ def test_create_recover_and_never_replay(self): self.assertTrue(result['Info']['BootstrapComplete']) self.assertTrue(result['Info']['CreateSettled']) startup = json.loads(self.cloud.files.write.call_args.args[1]) - self.assertEqual(startup['RuntimeBootstrap'], self.request['RuntimeBootstrap']) - self.assertEqual(startup['SandboxIO'], self.request['Bootstrap']['SandboxIO']) - self.assertNotIn('CoreURL', startup) - self.assertNotIn('Credential', startup) + self.assertEqual(startup, dict(self.request['Bootstrap'], InstallationID=self.config['InstallationID'])) self.assertEqual(self.call('create')['ErrorCode'], 'exists') self.assertTrue(self.call('inspect')['Info']['BootstrapComplete']) self.api.create.assert_called_once() @@ -85,7 +75,6 @@ def test_create_recover_and_never_replay(self): self.assertEqual(kwargs['metadata'], self.cloud.metadata) serialized = json.dumps(self.record()) self.assertNotIn(self.config['APIKey'], serialized) - self.assertNotIn(self.request['Bootstrap']['Credential'], serialized) self.assertNotIn(self.request['Bootstrap']['SandboxIO']['credential'], serialized) self.assertEqual(self.record()['connection']['envd_access_token'], 'private-envd-secret') self.api.connect.assert_not_called() @@ -152,7 +141,7 @@ def test_create_refuses_foreign_data_plane_before_envd(self): self.cloud.commands.run.assert_not_called() def test_template_invalid_refuses_before_credentials_and_retains_owned_cleanup(self): - with patch('provider.run', return_value={'ExitCode': 78, 'Stdout': '', 'Stderr': ''}): + with patch('provider.run', return_value=78): result = self.call('create') self.assertEqual(result['ErrorCode'], 'template_invalid') self.assertTrue(result['Info']['CreateSettled']) @@ -234,14 +223,14 @@ def test_unknown_bootstrap_only_settles_after_exact_kill(self): self.api.kill.assert_called_once_with('owned-id', **self.api.kill.call_args.kwargs) def test_bootstrap_failure_is_settled_but_not_ready(self): - with patch('provider.run', return_value={'ExitCode': 1}): + with patch('provider.run', return_value=1): result = self.call('create') self.assertTrue(result['Info']['CreateSettled']) self.assertFalse(result['Info']['BootstrapComplete']) self.assertEqual(self.call('inspect')['Info']['State'], 'running') def test_mismatched_receipt_cannot_prove_bootstrap_complete(self): - self.cloud.files.read.return_value = json.dumps({'identity': {}, 'status': 'daemon_started', 'daemon_pid': 2}) + self.cloud.files.read.return_value = json.dumps({'identity': {}, 'status': 'sandbox_io_started', 'sandbox_io_pid': 2}) self.assertEqual(self.call('create')['ErrorCode'], 'ownership') self.assertTrue(self.record()['settled']) self.assertFalse(self.record()['bootstrap_complete']) @@ -309,24 +298,19 @@ def test_constructor_restores_without_a_control_plane_operation(self): self.assertEqual(client.connection_config.sandbox_headers['X-Access-Token'], material['envd_access_token']) self.assertEqual(client.connection_config.retries, 0) - def test_stdin_is_separate_from_quoted_command_and_receives_eof(self): - import base64 + def test_startup_step_is_quoted_and_returns_only_its_exit_code(self): client = Mock() - client.commands.run.return_value.pid = 23 client.commands.run.return_value.wait.return_value = SimpleNamespace(stdout='ok', stderr='', exit_code=7) - result = run(client, {'Args': ['cat', 'a;touch /unwanted'], 'Directory': '/workspace', - 'Stdin': base64.b64encode(b'private-input').decode()}, lambda: 3) - self.assertEqual(result['ExitCode'], 7) + self.assertEqual(run(client, ['cat', 'a;touch /unwanted'], lambda: 3), 7) self.assertEqual(client.commands.run.call_args.args, ("cat 'a;touch /unwanted'",)) - client.commands.send_stdin.assert_called_once_with(23, b'private-input', request_timeout=3) - client.commands.close_stdin.assert_called_once_with(23, request_timeout=3) + self.assertEqual(client.commands.run.call_args.kwargs['user'], 'root') - def test_unknown_command_does_not_replay(self): + def test_unknown_startup_step_does_not_replay(self): client = Mock() client.commands.run.side_effect = TimeoutError('private command') with self.assertRaises(Failure) as result: - run(client, {'Args': ['true']}, lambda: 3) - self.assertEqual(result.exception.code, 'command_unconfirmed') + run(client, ['true'], lambda: 3) + self.assertEqual(result.exception.code, 'unconfirmed') client.commands.run.assert_called_once() diff --git a/services/core/tools/e2b-provider/sdk.py b/services/core/tools/e2b-provider/sdk.py index 4b2d3afb7..cc69090b1 100644 --- a/services/core/tools/e2b-provider/sdk.py +++ b/services/core/tools/e2b-provider/sdk.py @@ -1,5 +1,4 @@ """The version-pinned SDK boundary. No handwritten provider HTTP or envd RPC.""" -import base64 import shlex from e2b import Sandbox @@ -18,7 +17,7 @@ from state import Failure -from helper_contract_generated import SDK_VERSION, MAX_OUTPUT, MAX_COMMAND_INPUT +from helper_contract_generated import SDK_VERSION, MAX_OUTPUT def list_templates(config, remaining): @@ -155,44 +154,29 @@ def definitely_rejected(error): isinstance(error, SandboxException) and error.status_code in (400, 401, 403, 404, 422, 429)) -def run(sandbox, command, remaining, user='runtime'): - raw = command.get('Stdin') - data = base64.b64decode(raw, validate=True) if raw is not None else None - if data is not None and len(data) > MAX_COMMAND_INPUT: - raise Failure('invalid') - args = command.get('Args') - if not isinstance(args, list) or not args or any(not isinstance(arg, str) or '\0' in arg for arg in args): - raise Failure('invalid') - directory = command.get('Directory') or None - if directory is not None and not directory.startswith('/'): - raise Failure('invalid') +def run(sandbox, args, remaining): + """Run one startup step as root and return its exit code. Output is never + returned; output beyond the bound or a missing exit status is unconfirmed.""" counts = [0, 0] def bounded(index, text): counts[index] += len(text.encode()) if counts[index] > MAX_OUTPUT: - raise Failure('command_unconfirmed') + raise Failure('unconfirmed') try: - process = sandbox.commands.run(shlex.join(args), user=user, cwd=directory, - background=True, stdin=data is not None, + process = sandbox.commands.run(shlex.join(args), user='root', background=True, timeout=remaining(), request_timeout=remaining()) - if data is not None: - # Avoid an oversized unary SDK message; every chunk is submitted once. - for offset in range(0, len(data), 64 * 1024): - sandbox.commands.send_stdin(process.pid, data[offset:offset + 64 * 1024], - request_timeout=remaining()) - sandbox.commands.close_stdin(process.pid, request_timeout=remaining()) try: result = process.wait(on_stdout=lambda text: bounded(0, text), on_stderr=lambda text: bounded(1, text)) except CommandExitException as error: result = error - return {'Stdout': result.stdout, 'Stderr': result.stderr, 'ExitCode': result.exit_code} + return result.exit_code except Failure: raise except Exception: - raise Failure('command_unconfirmed') from None + raise Failure('unconfirmed') from None def verify_team_template(config, remaining): diff --git a/services/core/tools/e2b-provider/testdata/contract.json b/services/core/tools/e2b-provider/testdata/contract.json index 73aa4755e..d94ce9cb0 100644 --- a/services/core/tools/e2b-provider/testdata/contract.json +++ b/services/core/tools/e2b-provider/testdata/contract.json @@ -1,53 +1,42 @@ [ -{"kind":"managed","name":"disabled","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"disabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":true}, -{"kind":"managed","name":"enabled","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":true}, -{"kind":"managed","name":"invalid-AllowedDomains","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":"example.com","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-AllowedDomains","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":[1],"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-DeviceID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":null,"EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-DeviceID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"invalid","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-Extra","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","Extra":true,"InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-NetworkAccess","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":null,"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-NetworkAccess","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"unknown","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"invalid-NetworkAccess","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":true,"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-AllocationID","payload":{"AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-AllowedDomains","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-DeviceID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-EnvironmentID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-InstallationID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-NetworkAccess","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-RuntimeBootstrap","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-SandboxIO","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-SessionID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, -{"kind":"managed","name":"missing-TenantID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"enabled","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444"},"valid":false}, -{"kind":"managed","name":"restricted","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":["example.com"],"DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","NetworkAccess":"restricted","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":true}, -{"kind":"request","name":"command","payload":{"Version":1,"Operation":"command","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"config-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":null,"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"create","payload":{"Version":1,"Operation":"create","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"extra","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Extra":true,"Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"inspect","payload":{"Version":1,"Operation":"inspect","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"kill","payload":{"Version":1,"Operation":"kill","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"list_builds","payload":{"Version":1,"Operation":"list_builds","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"list_templates","payload":{"Version":1,"Operation":"list_templates","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"observe","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"operation-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":null,"Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"operation-unknown","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"unsupported","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"reference-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":null,"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"reference-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":"invalid","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"renew","payload":{"Version":1,"Operation":"renew","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"validate_deployment","payload":{"Version":1,"Operation":"validate_deployment","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"verify_credential","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"verify_credential-count-0","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"verify_credential-count-32","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000001-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000002-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000003-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000004-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000005-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000006-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000007-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000008-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000009-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000010-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000011-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000012-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000013-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000014-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000015-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000016-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000017-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000018-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000019-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000020-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, -{"kind":"request","name":"verify_credential-count-33","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000001-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000002-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000003-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000004-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000005-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000006-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000007-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000008-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000009-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000010-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000011-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000012-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000013-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000014-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000015-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000016-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000017-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000018-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000019-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000020-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000021-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SessionID":"44444444-4444-4444-8444-444444444444","DeviceID":"55555555-5555-4555-8555-555555555555","CoreURL":"https://core.example/api/v1","Credential":"fixture-only","NetworkAccess":"enabled","AllowedDomains":null,"SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"RuntimeBootstrap":{"version":1,"core_url":"https://core.example/api/v1","device_id":"55555555-5555-4555-8555-555555555555","credential":"fixture-only"},"Deadline":"2099-01-01T00:00:00Z"},"valid":false}, -{"kind":"request","name":"verify_credential-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"verify_credential","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":{},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"verify_credential-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"verify_credential","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":"invalid","RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"verify_credential-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"verify_credential","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":[null],"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":1},"valid":false}, -{"kind":"request","name":"version-bool","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":true},"valid":false}, -{"kind":"request","name":"version-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":null},"valid":false}, -{"kind":"request","name":"version-string","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":"1"},"valid":false}, -{"kind":"request","name":"version-unknown","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","AllowedDomains":null,"CoreURL":"https://core.example/api/v1","Credential":"fixture-only","DeviceID":"55555555-5555-4555-8555-555555555555","EnvironmentID":"22222222-2222-4222-8222-222222222222","NetworkAccess":"enabled","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"SessionID":"44444444-4444-4444-8444-444444444444","TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"RuntimeBootstrap":{"core_url":"https://core.example/api/v1","credential":"fixture-only","device_id":"55555555-5555-4555-8555-555555555555","version":1},"Version":2},"valid":false}, +{"kind":"managed","name":"invalid-AllocationID","payload":{"AllocationID":null,"EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"invalid-AllocationID","payload":{"AllocationID":"invalid","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"invalid-Extra","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","Extra":true,"InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"invalid-InstallationID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"00000000-0000-0000-0000-000000000000","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"invalid-SandboxIO","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":null,"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"invalid-SandboxIO","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":"invalid","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"missing-AllocationID","payload":{"EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"missing-EnvironmentID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"missing-InstallationID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"missing-SandboxIO","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","TenantID":"11111111-1111-4111-8111-111111111111"},"valid":false}, +{"kind":"managed","name":"missing-TenantID","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1}},"valid":false}, +{"kind":"managed","name":"valid","payload":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","InstallationID":"66666666-6666-4666-8666-666666666666","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"valid":true}, +{"kind":"request","name":"config-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":null,"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":1},"valid":false}, +{"kind":"request","name":"create","payload":{"Version":1,"Operation":"create","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"extra","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Extra":true,"Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":1},"valid":false}, +{"kind":"request","name":"inspect","payload":{"Version":1,"Operation":"inspect","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"kill","payload":{"Version":1,"Operation":"kill","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"list_builds","payload":{"Version":1,"Operation":"list_builds","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"list_templates","payload":{"Version":1,"Operation":"list_templates","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"observe","payload":{"Version":1,"Operation":"observe","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"operation-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":null,"Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":1},"valid":false}, +{"kind":"request","name":"operation-unknown","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"unsupported","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":1},"valid":false}, +{"kind":"request","name":"reference-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":null,"Version":1},"valid":false}, +{"kind":"request","name":"reference-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":"invalid","Version":1},"valid":false}, +{"kind":"request","name":"renew","payload":{"Version":1,"Operation":"renew","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"validate_deployment","payload":{"Version":1,"Operation":"validate_deployment","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"verify_credential","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"verify_credential-count-0","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"verify_credential-count-32","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000001-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000002-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000003-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000004-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000005-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000006-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000007-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000008-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000009-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000010-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000011-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000012-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000013-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000014-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000015-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000016-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000017-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000018-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000019-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000020-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":true}, +{"kind":"request","name":"verify_credential-count-33","payload":{"Version":1,"Operation":"verify_credential","Config":{"Binary":"","StateDir":"","InstallationID":"66666666-6666-4666-8666-666666666666","APIKey":"","Template":"","APIURL":"","Domain":"","TimeoutSeconds":0,"Resources":null},"Reference":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333"},"References":[{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000001-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000002-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000003-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000004-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000005-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000006-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000007-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000008-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000009-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000000f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000010-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000011-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000012-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000013-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000014-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000015-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000016-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000017-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000018-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000019-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001a-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001b-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001c-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001d-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001e-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"0000001f-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000020-3333-4333-8333-333333333333"},{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"00000021-3333-4333-8333-333333333333"}],"Bootstrap":{"TenantID":"11111111-1111-4111-8111-111111111111","EnvironmentID":"22222222-2222-4222-8222-222222222222","AllocationID":"33333333-3333-4333-8333-333333333333","SandboxIO":{"version":1,"link_url":"wss://core.example/api/v1/sandbox-link","credential":"fixture-serve-only","resource":{"tenant_id":"11111111-1111-4111-8111-111111111111","environment_id":"22222222-2222-4222-8222-222222222222","kind":"allocation","id":"33333333-3333-4333-8333-333333333333","generation":1}}},"Deadline":"2099-01-01T00:00:00Z"},"valid":false}, +{"kind":"request","name":"verify_credential-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"verify_credential","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":{},"Version":1},"valid":false}, +{"kind":"request","name":"verify_credential-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"verify_credential","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":"invalid","Version":1},"valid":false}, +{"kind":"request","name":"verify_credential-references-type","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"verify_credential","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"References":[null],"Version":1},"valid":false}, +{"kind":"request","name":"version-bool","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":true},"valid":false}, +{"kind":"request","name":"version-null","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":null},"valid":false}, +{"kind":"request","name":"version-string","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":"1"},"valid":false}, +{"kind":"request","name":"version-unknown","payload":{"Bootstrap":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","SandboxIO":{"credential":"fixture-serve-only","link_url":"wss://core.example/api/v1/sandbox-link","resource":{"environment_id":"22222222-2222-4222-8222-222222222222","generation":1,"id":"33333333-3333-4333-8333-333333333333","kind":"allocation","tenant_id":"11111111-1111-4111-8111-111111111111"},"version":1},"TenantID":"11111111-1111-4111-8111-111111111111"},"Config":{"APIKey":"","APIURL":"","Binary":"","Domain":"","InstallationID":"66666666-6666-4666-8666-666666666666","Resources":null,"StateDir":"","Template":"","TimeoutSeconds":0},"Deadline":"2099-01-01T00:00:00Z","Operation":"create","Reference":{"AllocationID":"33333333-3333-4333-8333-333333333333","EnvironmentID":"22222222-2222-4222-8222-222222222222","TenantID":"11111111-1111-4111-8111-111111111111"},"Version":2},"valid":false}, {"kind":"response","name":"error-","payload":{"Version":1,"ErrorCode":""},"valid":true}, -{"kind":"response","name":"error-command_unconfirmed","payload":{"Version":1,"ErrorCode":"command_unconfirmed"},"valid":true}, {"kind":"response","name":"error-exists","payload":{"Version":1,"ErrorCode":"exists"},"valid":true}, {"kind":"response","name":"error-invalid","payload":{"Version":1,"ErrorCode":"invalid"},"valid":true}, {"kind":"response","name":"error-not_found","payload":{"Version":1,"ErrorCode":"not_found"},"valid":true}, diff --git a/services/core/tools/microsandbox-provider/README.md b/services/core/tools/microsandbox-provider/README.md index a55d0e67e..7f8aef69e 100644 --- a/services/core/tools/microsandbox-provider/README.md +++ b/services/core/tools/microsandbox-provider/README.md @@ -12,15 +12,15 @@ Before every operation the helper checks that it was built with the published SD The runtime home must be private (mode 0700), short, on local persistent storage and used by no other installation, profile or manual lifecycle tool. microsandbox uses Unix sockets there, so the node installer refuses a home whose path would exceed their limit. The home holds confidential VM disks, memory snapshots and SDK state; preserve it with the node identity and Core's database when recovering a host. Every managed lifecycle change goes through the provider. -The Runtime image is an immutable `repository@sha256:<64 lowercase hex>` reference that matches the saved Runtime release; bare image IDs and mutable tags are rejected. The node installer imports the distribution's image under that reference. To load an image by hand, `msb image load --tag repository@sha256:` must register the digest reference explicitly, with the manifest digest from `image inspect`, not the Docker image config ID. The image carries the daemon, Python 3, the native Harnesses and the shared Runtime helpers. The provider installs no registry credentials. +The Runtime image is an immutable `repository@sha256:<64 lowercase hex>` reference that matches the saved Runtime release; bare image IDs and mutable tags are rejected. The node installer imports the distribution's image under that reference. To load an image by hand, `msb image load --tag repository@sha256:` must register the digest reference explicitly, with the manifest digest from `image inspect`, not the Docker image config ID. The image must carry `/usr/local/bin/oac-sandbox-io` and `/usr/bin/python3`, which the bootstrap runs. The provider installs no registry credentials. ## Create and bootstrap Create names the VM from a hash of the installation and allocation reference plus the compute generation; a name never serves another incarnation. It creates the VM with the saved CPUs and memory as both initial and maximum, a managed root disk of `root_disk_mib`, an owned ext4 disk of `environment_disk_mib` mounted at `/environment`, user 1000:1000, working directory `/` and the node's network policy ([`bootstrap.go`](bootstrap.go)). The resource checks run before bootstrap. -Workspace, staging and outputs share the `/environment` filesystem, which keeps the Runtime's cross-device and link checks intact; the layered root filesystem can report different device IDs for a directory and its upper-layer files, so it holds no workspace data. Full snapshots and sandbox removal capture, restore and reclaim this disk; there is no host path, external mount or separate storage lifecycle. +The workspace, initialization and package directories share the `/environment` filesystem, which keeps cross-device and link checks intact; the layered root filesystem can report different device IDs for a directory and its upper-layer files, so it holds no workspace data. Full snapshots and sandbox removal capture, restore and reclaim this disk; there is no host path, external mount or separate storage lifecycle. -VM creation does not run the image's entry point. The bootstrap runs as root through confidential standard input, with a two-minute limit. It creates the Runtime directories and the private control directory `/run/oac` (mode 0700, owned by UID 1000), writes the [Runtime bootstrap](../../../../docs/runtime-bootstrap.md) file to `/home/runtime/runtime-bootstrap.json`, bind-mounts `/environment/workspace` at `/workspace` and starts `oac-daemon connect --bootstrap-file` in the background as UID/GID 1000. It writes the [Sandbox bootstrap](../../../../docs/sandbox-bootstrap.md) file to `/home/runtime/sandbox-io-bootstrap.json` (mode 0600, owned by UID 1000) and starts `oac-sandbox-io --bootstrap-file` with that path, in the background as UID/GID 1000 with an empty environment. `OAC_RUNTIME_DAEMON_SUSPEND_PID_FILE=/run/oac/daemon-suspend.json` enables the daemon's park and wake control. The `io.oac.bootstrap` label then changes from `pending` to `complete` through the SDK's next-start modification policy, because v0.7.2 cannot update the labels of a running VM. That label confirms only these writes and the launch, not authentication or native readiness. +VM creation does not run the image's entry point. The bootstrap runs as root with a two-minute limit and reads its only input, the [Sandbox bootstrap](../../../../docs/sandbox-bootstrap.md), from standard input. It creates `/home/runtime` and the `/environment` workspace, initialization and package directories (mode 0700, owned by UID 1000), writes the input to `/home/runtime/sandbox-io-bootstrap.json` (mode 0600, owned by UID 1000), bind-mounts `/environment/workspace` at `/workspace` and starts `oac-sandbox-io --bootstrap-file` with that path in the background as UID/GID 1000 with an empty environment; it starts no other process. Only a zero exit after the whole input was written confirms the bootstrap. The `io.oac.bootstrap` label then changes from `pending` to `complete` through the SDK's next-start modification policy, because v0.7.2 cannot update the labels of a running VM. That label confirms only these writes and the launch, not that the sandbox Serves. A helper response carries `CreateSettled` with a configuration rejection only after native Create has completed, the first inspection has verified the exact compute ID and ownership, and the resource check has rejected the VM before bootstrap started. The adapter keeps the original error and validates the compute identity before passing the proof to Core. Ordinary inspection, uncertain Create outcomes, timeouts and ownership failures never produce it, and missing compute alone never proves that Create settled. @@ -38,21 +38,12 @@ Core persists operation IDs, source and target generations, exact identities and After a lost response Core uses `ObserveOnly`. It never starts a capture or restore, and observing a suspend operation never kills its source. Observation checks artifact integrity and source ownership independently of resource checks, so resource drift cannot hide a retained artifact from cleanup; Core can persist recovered snapshot evidence before KillCompute. If the artifact is absent but the exact source is still running or paused with a settled bootstrap, observation returns the source without a snapshot and Core can abort the suspension; thawing and further execution still require the resource checks. For an interrupted restore, `ObserveOnly` may finish the missing resource proof on the exact target but never restarts a stopped target, changes resources or restores again. Missing state never authorizes a replay. -GetCompute, commands, cleanup and the next suspension verify restored provenance from the persisted VM configuration after the consumed artifact is deleted. +GetCompute, cleanup and the next suspension verify restored provenance from the persisted VM configuration after the consumed artifact is deleted. -## Locks and commands +## Locks A helper holds a per-allocation lock, under `oac-locks/` in the runtime home, until its SDK call actually settles. Core's response deadline neither kills the helper nor cancels its FFI wait, because cancelling the wait does not prove that the native mutation stopped. On a timeout Core keeps an unknown operation and observes it; a later helper cannot pass the surviving lock holder. A stuck owner needs operator investigation, not lock deletion or another Create. -`RunCommand` and `RunCommandCompute` pass standard input on an anonymous pipe, run as UID 1000 with a deadline and a 1 MiB limit per output stream, and return a result only after the input is fully written and the guest reports its exit. Timeouts, output overflow and missing receipts return `ErrCommandUnconfirmed`; closing an SDK exec handle does not prove that the guest process exited ([`command.go`](command.go)). Core uses `RunCommandCompute` only to run the exact wake command it authorizes: - -```sh -oac-daemon resume --control-file /run/oac/daemon-suspend.json \ - --environment-id ENVIRONMENT_ID --suspend-id SUSPENSION_ID -``` - -The command checks the protected Environment and suspension identities and the parked daemon's PID and start time, then signals it through a Linux pidfd. The daemon reauthenticates and waits for Core's resume confirmation before it admits work. - ## Metrics The read-only metrics operation holds the allocation lock and verifies the exact compute ID through the SDK before and after it runs `msb metrics NAME --format json` with the same pinned runtime binary ([`metrics.go`](metrics.go)). The CLI report keeps the native sample timestamp and fractional-second uptime, so the helper reconstructs one run start consistently across polls and Core restarts, and a new run gets a new start. The Go SDK's projection drops the timestamp and truncates uptime to whole seconds, so it cannot provide this; sandbox creation time is not a run start time. In the pinned source (`v0.7.2`, commit `1c59b8dbf0ad47dda2f807c0214b529aceb81c74`), `crates/metrics/lib/registry.rs` reads `sampled_at_unix_ms` and `started_at_unix_ms` together and subtracts them for uptime, and `crates/cli/lib/commands/metrics.rs` serializes the timestamp and `uptime.as_secs_f64()`. diff --git a/services/core/tools/microsandbox-provider/backend.go b/services/core/tools/microsandbox-provider/backend.go index 00b078d2a..61c0bbc2f 100644 --- a/services/core/tools/microsandbox-provider/backend.go +++ b/services/core/tools/microsandbox-provider/backend.go @@ -30,20 +30,6 @@ func (b backend) run(ctx context.Context) (wire.Response, error) { case "resume_compute": s, e := b.resumeCompute(ctx, b.q.Compute) return wire.Response{State: &s}, e - case "command": - h, _, e := b.inspect(ctx, b.q.Compute) - if e != nil { - return wire.Response{}, e - } - live, e := h.Connect(ctx) - if e != nil { - return wire.Response{}, e - } - defer live.Detach(context.Background()) - commandCtx, cancel := context.WithDeadline(ctx, b.q.Deadline) - defer cancel() - result, e := runCommand(commandCtx, live, *b.q.Command, "1000:1000") - return wire.Response{Command: &result}, e case "suspend": s, e := b.suspend(ctx, *b.q.Suspend) return wire.Response{State: &s}, e diff --git a/services/core/tools/microsandbox-provider/bootstrap.go b/services/core/tools/microsandbox-provider/bootstrap.go index 6d3f9364f..ed6bac0fd 100644 --- a/services/core/tools/microsandbox-provider/bootstrap.go +++ b/services/core/tools/microsandbox-provider/bootstrap.go @@ -4,46 +4,38 @@ package main import ( "context" - "encoding/json" + "errors" + "io" "time" - "github.com/MiniMax-AI/OpenAgentCore/internal/agentnetwork" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" wire "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/microsandbox" sdk "github.com/superradcompany/microsandbox/sdk/go" ) -// Runtime and the Sandbox I/O service read their launch inputs; Runtime's -// private auth storage stays opaque. All credential bytes enter the guest on -// stdin before any native work is admitted. +// The bootstrap script writes the Sandbox I/O service's input, read from +// stdin so the credential never appears in arguments, and starts the service +// as the sandbox user. const bootstrapScript = ` -import ctypes,json,os,stat,subprocess,sys -b=json.load(sys.stdin) -for p in ['/home/runtime','/home/runtime/.oac','/environment','/environment/workspace','/environment/staging','/environment/initialization','/environment/packages','/run/oac']: +import ctypes,os,stat,subprocess,sys +data=sys.stdin.buffer.read() +for p in ['/home/runtime','/environment','/environment/workspace','/environment/initialization','/environment/packages']: os.makedirs(p,mode=0o700,exist_ok=True) if not stat.S_ISDIR(os.lstat(p).st_mode): raise RuntimeError('invalid bootstrap directory') os.chmod(p,0o700);os.chown(p,1000,1000) -def private(p,v): - fd=os.open(p,os.O_WRONLY|os.O_CREAT|os.O_EXCL|os.O_NOFOLLOW,0o600) - with os.fdopen(fd,'w') as f: - json.dump(v,f) - f.flush();os.fsync(f.fileno());os.fchown(f.fileno(),1000,1000) -p='/home/runtime/runtime-bootstrap.json' -private(p,b['Runtime']) s='/home/runtime/sandbox-io-bootstrap.json' -private(s,b['SandboxIO']) +fd=os.open(s,os.O_WRONLY|os.O_CREAT|os.O_EXCL|os.O_NOFOLLOW,0o600) +with os.fdopen(fd,'wb') as f: + f.write(data);f.flush();os.fsync(f.fileno());os.fchown(f.fileno(),1000,1000) if not stat.S_ISDIR(os.lstat('/workspace').st_mode): raise RuntimeError('invalid workspace alias') libc=ctypes.CDLL(None,use_errno=True) if libc.mount(b'/environment/workspace',b'/workspace',None,4096,None)!=0: raise OSError(ctypes.get_errno(),'workspace bind mount failed') -def runtime_user(): +def sandbox_user(): os.setgroups([]);os.setgid(1000);os.setuid(1000) -subprocess.run(['/usr/local/bin/oac-daemon','connect','--profile','default','--bootstrap-file',p,'-b'], - stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL, - cwd='/environment/workspace',preexec_fn=runtime_user,check=True) subprocess.Popen(['/usr/local/bin/oac-sandbox-io','--bootstrap-file',s],env={},start_new_session=True, stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL, - cwd='/environment/workspace',preexec_fn=runtime_user) + cwd='/environment/workspace',preexec_fn=sandbox_user) ` func (b backend) create(ctx context.Context) (wire.Response, error) { @@ -53,27 +45,22 @@ func (b backend) create(ctx context.Context) (wire.Response, error) { } else if !sdk.IsKind(e, sdk.ErrSandboxNotFound) { return wire.Response{}, e } - bootstrap := *b.q.Bootstrap - policy := agentnetwork.Policy{Access: bootstrap.NetworkAccess, AllowedDomains: bootstrap.AllowedDomains} - domains, _ := json.Marshal(policy.Hosts()) + input, e := b.q.Bootstrap.SandboxIO.Marshal() + if e != nil { + return wire.Response{}, sandbox.ErrInvalid + } labels := wire.Labels(b.q.Config, b.q.Reference) labels[bootstrapLabel] = "pending" live, e := sdk.CreateSandbox(ctx, c.Name, sdk.WithImage(b.q.Config.Image), sdk.WithMemory(b.q.Config.MemoryMiB), sdk.WithCPUs(b.q.Config.CPUs), sdk.WithMaxMemory(b.q.Config.MemoryMiB), sdk.WithMaxCPUs(b.q.Config.CPUs), sdk.WithRootDisk(sdk.RootDisk.Managed(b.q.Config.RootDiskMiB)), sdk.WithUser("1000:1000"), - // A native owned disk keeps workspace and staging on one filesystem. - // Bootstrap creates their directories before starting the daemon. + // The Environment lives on a native owned disk; bootstrap creates its + // directories before starting Sandbox I/O. sdk.WithWorkdir("/"), sdk.WithMounts(map[string]sdk.MountConfig{ "/environment": sdk.Mount.Owned(sdk.OwnedVolumeOptions{Kind: sdk.VolumeKindDisk, SizeMiB: b.q.Config.EnvironmentDiskMiB}), }), - sdk.WithLabels(labels), sdk.WithDetached(), sdk.WithQuietLogs(), sdk.WithNetwork(b.network()), - sdk.WithEnv(map[string]string{ - "HOME": "/home/runtime", "OAC_RUNTIME_HOME": "/home/runtime/.oac", - "OAC_RUNTIME_ENVIRONMENT_ID": bootstrap.EnvironmentID, "OAC_RUNTIME_SESSION_ID": bootstrap.SessionID, - "OAC_RUNTIME_NETWORK_ACCESS": policy.Access, "OAC_RUNTIME_ALLOWED_DOMAINS": string(domains), - "OAC_RUNTIME_DAEMON_SUSPEND_PID_FILE": "/run/oac/daemon-suspend.json", - })) + sdk.WithLabels(labels), sdk.WithDetached(), sdk.WithQuietLogs(), sdk.WithNetwork(b.network())) if e != nil { return wire.Response{}, e } @@ -87,19 +74,11 @@ func (b backend) create(ctx context.Context) (wire.Response, error) { if e != nil { return qualified, e } - data, e := launchInputs(bootstrap) - if e != nil { - return wire.Response{}, e - } initialization, cancel := context.WithTimeout(ctx, 2*time.Minute) defer cancel() - result, e := runCommand(initialization, live, sandbox.Command{Args: []string{"/usr/bin/python3", "-I", "-S", "-c", bootstrapScript}, Stdin: data}, "0:0") - if e != nil { + if e = runBootstrap(initialization, live, input); e != nil { return wire.Response{}, e } - if result.ExitCode != 0 { - return wire.Response{}, sandbox.ErrCommandUnconfirmed - } // Persist the final bootstrap receipt without restarting the live guest. // v0.7.2 cannot update active labels; ownership reads persisted config. _, e = live.Modify(ctx, sdk.ModifyOptions{Labels: map[string]string{bootstrapLabel: "complete"}, Policy: sdk.ModificationPolicyNextStart}) @@ -108,23 +87,77 @@ func (b backend) create(ctx context.Context) (wire.Response, error) { } _, state, e := b.inspect(ctx, c) if e == nil && !state.BootstrapComplete { - return wire.Response{}, sandbox.ErrCommandUnconfirmed + return wire.Response{}, sandbox.ErrComputeUnconfirmed } return wire.Response{State: &state}, e } -// launchInputs is the bootstrap script's stdin: the daemon's connection and -// the Sandbox I/O service's input. -func launchInputs(b sandbox.Bootstrap) ([]byte, error) { - connection, err := b.RuntimeConnection().Marshal() - if err != nil { - return nil, err +// runBootstrap runs the bootstrap script as root with input on its stdin. +// Only a confirmed zero exit after the whole input was written settles it. +func runBootstrap(ctx context.Context, live *sdk.Sandbox, input []byte) error { + deadline, ok := ctx.Deadline() + if !ok { + return sandbox.ErrInvalid } - serve, err := b.SandboxIO.Marshal() - if err != nil { - return nil, err + timeout := time.Until(deadline) + if timeout <= 0 { + return sandbox.ErrComputeUnconfirmed + } + handle, e := live.ExecStream(ctx, "/usr/bin/python3", []string{"-I", "-S", "-c", bootstrapScript}, + sdk.WithExecUser("0:0"), sdk.WithExecTimeout(timeout), sdk.WithExecStdinPipe()) + if e != nil { + return errors.Join(sandbox.ErrComputeUnconfirmed, e) + } + defer handle.Close() + sink := handle.TakeStdin() + if sink == nil { + return sandbox.ErrComputeUnconfirmed + } + // Write and receive concurrently to avoid full-pipe deadlocks. + written := make(chan error, 1) + go func() { + n, err := sink.WriteCtx(ctx, input) + if err == nil && n != len(input) { + err = io.ErrShortWrite + } + if err == nil { + err = sink.Close() + } + written <- err + }() + return awaitBootstrap(ctx, handle.Recv, written) +} + +func awaitBootstrap(ctx context.Context, receive func(context.Context) (*sdk.ExecEvent, error), written <-chan error) error { + exited, code := false, 0 + for { + event, err := receive(ctx) + if err != nil { + return errors.Join(sandbox.ErrComputeUnconfirmed, err) + } + switch event.Kind { + case sdk.ExecEventExited: + if exited { + return sandbox.ErrComputeUnconfirmed + } + exited, code = true, event.ExitCode + case sdk.ExecEventStdinError, sdk.ExecEventFailed: + return sandbox.ErrComputeUnconfirmed + case sdk.ExecEventDone: + if !exited || code != 0 { + return sandbox.ErrComputeUnconfirmed + } + select { + case err := <-written: + if err != nil { + return sandbox.ErrComputeUnconfirmed + } + return nil + case <-ctx.Done(): + return sandbox.ErrComputeUnconfirmed + } + } } - return json.Marshal(struct{ Runtime, SandboxIO json.RawMessage }{connection, serve}) } // Only the initial post-Create inspection uses this proof. Native creation has diff --git a/services/core/tools/microsandbox-provider/bootstrap_test.go b/services/core/tools/microsandbox-provider/bootstrap_test.go index 34e71aa72..eb9059109 100644 --- a/services/core/tools/microsandbox-provider/bootstrap_test.go +++ b/services/core/tools/microsandbox-provider/bootstrap_test.go @@ -14,6 +14,7 @@ import ( "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/contracttest" wire "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/microsandbox" + sdk "github.com/superradcompany/microsandbox/sdk/go" ) type captureCaller func(wire.Request) @@ -23,9 +24,8 @@ func (f captureCaller) Call(_ context.Context, q wire.Request) (wire.Response, e return wire.Response{}, errors.New("captured") } -// Core's adapter sends SandboxIO to the helper, and the helper hands the -// guest both launch inputs on stdin. -func TestCreateDeliversBothLaunchInputs(t *testing.T) { +// Core's adapter sends the Sandbox I/O input to the helper unchanged. +func TestCreateSendsTheSandboxIOInput(t *testing.T) { config := wire.Config{ InstallationID: "11111111-1111-4111-8111-111111111111", HelperPath: "/helper", RuntimeHome: "/private/msb", RuntimePath: "/private/bin/msb", FirmwarePath: "/private/lib/libkrunfw.so", RuntimeSHA256: strings.Repeat("a", 64), FirmwareSHA256: strings.Repeat("b", 64), Image: "registry/runtime@sha256:" + strings.Repeat("c", 64), @@ -44,20 +44,42 @@ func TestCreateDeliversBothLaunchInputs(t *testing.T) { decoder := json.NewDecoder(bytes.NewReader(sent)) decoder.DisallowUnknownFields() var q wire.Request - if err := decoder.Decode(&q); err != nil || wire.ValidateRequest(q) != nil || q.Bootstrap == nil || q.Bootstrap.SandboxIO != b.SandboxIO { + if err := decoder.Decode(&q); err != nil || wire.ValidateRequest(q) != nil || q.Bootstrap == nil || *q.Bootstrap != b { t.Fatal("the helper request lost the Sandbox I/O input", err) } - payload, err := launchInputs(*q.Bootstrap) - if err != nil { - t.Fatal(err) - } - runtime, _ := b.RuntimeConnection().Marshal() - serveInput, _ := b.SandboxIO.Marshal() - var got map[string]json.RawMessage - if err := json.Unmarshal(payload, &got); err != nil || len(got) != 2 || !bytes.Equal(got["Runtime"], runtime) || !bytes.Equal(got["SandboxIO"], serveInput) { - t.Fatalf("stdin payload has %v", err) - } - if !strings.Contains(bootstrapScript, "b['Runtime']") || !strings.Contains(bootstrapScript, "b['SandboxIO']") { - t.Fatal("the bootstrap script does not read both launch inputs") +} + +func TestBootstrapRequiresZeroExitAndWholeInput(t *testing.T) { + for _, tc := range []struct { + name string + events []sdk.ExecEvent + inputError error + confirmed bool + }{ + {"confirmed", []sdk.ExecEvent{{Kind: sdk.ExecEventStderr, Data: []byte("noise")}, {Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, nil, true}, + {"nonzero_exit", []sdk.ExecEvent{{Kind: sdk.ExecEventExited, ExitCode: 1}, {Kind: sdk.ExecEventDone}}, nil, false}, + {"missing_exit", []sdk.ExecEvent{{Kind: sdk.ExecEventDone}}, nil, false}, + {"failed_stdin", []sdk.ExecEvent{{Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, errors.New("lost stdin"), false}, + {"duplicate_exit", []sdk.ExecEvent{{Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, nil, false}, + {"stdin_event", []sdk.ExecEvent{{Kind: sdk.ExecEventStdinError}, {Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, nil, false}, + {"stream_lost", []sdk.ExecEvent{{Kind: sdk.ExecEventExited}}, nil, false}, + } { + t.Run(tc.name, func(t *testing.T) { + index := 0 + recv := func(context.Context) (*sdk.ExecEvent, error) { + if index >= len(tc.events) { + return nil, errors.New("stream lost") + } + event := tc.events[index] + index++ + return &event, nil + } + input := make(chan error, 1) + input <- tc.inputError + err := awaitBootstrap(context.Background(), recv, input) + if tc.confirmed != (err == nil) || err != nil && !errors.Is(err, sandbox.ErrComputeUnconfirmed) { + t.Fatalf("bootstrap outcome %v", err) + } + }) } } diff --git a/services/core/tools/microsandbox-provider/command.go b/services/core/tools/microsandbox-provider/command.go deleted file mode 100644 index 7cdb7f157..000000000 --- a/services/core/tools/microsandbox-provider/command.go +++ /dev/null @@ -1,105 +0,0 @@ -//go:build linux - -package main - -import ( - "bytes" - "context" - "errors" - "time" - - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" - wire "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/microsandbox" - sdk "github.com/superradcompany/microsandbox/sdk/go" -) - -func runCommand(ctx context.Context, live *sdk.Sandbox, c sandbox.Command, user string) (sandbox.CommandResult, error) { - var result sandbox.CommandResult - deadline, ok := ctx.Deadline() - if !ok || wire.ValidateCommand(c) != nil { - return result, sandbox.ErrInvalid - } - timeout := time.Until(deadline) - if timeout <= 0 { - return result, sandbox.ErrCommandUnconfirmed - } - options := []sdk.ExecOption{sdk.WithExecUser(user), sdk.WithExecCwd(c.Directory), sdk.WithExecTimeout(timeout), sdk.WithExecStdinPipe()} - handle, e := live.ExecStream(ctx, c.Args[0], c.Args[1:], options...) - if e != nil { - return result, errors.Join(sandbox.ErrCommandUnconfirmed, e) - } - defer handle.Close() - sink := handle.TakeStdin() - if sink == nil { - return result, sandbox.ErrCommandUnconfirmed - } - // Write and receive concurrently to avoid full-pipe deadlocks. - written := make(chan error, 1) - go func() { - data := c.Stdin - for len(data) > 0 { - n := len(data) - if n > 65536 { - n = 65536 - } - count, err := sink.WriteCtx(ctx, data[:n]) - if err != nil { - written <- err - return - } - if count != n { - written <- errors.New("short command input") - return - } - data = data[n:] - } - written <- sink.Close() - }() - return collectCommand(ctx, handle.Recv, written) -} - -func collectCommand(ctx context.Context, receive func(context.Context) (*sdk.ExecEvent, error), written <-chan error) (sandbox.CommandResult, error) { - var result sandbox.CommandResult - var stdout, stderr bytes.Buffer - exited := false - for { - event, err := receive(ctx) - if err != nil { - return result, errors.Join(sandbox.ErrCommandUnconfirmed, err) - } - switch event.Kind { - case sdk.ExecEventStdout: - if stdout.Len()+len(event.Data) > wire.MaxOutputBytes { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - _, _ = stdout.Write(event.Data) - case sdk.ExecEventStderr: - if stderr.Len()+len(event.Data) > wire.MaxOutputBytes { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - _, _ = stderr.Write(event.Data) - case sdk.ExecEventExited: - if exited { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - exited = true - result.ExitCode = event.ExitCode - case sdk.ExecEventStdinError, sdk.ExecEventFailed: - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - case sdk.ExecEventDone: - if !exited { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - select { - case err := <-written: - if err != nil { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - result.Stdout, result.Stderr = stdout.String(), stderr.String() - return result, nil - case <-ctx.Done(): - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - } - } -} diff --git a/services/core/tools/microsandbox-provider/command_test.go b/services/core/tools/microsandbox-provider/command_test.go deleted file mode 100644 index 696385c32..000000000 --- a/services/core/tools/microsandbox-provider/command_test.go +++ /dev/null @@ -1,53 +0,0 @@ -//go:build linux - -package main - -import ( - "context" - "errors" - "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox" - wire "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/microsandbox" - sdk "github.com/superradcompany/microsandbox/sdk/go" - "strings" - "testing" -) - -func TestCommandRequiresExitAndSuccessfulStdin(t *testing.T) { - for _, tc := range []struct { - name string - events []sdk.ExecEvent - inputError error - uncertain bool - }{ - {"confirmed", []sdk.ExecEvent{{Kind: sdk.ExecEventStdout, Data: []byte("ok")}, {Kind: sdk.ExecEventExited, ExitCode: 7}, {Kind: sdk.ExecEventDone}}, nil, false}, - {"missing_exit", []sdk.ExecEvent{{Kind: sdk.ExecEventDone}}, nil, true}, - {"failed_stdin", []sdk.ExecEvent{{Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, errors.New("lost stdin"), true}, - {"duplicate_exit", []sdk.ExecEvent{{Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, nil, true}, - {"stdin_event", []sdk.ExecEvent{{Kind: sdk.ExecEventStdinError}, {Kind: sdk.ExecEventExited}, {Kind: sdk.ExecEventDone}}, nil, true}, - {"overflow", []sdk.ExecEvent{{Kind: sdk.ExecEventStdout, Data: []byte(strings.Repeat("x", wire.MaxOutputBytes+1))}}, nil, true}, - } { - t.Run(tc.name, func(t *testing.T) { - index := 0 - recv := func(context.Context) (*sdk.ExecEvent, error) { - if index >= len(tc.events) { - return nil, errors.New("stream lost") - } - event := tc.events[index] - index++ - return &event, nil - } - input := make(chan error, 1) - input <- tc.inputError - got, e := collectCommand(context.Background(), recv, input) - if tc.uncertain { - if !errors.Is(e, sandbox.ErrCommandUnconfirmed) { - t.Fatalf("outcome confirmed: %+v %v", got, e) - } - return - } - if e != nil || got.ExitCode != 7 || got.Stdout != "ok" { - t.Fatalf("nonzero guest exit lost: %+v %v", got, e) - } - }) - } -} diff --git a/services/core/tools/microsandbox-provider/main.go b/services/core/tools/microsandbox-provider/main.go index 8d9adb29a..6682f5b59 100644 --- a/services/core/tools/microsandbox-provider/main.go +++ b/services/core/tools/microsandbox-provider/main.go @@ -80,8 +80,6 @@ func code(err error) string { return "exists" case errors.Is(err, sandbox.ErrNotFound), sdk.IsKind(err, sdk.ErrSandboxNotFound): return "not_found" - case errors.Is(err, sandbox.ErrCommandUnconfirmed): - return "command_unconfirmed" case errors.Is(err, runtimeobs.ErrUnavailable), sdk.IsKind(err, sdk.ErrMetricsDisabled), sdk.IsKind(err, sdk.ErrMetricsUnavailable): return "metrics_unavailable" default: