Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,26 @@ steps:
| `verbose` | No | Show verbose output (default: false) |
| `node-version` | No | Node.js version (default: 20) |

### Optional: Tool Outcome Attestation (TOA) after conformance

Protocol conformance and tool-delivery evidence are different checks. This framework does not grade tool outcomes. If your SDK CI already has a signed [toa](https://github.com/Carmel-Labs-Inc/toa) JSON (`toa/0.1`), you can optionally fail the job when offline verify fails. TOA is not a conformance scenario. No AgentStatus account is required to verify.

```yaml
- uses: modelcontextprotocol/conformance@v0.1.11
with:
mode: server
url: http://localhost:3001/mcp

# Optional. Provide toa.json from your emit step or an artifact.
- name: Verify tool delivery attestation
if: hashFiles('toa.json') != ''
run: |
pip install "git+https://github.com/Carmel-Labs-Inc/toa.git@99e2690fec24a5290d9542e58383a8bf753e8b74#subdirectory=python"
toa-verify toa.json --require-emitter agentstatus --require-layer functional=pass --max-age 7d
```

See [`examples/toa-after-conformance.yml`](./examples/toa-after-conformance.yml). Always pass `--require-emitter` (and `--max-age` when you need freshness). See the toa repo for `--public-key`.

## Example Clients

- `examples/clients/typescript/everything-client.ts` - Single client that handles all scenarios based on scenario name (recommended)
Expand Down
22 changes: 22 additions & 0 deletions SDK_INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,28 @@ jobs:

---

## Optional: Tool Outcome Attestation (TOA) after conformance

Conformance checks the wire protocol. [TOA](https://github.com/Carmel-Labs-Inc/toa) (`toa/0.1`) is separate signed JSON for tool delivery evidence (reach, invoke, functional, shape, and related layers). It is not a wire protocol and is not meant for every live `tools/call`.

If your CI produces a `toa.json` from AgentStatus (or another emitter whose key you pin), you can optionally verify it after the conformance action. Off by default. The example below requires `emitter.name=agentstatus` and uses the packaged AgentStatus key; pass `--public-key` for another issuer. No AgentStatus account is required to verify.

```yaml
- uses: modelcontextprotocol/conformance@v0.1.10
with:
mode: server
url: http://localhost:3001/mcp
suite: active

- name: Verify tool delivery attestation
if: hashFiles('toa.json') != ''
run: |
pip install "git+https://github.com/Carmel-Labs-Inc/toa.git@99e2690fec24a5290d9542e58383a8bf753e8b74#subdirectory=python"
toa-verify toa.json --require-emitter agentstatus --require-layer functional=pass --max-age 7d
```

Full copy-paste example: [`examples/toa-after-conformance.yml`](./examples/toa-after-conformance.yml).

## Writing Conformance Clients/Servers

### Example Client Pattern
Expand Down
34 changes: 34 additions & 0 deletions examples/toa-after-conformance.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Example only. Copy into an SDK or server repo as needed.
# Protocol conformance, then optional TOA verify (delivery evidence).
# TOA is not a conformance scenario.
name: MCP Conformance and optional TOA

on:
pull_request:
branches: [main]

jobs:
conformance-and-toa:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up and start server
run: |
# Replace with your SDK / server start.
echo "start your MCP server here"
# my-server --port 3001 &
# timeout 15 bash -c 'until curl -s http://localhost:3001/mcp; do sleep 0.5; done'

- uses: modelcontextprotocol/conformance@v0.1.11
with:
mode: server
url: http://localhost:3001/mcp
# expected-failures: ./conformance-baseline.yml

# Optional: only runs when toa.json is present in the workspace.
- name: Verify tool delivery attestation
if: hashFiles('toa.json') != ''
run: |
pip install "git+https://github.com/Carmel-Labs-Inc/toa.git@99e2690fec24a5290d9542e58383a8bf753e8b74#subdirectory=python"
toa-verify toa.json --require-emitter agentstatus --require-layer functional=pass --max-age 7d