Skip to content

For your consideration some testing fake seams risks RunWield found during init #1527

Description

@gandazgul

Question

when RunWield (https://github.com/gandazgul/runwield) initializes on a new repo it scans its tests for fake seams, places where the test should be using product owned machinery instead of a fake because you risk a green suite hiding a real problem. I havent examined the code to confirm these hence an issue and not a PR. Let me know if you would like me to dig deeper or if these are false positives.

Possible test-seam risks

  1. crates/openab-core/src/dispatch.rs — MockDispatchTarget

    • Replaceable behavior: real AdapterRouter, SessionPool, session creation/reset, and ACP turn streaming are replaced by a recorder.
    • Why it may be product-owned: dispatch buffering and batching are OpenAB-owned orchestration. The real session pool and router coupling could break while these tests
      stay green.
    • Fixture environment: use a real AdapterRouter and SessionPool with a small local ACP stub process; keep only the platform ChatAdapter fake because the platform is an
      external boundary.
    • Confidence: medium.
    • Uncertain facts: whether existing higher-level tests already cover this exact dispatcher/router/session path.
  2. src/acp_tunnel_source.rs — FakeTunnel

    • Replaceable behavior: real RootAcpTunnel, gateway tunnel registry lookup, detach/reattach behavior, and tool-list routing are simulated.
    • Why it may be product-owned: tunnel registry resolution and routing are OpenAB-owned, even though the remote MCP server is external.
    • Fixture environment: use the real tunnel registry with controlled in-process tunnel handles, then exercise discovery and call routing through AcpTunnelSource.
    • Confidence: low to medium.
    • Uncertain facts: how practical it is to construct real TunnelHandle fixtures without a live WebSocket.
  3. crates/openab-core/src/acp/pool.rs — CountingRegistrar

    • Replaceable behavior: real FacadeRegistrar and SessionTokens are replaced by a counter.
    • Why it may be product-owned: session token mint/revoke lifecycle is OpenAB-owned security machinery.
    • Fixture environment: use real SessionTokens through FacadeRegistrar, then assert token resolution and revocation behavior.
    • Confidence: medium.
    • Uncertain facts: existing SessionTokens unit tests may already cover enough of the real registry behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions