Skip to content

feat: add response-shape conformance codegen and test#9

Merged
gjtorikian merged 3 commits into
mainfrom
response-shape-conformance
Jun 26, 2026
Merged

feat: add response-shape conformance codegen and test#9
gjtorikian merged 3 commits into
mainfrom
response-shape-conformance

Conversation

@gjtorikian

Copy link
Copy Markdown
Collaborator

Summary

  • Extends the repo's "OpenAPI spec is source of truth" principle — already used for the webhook event catalog (gen:events) — from event payloads to REST response shapes.
  • Adds gen:shapes codegen (scripts/gen-shapes{,-lib}.ts), mirroring gen:events: reads the @workos/openapi-spec package and emits a committed, idempotent src/workos/generated/response-shapes.ts with per-resource property + required-field sets. A curated object → schema map selects the authoritative schema per object type; extraction fails loudly if a mapped schema lacks a matching object discriminator (rejects request DTOs like OrganizationDto).
  • Adds a conformance test (src/workos/response-shapes.spec.ts) asserting each hand-written format* helper: (1) returns every spec-required field, (2) returns nothing outside the spec schema, (3) never leaks a secret (e.g. password_hash).
  • Chose this "checked invariant" approach over generating the live formatters: generation would leak internal fields, cap the implementation at the generator's expressiveness, and still need this test to prove the wire output matches. The spec becomes an enforced contract while hand-written behavior stays authoritative.
  • Divergences are pinned in exact-match ledgers that ratchet: closing one forces deleting its entry, and any new one fails CI — drift can't accrue silently.
  • Surfaces three real emulator↔spec divergences today (pinned, not hidden): connection lacks required status; role lacks permissions/resource_type_slug and carries emulator-only organization_id/is_default_role/priority; permission lacks resource_type_slug/system.

Test plan

  • npm run typecheck — passes
  • npm run lint (oxlint) — passes
  • npm run fmt:check (oxfmt) — passes
  • npm test — full suite green (45 files, 444 tests)
  • npm run gen:shapes is idempotent (re-run produces no diff)
  • Reviewer: confirm OBJECT_SCHEMA_MAP curation + divergence ledgers read correctly

🤖 Generated with Claude Code

Extends the "OpenAPI spec is source of truth" principle (already used for the
event catalog) to REST response shapes. Rather than generating the live
formatters — which would leak internal fields and cap the implementation at the
generator's expressiveness — this makes the spec a *checked invariant*: the
hand-written format* helpers stay authoritative, and a test asserts their output
matches the spec.

- scripts/gen-shapes{,-lib}.ts: mirror gen:events — read the @workos/openapi-spec
  package (oxfmt-formatted, committed, idempotent) and emit
  src/workos/generated/response-shapes.ts (RESPONSE_SHAPE_REQUIREMENTS: per-resource
  property + required sets). A curated object->schema map selects the authoritative
  schema per object type; extraction fails loudly if a mapped schema lacks a
  matching `object` discriminator (rejects request DTOs like OrganizationDto).
- src/workos/response-shapes.spec.ts: per resource, assert (1) every spec-required
  field is returned, (2) no field outside the spec is returned, (3) no secret field
  ever leaks. Divergences live in exact-match ledgers that ratchet: closing one
  forces deleting its entry; a new one fails the build.

Surfaces three real emulator<->spec divergences today (pinned, not hidden):
connection lacks required `status`; role lacks `permissions`/`resource_type_slug`
and carries emulator-only org/priority fields; permission lacks
`resource_type_slug`/`system`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds OpenAPI-backed response shape checks for WorkOS formatter output. The main changes are:

  • A new gen:shapes script and schema extraction helper.
  • A committed generated response-shape catalog.
  • Unit coverage for the generator logic.
  • Formatter conformance tests against generated response requirements.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
scripts/gen-shapes.ts Adds the CLI that loads the OpenAPI spec, formats generated output, and writes the response-shape catalog.
scripts/gen-shapes-lib.ts Adds schema resolution, response-shape extraction, object discriminator validation, and generated-file rendering.
scripts/gen-shapes-lib.spec.ts Adds tests for schema references, composed schemas, discriminator checks, catalog sorting, and generated output.
src/workos/generated/response-shapes.ts Adds the generated response-shape requirements used by the conformance tests.
src/workos/response-shapes.spec.ts Adds formatter conformance tests for required fields, extra fields, and catalog-scoped internal fields.
package.json Adds the gen:shapes npm script.

Reviews (3): Last reviewed commit: "fix: reject a bare --out flag in gen-sha..." | Re-trigger Greptile


const CASES: ReadonlyArray<{ objectType: string; output: Record<string, unknown> }> = [
{ objectType: 'user', output: formatUser(user) },
{ objectType: 'organization', output: formatOrganization(organization, ws, { domains: [] }) },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Nested Domain Secrets Bypass Test

When formatOrganization is called without an explicit domains override, it formats stored organization domains through formatDomain; those domain objects can include internal verification fields. This test always passes an empty domain list and only checks top-level keys, so an organization response with a stored domain can leak nested verification data while this conformance test stays green.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/workos/response-shapes.spec.ts
Line: 162

Comment:
**Nested Domain Secrets Bypass Test**

When `formatOrganization` is called without an explicit `domains` override, it formats stored organization domains through `formatDomain`; those domain objects can include internal verification fields. This test always passes an empty domain list and only checks top-level keys, so an organization response with a stored domain can leak nested verification data while this conformance test stays green.

How can I resolve this? If you propose a fix, please make it concise.

Comment thread src/workos/response-shapes.spec.ts
Comment thread scripts/gen-shapes-lib.ts
Comment thread scripts/gen-shapes.ts
gjtorikian and others added 2 commits June 26, 2026 14:25
… catalog

Addresses two review findings on the response-shape conformance work:

- resolveSchema silently dropped fields when an allOf member resolved to a
  oneOf/anyOf (it has no top-level `properties` to merge). That could let the
  generated catalog understate the spec shape, defeating the "fail loudly"
  guarantee. Now throws, matching the top-level oneOf/anyOf handling. Adds a
  unit test.

- SECRET_FIELDS listed auth-code/token names (`code`, `token`, `secret`,
  `value`, `key`) that none of the eight catalog resources can produce, which
  overstated the leak guard's coverage. Trimmed to the internal fields the
  format* helpers actually strip, with a comment recording why flow-resource
  formatters (email verification, magic auth, password reset, client secrets,
  API keys) intentionally surface those values and are out of scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Without a value (or followed by another flag), `--out` would take the next
flag as the output filename — e.g. `--out --dry-run` writes to a file named
`--dry-run` and leaves the committed generated file stale. Fail loudly instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gjtorikian gjtorikian merged commit 358773b into main Jun 26, 2026
6 checks passed
@gjtorikian gjtorikian deleted the response-shape-conformance branch June 26, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant