feat(api): search, grep and call any public Base44 API from the spec

[amitbardos]

Note

Description

Adds a base44 api command group — catalog, search, grep and fetch — that lets a human or an agent discover and call any public Base44 API operation without per-endpoint CLI code. Everything is driven at runtime by the server's OpenAPI document (GET /api/openapi.json), cached per server under ~/.base44/cache/<server>/openapi.json for one hour (--refresh forces a download). The spec logic lives in src/core/api-spec/; the commands in src/cli/commands/api/ are thin wrappers over it, matching the existing core/cli layering.

Related Issue

None

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Other (please describe):

Changes Made

• src/core/api-spec/ (new): Zod-validated OpenAPI parsing (schema.ts, operations.ts), per-server spec cache with write-then-rename (spec.ts), area catalog (catalog.ts), MiniSearch-backed search (search.ts), regex grep over contracts (grep.ts), $ref-inlining contract builder (contract.ts) and the authenticated request sender (request.ts).
• base44 api catalog [area]: lists the API's areas (the path segment after {app_id}, else after /api) or one area's operations, capped at 200.
• base44 api search <text>: MiniSearch ranking with per-field boosts, stopword filtering and summary coverage/precision, so operations whose summary holds every query word rank first.
• base44 api grep <pattern>: regular-expression match against each operation's contract with component schemas inlined, so a field name such as webhook_secret finds the operations that send or return it.
• base44 api fetch <METHOD> <path>: sends the path as is through base44Client with no timeout and no local validation (the server validates); --help after a path prints the operation's contract; --read-only refuses unsafe operations before anything is sent; body from --body <json>, --body @file, --body @-, or --form name=value / name=@file for multipart uploads; --output <file> saves the response. Retries are limited to GET/HEAD, and a 429 surfaces Retry-After.
• Path handling: the path is normalized once (dot segments resolved the way the request URL resolves them) and that same path is both checked by --read-only and sent.
• Output: JSON is passed through byte for byte when piped (no int64 rounding) and pretty-printed only in a terminal; bytes that aren't valid UTF-8 with no content type are described as binary rather than printed.
• Redaction: a failed fetch reports to telemetry without the request or response body, --body, --form fields or the query string — any endpoint can carry secrets. The server's raw reply still reaches the caller in the error details.
• Read-only semantics: derived from each operation's security list (PersonalAccessTokenAuth: ["read_only"], the backend's @read_safe); the server stays the real guard.
• Resilience: a malformed route in the spec is skipped rather than fatal; component $refs to parameters, request bodies and responses resolve.
• Supporting changes: toJsonStdout() gained a compact option, getBase44GlobalDir() is now exported, writeFile() accepts Uint8Array, minisearch@7.1.2 was added to devDependencies (bundled, per the zero-dependency distribution rule), and the group is registered in program.ts.
• Docs: a new "API commands (spec-driven)" section in docs/commands.md and four rows in the CLI README command table.

Testing

• I have tested these changes locally
• I have added/updated tests as needed
• All tests pass (npm test)

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (if applicable)
• My changes generate no new warnings
• I have updated docs/ (AGENTS.md) if I made architectural changes

Additional Notes

About 1,400 lines of new tests: tests/core/api-spec.spec.ts (operation listing, read-only detection, search ranking, path-template matching, contract inlining, grep, catalog grouping, cache paths and TTL), tests/cli/api.spec.ts (end-to-end command behavior) and tests/cli/api_redaction.spec.ts (nothing secret-bearing reaches telemetry), plus an openapi.json fixture. The suite, typecheck and lint were not run in this description-generation environment, so the two unchecked testing boxes and the warnings box are unverified here rather than known failures — CI covers them.

fetch deliberately disables Commander's help option so --help after a method and path can print the operation's contract, and it reads stdin only for --body @- (an agent harness can hold stdin open forever, so reading it unasked would hang the call). The commit notes that the step-by-step history lives on amitb/api-command-history.

---

_{🤖 Generated by Claude | 2026-10-17 09:41 UTC | 2893932}

[github-actions]

🚀 Package Preview Available!

---

Install this PR's preview build with npm:

npm i @base44-preview/cli@0.1.32-pr.663.43d14f8

Prefer not to change any import paths? Install using npm alias so your code still imports base44:

npm i "base44@npm:@base44-preview/cli@0.1.32-pr.663.43d14f8"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "base44": "npm:@base44-preview/cli@0.1.32-pr.663.43d14f8"
  }
}

• 📦 Preview Package: @base44-preview/cli@0.1.32-pr.663.43d14f8
• 🔗 View this commit on GitHub

---

_{Preview published to npm registry — try new features instantly!}

[amitbardos]

Commands, cache behaviour and timings. The description is regenerated by pr-description.yml on every push, so these live here.

Commands

Every listing identifies an operation as {endpoint, summary}, where endpoint is METHOD /path/template. Read-only isn't an output field; --read-only filters by it.

Command	What it does
base44 api catalog [area] [--read-only]	With no area: every area. Areas with 2 or fewer operations are listed in full; the rest show a count and 3 sample summaries, GETs first. With an area: {area, operations, total, truncated}, at most 200 operations (the largest area today is google-ads with 89). An unknown area fails and lists the real ones. An area is the path segment after {app_id}, else the one after /api.
base44 api search <text> [--limit N] [--read-only]	Full-text search (MiniSearch), top 5. Results whose summary holds every query word come first: by how much of the summary the query names ("get app" puts "Get app" above "Get app user"), then reads before writes. The rest follow by score (method 1, path 8, summary 5, description 2, parameters 0.5), raised by the share of query words in the summary. Stopwords, id and one-letter words are dropped (who am i and x return []). Prefix matching from 3 letters, fuzzy from 4. On 66 review queries against production, 60 rank the wanted operation first; the misses are ambiguous (app-wide vs one workflow's runs) or need a synonym ("create" for "Add custom domain").
base44 api grep <pattern> [--area A] [--read-only] [-i] [--limit N]	Regex over each operation's contract with schemas inlined, so a field such as webhook_secret finds the operation that sends or returns it. Returns {endpoint, summary}, like catalog and search.
base44 api fetch <METHOD> <path> [--body '<json>' | @file.json | @-] [--form name=value | name=@file] [--output file]	An authenticated call. The path, query string included, goes as is through base44Client, so every auth mode works. No timeout. Nothing is validated locally. --form sends multipart (repeatable; name=@file;type=text/csv attaches a file), so CSV import and file uploads work; it can't be combined with --body. Stdin is read only with --body @-, because a harness may hold stdin open forever. A pipe or file on stdin without --body gets a warning that it's ignored. The path is normalized once (dot segments resolved) and that path is both checked and sent. Output: a JSON body byte for byte when piped, pretty-printed in a terminal; other text as is, or {status, contentType, body} under --json; {"status": N} for an empty body; {status, contentType, bytes, note} for a binary body (including bytes that aren't UTF-8 with no content type). --output saves any body as is. On an HTTP error, the server's raw reply is in the error's details, --json includes statusCode, and a 429 shows the server's Retry-After wait. Writes are never retried; GET and HEAD are. Neither body, nor --body, --form or the query string, goes to telemetry.
… fetch <METHOD> <path> --help	Matches the path to its template (a literal segment wins over a placeholder) and prints {endpoint, method, path, summary, description, parameters, requestBodySchema, responseSchema}. Parameters are {name, in, required, schema, example}, including path-level ones and component $refs (parameters, request bodies, responses). Only the success response's schema is kept, with $refs inlined to depth 4 (a cycle keeps its $ref). A non-JSON body is named in requestBodyContentType, and a multipart one adds a hint to use --form. Any schema a leftover $ref points to is listed in refs. Schemas drop FastAPI's generated titles and write nullable types as type: [X, "null"]; descriptions drop docs-site callout tags and relative links (5–15% smaller). Search reads the same cleaned descriptions. Error shapes are left out, as in the MCP bridge; a failed call returns the server's own error. The help notes the contract can be cut down with jq.
… fetch … --read-only	Refuses, before sending, anything the spec doesn't mark read-only, including unknown paths. Read-only means security grants PersonalAccessTokenAuth: ["read_only"] (the backend's @read_safe): 132 of 491 operations in production.

--help and the --read-only refusal send nothing, so they need no login. That's why fetch turns off the command's built-in login and Commander's own --help. Output is JSON: pretty in a terminal, compact when piped. An HTTP error prints the server's reply once.

Cache

• The CLI fetches GET /api/openapi.json from getBase44ApiUrl() and saves it as is to ~/.base44/cache/<server>/openapi.json.
• The saved copy is reused while it's under an hour old. --refresh forces a fetch. An unreadable copy is downloaded again.
• Each route is validated on its own, so a malformed one is skipped instead of failing every command.
• It's written to a temp file and renamed, so concurrent runs never read a partial file. The download allows 60 s.
• A spec that fails to parse is never saved.
• The file is parsed on every run; there's no separate index file.
• fetch reads the spec only for --help and --read-only, so a route newer than the cache still works.

Timings

Measured on the production spec (2.31 MB, 491 operations, about 390 KB compressed on the wire), Node 22, cold process, median of 9 runs:

Step	ms
Read the saved file	3
Parse (JSON.parse + Zod)	15
List operations	3
Build the MiniSearch index + one search	55
grep over everything	11

End to end, api search takes 0.35 s, against 0.26 s for --version. A run that downloads the spec takes 2.6–3.0 s. Saving the index instead of rebuilding it would cost a 611 KB file and save about 35 ms (10 ms to load against 45 ms to build), so it's not worth it.

Dependency

minisearch@7.1.2 is added as a devDependency, so it gets bundled. The bun.lock diff is that package only.

Known issue (not from this PR)

On Node 20.11, which is below the package's engines of >=20.19.0, a large response through base44Client never lets the process exit. Any ky afterResponse hook triggers it; plain fetch doesn't. The same run exits normally on Node 20.19 and 22.

Test status

typecheck, lint and knip pass, and the 105 new tests pass. 29 existing tests in 9 files fail on my machine, and the same 29 fail on a clean main checkout there: local Node 20.11 takes --env-file as its own flag, and deploy tests time out.

History

The branch is squashed into one commit for review. The step-by-step history (20 commits, review rounds included) is kept on amitb/api-command-history; this branch adds the later --form, Retry-After and malformed-route changes on top.

[ronnyrin]

Nice work.
A few things before merge:

Blocking

1. --read-only checks a different path than it sends. matchOperation runs on the raw path, but ky builds a WHATWG Request that resolves ., .. and %2e%2e. So GET /api/apps/x/workflows/.. matches {workflow_id} but actually hits GET /api/apps/x/. I'd either reject dot/empty segments outright (simplest), or normalize once and use the result for both matching and sending:

   const url = new URL(path.replace(/^\/*/, "/"), "http://placeholder");
   const sendPath = `${url.pathname}${url.search}`;

2. JSON round-trip loses int64 precision. formatResponse does JSON.parse → stringify, which silently rounds ints > 2^53 and reformats the server's output. Could we pass bytes through as-is when piped and only pretty-print in a TTY?

Worth fixing

3. The contract reports requestBodyContentType (e.g. multipart), but sendApiRequest always sends Content-Type: application/json and reads @file as text. So upload endpoints show up in search but can't actually be called. Either add --content-type + binary @file, or flag them as unsupported in the contract.
4. isText("") returns true, so a binary body with no content type gets decoded and printed. A strict decode (new TextDecoder("utf-8", { fatal: true })) with fallback to the binary description would catch it.
5. $refs to #/components/parameters|requestBodies|responses fail parsing and silently drop out of the contract. FastAPI doesn't emit these today, but keeping the raw $ref would beat losing them.

Nits

6. formatJson goes compact when piped, while other commands' --json is always pretty. Smaller agent output, but inconsistent across the CLI. Intentional?
7. grep rebuilds every contract on each call, and matches against JSON-escaped text (\", \n). Fine at current spec size; maybe memoize later and mention the escaping in help.
8. minisearch is pinned to exactly 7.1.2 while everything else uses ^. Intentional?

[amitbardos]

@ronnyrin thanks, all addressed in 2893932:

1. --read-only path: the path is now normalized once with URL (dot segments and %2e%2e resolve as the request resolves them), and that same path is both matched and sent. GET /api/apps/x/workflows/.. is checked as GET /api/apps/x/ (Get app). A path with an empty segment (//) matches no template, so --read-only refuses it. Tests cover both.
2. int64: a JSON body is passed through byte for byte when piped, and pretty-printed only in a terminal. Test with 12345678901234567890.
3. Multipart: fixed before this review landed: --form name=value / --form name=@file;type=… sends multipart (CSV import, uploads, fonts, deploy-dist), and a multipart contract adds a hint to use it. Checked with a real upload.
4. No content type: decoded with TextDecoder("utf-8", { fatal: true }); bytes that aren't UTF-8 are described as binary.
5. Other component $refs: parameters, request bodies and responses resolve from components; an unresolvable parameter $ref is kept as is instead of dropped.
6. Compact when piped: intentional, to keep agent output small. It applies only to the api commands' own JSON; fetch now prints the server's bytes as is.
7. grep: help now says it matches each contract as JSON text, with quotes and newlines escaped. Agreed on memoizing later if the spec grows.
8. minisearch: not intentional; now ^7.1.2 (the lockfile still pins 7.1.2).

[ronnyrin]

Very thorough PR. Great job 👏🏻
Left a couple of human comments but none critical.

[ronnyrin]

Ouch, many undefined checks around the code

[ronnyrin]

Not a fan of the naming but not critical

[ronnyrin]

Overkill?

[ronnyrin]

Feels like extreme to me. Not sure we need to know who called us and how

[ronnyrin]

Don't we need all headers, like request id?

[ronnyrin]

Do we write logs whenever we download it (or fail to save)? It would be interesting to know if we need to support partial download of specific area

[ronnyrin]

I must say that the term spec is a little confusing to me. Is it the standard? In wix it's called schema afaik

[ronnyrin]

Is "area" the right term?