[docs] Reorder the docs around the reliability loop, and give the cloud an onboarding path - #687
Open
NiveditJain wants to merge 7 commits into
Open
[docs] Reorder the docs around the reliability loop, and give the cloud an onboarding path#687NiveditJain wants to merge 7 commits into
NiveditJain wants to merge 7 commits into
Conversation
NiveditJain
added a commit
that referenced
this pull request
Aug 12, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js
|
Important Review skippedToo many files! This PR contains 1287 files, which is 1137 over the limit of 150. To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (24)
📒 Files selected for processing (1287)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Hermes
No summary yet. What this changesNo component map for this revision. RoundsNo review has finished on this pull request yet. FindingsNothing raised yet.
|
Contributor
Hermes
Found four documentation correctness issues: the custom-agent path cannot upload events as written, the legacy history page names a nonexistent dashboard route, the 14 localized sidebars omit the new deployment task, and the README understates the policy count. What this changesflowchart LR
n0Documentationnavigation["~ Documentation navigation"]
n1Cloudonboarding["+ Cloud onboarding"]
n2Cloudreference["+ Cloud reference"]
n3Localizeddocumentation["~ Localized documentation"]
n4Docstranslationtooling["~ Docs translation tooling"]
n5CLIhelpandREADME["~ CLI help and README"]
n0Documentationnavigation -- "routes readers to tasks" --> n1Cloudonboarding
n0Documentationnavigation -- "lists localized pages" --> n3Localizeddocumentation
n1Cloudonboarding -- "links to setup details" --> n2Cloudreference
n3Localizeddocumentation -- "uses localized Cloud routes" --> n2Cloudreference
n5CLIhelpandREADME -- "links users into docs" --> n0Documentationnavigation
Rounds
FindingsOpen
Resolved
|
hermes-exosphere
previously approved these changes
Aug 12, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found no blocking issues in this revision.
2 advisory findings
- Low/High Restore localized labels for the new navigation — Every non-English entry now renders the tab as "Documentation" and groups such as "Guardrails", "See what happened", and "Analyze and act" in English (for example docs/docs.json:144-218 for zh). The generator falls back to the English text for any unmapped name; its map only adds "Start here" and "CLI reference" (scripts/translate-docs/mintlify-nav.ts:86-108). Thus regenerating localized navigation will preserve this regression. (
scripts/translate-docs/mintlify-nav.ts:86) - Low/High Document that plain uninstall can retain the daemon — The new reference says
failproofai uninstallremoves the service and lists service removal as an unconditional step (docs/cli/uninstall.mdx:13-14,49-50). In the implementation, an interactive non-purge uninstall asks separately and keeps the service when declined (src/hooks/uninstall-cli.ts:305-313); the CLI help states this explicitly (bin/failproofai.mjs:994-998). (docs/cli/uninstall.mdx:13)
Contributor
|
I could not establish complete review coverage for What the review did establish: The documentation migration’s navigation, redirects, and internal links resolve across all 825 documentation pages. One edited public README entry still reports the wrong built-in-policy count. Full MDX compilation and Vitest verification could not run because dependencies were unavailable in the isolated container. Re-run with |
chhhee10
changed the title
This was referenced Aug 13, 2026
The site had two top-level tabs, "Enforcement" and "Observability", which asked every reader to work out which half of a single product they were in before reading anything. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability half is repositioned as FailproofAI Cloud and moves from the agenteye section to the cloud section, with a redirect for every old URL. Documents the integration surface that had no docs at all. `failproofai config --connect` existed only in --help: new pages cover connecting a machine (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), managed policies (deployments, digest verification, observe-mode rollout), the fleet view, and session capture across all 12 CLIs -- replacing three per-CLI capture pages written for a separate collector that the daemon has since absorbed. Adds the pages a reader kept needing and could not find: How it works (tool call to decision to dashboard, with the failure-mode table), the failproofaid service (fail-closed, supervision, how the binary arrives, upgrades), Supported agents (per-CLI matrix of what a deny actually blocks, derived from the same source the product asserts against), Policies as a hub, one merged Concepts glossary, a files-and-paths reference, and CLI pages for config, harness, backfill, flush and uninstall. Cloud pages are value- and contract-first: internal architecture claims are out, what a user needs to operate the product stays. Per-CLI hook-schema internals move off the Configuration page and into the support matrix. Two tests hardcoded the old docs directory as a fixture path and are updated to the new location. The 14 language trees are moved and relinked; pages with no counterpart are seeded from English so every nav entry resolves, and the scheduled translation job replaces them on its next run (its cache is keyed on English source hashes, all of which changed). Adds one screenshot of the local session viewer, captured through headless Chrome against synthetic fixture data rather than a real machine's sessions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js
The restructure was sound; its migration and several of its claims were not. Redirects covered English only. All 14 localized trees lost 30 URLs each with no redirect at all, and 13 redirects that were live on main had been rewritten at the source rather than the destination — so the URLs readers actually hold stopped resolving while 27 new entries pointed from paths that had never been valid. Coverage is now complete in all 15 trees, and 1,792 links in translated pages that pointed into English were localized. Anchors are derived from what Mintlify actually renders, which keeps the em dash rather than stripping it. The claims were checked against the code rather than against each other: the built-in count (40, not 39), the Recommended preset (15, not 16), the hook exit codes, credentials.json, audit/cache/, logs/hooks.log, the deny string, that allows are logged, and that state/ holds the undelivered upload spool and must not be deleted. The sanitize policies detect secrets but on ten of twelve CLIs cannot remove them, so the page no longer promises redaction; the turn-end gap on Hermes and Goose is a scope decision, not a platform limit. The files reference now says when each path appears, because a fresh install creates seven entries and the rest are conditional. The agent-skills page documented three skills where four ship. "Guardrails" is now "policies" throughout, including the nav group and the transliterated forms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…arding path The docs were technically accurate and hard to start from. Three things were wrong, and they compounded. The value proposition was inverted. The landing page opened "FailproofAI is the policy layer for that problem" and treated the cloud as a second half. The product is a loop — observe every run, score it, find the failures nobody wrote a rule for, prevent them, confirm the next audit is clean — and policy is the fix step inside it, not the pitch. introduction.mdx is now "What is FailproofAI" and says that; how-it-works and concepts follow the same order rather than the hook path's. There was no cloud onboarding at all. /cloud/getting-started redirected to an overview page, getting an API key was an off-site link on four pages, and "send your first trace" existed only inside a 447-line SDK reference whose install step is "talk to your contact". A new user could not self-serve. Five sequential pages now run from nothing to a deployed policy: create your workspace, send your first session, bring in the history you already have, find your first failures, prevent it happening again. They carry the minimum commands and link out for depth rather than restating the reference pages. And the sidebar described a product that no longer exists. cloud/overview enumerated an Observe/Analyze/Govern sidebar, four other pages named that grouping in prose, and four page slugs disagreed with what the dashboard calls them. The nav is now two tabs — the journey, and reference — and cloud/overview is one table mapping every real sidebar item to its page. Renames, with redirects in all 15 language trees: /cloud/incidents -> /cloud/issues /cloud/managed-policies -> /cloud/policy-editor /cloud/fleet -> /cloud/deploy-policies /cloud/access -> /cloud/admin 35 existing redirects pointed at two of those old destinations and were repointed; 461 in-body links across all 15 trees were rewritten, a class CI does not catch (mintlify validate resolves nav links only, validate:mdx resolves assets only). Two editorial rules now run through every page. There are exactly two ways to plug in — a supported harness via `failproofai config --connect`, or a custom agent via the Python SDK — and that fork is stated wherever a reader has to choose, instead of living in one buried note. And the product is FailproofAI Cloud in prose everywhere, with `agenteye` appearing only as something you type; both places a reader meets that name now say why. `--help` printed a third spelling, "FAILPROOF CLOUD". Also corrected against the shipped CLI: the cloud CLI's issue group is `issues`, not `incidents`, and the page claiming to list all 18 commands was missing `audits` and `usage`. Both admin surfaces those back — Usage and Settings — had no docs and now do. The local one-shot scan is "Local audit" against the fleet-wide "Audits", which retires the note in concepts.mdx that had been apologising for the collision. Locale page bodies are unchanged and stay in their current wording; the nightly translation job sees the path-keyed cache misses and rewrites them. No API spend here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4teTZnSkV9KESL6rD74on
NiveditJain
force-pushed
the
luv-legion-687
branch
from
August 14, 2026 11:35
fd19987 to
d266d91
Compare
NiveditJain
changed the title
…nchor `### Policies run, in order` renders an anchor carrying the comma, while the link on cloud/policy-editor asks for `#policies-run-in-order` — so it landed at the top of the page instead of the section. Removing the punctuation makes the anchor unambiguous rather than betting on one slug rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4teTZnSkV9KESL6rD74on
hermes-exosphere
requested changes
Aug 14, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found blocking issues that should be addressed.
1 advisory finding
- Low/High Localized key link has an invalid English fragment — All 14 non-English
start/sign-up.mdxpages link to/[locale]/cloud/admin#the-three-keys-most-teams-need(for example,docs/ar/start/sign-up.mdx:34). The Arabic destination heading is## المفاتيح الثلاثة التي تحتاجها معظم عمليات النشر(docs/ar/cloud/admin.mdx:8), so it has no English anchor. The fragment therefore lands readers at the page top rather than the machine-key section. (docs/ar/start/sign-up.mdx:34)
hermes-exosphere
previously approved these changes
Aug 14, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found no blocking issues in this revision.
3 advisory findings
- Medium/High Custom-agent onboarding never configures the uploader — The custom-agent branch presents the Python SDK itself as the connection method, then only calls
agenteye.configure()and emits events before stating that “The collector uploads those files” (docs/start/first-trace.mdx:53 and :107). It never installs/configures the collector or supplies the key from step 1. The collector credential/configuration is written byfailproofai config --connect(src/hooks/cloud-enrollment-cli.ts and src/hooks/collector-config.ts), whose header states it is the only writer of the collector credential. A custom-agent user following this new path will leave JSONL files on disk and never see the promised Sessions row. (docs/start/first-trace.mdx:53) - Medium/High History onboarding points to a dashboard location that no longer exists — The new onboarding tells readers that the fleet view is under “analyse → Policies” (docs/start/history.mdx:79). The new dashboard map instead places the fleet surface, Deploy Policies, under the
fixsection (docs/cloud/overview.mdx:44-50);analyse → Policiesonly leads to the policy editor. The same incorrect instruction is replicated in every localized history page. (docs/start/history.mdx:79) - Low/High README advertises 39 policies although the shipped catalog has 40 — This PR changes the README documentation table to say “All 39 policies” (README.md:193). The exported
BUILTIN_POLICIEScatalog contains 40 entries (src/hooks/builtin-policies.ts, beginning at line 2038), and the rewritten policy reference also states 40. (README.md:193)
hermes-exosphere
approved these changes
Aug 14, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found no blocking issues in this revision.
4 advisory findings
- Medium/High Localized sidebars omit the deployment task — The English Get started sequence includes
start/deploy-policy(docs/docs.json:47-53), but the representative Chinese sidebar still lists the old sign-up/history sequence and ends atstart/first-policy(docs/docs.json:175-180). The same missing deployment page occurs in all 14 non-English navigation trees, so localized readers cannot discover the final onboarding step. (docs/docs.json:175) - Medium/High Custom-agent onboarding never configures the uploader — The custom-agent path only calls
agenteye.configure()and then says that “The collector uploads those files” (docs/start/first-trace.mdx:73-107). It never installs/configuresfailproofaidor supplies itsevents:addcredential. The supported-harness branch does this withfailproofai config --connect; the collector configuration source identifies that connection flow as the writer of the ingest credential. A custom-agent user following this branch therefore only creates local JSONL files. (docs/start/first-trace.mdx:106) - Medium/High History onboarding points to a dashboard location that no longer exists — The history page directs readers to “analyse → Policies → the fleet view” (docs/start/history.mdx:79). The new dashboard map places Policy editor under analyse, while fleet assignments are under “fix → Deploy Policies”; there is no fleet view at the stated location. (
docs/start/history.mdx:79) - Low/High README advertises 39 policies although the shipped catalog has 40 — README.md:193 says “All 39 policies,” while
BUILTIN_POLICIEScontains 40 entries and the updated policy documentation consistently states 40. (README.md:193)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Rebuilds the documentation as one product, then reorders it around the loop that product actually is: observe every run → score it → find the failures nobody wrote a rule for → prevent them → confirm the next audit is clean.
Two rounds of work, one PR:
/cloud/*, and documenting the integration surface that lived only in--help.Why round 2
Three problems compounded, and none of them was a factual error:
/cloud/getting-startedredirected to an overview page, getting an API key was an off-site link on four pages, and "send your first trace" existed only inside a 447-line SDK reference whose install step is "talk to your contact". A new cloud user could not self-serve.cloud/overviewenumerated an Observe/Analyze/Govern sidebar, four other pages named that grouping in prose, and four slugs disagreed with what the dashboard calls them.The new shape
Two tabs — the journey, and reference. That alone takes 28 pages out of the first sidebar a new reader scrolls.
Five sequential onboarding pages, from nothing to a deployed policy in about ten minutes. Each carries the minimum commands and links out for depth rather than restating the reference pages.
start/sign-upevents:add+policies:pull) and why only those twostart/first-tracefailproofai config --connect; custom agent → Python SDK. Ends with "you should see a row in Sessions" and what to do if you don'tstart/historyharness add-path,backfill --since, and what is already gone from diskstart/first-auditnpx -y failproofai auditas the free local on-rampstart/first-policyTwo rules that now run through every page
There are exactly two ways to plug in, and the docs say so wherever a reader has to choose — on
introduction, inhow-it-works, at the top ofstart/first-trace, inconcepts, and in the nav ordering. It previously existed as one buried<Note>. The SDK is Python only; the docs say that rather than implying parity.The product is FailproofAI Cloud in prose;
agenteyeappears only as something you type. Both places a reader meets that name (cloud/cli,cloud/sdk) now say why in one clause.failproofai config --helpprinted a third spelling,FAILPROOF CLOUD.Renames
Four slugs now match what the dashboard calls them, with redirects in all 15 language trees:
/cloud/incidents/cloud/issues/cloud/managed-policies/cloud/policy-editor/cloud/fleet/cloud/deploy-policies/cloud/access/cloud/admin35 existing redirects pointed at two of those old destinations and were repointed. 461 in-body links across all 15 trees were rewritten — a class CI does not catch (
mintlify validateresolves nav links only;validate:mdxresolves assets only).Corrections against the shipped product
issues, notincidents(add_typer(..., name="issues")).auditsandusage. Both are now documented, verified againstaudits_cmds.py/usage_cmds.py.cloud/admin.cloud/performancedescribed Models / Tools / Hooks as standalone pages; they are tabs inside Dashboards.Telemetry,Error tracking,CLI and agents) now match the pages they point at.Name collisions, retired
concepts.mdxended with a<Note>apologising that "audit" meant two features. It is gone, because the titles now disambiguate: Local audit vs Audits, Local dashboard vs Charts and boards.Evaluations→ Online evals andEvaluators→ Build an evaluator, so the benefits page and the build guide no longer share a name.Translations
Locale page bodies are unchanged and stay in their current wording. Files were
git mv'd in lockstep across all 14 locales, the 5 new pages were seeded from English with/<lang>/link rewriting, and locale navs were regenerated withcli.ts --update-nav. The scheduled translation job sees the path-keyed cache misses and rewrites them on its next pass. No LLM spend in this PR.Verification
Read end to end against a local
mintlify dev: all five onboarding pages resolve, every "Next →" lands, and all four old URLs 307 — including their locale forms (/zh/cloud/incidents).Known gap, flagged not fixed
cloud/sdkstates the SDK wheel is private — "talk to your Failproof AI contact" — whilecloud/clisays the CLI is on public PyPI under the same distribution name. So the custom-agent half ofstart/first-tracecannot be completed self-serve. The page leads with the harness path, which can, and marks the SDK path as requiring access. Making the SDK self-serve is a product change.🤖 Generated with Claude Code
https://claude.ai/code/session_01B4teTZnSkV9KESL6rD74on
Concision and sequence pass
/issuesdashboard route while retaining theincidents:*API permission names.Four-task onboarding
cloud/connectandcloud/captureremain available as reference pages but no longer appear as separate onboarding choices.Hermes review
1bb2f2262a42155ec6ca046ae105a5d2e18e7b9b1d8f31d926828f3bae215c58f5b35baa44acbff0gpt-5.6-terraSummary
Found four documentation correctness issues: the custom-agent path cannot upload events as written, the legacy history page names a nonexistent dashboard route, the 14 localized sidebars omit the new deployment task, and the README understates the policy count.
Changes
Validation
Passedjq empty docs/docs.json; static resolver for MDX internal routes and Markdown assets— docs.json parsed successfully; all 773 direct internal MDX routes resolve to an existing page or redirect, and referenced Markdown assets exist. (4s)Skippeddocker run … oven/bun:latest … bun install --frozen-lockfile --ignore-scripts; bun run validate:mdx; bun scripts/translate-docs/cli.ts --validate; focused Vitest files— No centralized validation command was configured. The isolated container dependency installation did not complete in this harness, so repository validations were not executed. (26s)Findings
No blocking findings.
4 advisory findings
start/deploy-policy(docs/docs.json:47-53), but the representative Chinese sidebar still lists the old sign-up/history sequence and ends atstart/first-policy(docs/docs.json:175-180). The same missing deployment page occurs in all 14 non-English navigation trees, so localized readers cannot discover the final onboarding step. (docs/docs.json:175)agenteye.configure()and then says that “The collector uploads those files” (docs/start/first-trace.mdx:73-107). It never installs/configuresfailproofaidor supplies itsevents:addcredential. The supported-harness branch does this withfailproofai config --connect; the collector configuration source identifies that connection flow as the writer of the ingest credential. A custom-agent user following this branch therefore only creates local JSONL files. (docs/start/first-trace.mdx:106)docs/start/history.mdx:79)BUILTIN_POLICIEScontains 40 entries and the updated policy documentation consistently states 40. (README.md:193)Open questions
None.
Policy overrides
None.