diff --git a/.github/workflows/automation-documentation-quality-ci.yml b/.github/workflows/automation-documentation-quality-ci.yml new file mode 100644 index 000000000..2167f7003 --- /dev/null +++ b/.github/workflows/automation-documentation-quality-ci.yml @@ -0,0 +1,37 @@ +name: Automation Documentation Quality CI + +on: + pull_request: + branches: [main] + paths: + - "docs/automation/**" + - "tests/test_automation_documentation.py" + - ".github/workflows/automation-documentation-quality-ci.yml" + - "AGENTS.md" + - "CLAUDE.md" + - "CHANGELOG.md" + +permissions: + contents: read + +concurrency: + group: automation-documentation-quality-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + documentation-contract: + name: documentation-contract + runs-on: ubuntu-24.04 + timeout-minutes: 5 + steps: + - name: Checkout exact source revision + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + persist-credentials: false + - name: Verify canonical automation documentation + shell: bash --noprofile --norc -e -o pipefail {0} + run: | + test "$(git rev-parse HEAD)" = "${{ github.event.pull_request.head.sha || github.sha }}" + python tests/test_automation_documentation.py + git diff --exit-code diff --git a/AGENTS.md b/AGENTS.md index 688b33035..4f62d5272 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,3 +2,5 @@ > **Agents: read the master context FIRST.** Before any work, read [`docs/CWL-MASTER-CONTEXT.md`](docs/CWL-MASTER-CONTEXT.md) (mission · naruon-as-platform + inter-component UML · cross-cutting disciplines · conventions · roadmap · current state), the live **GitHub Project #1** (work/roadmap source of truth), the full spec **ContextualWisdomLab/naruon#974**, and operate the Project per [`docs/agent-github-project-protocol.md`](docs/agent-github-project-protocol.md). The repo/Project — not any private agent memory — is the source of truth. + +Canonical organization automation/control-plane requirements, architecture, security, operations, diagrams, data model, traceability, and ADRs live in [`docs/automation/README.md`](docs/automation/README.md). Treat document status as evidence scope; `active_pr` is not protected-main implementation. diff --git a/CHANGELOG.md b/CHANGELOG.md index bf30091dd..3d8653c63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,8 @@ Semantic Versioning where the repository publishes a release. - Added a trusted pull-request comment router for `@cwl-noema-review` and review-only `@opencode-agent` dispatches, with an organization sweep, exact-head receipts, repository allowlisting, fixed runners, immutable checkout pins, and a permanent 100% statement/branch/docstring quality gate. - Added exact-base `uv.lock` materialization that reconstructs standalone nested projects with a checksum-pinned official `uv` exporter, isolated frozen/offline execution, strict exact-pin and SHA-256 output validation, and complete Python 3.10/3.14 quality evidence. +- Add a canonical automation control-plane PRD/TRD/architecture/security/operations/traceability graph, detailed ADR set, diagram/data-model coverage, and an exact-head documentation fitness workflow. +- Added a conversation-wide documentation fitness audit, external-orchestration/GitHub-evidence authority split, same-invocation continuation handoff, double-exit termination contract, expanded conceptual ERD/UML views, and ADR-0010 so prompt or documentation updates cannot become report-as-completion while safe queue work remains. ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index 1c7bdb2f6..7b73583e0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -126,3 +126,7 @@ repeatable compile command. cross-repo references as `owner/repo#num` or full URLs; durable knowledge in the repo/Project, not private memory; one roadmap phase at a time) are defined in `docs/CWL-MASTER-CONTEXT.md` §7 and apply here. + +## Automation control plane + +Use [`docs/automation/README.md`](docs/automation/README.md) as the canonical graph for organization automation requirements, architecture, security, operations, diagrams, evidence model, traceability, and ADRs. Preserve its implementation-status vocabulary and never promote `active_pr` to shipped behavior. diff --git a/docs/automation/ARCHITECTURE.md b/docs/automation/ARCHITECTURE.md new file mode 100644 index 000000000..4d4b9c513 --- /dev/null +++ b/docs/automation/ARCHITECTURE.md @@ -0,0 +1,110 @@ +# CWL Automation Control Plane Architecture + +Status: active_pr + +## Bounded contexts + +```mermaid +flowchart LR + X[External orchestration plane] --> O[Organization control plane] + O --> Q[Live executable queue] + Q --> E[Evidence collectors] + E --> R[RCA and feasibility] + R --> W[Repository writer lease] + W --> P[PR maintenance] + W --> D[Product development] + P --> G[Gate evaluator] + D --> G + G --> M[Merge authority] + M --> A[Protected-main acceptance] + O -. read-only .-> F[Fleet auditor] + O --> L[Thin leaf callers] + O --> C[Canonical documentation graph] +``` + +The External orchestration plane owns scheduled agent invocation, continuation policy, and external writer-lease state. The GitHub execution and evidence plane owns repositories, PRs, branches, workflows, checks, statuses, reviews, rulesets, artifacts, and merge/release evidence. These planes interact but are not interchangeable authorities. + +The central repository owns reusable GitHub automation semantics and the canonical documentation graph. Product repositories own product code and thin local caller/contracts. Dedicated repository loops own writes to their repositories; the fleet auditor remains read-only. A conversational decision or automation prompt is an input to reconciliation, not a replacement for repository documentation. + +## Plane boundaries + +### External orchestration plane + +Approved scheduled agent/orchestrator services may hold the authoritative writer lease for a repository and may choose the next execution lane. Their enabled state, schedule, prompt/configuration, and most recent run are external control records. They cannot turn a GitHub check, review, merge, or protected-main runtime result into success by declaration. + +### GitHub execution and evidence plane + +GitHub-native scheduled/manual/event workflows, reusable workflows, repository-local thin callers, PRs, checks, review threads, artifacts, rulesets, and protected branches are the authoritative source of repository state and execution evidence. Cross-repository reusable behavior belongs centrally unless an accepted architecture decision assigns it elsewhere. + +### Canonical documentation plane + +`docs/automation/**`, the ADR index, AGENTS/CLAUDE/CHANGELOG links, and their machine-checkable fitness gate are the durable design authority. Pull-request bodies, incident comments, conversation history, and downloadable planning artifacts supply evidence and candidate decisions, but material durable decisions must be reconciled into this graph with explicit maturity. + +## Trust boundaries + +1. GitHub event and repository metadata are inputs, not proof of current state until refetched. +2. External automation configuration is lease/control evidence, not source/check/review evidence. +3. PR-controlled source, comments, logs, and model prompts are untrusted content. +4. Workflow source is trusted only when immutably identified. +5. Checks, statuses, formal reviews, model judgments, merge authority, and protected-main runtime evidence are separate channels. +6. Model credentials are privileged secrets and are not needed for deterministic gates. +7. A source merge and a protected-main runtime execution are separate acceptance boundaries. +8. Conversation history and active-PR documentation cannot be presented as protected-main implementation. + +## Failure domains + +- Repository-local product/test defect. +- Central reusable-workflow defect. +- External scheduler/orchestrator continuation defect. +- Reviewer methodology/provider failure. +- Runner/network/bootstrap infrastructure failure. +- Permission/governance configuration. +- Writer conflict or stale evidence. +- Documentation authority or traceability drift. + +A failure freezes only the smallest affected domain/lane unless evidence proves a broader boundary. + +## Control flow + +```mermaid +sequenceDiagram + participant X as External Orchestrator + participant C as Control Plane + participant G as GitHub + participant R as Repository + participant V as Reviewer/Checks + X->>C: Start finite invocation with live continuation policy + C->>G: Refetch head, live base, policy, evidence + C->>C: RCA + distinct remedies + feasibility + alt safe repository mutation + C->>R: Exact-head/blob/ref-bound change + R->>V: Run exact-head verification + V-->>C: Evidence by authority channel + else lane blocked + C->>C: Defer exact identity and rotate + end + C->>G: Merge only if real gates pass + C->>G: Verify protected-main operation when required + C->>C: Continue next executable lane + C->>C: Double exit sweep before any terminal response + C-->>X: continuation_handoff or bounded termination +``` + +## Documentation reconciliation flow + +```mermaid +flowchart TD + H[Conversation / prompt / incident / PR evidence] --> R[Refetch protected main and active PRs] + R --> K{Canonical documentation line exists?} + K -->|yes| U[Update existing canonical line] + K -->|no| N[Create one canonical line] + U --> S[Assign controlled maturity state] + N --> S + S --> T[Update PRD/TRD/Architecture/ADR/UML/Data Model/Security/Operations/Traceability as affected] + T --> V[Run documentation fitness contract] + V --> Q[Return to executable queue] +``` + +## Deployment topology + +The control plane is hybrid rather than purely GitHub-native. External scheduled agent/orchestrator services provide finite invocation and writer-lease coordination; GitHub provides repository execution, evidence, policy, collaboration, and protected integration; external model/reviewer providers sit behind explicit credential and network boundaries. No durable database is assumed by this architecture; the data model is conceptual unless a persistence implementation is separately accepted. diff --git a/docs/automation/DATA_MODEL.md b/docs/automation/DATA_MODEL.md new file mode 100644 index 000000000..6d4a04c88 --- /dev/null +++ b/docs/automation/DATA_MODEL.md @@ -0,0 +1,67 @@ +# CWL Automation Control Plane — Conceptual Data Model + +Status: active_pr + +This is a logical evidence/domain model. It does **not** assert that the central control plane currently persists these entities in a database. External scheduler state, GitHub repository state, and canonical documentation state are separate authorities even when one automation run observes all three. + +```mermaid +erDiagram + repository_target ||--o{ pull_request_snapshot : contains + pull_request_snapshot ||--|| source_revision : identifies + pull_request_snapshot ||--|| base_revision : targets + pull_request_snapshot ||--o{ check_evidence : has + pull_request_snapshot ||--o{ review_evidence : has + pull_request_snapshot ||--o{ status_evidence : has + pull_request_snapshot ||--o{ workflow_evidence : has + pull_request_snapshot ||--o{ dependency_evidence : has + automation_run ||--o{ execution_lane : schedules + execution_lane ||--o{ deferred_item : defers + automation_run ||--o{ incident_hypothesis : evaluates + automation_run ||--o{ handoff_record : records + automation_run ||--o{ continuation_handoff : continues_with + automation_run ||--o{ automation_control_record : observes + automation_run ||--o{ operational_acceptance : verifies + automation_run ||--o{ secret_requirement : constrains + repository_target ||--o{ writer_lease : governed_by + automation_control_record ||--o{ writer_lease : may_grant + documentation_baseline ||--o{ documentation_fitness_result : evaluated_by + documentation_baseline ||--o{ decision_record : contains +``` + +## Entities + +- `automation_run`: one finite invocation of a maintenance, development, or audit loop. +- `automation_control_record`: observed external scheduler/orchestrator identity, enabled state, cadence, ownership scope, and configuration revision relevant to a run; it is not GitHub source evidence. +- `repository_target`: repository identity and control-plane ownership class. +- `execution_lane`: one independently executable unit such as a PR/head, issue, documentation line, operational acceptance probe, release task, or bounded product slice. +- `deferred_item`: an exact lane identity temporarily non-actionable because of queued evidence, approval, provider capacity, read-only dependency, writer conflict, or another bounded wait condition. +- `continuation_handoff`: records the preceding substantive action or defer decision and the next selected executable lane, or the bounded termination reason after the required exit sweeps. +- `pull_request_snapshot`: current PR metadata captured for a decision. +- `source_revision`: exact source/head commit identity. +- `base_revision`: independently resolved current base-ref tip identity. +- `check_evidence`: named GitHub Check evidence bound to a revision. +- `status_evidence`: commit-status context, creator, state, target URL where applicable, and revision. +- `review_evidence`: formal review/reviewer/thread evidence. +- `workflow_evidence`: workflow/run/job/checkout identity and outcome. +- `dependency_evidence`: state of a central or stacked prerequisite. +- `incident_hypothesis`: falsifiable causal hypothesis and disposition. +- `handoff_record`: read-only transfer to the authoritative owner when mutation is outside the lease. +- `operational_acceptance`: protected-main consumer execution evidence. +- `secret_requirement`: purpose, scope, materialization boundary, and least-privilege requirement for a secret. +- `writer_lease`: authoritative writer scope, source of authority, branch/repository boundary, and conflict evidence. +- `documentation_baseline`: one canonical documentation authority for a bounded control-plane/product scope. +- `documentation_fitness_result`: adequacy assessment by artifact class, including missing, stale, partial, adequate, conflicting, or not-applicable findings and required corrective actions. +- `decision_record`: material decision reconciled from protected-main source, active PRs, incidents, research, or conversation evidence with an explicit maturity state. + +## Invariants + +- `source_revision` and `base_revision` are never collapsed into one identity. +- Review, check, status, workflow, model, merge, external-automation, and runtime evidence remain separate authorities. +- A stale snapshot cannot authorize a write. +- A `deferred_item` blocks only its exact `execution_lane` unless broader evidence proves shared scope. +- Every substantive action that does not exhaust the invocation produces a `continuation_handoff` to the next executable lane. +- A user-visible status or prompt update cannot satisfy `continuation_handoff` by itself. +- A `documentation_baseline` is singular for its declared scope; conversation or PR-body text does not silently become a second authority. +- A `documentation_fitness_result` marked missing, stale, partial, or conflicting requires a repository mutation or explicit non-actionable disposition when the current writer owns the documentation line. +- A conceptual entity is not evidence that persistence exists. +- Durable database object names, if later introduced, use descriptive two-or-more-word `snake_case` names. diff --git a/docs/automation/DOCUMENTATION_AUDIT.md b/docs/automation/DOCUMENTATION_AUDIT.md new file mode 100644 index 000000000..90dd10063 --- /dev/null +++ b/docs/automation/DOCUMENTATION_AUDIT.md @@ -0,0 +1,80 @@ +# Automation documentation fitness audit + +Status: active_pr + +## Scope and conclusion + +This audit evaluates whether the canonical `ContextualWisdomLab/.github` automation-control-plane documentation can faithfully absorb the durable automation/governance decisions established across the current CWL project conversation without turning chat history into shipped claims. + +**Conclusion:** the original PR #886 baseline was structurally strong but not sufficient for the full conversation-derived control-plane contract. It covered PR maintenance, product development, exact-head/live-base identity, evidence separation, writer leases, RCA, independent approval, protected-main acceptance, central/leaf ownership, security, threats, tests, operability, incidents, traceability, UML, and a conceptual ERD. It did not explicitly model the external scheduled agent/orchestrator plane, the same-invocation continuation/handoff rule after prompt or documentation changes, a double exit sweep, or the conversation-to-canonical-GitHub reconciliation boundary. Those gaps are corrected in this documentation line and protected by the documentation fitness test. + +This audit does **not** claim that every product-specific design discussed anywhere in the wider CWL project is duplicated into `.github`. Product-specific PRD/TRD/ADR/UML/ERD remain authoritative in their owning repositories. The central control plane records ownership and handoff semantics instead of creating a second specification authority. + +## Fitness matrix + +| Artifact | Before this audit | Required correction | Current status in this PR | +|---|---|---|---| +| PRD | Partial | Make user-visible status/prompt/doc updates non-terminal; require same-invocation continuation and conversation reconciliation | active_pr | +| TRD | Partial | Add external automation control evidence, `continuation_handoff`, queue-wide double exit sweep, reconciliation contract | active_pr | +| Architecture | Partial | Separate External orchestration plane from GitHub execution and evidence plane; add canonical documentation plane | active_pr | +| Conceptual ERD / data model | Partial | Add `automation_control_record`, `execution_lane`, `deferred_item`, `continuation_handoff`, documentation baseline/fitness entities | active_pr | +| UML | Partial | Add continuation state machine, external-scheduler authority view, and conversation-to-repository reconciliation sequence | active_pr | +| ADR index | Partial | Existing ADR-0003 covered work conservation, but no dedicated handoff/reconciliation authority decision existed | active_pr | +| Security | Adequate for central scope | Preserve credential/evidence/writer boundaries; no new secret authority is introduced by reconciliation | active_pr | +| Threat model | Adequate for central scope | Premature termination and split documentation authority are governance/availability threats and should remain traceable through ADR/operability | active_pr | +| Test strategy | Partial | Documentation test must require the audit, all ten ADR families, and continuation/reconciliation invariants | active_pr | +| Operability | Adequate but needs traceability | Premature termination is an operational defect requiring prompt/control repair plus resumed queue execution | active_pr | +| Incident runbook | Adequate for RCA mechanics | Treat early stop as a control-plane incident when safe work demonstrably remained | active_pr | +| Traceability | Partial | Map continuation/reconciliation and external automation authority without claiming GitHub-native implementation | active_pr | +| AGENTS / CLAUDE / CHANGELOG | Structurally adequate | Keep links and behavioral summary aligned with the amended canonical graph | active_pr | + +## Central-versus-leaf documentation boundary + +The following durable topic families have appeared in the wider CWL project conversation. They are **not** specifications owned by `.github`; they are handoff obligations to their product repositories or dedicated documentation lines. + +| Product/domain family | Central responsibility | Leaf responsibility | +|---|---|---| +| Psychometrics / fast-mlsirm / psychometrics-commons | shared automation, evidence, CI/security, writer-lease policy | mathematical model, Rust CPU/GPU estimator, multilevel/multiple-membership/time modeling, simulation/recovery evidence | +| Temporal Event Psychometrics / TEPP | shared automation and standards/doctoring conventions | temporal/event ontology, multi-clock model, TDT/CHRONOS integration, longitudinal ESEM/DSEM, product data model and UI | +| naruon | shared CI/review/release controls | language/product functionality such as topic modeling, grammar/spell checking, connectors and end-user workflows | +| pg-erd-cloud | shared CI/review/release controls | forward engineering, saved-view workspace, schema-change workflows, ERD/data product UX and Figma contracts | +| BandScope | shared CI/review/release controls | real-audio/stem accuracy, rehearsal handoff, accessibility and music-analysis product contracts | +| Inkspan | shared CI/review/release controls | deterministic document rendering, collaboration and document-product behavior | +| OriginWeave | shared CI/review/release controls | agentic browser runtime, Chromium compatibility boundary, resource governor, provenance and browsing policy | +| EmbedRelay | shared CI/review/release controls | embedding-space identity, directed adapters, migration/backfill, fidelity benchmarks and vector-store connectors | +| MHTML ETL Gateway | shared CI/review/release controls | immutable raw ingestion, MHTML parsing, schema inference, PostgreSQL loading, lineage and enterprise data-model contracts | +| LifeOS | shared CI/review/release controls | personal goal/project/habit/task product model, cloud/self-host deployment, auth and user-experience contracts | +| AppGuardrail and security products | shared security/review evidence | scanner/rule-engine detection coverage and product-specific adversarial regression corpus | + +A central automation change may link to or verify a leaf requirement, but it must not silently restate a leaf design as central ownership. Conversely, a durable shared automation requirement must not remain only in a product PR, conversation, or downloadable planning pack. + +## Required reconciliation algorithm + +1. Identify a material decision in conversation, an automation prompt, an incident, an active PR, or a planning artifact. +2. Refetch protected-main implementation and the current active PRs that may already own the decision. +3. Resolve the canonical owner: `.github` for shared automation/control-plane semantics; the product repository for product behavior. +4. Reuse the existing canonical documentation branch/PR when one exists. +5. Classify the decision using the controlled maturity vocabulary. +6. Update every affected artifact class, not only an ADR or PR body. +7. Update traceability and machine-checkable documentation contracts. +8. Return through `continuation_handoff` to the live executable queue; documentation completion is not run completion. + +## Residual gaps and non-claims + +- This PR does not prove that every leaf repository already contains its complete product-specific PRD/TRD/ADR/UML/ERD pack. That requires repository-by-repository live audits under each repository's writer lease. +- This PR does not make external scheduled-agent state GitHub-native or persist it in a central database. +- This PR does not implement the active review-quality, model-routing, fleet-coordination, sandbox-redaction, or security/coverage changes represented by other active PRs. It documents authority and maturity so those lines can be integrated without evidence conflation. +- A documentation file being present is not evidence that the described behavior is shipped. Only protected-main source and required operational acceptance can promote a decision to `implemented_on_protected_main`. + +## Acceptance + +The central documentation graph is sufficient for this control-plane conversation slice when: + +- every required artifact is indexed and uses the controlled maturity vocabulary; +- the external orchestration, GitHub execution/evidence, and canonical documentation planes are explicit; +- work-conserving same-invocation continuation and double exit sweeps are normative; +- the conceptual ERD contains continuation/defer/handoff and documentation-fitness entities without inventing persistence; +- ADR-0010 records conversation/prompt/documentation-to-executable handoff authority; +- traceability distinguishes active/accepted behavior from protected-main implementation; +- the dependency-free documentation contract passes on the unchanged exact head; and +- the invocation returns to other safe queue work instead of treating this audit as completion. diff --git a/docs/automation/INCIDENT_RUNBOOK.md b/docs/automation/INCIDENT_RUNBOOK.md new file mode 100644 index 000000000..833cb48d5 --- /dev/null +++ b/docs/automation/INCIDENT_RUNBOOK.md @@ -0,0 +1,47 @@ +# Automation control-plane incident runbook + +Status: active_pr + +## 1. Establish current identity + +Refetch repository policy, protected base ref, PR source head, target blobs/refs, workflow source revision, requested/formal reviews, unresolved threads, statuses, runs/jobs, external scheduler/automation writer state, canonical documentation owner, and writer lease. Record source head and live base as separate values. + +## 2. Find the first failing boundary + +Separate symptom, immediate cause, root cause, and owner. Classify the boundary as source/test, trusted bootstrap/materialization, workflow contract, external scheduler/continuation, runner/provider, credential/permission, reviewer/governance, concurrency/queue, documentation authority, or protected-main runtime. + +For an early-stop symptom, prove whether another safe lane was executable at termination. If yes, the root cause is the prompt/control rule that incorrectly promoted an intermediate event to terminal, not the blocked PR/check/review itself. + +## 3. Test remedies for feasibility + +Enumerate materially distinct minimal remedies. Reject any remedy that needs invented credentials or reviewers, weakens a gate, uses stale evidence, races a writer, changes unrelated behavior, creates a parallel documentation authority, or lacks an observable acceptance test and rollback. + +## 4. Retry classification + +Retry only evidence-classified transient DNS/connectivity, rate-limit, or provider-capacity failures within a bounded budget. Do not retry integrity/hash/signature, authorization, TLS/certificate, immutable ref, untrusted origin/redirect, schema, product-test, or deterministic continuation-contract failures as infrastructure noise. + +## 5. Execute and verify + +Implement the smallest root-cause-changing repair test-first. Refetch immediately before each write and use CAS/blob/ref-bound or non-force fast-forward mutation. Verify the exact resulting head with deterministic, security, dependency, automated-review, documentation-fitness, and relevant external-control evidence. + +For a premature-termination defect, amend the authoritative scheduler/orchestrator prompt/configuration, then resume the missed or next safe GitHub lane in the same invocation when possible. Prompt repair alone is not completion evidence. + +## 6. Protect queue health + +Defer the exact pending identity after one read. Preserve the sole current-head evidence lane and cancel only runs proven obsolete by current PR/head/workflow identity. Interactive dispatch and scheduled sweeps use isolated concurrency so a sweep cannot replace interactive work. + +After every substantive action or defer decision, reselect the queue. Before termination perform the double exit sweep; if either sweep finds an execute-now item, execute it and continue. + +## 7. Reconcile durable decisions + +If the incident changes automation behavior, evidence authority, writer leases, security, operations, or acceptance semantics, reconcile the durable decision into the existing canonical GitHub documentation line. Conversation text, prompt text, PR bodies, and incident comments remain evidence inputs. Product-specific behavior stays in the owning product repository. + +## 8. Merge and operational acceptance + +Merge only after actual protection and counted independent exact-head approval pass. For operational defects, exercise the protected-main scheduled/manual consumer, verify downstream acknowledgement and negative control, and rehearse rollback before closure. + +A documentation or prompt change can be source-level acceptance for the control definition, but an incident about actual scheduler behavior closes only after runtime evidence demonstrates correct lane rotation/termination behavior. + +## 9. Reopen conditions + +Reopen when the same failure class recurs, a run again terminates while a safe lane remains, the protected-main consumer did not execute the repaired boundary, documentation authority splits again, evidence was stale/synthetic, rollback was not viable, or a security control was weakened to obtain green status. diff --git a/docs/automation/OPERABILITY.md b/docs/automation/OPERABILITY.md new file mode 100644 index 000000000..a710d0a3d --- /dev/null +++ b/docs/automation/OPERABILITY.md @@ -0,0 +1,40 @@ +# Automation control-plane operability + +Status: active_pr + +## Service objectives + +- Every mutation is bound to a freshly observed exact head/blob/ref. +- No required evidence is promoted from stale, pending, cancelled, skipped, absent, or predecessor state. +- A blocked lane does not prevent execution of another safe lane. +- A prompt update, documentation update, RCA, review request, merge, or user-visible status does not terminate a run while another safe executable lane exists. +- Every substantive action or defer decision produces queue reselection and a `continuation_handoff` unless the practical invocation/tool budget is exhausted. +- Termination is preceded by the required fresh double exit sweep. +- Current-head required checks queued beyond the declared repository SLO are diagnosable by repository, workflow, run, event, head, queue age, and external prerequisite. +- Operational incidents close only after a current protected-main consumer and negative control pass. + +## Signals + +Track queue age, executable-lane count, deferred-lane count and exact defer identity, last substantive action, next selected lane, continuation-handoff count, exit-sweep result, premature-termination detection, running/pending lanes per PR and workflow, obsolete-run count, provider failure class, retry count, dispatch acknowledgement, reviewer eligibility, unresolved thread count, exact-head check completeness, external scheduler/automation writer-lease state, secret materialization, redaction events, merge result, canonical documentation fitness, and protected-main acceptance identity. + +## Premature-termination incident + +Classify a run as prematurely terminated when its terminal/user-visible output is followed by fresh evidence that a safe, policy-compliant lane was executable under the current writer lease. Treat the emitted status as the symptom; the root cause is the prompt/control condition that promoted an intermediate state to terminal. + +Remediation is work-conserving: repair the authoritative scheduler prompt/configuration, refetch the missed queue, execute the highest-value safe lane in the same invocation when possible, and update the canonical documentation/test contract when a durable invariant was missing. Prompt repair by itself is not incident closure. + +## Degraded operation + +- Provider outage/rate limit: defer model evidence; continue deterministic and disjoint work. +- Runner saturation: preserve the sole current-head required run; cancel only proven obsolete runs. +- DNS/network bootstrap failure: retry only bounded transient classes with backoff; integrity, auth, TLS, ref, and origin failures fail immediately. +- External human approval: enable expected-head-safe auto-merge when appropriate and continue other work. +- OIDC/App failure: keep credentials undisclosed, record the failed envelope boundary, and fall back only to a separately authorized path. +- Documentation authority conflict: freeze only the conflicting documentation lane, resolve canonical ownership from live repository state, retain superseded history, and continue disjoint work. +- External scheduler/control read failure: fail closed on writer-lease mutation for the affected repository but continue read-only audit or separately owned lanes. + +## Replay and rollback + +Rerun only the failed exact-head job when evidence remains valid and the workflow contract supports it. Never use rerun churn to mask a source defect. Roll back the smallest protected-main change, disable the narrow caller, and retain incident evidence for reacceptance. + +For continuation-policy regressions, roll back only the external prompt/configuration delta when it causes unsafe selection; writer leases and GitHub gates remain intact. Re-enable the repaired continuation policy only after a deterministic queue scenario proves a blocked lane rotates to another safe lane and the exit-sweep contract behaves as specified. diff --git a/docs/automation/PRD.md b/docs/automation/PRD.md new file mode 100644 index 000000000..bc7dd0b5f --- /dev/null +++ b/docs/automation/PRD.md @@ -0,0 +1,68 @@ +# CWL Automation Control Plane — Product Requirements + +Status: active_pr + +## Purpose + +The ContextualWisdomLab organization automation control plane continuously moves repositories toward safe integration and commercial readiness without turning blocked PRs, queued checks, reviewer latency, or status narration into repository-wide idle time. + +## Stakeholders + +- Repository maintainers who need review → repair → exact-head verification → merge continuity. +- Product owners who need development to continue when one integration lane is blocked. +- Security and governance reviewers who require evidence separation, least privilege, auditable automation, and fail-closed behavior. +- Operators who need actionable incident state rather than repeated blocker reports. + +## Product modes + +### PR maintenance + +1. Refetch live PR head and independently resolved live base tip. +2. Collect human and automated review evidence, checks, statuses, rulesets, and security gates. +3. Perform RCA for every non-passing or surprising gate. +4. Enumerate materially distinct remedies and verify real-world feasibility. +5. Execute the smallest safe root-cause-changing remedy test-first. +6. Revalidate the unchanged exact head. +7. Merge only when actual repository policy is satisfied. +8. Rotate immediately to another safe lane while any affected item waits. + +### Product development + +When integration work cannot consume the available execution budget, select a bounded high-value repository or control-plane gap, implement it test-first, document it, publish it for review, and return to the integration queue. + +### Continuation and conversation handoff + +The control plane treats a prompt update, documentation update, blocked PR, or user-visible status as an intermediate event rather than a completion signal. If another safe executable lane exists, the same invocation continues into that lane. A status response that merely restates one blocked item while other executable work exists is a premature-termination defect in the control plane. + +Durable decisions established in the CWL project conversation are evidence inputs, not the canonical specification. Material decisions that affect automation behavior, evidence authority, writer leases, product boundaries, security, operations, or acceptance criteria must be reconciled into the existing canonical GitHub documentation line with an explicit implementation-status classification. Conversation history and downloadable planning artifacts must never be silently promoted to `implemented_on_protected_main`. + +## Non-goals + +- Manufacturing approvals or weakening branch protection. +- Treating model verdicts, statuses, or predecessor evidence as independent human approval. +- Racing a repository's dedicated writer. +- Using status reports as completion. +- Copying central implementation into every product repository. +- Claiming protected-main operational closure from a source PR alone. +- Creating a parallel documentation authority when a canonical documentation branch already owns the topic. + +## Acceptance criteria + +- Waiting is scoped to the exact affected PR/action. +- Every actionable failure follows RCA → distinct remedies → feasibility → action → proof. +- Source-head, live-base, workflow, check, status, review, model, merge, release, and protected-main runtime evidence remain distinct authorities. +- Writer leases are explicit and branch-local where possible. +- Model credentials are materialized only on model-backed paths; autonomous development uses `NVIDIA_NIM_API_KEY`, not `COPILOT_GITHUB_TOKEN`. +- Routine runs do not stop after one commit, review request, merge, documentation update, prompt update, user-visible status, or blocker. +- Prompt repair and documentation fitness work hand off to the next safe GitHub action in the same invocation when one exists. +- Before termination, a first fresh whole-queue sweep finds no safe executable work; after any action triggered by that sweep, a second fresh sweep must also find no execute-now item, unless the practical invocation budget is exhausted. +- Canonical PRD/TRD/Architecture/ADR/UML/domain-model/security/operability/traceability/documentation-audit material is code-current and machine-checked. +- External scheduled agent/orchestrator state and GitHub-native workflow execution/evidence are modeled as distinct control-plane boundaries. + +## Degraded behavior + +Provider outages, rate limits, queued checks, missing external approval, and read-only dependencies defer only the affected lane. Deterministic work, other PRs, documentation repair, operational acceptance, and bounded product work continue when safe. + +## Implementation status vocabulary + +Canonical documents use only: `implemented_on_protected_main`, `active_pr`, `accepted_architecture`, `planned`, `research_only`, `superseded`, and `out_of_scope`. diff --git a/docs/automation/README.md b/docs/automation/README.md new file mode 100644 index 000000000..4beef511f --- /dev/null +++ b/docs/automation/README.md @@ -0,0 +1,41 @@ +# Automation control-plane documentation + +Status: active_pr + +This directory is the canonical documentation graph for the organization automation control plane. It separates shipped behavior, active work, accepted architecture, plans, research, superseded decisions, and out-of-scope proposals. Pull-request bodies, conversational history, automation prompts, and downloadable planning artifacts are evidence inputs, not the durable specification. + +The control plane spans three distinct authorities: external scheduled agent/orchestrator state, GitHub execution/evidence/policy state, and this canonical documentation graph. A durable decision is reconciled here only after live implementation and active ownership are refetched; product-specific specifications remain in their owning repositories. + +## Canonical documents + +- [Product requirements](PRD.md) +- [Technical requirements](TRD.md) +- [Architecture](ARCHITECTURE.md) +- [Conceptual data model](DATA_MODEL.md) +- [UML diagrams](UML.md) +- [Security](SECURITY.md) +- [Threat model](THREAT_MODEL.md) +- [Test strategy](TEST_STRATEGY.md) +- [Operability](OPERABILITY.md) +- [Incident runbook](INCIDENT_RUNBOOK.md) +- [Traceability](TRACEABILITY.md) +- [Documentation fitness audit](DOCUMENTATION_AUDIT.md) +- [Architecture decision records](adr/README.md) + +## Status vocabulary + +Every canonical document declares exactly one status: + +- `implemented_on_protected_main`: directly verified on the current protected branch. +- `active_pr`: proposed by an open pull request and not yet shipped. +- `accepted_architecture`: accepted direction whose implementation may be incomplete. +- `planned`: approved backlog without implementation evidence. +- `research_only`: exploratory material that does not authorize production behavior. +- `superseded`: retained only for historical traceability. +- `out_of_scope`: explicitly excluded from the control plane. + +## Change discipline + +A behavioral change updates requirements, architecture, security, tests, operations, traceability, documentation fitness, and affected ADRs in the same integration line. Exact commit identities belong in dated evidence or pull-request records, not timeless architecture. Operational incidents close only after protected-main consumer evidence proves the repaired boundary. + +A prompt update or documentation update is not run completion when another safe executable lane exists. After canonical documentation mutation, the authoritative writer returns to the live executable queue and applies the continuation/exit-sweep contract in ADR-0003 and ADR-0010. diff --git a/docs/automation/SECURITY.md b/docs/automation/SECURITY.md new file mode 100644 index 000000000..8998aa43d --- /dev/null +++ b/docs/automation/SECURITY.md @@ -0,0 +1,35 @@ +# Automation control-plane security + +Status: active_pr + +## Security objectives + +The control plane preserves source integrity, least privilege, evidence provenance, reviewer independence, credential confidentiality, and fail-closed merge/release decisions while continuing unrelated safe work. + +## Trust boundaries + +1. Pull-request source, titles, bodies, comments, patches, artifacts, logs, and model prompts are untrusted. +2. A workflow is trusted only when its source repository and immutable revision are established independently of PR-controlled input. +3. GitHub checks, commit statuses, formal reviews, model outputs, merge authority, release authority, and runtime acceptance are distinct channels. +4. Repository tokens, GitHub Apps, OIDC identities, and model credentials have separate purposes and cannot substitute for one another. +5. Leaf repositories may supply bounded inputs but cannot self-enable privileged central policy. + +## Credential contract + +- Deterministic gates run before model credentials are required or materialized. +- Reusable workflows declare named minimal secrets; blanket `secrets: inherit` is rejected unless a separately reviewed interface proves every inherited value necessary. +- Autonomous model development uses `NVIDIA_NIM_API_KEY`; `COPILOT_GITHUB_TOKEN` is not a development-model credential. +- OIDC exchanges validate audience, issuer, repository/ref claims, expiry, and the documented response envelope without logging tokens. +- Provider credentials are passed only to the child process that calls that provider and are absent from arguments, reports, summaries, and artifacts. + +## Source and evidence integrity + +Writes bind exact branch head, target blob/ref, and independently resolved live base. Stale PR base metadata, synthetic merge commits, predecessor runs, skipped required checks, and status-only reviewer text cannot authorize a write or merge. + +## Supply-chain and execution controls + +Actions and trusted bootstrap inputs use immutable pins and verified hashes where practical. PR-controlled execution runs tokenless, with bounded filesystem access, output limits, timeouts, process-group cleanup, and network denial unless an explicit reviewed operation requires egress. + +## Rollback + +Disable the narrow caller or revert the exact protected-main integration commit. Preserve evidence, revoke affected credentials, cancel only obsolete runs, and reopen the incident until a current protected-main consumer proves recovery. diff --git a/docs/automation/TEST_STRATEGY.md b/docs/automation/TEST_STRATEGY.md new file mode 100644 index 000000000..78bd0cab6 --- /dev/null +++ b/docs/automation/TEST_STRATEGY.md @@ -0,0 +1,41 @@ +# Automation control-plane test strategy + +Status: active_pr + +## Test layers + +1. Static contracts validate immutable pins, explicit secret interfaces, workflow provenance, required paths, status vocabulary, external-orchestrator/GitHub authority separation, canonical documentation ownership, and forbidden stale literals. +2. Unit/property tests cover normalization, evidence classification, exact-head/live-base identity, retry classes, queue lanes, writer leases, defer identities, continuation handoffs, double-exit semantics, and output redaction. +3. Integration tests exercise workflow event envelopes, OIDC/App claims, artifact handoff, concurrency isolation, negative controls, and lane rotation when one exact PR/action is blocked. +4. Exact-head GitHub evidence proves the unchanged PR head passed required product/security/dependency checks. +5. Protected-main acceptance exercises scheduled/manual real consumers after integration. +6. External automation acceptance verifies that a prompt/control update or status event returns through queue selection rather than terminating while a separate safe lane remains. + +## Permanent negative cases + +Tests must reject stale heads, synthetic merge evidence, skipped-required checks, status-only approval, author approval, spoofed reviewer identity, prompt-injected shell fragments, untrusted redirects, ref/auth/TLS retry, arbitrary output paths, secret-shaped stdout/stderr/service tails, PR self-enablement of privileged policy, and conversation/prompt text presented as protected-main implementation. + +The continuation contract additionally treats these as failures: + +- terminating after a prompt update while a safe GitHub lane is executable; +- terminating after a documentation audit/update while another safe lane is executable; +- terminating after one merge, review request, RCA, or defer decision without queue reselection; +- treating a user-visible status response as an exit condition; +- performing only one exit sweep when it discovers work; and +- creating a competing documentation authority when a canonical line already owns the scope. + +## Documentation fitness + +`tests/test_automation_documentation.py` verifies required documents and indexes, controlled implementation status, Mermaid fence integrity, absence of ephemeral full SHAs in timeless documents, detailed ADR coverage through ADR-0010, the documentation-fitness audit, continuation/reconciliation invariants in core documents, and traceability to live workflow paths. `.github/workflows/automation-documentation-quality-ci.yml` runs the contract on the exact source revision. + +Documentation tests are intentionally dependency-free and should fail before implementation/documentation repair when a new canonical artifact or invariant is introduced. A documentation file being present is necessary but not sufficient; the controlled maturity state prevents active or planned behavior from being mistaken for protected-main implementation. + +## Acceptance evidence + +A green run is valid only for the commit actually checked out. Pending, cancelled, rate-limited, stale, predecessor, synthetic-merge, skipped-required, absent, or infrastructure-only results are not success. Operational repairs additionally require a protected-main consumer run and a negative control. + +For premature-termination repairs, acceptance also requires runtime evidence that an exact blocked lane is deferred and a different safe lane is selected before termination. A prompt/configuration diff alone is not operational closure. + +## Coverage and docstrings + +Owned production code targets 100% statement and branch coverage and public callable docstrings. Test-only documentation contracts are deterministic and dependency-free so they can run before model credentials or network access. diff --git a/docs/automation/THREAT_MODEL.md b/docs/automation/THREAT_MODEL.md new file mode 100644 index 000000000..23cf101bf --- /dev/null +++ b/docs/automation/THREAT_MODEL.md @@ -0,0 +1,33 @@ +# Automation control-plane threat model + +Status: active_pr + +## Assets and adversaries + +Assets include protected refs, workflow definitions, reviewer identity, merge/release authority, OIDC/App credentials, model secrets, evidence artifacts, writer-lease state, finite scheduled execution capacity, the canonical documentation authority, and operator trust. Adversaries include malicious pull-request authors, compromised dependencies/actions, prompt-injected comments or source, confused-deputy callers, spoofed reviewers, accidental automation races, and control-plane configuration drift. + +## Threats and controls + +| Threat | First failing boundary | Preventive controls | Detection and recovery | +|---|---|---|---| +| Token or model-secret disclosure | Secret materialization/logging | Minimal named secrets, child-only environment, redaction, no PR-controlled echo | Secret scan, synthetic credential-shaped tests, revoke and replay | +| Excessive `secrets: inherit` | Reusable-workflow interface | Explicit per-secret contracts and deterministic gates before secrets | Documentation/test contract and workflow review | +| Confused deputy via dispatch | Caller/target identity binding | Allowlisted repository, event, ref, source revision, audience and permission checks | Reject receipt with bounded identity evidence | +| Stale or synthetic evidence | Evidence-to-head binding | Exact source head plus independently resolved live base | Recollect exact-head evidence; never transfer predecessor success | +| Malicious PR content or prompt injection | Untrusted content enters model/shell | Fixed argv, quoted data channels, no arbitrary evaluation, schema validation | Adversarial fixtures and abstaining model verdicts | +| Reviewer spoofing | Review eligibility | GitHub-counted non-author formal review on the exact head | Ignore comments, statuses, reactions, dismissed and stale reviews | +| Supply-chain compromise | Action/bootstrap provenance | Immutable action pins, hash verification, origin/redirect validation | Fail closed, isolate provider, restore trusted pin | +| Unbounded egress or SSRF | Network request construction | Trusted-origin policy, redirect revalidation, allowlists and timeouts | Egress logs, negative redirect/private-address tests | +| Writer race | Branch/ref mutation | Branch-local lease, refetch-before-write, non-force CAS/fast-forward | Abort on moved head; reconcile read-only | +| Gate weakening during outage | Failure classification | Integrity/auth/TLS/ref errors never retried or downgraded | Incident classification, rollback and protected-main replay | +| Premature termination / queue starvation | External scheduler continuation policy | Work-conserving queue, branch-local defer identity, same-invocation reselection, double exit sweep | Detect safe executable lane after terminal output; repair prompt/control and resume queue | +| Split documentation authority | Conversation/prompt/PR body bypasses canonical owner | One indexed canonical documentation line, explicit central-vs-leaf ownership, maturity vocabulary, reconciliation algorithm | Documentation fitness audit, supersession trace, freeze only conflicting documentation lane | +| External automation authority conflation | Scheduler state treated as GitHub evidence | Separate external orchestration, GitHub execution/evidence, and canonical documentation planes | Reject cross-channel substitution; require source/check/review/runtime evidence from its own authority | + +## Assumptions + +GitHub protection and organization policy remain external governance authorities. Automated reviewers are advisory and cannot create human approval. A provider outage can delay one lane but cannot authorize bypass. External scheduler/orchestrator configuration can assign an accepted writer lease but cannot manufacture GitHub source, check, review, merge, or protected-main acceptance evidence. + +## Residual risks + +Hosted-runner saturation, provider outages, organization billing/policy, reviewer capacity, external scheduler outages, and incomplete leaf-repository documentation audits may remain external or distributed. They are recorded as exact prerequisites while other safe lanes continue. Conversation-wide product documentation completeness cannot be centrally claimed until each owning repository has been audited under its own writer lease. diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md new file mode 100644 index 000000000..191fd5bc3 --- /dev/null +++ b/docs/automation/TRACEABILITY.md @@ -0,0 +1,45 @@ +# Automation control-plane traceability + +Status: active_pr + +## Implementation map + +| Requirement or decision | Protected-main / authority surface | Verification surface | Status | +|---|---|---|---| +| Central reusable review contract | `.github/workflows/opencode-review.yml` | workflow contract and exact-head security tests | implemented_on_protected_main | +| Review dispatch and consumer evidence | `.github/workflows/opencode-review-dispatch.yml` | dispatch-envelope, OIDC, concurrency and protected-main canaries | implemented_on_protected_main | +| Merge/reviewer authority separation | `.github/workflows/pr-review-merge-scheduler.yml` | scheduler unit/contracts plus counted review evidence | implemented_on_protected_main | +| Ref-safe automated maintenance | `.github/workflows/pr-auto-rebase.yml` | stale-head, label and non-force update tests | implemented_on_protected_main | +| Canonical documentation graph | `docs/automation/**` | `tests/test_automation_documentation.py` | active_pr | +| Documentation fitness audit | `docs/automation/DOCUMENTATION_AUDIT.md` | required-doc/index/status/continuation contract | active_pr | +| Writer lease and work-conserving queue | ADR-0001 and ADR-0003 | external scheduler state + runtime queue/branch-head CAS behavior | accepted_architecture | +| Same-invocation continuation and conversation/documentation handoff | ADR-0010, PRD/TRD/Architecture/UML/Data Model | external prompt/configuration plus runtime lane-rotation evidence and documentation fitness | active_pr | +| External orchestration vs GitHub execution/evidence authority | Architecture/TRD/Data Model | documentation contract plus future runtime acceptance | accepted_architecture | +| Trusted bootstrap retry classes | ADR-0005 | transient/permanent classification tests and consumer replay | active_pr | +| Explicit secret interfaces | ADR-0006 | workflow/static contracts and secret-shaped negative tests | active_pr | +| Independent review governance | ADR-0007 and issue #772 | GitHub formal review plus exact-head eligibility tests | accepted_architecture | +| Protected-main closure | ADR-0008 | scheduled/manual consumer receipt, negative control and rollback | accepted_architecture | +| Central versus leaf ownership | ADR-0009 | reusable-workflow callers, source-repository tests, and documentation handoff audit | accepted_architecture | +| Product-specific PRD/TRD/ADR/UML/ERD | owning leaf repositories | repository-specific documentation fitness under each writer lease | planned | + +## Authority boundaries + +External scheduled agent/orchestrator configuration can establish invocation cadence and writer-lease ownership, but it does not substitute for GitHub source, check, status, review, merge, or protected-main runtime evidence. GitHub-native evidence does not by itself prove that an external work-conserving loop continued after an intermediate event. The canonical documentation graph records both authorities and their handoff without collapsing them. + +Conversation history, prompt text, PR bodies, incident comments, and downloadable planning artifacts are candidate evidence. A durable shared decision becomes canonical only after live implementation/ownership is refetched and the existing documentation line is updated with the controlled maturity state. Product-specific decisions are handed to the owning product repository instead of being duplicated centrally. + +## Incident linkage + +Current incidents and active pull requests remain dated evidence, not shipped claims. Their exact SHAs and run identifiers belong in the issue/PR record. This matrix changes to `implemented_on_protected_main` only after protected integration and, for operational defects, real consumer acceptance. + +A premature-termination incident remains open until the authoritative external continuation policy is corrected **and** runtime evidence proves that a blocked/intermediate lane rotates to another safe executable lane before a valid double-exit termination. A prompt diff or documentation update alone is not operational closure. + +## Primary references + +GitHub. (n.d.). *About protected branches*. https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches + +GitHub. (n.d.). *Reusing workflow configurations*. https://docs.github.com/actions/using-workflows/reusing-workflows + +National Institute of Standards and Technology. (2022). *Secure Software Development Framework (SSDF) version 1.1* (SP 800-218). https://doi.org/10.6028/NIST.SP.800-218 + +Open Source Security Foundation. (2025). *SLSA specification, version 1.1*. https://slsa.dev/spec/v1.1/ diff --git a/docs/automation/TRD.md b/docs/automation/TRD.md new file mode 100644 index 000000000..f835561ba --- /dev/null +++ b/docs/automation/TRD.md @@ -0,0 +1,82 @@ +# CWL Automation Control Plane — Technical Requirements + +Status: active_pr + +## Evidence identity + +A decision must bind separately to: + +- `source_revision`: exact PR head commit. +- `base_revision`: independently resolved current tip of the target base ref. +- `workflow_evidence`: workflow/run/job identity and the commit actually checked out. +- `check_evidence`: check name, state, conclusion, and checked revision. +- `review_evidence`: reviewer identity, formal state, reviewed revision where known, and thread state. +- `status_evidence`: commit-status context, creator, state, and revision. +- `operational_acceptance`: protected-main scheduled/manual consumer execution after integration. +- `automation_control_record`: external scheduler/orchestrator configuration and enabled state when it grants or withholds a writer lease. +- `continuation_handoff`: the proof that an intermediate action was followed by the next selected executable lane, or by a valid double-exit conclusion. + +No evidence type silently substitutes for another. + +## Gate taxonomy + +`success` is exact-current evidence only. `queued`, `pending`, `cancelled`, `skipped-required`, `neutral-required`, `absent`, `failed`, stale-head, predecessor-head, synthetic-only, and infrastructure-only evidence are not success. + +## RCA contract + +For each non-passing gate capture the first failing boundary, trigger, rendered inputs, permissions, credential boundary, immediate cause, technical root cause, systemic/control cause when material, correction owner, and a falsifiable hypothesis. + +Generate materially distinct remedies. Before mutation verify current API/tool support, authority, credential scope, reviewer eligibility, rulesets, workflow semantics, dependency order, writer lease, rate/runtime constraints, blast radius, rollback, security/privacy impact, and exact acceptance evidence. + +## Writer lease + +A repository with an enabled dedicated writer loop is read-only to other writer loops. Before every write, refetch exact target head/base/blob/ref and relevant review state. Source/ref/blob movement or another active writer targeting the same branch invalidates the write lease for that branch. Review/check completion alone is not a writer conflict. + +External automation state is a real lease input but is not GitHub source evidence. A scheduled agent/orchestrator can own a repository write lease while GitHub Actions, reviews, and checks provide execution or validation evidence. The control plane records those authorities separately and revalidates both before mutation. + +## Concurrency and retries + +- Pending external evidence is deferred by exact identity; unchanged deferred items are not repeatedly polled. +- Transient transport/bootstrap failures may receive bounded classified retries. +- Integrity, authentication, TLS, ref mismatch, stale-head, permission, and policy failures remain fail-closed. +- A retry must be capable of changing the diagnosed causal boundary. +- A blocked execution lane never consumes the entire run while another independent safe lane is executable. + +## Continuation and termination contract + +Every finite invocation maintains a live executable queue. After each mutation, merge, review request, RCA, documentation change, prompt change, defer decision, or operational proof, the scheduler selects the next executable lane before considering termination. + +A `continuation_handoff` records at least the completed lane identity, the newly observed state that caused defer or completion, and the next selected lane or termination reason. User-visible output is not itself a valid termination reason. + +Termination requires a double exit sweep: + +1. the first fresh queue-wide sweep must find no execute-now item; if it finds one, execute the highest-value safe item and restart continuation selection; +2. a second fresh queue-wide sweep after the final substantive action must also find no execute-now item, unless the practical invocation/tool budget is exhausted. + +The double exit sweep covers source repairs, merges, addressed review-thread resolution, duplicate closure, Draft/issue advancement, protected-main acceptance, canonical documentation repair, tests/security/reliability work, release preparation, and bounded product/control-plane development. + +## Conversation-to-repository reconciliation + +Conversation history, uploaded planning packs, automation prompts, PR bodies, and incident comments are evidence inputs. They do not become durable specification until reconciled into the canonical GitHub documentation line. Reconciliation must: + +- refetch protected-main implementation and relevant active PRs first; +- update the existing canonical documentation branch rather than create a parallel authority; +- assign the controlled maturity state to every material decision; +- preserve disagreement or supersession instead of silently combining incompatible claims; and +- update machine-checkable documentation contracts when a new required authority, document, or invariant is introduced. + +## Secrets and model paths + +Deterministic gates execute before model-secret materialization whenever possible. Reusable workflows declare minimal explicit secret contracts. Blanket inheritance is avoided unless independently justified. Development/model-backed automation uses `NVIDIA_NIM_API_KEY`; `COPILOT_GITHUB_TOKEN` is not a development-model credential. + +## Approval authority + +Automated reviewer/model evidence is advisory unless live repository policy explicitly says otherwise. COMMENTED reviews, statuses, checks, reactions, author evidence, dismissed reviews, textual approval, and predecessor-head reviews never become qualifying independent approval by inference. + +## Operational closure + +A merged source repair is not operational closure. For control-plane incidents, closure requires protected-main scheduled/manual consumer evidence on the integrated implementation, with the expected failure mode absent and the intended bounded behavior observed. + +## Documentation fitness + +Canonical documentation must distinguish implementation maturity and map requirements/ADRs to workflows, scripts, tests, incidents, external automation boundaries, and acceptance evidence. Documentation tests should validate links, state names, secret contracts, continuation terms, diagram integrity, ADR index consistency, documentation-audit presence, and stale terminology where practical. diff --git a/docs/automation/UML.md b/docs/automation/UML.md new file mode 100644 index 000000000..20a00a542 --- /dev/null +++ b/docs/automation/UML.md @@ -0,0 +1,155 @@ +# CWL Automation Control Plane — UML Views + +Status: active_pr + +## PR maintenance sequence + +```mermaid +sequenceDiagram + participant S as Scheduler + participant E as Evidence + participant F as Feasibility + participant W as Writer + participant C as Checks/Reviews + participant M as Merge + S->>E: Refetch exact head + live base + gates + E->>F: Non-passing evidence + F->>F: RCA and distinct remedy evaluation + alt executable remedy + F->>W: Exact bounded mutation + W->>C: Exact-head verification + C-->>S: Authority-separated evidence + else waiting/external/read-only + F-->>S: Defer exact lane identity + end + S->>M: Merge only if real gates pass + S->>S: Rotate to next executable lane +``` + +## Product development sequence + +```mermaid +sequenceDiagram + participant S as Scheduler + participant D as Deterministic gates + participant L as Optional model path + participant R as Repository + S->>D: Confirm no higher-priority executable integration work + D->>D: Scope, lease, security and release gates + alt model is materially required + D->>L: Materialize NVIDIA_NIM_API_KEY only here + L-->>D: Bounded proposal/evidence + end + D->>R: Test-first bounded implementation + R-->>S: Exact-head evidence + S->>S: Return to integration queue +``` + +## Continuation and handoff state machine + +```mermaid +stateDiagram-v2 + [*] --> QueueRefetch + QueueRefetch --> Execute: executable lane exists + QueueRefetch --> ExitSweepOne: no executable lane observed + Execute --> RefetchAffectedState: action, merge, RCA, doc or prompt mutation + RefetchAffectedState --> Execute: another safe lane exists + RefetchAffectedState --> ExitSweepOne: no execute-now lane observed + ExitSweepOne --> Execute: first sweep finds work + ExitSweepOne --> ExitSweepTwo: first sweep finds none + ExitSweepTwo --> Execute: second sweep finds work + ExitSweepTwo --> BoundedTermination: second sweep finds none + BoundedTermination --> [*] +``` + +A user-visible status, prompt update, review request, merge, documentation edit, or defer decision never transitions directly to `BoundedTermination`. It must return through queue selection and the required exit sweeps unless the practical invocation/tool budget is exhausted. + +## Conversation-to-repository reconciliation + +```mermaid +sequenceDiagram + participant H as Conversation/Prompt/Artifact + participant G as GitHub Live State + participant C as Canonical Documentation + participant T as Documentation Contract + participant Q as Executable Queue + H->>G: Identify material durable decision candidate + G->>G: Refetch protected main + active PR implementation + G->>C: Classify shipped/active/accepted/planned/research/superseded/out-of-scope + alt canonical line already exists + C->>C: Extend existing authority + else no canonical line exists + C->>C: Create one discoverable authority + end + C->>T: Update affected PRD/TRD/Architecture/ADR/UML/Data Model/Security/Operations/Traceability + T-->>C: Machine-check fitness + C->>Q: continuation_handoff to next executable lane +``` + +## Evidence/gate state machine + +```mermaid +stateDiagram-v2 + [*] --> Observed + Observed --> Actionable: repository-owned root cause + Observed --> Deferred: pending/external/read-only + Actionable --> Verifying: mutation executed + Verifying --> Actionable: failed with new evidence + Verifying --> GateClean: exact-head gates pass + GateClean --> MergeWaiting: counted external approval only + GateClean --> Merged: all merge gates pass + MergeWaiting --> Merged: approval arrives, head unchanged + Merged --> OperationalAcceptance: runtime proof required + OperationalAcceptance --> Closed: protected-main evidence passes + Merged --> Closed: no runtime proof required + Deferred --> Observed: material state change +``` + +## Reviewer and merge authority + +```mermaid +flowchart LR + A[Automated model/reviewer] -->|advisory evidence| G[Gate evaluator] + H[Independent formal review] -->|counted only if eligible/current| G + C[Checks/statuses] -->|separate evidence| G + P[Rulesets/branch protection] --> G + G -->|all actual gates pass| M[Merge authority] + G -->|otherwise| D[Defer or remediate] +``` + +## External scheduler and GitHub authority + +```mermaid +flowchart LR + X[External scheduler/orchestrator] -->|invocation + writer lease| Q[Execution queue] + Q -->|read/write under lease| R[GitHub repository] + R --> C[Checks] + R --> V[Formal reviews] + R --> S[Commit statuses] + R --> W[Workflow runs] + C --> G[Gate evaluator] + V --> G + S --> G + W --> G + X -. cannot substitute .-> G +``` + +## Incident classification + +```mermaid +flowchart TD + F[Observed failure] --> R[RCA] + R --> T{Failure class} + T -->|transient transport/bootstrap| B[Bounded classified retry] + T -->|integrity/auth/TLS/ref/policy| X[Fail closed] + T -->|repository product defect| P[Test-first product repair] + T -->|central dependency| H[Read-only handoff to owner] + T -->|reviewer/provider capacity| D[Defer lane and rotate] + T -->|premature termination| C[Repair continuation policy and resume queue] + B --> V[Exact acceptance evidence] + X --> V + P --> V + H --> V + D --> V + C --> V +``` diff --git a/docs/automation/adr/0001-writer-leases.md b/docs/automation/adr/0001-writer-leases.md new file mode 100644 index 000000000..7799bd775 --- /dev/null +++ b/docs/automation/adr/0001-writer-leases.md @@ -0,0 +1,37 @@ +# ADR-0001 — Dedicated repository writer leases and read-only fleet audit + +Status: active_pr + +## Context + +Concurrent autonomous writers can invalidate exact-head evidence, overwrite unrelated work, and create self-amplifying repair loops. + +## Alternatives + +1. Allow all loops to write everywhere. +2. Use a single organization-wide writer. +3. Assign one authoritative writer per repository and keep fleet audit read-only. + +## Decision + +Use option 3. Enabled dedicated loops own writes to their repository. General fleet development skips those repositories. Fleet incident auditing remains read-only. Conflicts are branch-local when evidence supports that scope. + +## Consequences + +This reduces races and preserves modular repository ownership, at the cost of explicit handoffs for central dependencies. + +## Failure and recovery + +If another writer moves the target ref/blob/head between required reads, discard stale assumptions, freeze writes to that branch for the invocation, and rotate. Reacquire from fresh evidence later. + +## Security and governance + +Never bypass the lease using one-shot workflows, force-push, alternate credentials, or a second autonomous writer. + +## Acceptance + +Tests and automation contracts must demonstrate lease derivation, stale-write refusal, and read-only fleet audit behavior. + +## Supersession + +Supersede only if a stronger transaction/lease mechanism provides equivalent or better race prevention across all supported write paths. diff --git a/docs/automation/adr/0002-exact-head-live-base.md b/docs/automation/adr/0002-exact-head-live-base.md new file mode 100644 index 000000000..ee09a5cdf --- /dev/null +++ b/docs/automation/adr/0002-exact-head-live-base.md @@ -0,0 +1,37 @@ +# ADR-0002 — Exact source-head plus independently resolved live-base evidence + +Status: active_pr + +## Context + +A PR API base SHA can be a historical snapshot and does not necessarily represent the current protected base-ref tip. Reviews and checks may also refer to older heads. + +## Alternatives + +1. Trust PR metadata snapshots. +2. Track only source head. +3. Bind decisions to exact source head and independently resolve the current live base tip. + +## Decision + +Use option 3. Source revision and live base revision are separate evidence identities. Exact-head verification never transfers after a source-head change; base-sensitive acceptance is re-evaluated after material base movement. + +## Consequences + +More refetching is required, but stale-base and predecessor-evidence errors become explicit. + +## Failure and recovery + +If head/base/ref changes before mutation or merge, invalidate the affected decision and recompute from live state. + +## Security and governance + +Stale revision evidence cannot authorize source writes, approval sufficiency, or protected merge. + +## Acceptance + +Automation tests must include stale PR base metadata, moved heads, predecessor checks/reviews, and independently resolved base tips. + +## Supersession + +Supersede only if GitHub provides an equivalent atomic decision primitive that binds source, live target base, gates, and merge authority. diff --git a/docs/automation/adr/0003-work-conserving-execution.md b/docs/automation/adr/0003-work-conserving-execution.md new file mode 100644 index 000000000..99ac6a062 --- /dev/null +++ b/docs/automation/adr/0003-work-conserving-execution.md @@ -0,0 +1,41 @@ +# ADR-0003 — Work-conserving execution and no report-as-completion + +Status: active_pr + +## Context + +Autonomous maintenance previously risked ending after a single useful action or repeatedly narrating a blocker while other safe work remained. The same failure can occur after a prompt update, documentation audit, review request, merge, RCA, or user-visible status when the control plane mistakes an intermediate event for a valid invocation endpoint. + +## Alternatives + +1. One task per scheduled invocation. +2. Wait on the highest-priority blocked item. +3. Maintain a live executable queue and rotate around blocked lanes. + +## Decision + +Use option 3. A commit, review request, queued check, merge, documentation update, prompt update, RCA, defer decision, external blocker, or user-visible status is intermediate while another safe action exists. The hourly schedule is continuation after genuine finite budget exhaustion, not a one-action quota and not a substitute for same-invocation queue utilization. + +After every substantive action or defer decision, refetch enough live state to choose the next executable lane. Before termination, perform a fresh queue-wide sweep. If that sweep finds executable work, act and sweep again. A valid normal termination therefore requires a second fresh sweep with no execute-now item, unless the practical invocation/tool budget is exhausted. + +ADR-0010 extends this rule to conversation/prompt/documentation reconciliation and the explicit `continuation_handoff` authority boundary. + +## Consequences + +Invocations use more of their practical execution budget and reduce queue starvation. State management, defer identity, continuation handoff, and writer conflict handling must be stricter. External waits are intentionally local rather than invocation-global. + +## Failure and recovery + +Defer blocked items by exact identity and revisit only after material state change, another substantive action, or an exit sweep. If a terminal response is emitted while fresh evidence shows another safe lane was executable, classify the run as prematurely terminated, repair the authoritative continuation condition, and resume the missed or next safe lane in the same invocation when possible. + +## Security and governance + +Work conservation never authorizes weaker gates, unsafe parallel writers, fabricated approvals, speculative mutations, invented credentials, or cross-lease writes. Unsafe work is deferred; the queue rotates only to a separately safe lane. + +## Acceptance + +Automation contracts require double whole-queue exit sweeps and prohibit termination when any safe merge, fix, addressed-thread resolution, duplicate closure, Draft/issue advancement, documentation repair, operational proof, release preparation, test/security/reliability action, or bounded product/control-plane action remains. Runtime acceptance should include a scenario where one lane is blocked and another is demonstrably selected before termination. + +## Supersession + +Supersede only if a scheduler can prove equivalent or stronger queue utilization, same-invocation continuation, termination evidence, and safety with a different execution policy. diff --git a/docs/automation/adr/0004-evidence-authority-separation.md b/docs/automation/adr/0004-evidence-authority-separation.md new file mode 100644 index 000000000..bebefe374 --- /dev/null +++ b/docs/automation/adr/0004-evidence-authority-separation.md @@ -0,0 +1,37 @@ +# ADR-0004 — Separate evidence and decision authorities + +Status: active_pr + +## Context + +A green check, model verdict, COMMENTED review, source merge, or runtime execution answers a different question. Conflating these channels can create false source findings or unsafe merge conclusions. + +## Alternatives + +1. Collapse all green signals into one readiness flag. +2. Let automated reviewers determine merge authority. +3. Preserve independent evidence channels and compute decisions from explicit policy. + +## Decision + +Use option 3. Source semantic findings, infrastructure/policy blockers, checks, commit statuses, formal reviews, model judgments, merge authority, release authority, and protected-main operational acceptance remain separate. + +## Consequences + +Decision envelopes are more explicit and auditable. Consumers must handle `blocked` and `unknown` states without fabricating source defects. + +## Failure and recovery + +When evidence is stale, missing, contradictory, or unavailable, fail closed for the affected authority while continuing unrelated work. + +## Security and governance + +Automated/model evidence cannot impersonate qualifying independent approval. Infrastructure failures cannot gain source path/line authority merely because they block merge readiness. + +## Acceptance + +Tests must prove coverage/check failures do not synthesize source findings and that predecessor/status/model evidence cannot satisfy independent review requirements. + +## Supersession + +Supersede only with an evidence model that retains the same authority separations and auditability. diff --git a/docs/automation/adr/0005-bounded-bootstrap-retries.md b/docs/automation/adr/0005-bounded-bootstrap-retries.md new file mode 100644 index 000000000..0f4119efc --- /dev/null +++ b/docs/automation/adr/0005-bounded-bootstrap-retries.md @@ -0,0 +1,43 @@ +# ADR-0005 — Classify and bound trusted-bootstrap retries + +Status: active_pr + +## Context + +Fleet failures show that source materialization and trusted bootstrap can fail transiently, while integrity, authorization, TLS, immutable-ref, and origin failures indicate a different security boundary. + +## Drivers + +Improve reliability without converting permanent or adversarial failures into green evidence. Keep retry budgets observable and finite. + +## Alternatives + +1. Never retry. 2. Retry every failure. 3. Retry only classified transient failures within a fixed budget. + +## Decision + +Choose option 3. DNS resolution, connection reset, provider capacity, and explicit rate-limit classes may retry with bounded backoff. Hash/signature, auth, TLS, ref, origin/redirect, schema, and product-test failures fail immediately. + +## Consequences + +Transient incidents recover without widening trust. Classifiers and budgets become part of the reviewed interface. + +## Failure and recovery + +On budget exhaustion, report the exact external prerequisite and defer the lane. A misclassified permanent failure reopens the incident and rolls back the classifier. + +## Security and governance impact + +No retry may bypass provenance, integrity, or credential checks; logs remain redacted and bounded. + +## Tests and acceptance + +Permanent-class negative cases must show one attempt. Transient cases prove bounded attempts, backoff, final classification, and protected-main consumer recovery. + +## Migration and rollback + +Introduce classifiers behind existing fail-closed behavior. Revert the classifier or disable the narrow retry path if false recovery or amplification appears. + +## Supersession conditions + +Supersede only with a stronger typed failure protocol that preserves finite budgets and immediate security failures. diff --git a/docs/automation/adr/0006-explicit-secret-contracts.md b/docs/automation/adr/0006-explicit-secret-contracts.md new file mode 100644 index 000000000..a8f047ef6 --- /dev/null +++ b/docs/automation/adr/0006-explicit-secret-contracts.md @@ -0,0 +1,43 @@ +# ADR-0006 — Use explicit minimal reusable-workflow secrets + +Status: active_pr + +## Context + +Blanket secret inheritance obscures which credentials a reusable workflow needs and can materialize model or merge authority before deterministic gates. + +## Drivers + +Least privilege, auditable interfaces, confused-deputy resistance, and lower blast radius. + +## Alternatives + +1. Continue blanket inheritance. 2. Copy secrets into leaf repositories. 3. Declare named purpose-bound secrets and materialize them only on authorized jobs. + +## Decision + +Choose option 3. Deterministic open-PR, policy, source and product gates run without model secrets. OIDC/App authority is preferred where it satisfies the exact operation. + +## Consequences + +Callers become explicit and migrations may require coordinated central/leaf changes, but secret exposure and accidental coupling decrease. + +## Failure and recovery + +Missing optional model secrets defer only model-backed work. Missing required mutation authority fails closed with the exact permission boundary. + +## Security and governance impact + +Credentials cannot substitute across model, status, review, merge, release, or deployment authorities. Secret values never enter reports or arguments. + +## Tests and acceptance + +Static contracts reject unnecessary inheritance; synthetic credential-shaped tests cover stdout, stderr, timeout and service-tail paths; real OIDC/App canaries prove envelope consumption. + +## Migration and rollback + +Migrate one reusable interface at a time with compatibility telemetry, then remove inherited secrets. Roll back the caller/interface pair together. + +## Supersession conditions + +Supersede only with an equally explicit capability-based interface whose effective permissions are no broader. diff --git a/docs/automation/adr/0007-independent-review-governance.md b/docs/automation/adr/0007-independent-review-governance.md new file mode 100644 index 000000000..df7938c47 --- /dev/null +++ b/docs/automation/adr/0007-independent-review-governance.md @@ -0,0 +1,43 @@ +# ADR-0007 — Require counted independent exact-head review + +Status: active_pr + +## Context + +Automated comments, statuses and model verdicts are useful evidence but are not GitHub-counted independent human approval. Reviews can also become stale when the head moves. + +## Drivers + +Preserve real governance, reviewer identity, head binding and branch protection without idling other lanes. + +## Alternatives + +1. Count any positive text. 2. Allow author or alternate credentials. 3. Require a qualifying non-author formal approval that applies to the exact head. + +## Decision + +Choose option 3. Advisory automation remains separate. When approval is the sole gate, expected-head-safe auto-merge may remain enabled while other work continues. + +## Consequences + +Some integrations wait for human capacity; the repository never manufactures governance evidence. + +## Failure and recovery + +A moved head invalidates prior acceptance according to repository policy. Dismissed, commented, reaction, status and predecessor evidence are ignored. + +## Security and governance impact + +Prevents reviewer spoofing, self-approval and protection bypass. Reviewer eligibility is evaluated from GitHub identity and current policy. + +## Tests and acceptance + +Tests cover author, bot, dismissed, stale-head and status-only cases plus one qualifying reviewer. Merge evidence must bind the expected head. + +## Migration and rollback + +Adopt in the scheduler and documentation before removing legacy positive-text paths. Roll back by disabling merge automation, never by weakening protection. + +## Supersession conditions + +Supersede only if organization policy provides a stronger independently auditable approval mechanism. diff --git a/docs/automation/adr/0008-protected-main-acceptance.md b/docs/automation/adr/0008-protected-main-acceptance.md new file mode 100644 index 000000000..fe9695951 --- /dev/null +++ b/docs/automation/adr/0008-protected-main-acceptance.md @@ -0,0 +1,43 @@ +# ADR-0008 — Require protected-main operational acceptance + +Status: active_pr + +## Context + +A source PR can pass static and exact-head tests while a scheduled, manual, OIDC, dispatch, concurrency or downstream-consumer boundary still fails after merge. + +## Drivers + +Close operational incidents on real behavior, retain negative controls, and prove rollback. + +## Alternatives + +1. Close on PR checks. 2. Close on merge. 3. Require a protected-main consumer run through the repaired boundary. + +## Decision + +Choose option 3 for operational defects. Acceptance records the protected revision, event, consumer, acknowledgement, negative control and rollback rehearsal. + +## Consequences + +Incident closure takes longer but reflects deployed control-plane behavior rather than source intent. + +## Failure and recovery + +If the consumer does not exercise the boundary or evidence is stale, the incident remains open. Roll back the narrow integration on regression. + +## Security and governance impact + +Prevents false closure, stale evidence transfer and untested credential/dispatch behavior. + +## Tests and acceptance + +Require real scheduled/manual receipt, downstream acknowledgement, zero secret disclosure, failure-mode negative control, and observable rollback. + +## Migration and rollback + +Add acceptance criteria before merge and run the consumer immediately after integration. Revert the protected commit or disable the caller if acceptance fails. + +## Supersession conditions + +Supersede only with deployment verification that provides at least the same protected-runtime evidence. diff --git a/docs/automation/adr/0009-central-control-plane-ownership.md b/docs/automation/adr/0009-central-control-plane-ownership.md new file mode 100644 index 000000000..20c37543d --- /dev/null +++ b/docs/automation/adr/0009-central-control-plane-ownership.md @@ -0,0 +1,43 @@ +# ADR-0009 — Central control-plane ownership and thin leaf callers + +Status: active_pr + +## Context + +Copying automation into products causes policy drift, duplicated secrets, incompatible evidence and slow incident repair. Products still need independent operation and product-owned tests. + +## Drivers + +One reviewed organization control plane, stable explicit interfaces, modular products, and limited failure domains. + +## Alternatives + +1. Copy full workflows into every repository. 2. Centralize all product behavior. 3. Keep reusable policy centrally and use thin leaf callers/contracts. + +## Decision + +Choose option 3. The central repository owns cross-repository governance semantics; leaf repositories own product code, repository-specific gates and bounded caller inputs. + +## Consequences + +Central changes have broad leverage and require stronger compatibility testing. Leaf repositories remain independently operable when optional central/model services are unavailable. + +## Failure and recovery + +A central regression is isolated by versioned provenance and caller rollback. A leaf-specific defect remains in the leaf lease. + +## Security and governance impact + +Central interfaces enforce least privilege, exact caller identity and no privilege self-enablement from PR content. + +## Tests and acceptance + +Source-repository, fork, dispatch-envelope, secret-interface and compatibility tests cover central and leaf sides; protected-main consumers prove integration. + +## Migration and rollback + +Publish versioned interfaces, migrate leaf callers incrementally, and retain the last compatible pin during rollback. + +## Supersession conditions + +Supersede only if a new ownership model preserves single-source policy, modular product operation and explicit trust boundaries. diff --git a/docs/automation/adr/0010-continuation-documentation-handoff.md b/docs/automation/adr/0010-continuation-documentation-handoff.md new file mode 100644 index 000000000..78af2845c --- /dev/null +++ b/docs/automation/adr/0010-continuation-documentation-handoff.md @@ -0,0 +1,83 @@ +# ADR-0010 — Conversation, prompt, and documentation changes must hand off to executable work + +Status: active_pr + +## Context + +The control plane already adopted work-conserving execution in ADR-0003, but an observed failure mode remained: an invocation could correctly identify a blocked PR, update a prompt, assess documentation, or produce a user-visible status and then stop even though another safe repository action remained. The design also allowed durable decisions to remain scattered across conversation history, automation prompts, PR bodies, incident comments, and downloadable planning packs instead of being reconciled into the canonical GitHub documentation graph. + +This is not merely a reporting-style problem. Premature termination consumes scarce scheduled execution capacity, starves independent lanes, delays integration, and makes the declared work-conserving policy false in operation. Split documentation authority similarly creates contradictory product and security contracts. + +## Alternatives + +1. Treat user-visible status, prompt updates, or documentation assessments as valid end states and rely on the next hourly run. +2. Continue execution, but keep chat/prompt/PR-body decisions as equal specification authorities. +3. Require same-invocation continuation after every intermediate control/documentation action and require durable decisions to be reconciled into one canonical repository documentation authority. + +## Decision + +Use option 3. + +A prompt update, documentation audit, ADR/PRD/TRD/UML/ERD edit, review request, merge, RCA, defer decision, queued check, approval wait, rate limit, or user-visible status is an intermediate event while another safe executable lane exists. The finite invocation must immediately refetch enough state to choose and execute the next lane. + +Before any terminal response, the invocation performs a double exit sweep. If the first fresh sweep finds an execute-now item, it executes the highest-value safe item and resumes continuation. A second fresh sweep after the final substantive action must also find no execute-now item unless the practical invocation/tool budget is exhausted. + +Durable decisions from conversation, automation prompts, planning artifacts, PR bodies, and incident records are evidence inputs. They become specification only after live implementation is refetched, the canonical owner is resolved, the existing documentation line is extended rather than duplicated, and the decision is assigned the controlled maturity state. Shared control-plane decisions belong in the `.github` canonical graph; product behavior remains in the owning product repository. + +## Consequences + +### Positive + +- Scheduled execution capacity is consumed by repository progress rather than repeated narration. +- A blocked lane cannot implicitly block unrelated safe work. +- Prompt repair itself becomes testable control-plane work with a required repository handoff. +- Conversation-derived decisions become discoverable, reviewable, versioned, and traceable. +- Central and leaf repositories retain clear specification ownership. + +### Negative + +- Invocations perform more live-state reads and require stronger exact-identity/defer bookkeeping. +- Documentation changes can no longer be considered complete in isolation; the loop must return to the executable queue. +- Repository-by-repository conversation reconciliation is incremental and cannot be honestly declared complete without live leaf audits. + +## Failure and recovery + +A premature-termination incident exists when a run emits a terminal/status response while a fresh queue view proves another safe action was executable under the current writer lease and authority. + +Recovery: + +1. refetch the queue and identify the highest-value missed lane; +2. identify which prompt/control condition incorrectly treated the preceding event as terminal; +3. amend the authoritative automation prompt/configuration without weakening safety gates; +4. continue into the missed lane or next safe lane in the same invocation when possible; +5. update the canonical documentation/test contract if the failure exposed a missing durable invariant; +6. perform the double exit sweep before termination. + +If documentation authority is split, freeze only the conflicting documentation lane, identify the canonical owner from live repository state and accepted ADRs, migrate or supersede duplicate claims without erasing audit history, and restore one indexed authority. + +## Security and governance impact + +Continuation never authorizes bypassing branch protection, counted review, writer leases, security gates, secret boundaries, dependency order, or exact-head/live-base validation. A lane that is unsafe becomes deferred; work conservation selects a different safe lane. + +Conversation and prompt text are untrusted as implementation evidence. They cannot grant credentials, reviewer eligibility, write authority, merge authority, or `implemented_on_protected_main` status. External scheduler state can grant a writer lease only according to the accepted control-plane configuration; it does not become GitHub check/review/source evidence. + +## Acceptance evidence + +- The authoritative scheduled `.github` writer prompt explicitly states that prompt updates and user-visible output are non-terminal while safe work exists. +- PRD and TRD require same-invocation continuation and a double exit sweep. +- Architecture distinguishes the external orchestration, GitHub execution/evidence, and canonical documentation planes. +- The conceptual data model includes `execution_lane`, `deferred_item`, `continuation_handoff`, external automation control records, and documentation fitness. +- UML includes continuation and conversation-to-repository reconciliation flows. +- A documentation fitness audit records the previously missing boundaries and leaf-repository non-claims. +- The dependency-free documentation test requires these artifacts and invariants. +- Future runtime evidence demonstrates that a blocked PR causes lane rotation rather than a status-only termination. + +## Migration and rollback + +No GitHub runtime workflow, reviewer identity, credential, branch protection rule, or merge authority is changed by adopting this documentation decision. The external scheduled writer prompt is updated independently and can be rolled back to the prior configuration if continuation logic causes unsafe lane selection; writer leases and GitHub gates remain the safety boundary during rollback. + +If a future scheduler provides an equivalent formally verified queue/continuation mechanism, this ADR may be superseded together with ADR-0003. Supersession must preserve single documentation authority, same-invocation work conservation, explicit termination proof, and evidence-channel separation. + +## Supersession + +Supersede only with an accepted decision that proves an equal or stronger no-early-stop continuation contract and a versioned single-authority conversation-to-repository reconciliation mechanism. diff --git a/docs/automation/adr/README.md b/docs/automation/adr/README.md new file mode 100644 index 000000000..a82391f0f --- /dev/null +++ b/docs/automation/adr/README.md @@ -0,0 +1,14 @@ +# Automation architecture decision records + +Status: active_pr + +- [ADR-0001 — Dedicated repository writer leases](0001-writer-leases.md) +- [ADR-0002 — Bind exact source head and current live base](0002-exact-head-live-base.md) +- [ADR-0003 — Work-conserving execution](0003-work-conserving-execution.md) +- [ADR-0004 — Separate evidence and decision authorities](0004-evidence-authority-separation.md) +- [ADR-0005 — Classify and bound trusted-bootstrap retries](0005-bounded-bootstrap-retries.md) +- [ADR-0006 — Use explicit minimal reusable-workflow secrets](0006-explicit-secret-contracts.md) +- [ADR-0007 — Require counted independent exact-head review](0007-independent-review-governance.md) +- [ADR-0008 — Require protected-main operational acceptance](0008-protected-main-acceptance.md) +- [ADR-0009 — Central control-plane ownership and thin leaf callers](0009-central-control-plane-ownership.md) +- [ADR-0010 — Conversation, prompt, and documentation changes must hand off to executable work](0010-continuation-documentation-handoff.md) diff --git a/tests/test_automation_documentation.py b/tests/test_automation_documentation.py new file mode 100644 index 000000000..385339d38 --- /dev/null +++ b/tests/test_automation_documentation.py @@ -0,0 +1,126 @@ +#!/usr/bin/env python3 +"""Machine-check the canonical automation documentation graph.""" + +from __future__ import annotations + +import re +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +DOC_ROOT = ROOT / "docs" / "automation" +STATUS_VALUES = { + "implemented_on_protected_main", + "active_pr", + "accepted_architecture", + "planned", + "research_only", + "superseded", + "out_of_scope", +} +REQUIRED_DOCS = ( + "README.md", + "PRD.md", + "TRD.md", + "ARCHITECTURE.md", + "DATA_MODEL.md", + "UML.md", + "SECURITY.md", + "THREAT_MODEL.md", + "TEST_STRATEGY.md", + "OPERABILITY.md", + "INCIDENT_RUNBOOK.md", + "TRACEABILITY.md", + "DOCUMENTATION_AUDIT.md", +) +REQUIRED_ADRS = tuple(f"{number:04d}-" for number in range(1, 11)) +WORKFLOW_REFERENCES = ( + ".github/workflows/opencode-review.yml", + ".github/workflows/opencode-review-dispatch.yml", + ".github/workflows/pr-review-merge-scheduler.yml", + ".github/workflows/pr-auto-rebase.yml", +) +CONTINUATION_TERMS = { + "PRD.md": ( + "user-visible status", + "same invocation", + "conversation", + ), + "TRD.md": ( + "continuation_handoff", + "external automation", + "double exit sweep", + ), + "ARCHITECTURE.md": ( + "External orchestration plane", + "GitHub execution and evidence plane", + "canonical documentation", + ), + "DATA_MODEL.md": ( + "execution_lane", + "deferred_item", + "continuation_handoff", + ), + "UML.md": ( + "Continuation and handoff state machine", + "Conversation-to-repository reconciliation", + ), +} + + +class AutomationDocumentationContract(unittest.TestCase): + """Protect the code-current documentation graph from silent drift.""" + + def test_required_documents_and_statuses(self) -> None: + """Every canonical document exists and uses the controlled status vocabulary.""" + for relative_path in REQUIRED_DOCS: + path = DOC_ROOT / relative_path + self.assertTrue(path.is_file(), relative_path) + text = path.read_text(encoding="utf-8") + match = re.search(r"^Status: ([a-z_]+)$", text, re.MULTILINE) + self.assertIsNotNone(match, relative_path) + self.assertIn(match.group(1), STATUS_VALUES, relative_path) + + def test_indexes_cover_documents_and_adrs(self) -> None: + """Indexes link every required document and detailed ADR.""" + readme = (DOC_ROOT / "README.md").read_text(encoding="utf-8") + for relative_path in REQUIRED_DOCS[1:]: + self.assertIn(f"]({relative_path})", readme, relative_path) + + adr_root = DOC_ROOT / "adr" + index = (adr_root / "README.md").read_text(encoding="utf-8") + adr_names = sorted(path.name for path in adr_root.glob("[0-9][0-9][0-9][0-9]-*.md")) + for prefix in REQUIRED_ADRS: + self.assertTrue(any(name.startswith(prefix) for name in adr_names), prefix) + for name in adr_names: + self.assertIn(f"]({name})", index, name) + + def test_diagrams_and_immutable_architecture_are_well_formed(self) -> None: + """Markdown fences close and timeless architecture does not pin ephemeral SHAs.""" + full_sha = re.compile(r"(? None: + """Traceability maps the protected-main workflow surfaces that implement policy.""" + traceability = (DOC_ROOT / "TRACEABILITY.md").read_text(encoding="utf-8") + for workflow in WORKFLOW_REFERENCES: + self.assertTrue((ROOT / workflow).is_file(), workflow) + self.assertIn(workflow, traceability, workflow) + + def test_continuation_and_conversation_handoff_are_canonical(self) -> None: + """No-report continuation and chat-to-GitHub handoff stay explicit in core docs.""" + for relative_path, required_terms in CONTINUATION_TERMS.items(): + text = (DOC_ROOT / relative_path).read_text(encoding="utf-8") + for required_term in required_terms: + self.assertIn(required_term, text, f"{relative_path}: {required_term}") + + +if __name__ == "__main__": + unittest.main()