Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 38 additions & 8 deletions .github/workflows/bash-lint-auditor.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,23 @@ Confirm `shellcheck --version` runs and the version is `>= 0.9`.

## Step 3 — Baseline the Lint

Run the existing integration test under enforce mode and capture the result:
Shell is linted at two levels. Run both under enforce mode and capture the results:

```bash
# 1. Every registered script, in isolation, straight from the registry.
ENFORCE_BASH_LINT=1 cargo test --bin ado-aw compile::shell -- --nocapture 2>&1 | tee /tmp/lint-registry.log

# 2. Every bash body that actually reaches the emitted YAML.
ENFORCE_BASH_LINT=1 cargo test --test bash_lint_tests -- --nocapture 2>&1 | tee /tmp/lint-baseline.log
echo "exit=$?"
```

The registry lint (`src/compile/shell/lint.rs`) proves the shell is *correct*;
the integration test proves it is *emitted*. Coverage of the first is total by
construction, so a "no fixture reaches this generator" gap can no longer hide a
broken script — but it can still hide a script that is never emitted at all,
which is what the second catches.

There are three possible outcomes; each takes a different path.

**A. Lint is green (exit 0).** The PR gate is doing its job. Move to Step 4 and look for proactive improvements.
Expand All @@ -100,28 +110,48 @@ When the lint is already green, audit the *quality* of the bash hygiene story. D

### 4a. Stale disable directives

Find `# shellcheck disable=SCxxxx` directives that no longer fire on the bash body that contains them:
Find `# shellcheck disable=SCxxxx` directives that no longer fire on the shell body that contains them:

```bash
grep -rn "shellcheck disable=" src/data/ src/runtimes/ src/compile/ src/tools/ src/engine.rs 2>/dev/null
grep -rn "shellcheck disable=" src/data/ src/runtimes/ src/compile/ src/tools/ src/safe_outputs/ src/engine.rs 2>/dev/null
```

For each hit, temporarily delete the directive, rerun `cargo test --test bash_lint_tests -- --nocapture` (with `ENFORCE_BASH_LINT=1`), and check whether the test still passes. If the directive is now unnecessary (test still passes), remove it permanently. Restore the source file if the test fails.
For each hit, temporarily delete the directive, rerun both lint commands from Step 3 (with `ENFORCE_BASH_LINT=1`), and check whether they still pass. If the directive is now unnecessary (both still pass), remove it permanently. Restore the source file if either fails.

### 4b. Lint exclude-list audit

The lint excludes `SC1090,SC1091` globally (documented in `tests/bash_lint_tests.rs`). Check whether tightening would surface new findings:
The integration lint keeps a deliberately minimal global exclude list (documented in `tests/bash_lint_tests.rs`). Check whether tightening would surface new findings:

```bash
# Probe a stricter rule set
ENFORCE_BASH_LINT=1 cargo test --test bash_lint_tests 2>&1 | head -50
```

If you propose tightening, add a per-line `# shellcheck disable=` comment inside the offending bash body rather than expanding the global exclude list. Keep the exclude list minimal.
If you propose tightening, add a per-line `# shellcheck disable=` comment inside the offending body rather than expanding the global exclude list. Keep the exclude list minimal.

### 4c. Unstructured shell

Generated shell must go through `ShellScript` (`src/compile/shell/`, see the
*Generated shell scripts* section of `docs/extending.md`), which registers it
for linting and restricts substitution to a typed, quoted prelude. Shell built
with `format!` is invisible to the registry lint and reintroduces the escaping
that made these bodies unreviewable.

Look for shell still being assembled by hand:

```bash
# `\n\` continuations are the signature of a format!-built shell body
grep -rn '\\n\\' src/ --include=*.rs | grep -v '^src/compile/shell/'
```

If you find any, migrate it: move the body into a `shell_script!` const written
verbatim, and pass each interpolated value as a typed `Binding`. Do one file
per run — this is a mechanical change but a reviewable diff matters more than
volume.

### 4c. Expand fixture coverage
### 4d. Expand fixture coverage

Walk `src/runtimes/`, `src/tools/`, `src/compile/extensions/` and check whether every code path that emits a `- bash: |` step is exercised by some fixture. A generator that the lint never reaches is a generator with no quality story. Add a fixture (or extend an existing one) only if you find a real, currently-unreached generator.
Walk `src/runtimes/`, `src/tools/`, `src/compile/extensions/` and check whether every code path that emits a `- bash: |` step is exercised by some fixture. The registry lint already proves each script is *correct*; a fixture proves it is actually *emitted*. Add a fixture (or extend an existing one) only if you find a real, currently-unreached generator.

If none of 4a / 4b / 4c finds anything, **exit cleanly** — use the `noop` safe output with the message "Bash hygiene is current; no actionable findings."

Expand Down
35 changes: 29 additions & 6 deletions .github/workflows/review-compiler-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,12 +160,35 @@ New `CompilerExtension` implementations must be registered in
`collect_extensions()`, and new runtimes/tools documented in
`docs/runtimes.md` / `docs/tools.md` per `docs/extending.md`.

### Generated bash

Any new literal `bash:` body in generated pipeline YAML must survive
`cargo test --test bash_lint_tests` (shellcheck). Watch for `cd "$X"` without
`|| exit`, tilde inside double quotes, and masked return codes in assignments —
ADO's "fail on last command" default hides all three.
### Generated shell

Compiler-generated shell must go through `ShellScript` (`src/compile/shell/`,
documented in `docs/extending.md` under *Generated shell scripts*). Findings:

- **Shell built with `format!`.** A `\n\` continuation, a doubled `{{` brace or
an escaped `\"` inside a shell body means the script is not registered, so
neither the registry lint nor `export-bash-scripts` can see it — and the
escaping is what made these bodies unreviewable in the first place.
- **An interpolated value that is not a `Binding`.** Substitution must be a
`Binding::text` / `::number` / `::boolean` / `::words` / `::ado_macro` /
`::document`. Those land in exactly one position — the right-hand side of a
prelude assignment — and therefore cannot alter the structure of the script.
A value spliced anywhere else is an injection surface.
- **A credential in a binding.** The prelude is written verbatim into the
committed `*.lock.yml`. Credentials belong on
`.with_env(…, EnvValue::secret(…))`, which ADO masks.
- **An undeclared variable.** Everything a body reads must be declared in
`bindings:` or `externals:`. A missing `externals:` entry hides an
undocumented runtime coupling.
- **A `fragment` carrying control flow that the outline body depends on.** The
outline must remain valid shell without the fragment, or it cannot be linted.

Any body must survive both `ENFORCE_BASH_LINT=1 cargo test --bin ado-aw
compile::shell` (every registered script, in isolation) and
`ENFORCE_BASH_LINT=1 cargo test --test bash_lint_tests` (every body that
reaches emitted YAML). Watch for `cd "$X"` without `|| exit`, tilde inside
double quotes, and masked return codes in assignments — ADO's "fail on last
command" default hides all three.

## Step 4 — Documentation sync

Expand Down
70 changes: 60 additions & 10 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,12 @@ fail-closed and only pauses when the agent actually proposed a reviewed output.
│ │ ├── stage_ir.rs # Stage target typed-IR builder
│ │ ├── az_wrapper.rs # Renders the `az` CLI redirect wrapper installed into the agent sandbox (env-based `HTTPS_PROXY` redirect, not argument rewriting)
│ │ ├── source_path_guard.rs # Validation guard for untrusted workflow source-path inputs used by audit + mcp_author
│ │ ├── shell/ # Typed generation of every shell script the compiler emits (see docs/extending.md "Generated shell scripts")
│ │ │ ├── mod.rs # ShellScript: raw-string bodies + a typed shell-quoted binding prelude; `# ado-aw:fragment` splicing; into_step()
│ │ │ ├── bindings.rs # Binding constructors (text/number/boolean/words/ado_macro/document) — the single injection chokepoint; rejects credentials
│ │ │ ├── registry.rs # shell_script! macro + `inventory` auto-registration; ShellScriptDef; all_scripts()
│ │ │ ├── export.rs # `ado-aw export-bash-scripts` — materializes every registered script as reviewable .sh / JSON
│ │ │ └── lint.rs # Registry-wide shellcheck + declared-variable-surface guards (reaches scripts no fixture emits)
│ │ ├── gitattributes.rs # .gitattributes management for compiled pipelines
│ │ ├── filter_ir.rs # Filter expression IR: Fact/Predicate types, lowering, validation, codegen
│ │ ├── pr_filters.rs # PR trigger filter generation (native ADO + gate steps)
Expand Down Expand Up @@ -398,8 +404,8 @@ index to jump to the right page.
`remove`, `list`, `status`, `run`, `audit`, `mcp-author`, `trace`,
`inspect`, `graph`, `whatif`, `lint`, `catalog`; `configure` is a
deprecated hidden alias; `export-gate-schema`, `export-fact-catalog`,
`export-ado-proxy-catalog-schema`, and `export-ado-proxy-catalog` are hidden
build-time tools).
`export-ado-proxy-catalog-schema`, `export-ado-proxy-catalog`, and
`export-bash-scripts` are hidden build-time tools).
- [`docs/agency-plugin.md`](docs/agency-plugin.md) — the Agency / Claude Code
plugin (`agency/plugins/ado-aw/`): canonical layout, six skills, `mcp-author`
wiring, the self-contained root marketplace catalogs, `init --agency`
Expand Down Expand Up @@ -536,24 +542,68 @@ cargo test
cargo clippy
```

### Generated shell

Compiler-generated shell is **not** built with `format!`. Every script is a
raw-string constant registered with `shell_script!` in the module that
produces it, with substitution restricted to a typed, shell-quoted prelude —
see `src/compile/shell/` and the *Generated shell scripts* section of
[`docs/extending.md`](docs/extending.md).

The body is the shell exactly as it runs: no `\n\` continuations, no doubled
braces, no escaped quotes. A value reaches a script only as `Binding::text`,
`::number`, `::boolean`, `::words`, `::ado_macro` or `::document`, all of
which land in a single position (the right-hand side of a prelude assignment)
and therefore cannot alter the structure of the script. A credential must
never become a binding — the prelude is committed to the repository — so it
stays on `.with_env(…, EnvValue::secret(…))`.

Every variable a body reads must be declared as a `binding` or an `external`.
Both the render path and a registry-wide test enforce it.

`tests/generated_shell_guard.rs` fails the build if shell regresses to the old
shape — a `BashStep::new` whose script argument is built with `format!`, an
escaped continuation inside a `shell_script!` body, or a reintroduced
`bash()` / `dedent()` helper.

### Bash step lint

Shell is linted at two levels.

`src/compile/shell/lint.rs` shellchecks **every registered script in
isolation**, straight from the registry. Coverage is total by construction:
before this, lint coverage was a function of fixture reachability, so a
generator no fixture exercised was linted by nothing.

The `tests/bash_lint_tests.rs` integration test compiles a representative set
of fixtures and runs `shellcheck` against every literal `bash:` body in the
generated YAML. It catches silent-failure patterns that ADO's "fail on last
command" default would let through (e.g. `cd "$X"` without `|| exit`, tilde
inside double quotes, masked-return assignments).
generated YAML — proving scripts are *emitted*, where the registry lint proves
they are *correct*. It catches silent-failure patterns that ADO's "fail on
last command" default would let through (e.g. `cd "$X"` without `|| exit`,
tilde inside double quotes, masked-return assignments).

The test is skipped if `shellcheck` is not on PATH. Install locally with
Both are skipped if `shellcheck` is not on PATH. Install locally with
`brew install shellcheck` (macOS) or `apt-get install -y shellcheck` (Debian
/ Ubuntu); CI installs it in `.github/workflows/rust-tests.yml` and sets
`ENFORCE_BASH_LINT=1` so a missing shellcheck becomes a hard failure rather
than a silent skip.

When adding a new bash step, run `cargo test --test bash_lint_tests` and fix
anything it flags. If a finding is genuinely intentional, add a
`# shellcheck disable=SCxxxx` comment immediately above the offending line in
the bash body — shellcheck honours the directive and it's inert at runtime.
When adding a new shell script, run both and fix anything they flag:

```bash
ENFORCE_BASH_LINT=1 cargo test --bin ado-aw compile::shell
ENFORCE_BASH_LINT=1 cargo test --test bash_lint_tests
```

If a finding is genuinely intentional, add a `# shellcheck disable=SCxxxx`
comment immediately above the offending line in the body — shellcheck honours
the directive and it's inert at runtime.

To review the generated shell as ordinary files:

```bash
cargo run -- export-bash-scripts --out /tmp/ado-aw-shell
```

### Markdown-only smoke suite

Expand Down
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ sha2 = "0.11.0"
indexmap = "2"
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
semver = "1.0.28"
inventory = "0.3.24"

[dev-dependencies]
reqwest = { version = "0.12", features = ["blocking"] }
Expand Down
6 changes: 6 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,12 @@ These commands are not shown in `--help` but are available for contributors work
- `export-ado-proxy-catalog` - Export the `ado-proxy` catalog data as JSON. Build-time drift guard for the bundle's committed catalog snapshot (see [`docs/ado-proxy-design.md`](ado-proxy-design.md)).
- `--output, -o <path>` - Write the catalog to a file instead of stdout.

- `export-bash-scripts` - Materialize every shell script the compiler can emit as ordinary files, so generated shell can be reviewed and analysed without reading Rust. Reads the `src/compile/shell/` registry directly, so unlike the fixture-driven bash lint it reaches *every* script — including ones no pipeline currently emits.
- `--output, -o <dir>` - Directory to write into. Created if it does not exist. Required.
- `--format <files|json>` - `files` (default) writes one `.sh` per script with a provenance header naming the producing Rust source; `json` writes a single `bash-scripts.json` carrying the same content plus each script's declared binding surface.
- What is written is the *lint source*: the body with declared variables stub-assigned, which is the form that stands alone and the form the shellcheck harness judges. A rendered script needs real bindings, which only the producing call site has.
- Typical use: `cargo run -- export-bash-scripts --output /tmp/ado-aw-shell && shellcheck /tmp/ado-aw-shell/*.sh`

### Hidden Pipeline-Internal Commands

These commands are started by the pipeline itself (or by AWF on its behalf) and are not part of the authoring surface:
Expand Down
Loading
Loading