Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ado-script.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ node import.js <prompt-file> --base "$(Build.SourcesDirectory)" \
compiler always passes `$(Build.SourcesDirectory)` (ADO expands the macro
before node runs), and the compiler-emitted marker for the agent body is a
**trigger-repo-relative** path (e.g. `agents/foo.md`, or
`$(Build.Repository.Name)/agents/foo.md` under multi-checkout) — not
`self/agents/foo.md` under multi-checkout) — not
absolute. `import.js` rejects absolute and `..` paths.
- `--var name=value` (repeatable) — a small, **compiler-owned allowlist** of
ADO path-anchor variables (currently `Build.SourcesDirectory` and
Expand Down
1 change: 1 addition & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Global flags (apply to all subcommands): `--verbose, -v` (enable info-level logg
- Useful for CI checks to ensure pipelines are regenerated after source changes
- `mcp <output_directory> <bounding_directory>` - Run SafeOutputs as a stdio MCP server. **This is what compiled pipelines use**: MCPG spawns it as a hardened, network-isolated sibling container entrypoint (see [`docs/mcpg.md`](mcpg.md)).
- `--enabled-tools <name>` - Restrict available tools to those named (repeatable)
- `--self-repository-directory <path>` - Exact `self` checkout used for Git patch generation when the bounding workspace is a multi-checkout root or another repository.
- `mcp-author` - Run the author-facing stdio MCP server for IDE/Copilot Chat integrations. See [`mcp-author.md`](mcp-author.md) for the full tool surface and trust model.
- `execute` - Execute safe outputs from Stage 1 (Stage 3 of pipeline)
- `--source, -s <path>` - Path to source markdown file
Expand Down
2 changes: 1 addition & 1 deletion docs/codemods.md
Original file line number Diff line number Diff line change
Expand Up @@ -369,7 +369,7 @@ fold substituted the directory markers:

| Marker | Resolved to |
|--------|-------------|
| `{{ workspace }}`, `{{ working_directory }}` | the resolved working directory (`$(Build.SourcesDirectory)`, `$(Build.SourcesDirectory)/$(Build.Repository.Name)`, or `$(Build.SourcesDirectory)/<alias>`) |
| `{{ workspace }}`, `{{ working_directory }}` | the resolved working directory (`$(Build.SourcesDirectory)`, `$(Build.SourcesDirectory)/self`, or `$(Build.SourcesDirectory)/<alias>`) |
| `{{ trigger_repo_directory }}` | the trigger ("self") repo dir |

After the IR migration these markers flow through verbatim and are no
Expand Down
25 changes: 20 additions & 5 deletions docs/front-matter.md
Original file line number Diff line number Diff line change
Expand Up @@ -385,10 +385,19 @@ Reference the explicit ADO path instead:

- `$(Build.SourcesDirectory)` — the checkout root (the trigger repo root when
only `self` is checked out).
- `$(Build.SourcesDirectory)/$(Build.Repository.Name)` — the trigger repo when
one or more additional repositories are checked out.
- `$(Build.SourcesDirectory)/self` — the compiler-owned `self` checkout path
when one or more additional repositories are checked out.
- `$(Build.SourcesDirectory)/<alias>` — a specific checked-out repository.

> **`self` identity.** The compiler resolves the `self` repository's name at
> compile time from the Azure DevOps git remote (or
> `ADO_AW_COMPILE_REMOTE_URL`) and bakes it into the compiled pipeline, so
> safe outputs targeting `repository: self` never depend on
> `Build.Repository.*` — which names the *triggering* repository and differs
> from `self` on repository-resource-triggered runs. If no ADO remote can be
> resolved at compile time the compiler warns and falls back to
> `$(Build.Repository.Name)`; compile from an Azure DevOps clone to avoid it.

The `legacy_path_markers` codemod automatically rewrites any remaining markers
in front matter to the path they resolved to on the next `compile` (see
[`docs/codemods.md`](codemods.md)). Markers left in the **agent body** cannot be
Expand Down Expand Up @@ -423,6 +432,10 @@ Object fields:
| `fetch-depth` | *(ADO default)* | Shallow-clone depth for this repo's checkout (ADO `fetchDepth`). `0` = full history |
| `fetch-tags` | *(ADO default)* | Whether to fetch git tags during checkout (ADO `fetchTags`) |

Aliases must be unique case-insensitively because they become checkout
directory names on Windows agents. `root`, `repo`, and `self` are reserved in
every casing; `self` is the compiler-owned path for the pipeline repository.

### Tuning checkout fetch behavior (`fetch-depth` / `fetch-tags`)

On large monorepos the checkout step can dominate the run. Azure DevOps can
Expand Down Expand Up @@ -614,9 +627,11 @@ The trade-off is that the generated YAML is larger, and prompt-body
edits require `ado-aw compile` plus committing the updated pipeline
file.

A small, fixed set of ADO path-anchor variables —
`$(Build.SourcesDirectory)` and `$(Build.Repository.Name)` — is
substituted into the prompt consistently in **both** modes. Arbitrary
A small, fixed set of ADO path-anchor variables — including
`$(Build.SourcesDirectory)` and `$(Build.Repository.Name)` — is substituted
into the prompt consistently in **both** modes. `Build.Repository.Name`
identifies the triggering repository and is not the compiler-owned `self`
checkout path. Arbitrary
`$(...)` macros and pipeline/secret variables are not expanded; see
[ADO variables in the prompt](runtime-imports.md#ado-variables-in-the-prompt).

Expand Down
7 changes: 3 additions & 4 deletions docs/runtime-imports.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ along with any author-written markers.
`{{#runtime-import ../../../../etc/passwd}}` are both compile-time errors.
- **Compiler-generated marker for the agent body** uses a **trigger-repo-relative**
path resolved against `--base "$(Build.SourcesDirectory)"` — e.g. `agents/foo.md`,
or `$(Build.Repository.Name)/agents/foo.md` under multi-checkout (ADO expands the
macro before the resolver runs). It is deliberately relative, not absolute:
or `self/agents/foo.md` under multi-checkout. It is deliberately relative,
not absolute:
`import.js` rejects absolute paths for everyone. The compile-time `..`/absolute
restriction is a compile-host guard and does not constrain this tooling-generated
path beyond the runtime resolver's own absolute/`..` rejection.
Expand All @@ -59,7 +59,7 @@ by the runtime resolver:
| Variable | Expands to |
|----------|------------|
| `$(Build.SourcesDirectory)` | the checkout root |
| `$(Build.Repository.Name)` | the trigger repo's subfolder name (multi-checkout) |
| `$(Build.Repository.Name)` | the triggering repository name (not the compiler-owned `self` path) |

The compiler passes these to `import.js` as `--var name=$(name)` flags (ADO expands
the macro at runtime); the resolver then replaces every `$(name)` occurrence in the
Expand Down Expand Up @@ -113,4 +113,3 @@ compile time instead of on the pipeline runner.
- **Compile time**: `resolve_imports_inline()` in
`src/compile/extensions/ado_script.rs` performs the inline expansion
when `inlined-imports: true`.

Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ function baseEnv(overrides: Record<string, string | undefined> = {}): NodeJS.Pro
COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: "2602",
COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: "2603",
COMPILER_SMOKE_REPORTER_DEFINITION_ID: "2605",
COMPILER_SMOKE_MULTI_REPO_DEFINITION_ID: "2606",
...overrides,
};
}
Expand All @@ -35,6 +36,7 @@ describe("loadConfig", () => {
"azure-cli": 2602,
"noop-target": 2603,
"smoke-failure-reporter": 2605,
"multi-repo": 2606,
});
expect(config.concurrency).toBe(5);
expect(config.childTimeoutMs).toBe(7_200_000);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
import { describe, expect, it } from "vitest";

import { ALL_FIXTURES, allowedChangedPaths, fixturePaths, FIXTURE_DIR } from "../fixtures.js";
import {
ALL_FIXTURES,
allowedChangedPaths,
CANDIDATE_FIXTURE_DIR,
fixturePaths,
FIXTURE_DIR,
} from "../fixtures.js";

describe("fixturePaths", () => {
it("builds repo-relative md/lock paths under tests/safe-outputs", () => {
Expand All @@ -10,30 +16,40 @@ describe("fixturePaths", () => {
relLock: "tests/safe-outputs/canary.lock.yml",
});
});

it("keeps candidate-only fixtures out of the release-owned directory", () => {
expect(fixturePaths("multi-repo")).toEqual({
name: "multi-repo",
relMd: "tests/compiler-smoke-e2e/fixtures/multi-repo.md",
relLock: "tests/compiler-smoke-e2e/fixtures/multi-repo.lock.yml",
});
});
});

describe("ALL_FIXTURES", () => {
it("has exactly the four fixtures in the required stable order", () => {
it("has exactly the five fixtures in the required stable order", () => {
expect(ALL_FIXTURES.map((f) => f.name)).toEqual([
"canary",
"azure-cli",
"noop-target",
"smoke-failure-reporter",
"multi-repo",
]);
});

it("every fixture path lives under the shared FIXTURE_DIR", () => {
it("every fixture path lives under a known fixture directory", () => {
for (const f of ALL_FIXTURES) {
expect(f.relMd.startsWith(`${FIXTURE_DIR}/`)).toBe(true);
expect(f.relLock.startsWith(`${FIXTURE_DIR}/`)).toBe(true);
const dir = f.name === "multi-repo" ? CANDIDATE_FIXTURE_DIR : FIXTURE_DIR;
expect(f.relMd.startsWith(`${dir}/`)).toBe(true);
expect(f.relLock.startsWith(`${dir}/`)).toBe(true);
}
});
});

describe("allowedChangedPaths", () => {
it("contains exactly the four md files, four lock files, and .gitattributes", () => {
it("contains exactly each md file, each lock file, and .gitattributes", () => {
const allowed = allowedChangedPaths();
expect(allowed.size).toBe(9);
expect(allowed.size).toBe(ALL_FIXTURES.length * 2 + 1);
expect(allowed.has(".gitattributes")).toBe(true);
for (const f of ALL_FIXTURES) {
expect(allowed.has(f.relMd)).toBe(true);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ const baseEnv = {
COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: "3002",
COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: "3003",
COMPILER_SMOKE_REPORTER_DEFINITION_ID: "3005",
COMPILER_SMOKE_MULTI_REPO_DEFINITION_ID: "3006",
COMPILER_SMOKE_CHILD_TIMEOUT_MS: "5000",
COMPILER_SMOKE_POLL_MS: "1",
};
Expand Down
2 changes: 2 additions & 0 deletions scripts/ado-script/src/compiler-smoke-e2e/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ export const FIXTURE_NAMES = [
"azure-cli",
"noop-target",
"smoke-failure-reporter",
"multi-repo",
] as const;

export type FixtureName = (typeof FIXTURE_NAMES)[number];
Expand Down Expand Up @@ -89,6 +90,7 @@ const DEFINITION_ID_ENV_BY_FIXTURE: Readonly<Record<FixtureName, string>> = {
"azure-cli": "COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID",
"noop-target": "COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID",
"smoke-failure-reporter": "COMPILER_SMOKE_REPORTER_DEFINITION_ID",
"multi-repo": "COMPILER_SMOKE_MULTI_REPO_DEFINITION_ID",
};

/** ADO macros that failed to expand look like `$(NAME)`; treat them as unset. */
Expand Down
52 changes: 36 additions & 16 deletions scripts/ado-script/src/compiler-smoke-e2e/fixtures.ts
Original file line number Diff line number Diff line change
@@ -1,26 +1,45 @@
/**
* Manifest of the four fixed compiler-smoke fixtures.
* Manifest of the fixed compiler-smoke fixtures.
*
* These are selected release-backed agentic-pipeline sources documented in
* Most are selected release-backed agentic-pipeline sources documented in
* `tests/safe-outputs/README.md` (canary, azure-cli, noop-target, and
* smoke-failure-reporter). The weekly janitor remains in the release lane but
* is not part of candidate checks. The harness reads the exact files from the
* detached candidate worktree (an exact checkout of `BUILD_SOURCEVERSION`, at
* `<worktree>/tests/safe-outputs/<name>.md` — never from
* smoke-failure-reporter); the weekly janitor remains in the release lane but
* is not part of candidate checks. `multi-repo` is candidate-only — it has no
* released lock file and lives under `tests/compiler-smoke-e2e/fixtures/`.
*
* The harness reads the exact files from the detached candidate worktree (an
* exact checkout of `BUILD_SOURCEVERSION` — never from
* `BUILD_SOURCESDIRECTORY`, which may sit at a different commit), stages a
* pinned `supply-chain.pipeline-artifact` transform of each onto the mirror
* repo, recompiles, and queues the four FIXED "candidate lane" pipeline
* definitions tracked in `tests/compiler-smoke-e2e/REGISTERED.md` (distinct
* from the release-backed definitions those same sources also feed).
* repo, recompiles, and queues the FIXED "candidate lane" pipeline definitions
* tracked in `tests/compiler-smoke-e2e/REGISTERED.md` (distinct from the
* release-backed definitions those same sources also feed).
*
* Test-harness module; not shipped in `ado-script.zip`.
*/
import type { FixtureName } from "./config.js";
import { FIXTURE_NAMES } from "./config.js";

/** Repo-relative directory containing every fixture source + compiled lock. */
/** Repo-relative directory containing the release-backed fixture sources. */
export const FIXTURE_DIR = "tests/safe-outputs";

/**
* Repo-relative directory holding fixtures that exist **only** for the
* candidate lane. These have no released lock file and no release-backed
* definition, so they must not live beside the release-owned sources in
* {@link FIXTURE_DIR}.
*/
export const CANDIDATE_FIXTURE_DIR = "tests/compiler-smoke-e2e/fixtures";

/** Directory each fixture's source lives in. */
const FIXTURE_DIR_BY_NAME: Readonly<Record<FixtureName, string>> = {
canary: FIXTURE_DIR,
"azure-cli": FIXTURE_DIR,
"noop-target": FIXTURE_DIR,
"smoke-failure-reporter": FIXTURE_DIR,
"multi-repo": CANDIDATE_FIXTURE_DIR,
};

export interface FixturePaths {
readonly name: FixtureName;
/** Repo-relative path to the fixture markdown source, e.g. tests/safe-outputs/canary.md. */
Expand All @@ -31,21 +50,22 @@ export interface FixturePaths {

/** Repo-relative path for a fixture's markdown source. Always POSIX-separated (a git path, not a filesystem path). */
export function fixturePaths(name: FixtureName): FixturePaths {
const dir = FIXTURE_DIR_BY_NAME[name];
return {
name,
relMd: `${FIXTURE_DIR}/${name}.md`,
relLock: `${FIXTURE_DIR}/${name}.lock.yml`,
relMd: `${dir}/${name}.md`,
relLock: `${dir}/${name}.lock.yml`,
};
}

/** All four fixtures in the stable declaration order used throughout the harness. */
/** Every fixture in the stable declaration order used throughout the harness. */
export const ALL_FIXTURES: readonly FixturePaths[] = FIXTURE_NAMES.map(fixturePaths);

/**
* The exact set of repo-relative paths the candidate-staging commit is
* allowed to touch: the four markdown sources, their four compiled locks,
* and the compiler-managed `.gitattributes` block. Any other changed path
* fails the run before it pushes anything.
* allowed to touch: every markdown source, its compiled lock, and the
* compiler-managed `.gitattributes` block. Any other changed path fails the
* run before it pushes anything.
*/
export function allowedChangedPaths(): Set<string> {
const paths = new Set<string>([".gitattributes"]);
Expand Down
10 changes: 10 additions & 0 deletions scripts/ado-script/src/executor-e2e/__tests__/index.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { describe, expect, it } from "vitest";

import { summarise } from "../index.js";
import { allScenarios } from "../scenarios/index.js";
import type { ScenarioResult } from "../scenario.js";

describe("summarise", () => {
Expand All @@ -17,3 +18,12 @@ describe("summarise", () => {
expect(text).toContain("Total: 3 | Passed: 1 | Failed: 1 | Skipped: 1");
});
});

describe("scenario registry", () => {
it("registers both create-pull-request checkout layouts with unique ids", () => {
const ids = allScenarios.map((scenario) => scenario.id ?? scenario.tool);
expect(new Set(ids).size).toBe(ids.length);
expect(ids).toContain("create-pull-request");
expect(ids).toContain("create-pull-request-self-multi-checkout");
});
});
Loading
Loading