diff --git a/.gitattributes b/.gitattributes index 32749f294c..e2e6931b66 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,7 @@ * text=auto eol=lf -.github/workflows/*.lock.yml linguist-generated=true merge=ours -whitespace \ No newline at end of file +.github/workflows/*.lock.yml linguist-generated=true merge=ours -whitespace +# The project constitution is the one dogfooding artifact carried forward. +# Keep it exempt from git's whitespace checks (git diff --check / CI) since its +# generated formatting is not hand-edited. +.specify/memory/constitution.md -whitespace diff --git a/.specify/memory/constitution.md b/.specify/memory/constitution.md new file mode 100644 index 0000000000..0921f18ee6 --- /dev/null +++ b/.specify/memory/constitution.md @@ -0,0 +1,214 @@ + + +# Spec Kit Constitution + +Spec Kit (the `specify-cli` package and its bundled assets) is a local, offline-capable +developer CLI that bootstraps and operates Spec-Driven Development workflows for AI coding +agents. These principles are derived from the patterns the codebase already enforces. They +are binding on all changes — including the `specify bundle` subcommand and any future +command group, integration, extension, preset, or workflow. + +## Core Principles + +### I. Code Quality & Architectural Discipline + +The codebase follows a strict, registry-driven, layered architecture, and all changes MUST +preserve it. + +- **Separate the CLI surface from importable logic.** User-facing commands live in Typer + sub-apps (e.g. `commands/`, `*/_commands.py`); business logic lives in plain, importable + modules with no `@app.command()` decorators. New features MUST keep orchestration logic + testable independently of Typer. +- **Use the established extension pattern.** New agents/integrations MUST subclass one of the + standard base classes (`MarkdownIntegration`, `TomlIntegration`, `YamlIntegration`, + `SkillsIntegration`) and declare the required class attributes (`key`, `config`, + `registrar_config`, and `context_file` where applicable). Extending `IntegrationBase` + directly is permitted only when no base class fits, and the deviation MUST be justified. +- **Honor the single source of truth.** Built-ins are wired through the relevant registry + (e.g. `INTEGRATION_REGISTRY` via `_register_builtins()`), with imports and registrations + kept in alphabetical order. Duplicate keys MUST fail loudly rather than silently override. +- **Naming and typing are not optional.** Private modules/functions are `_`-prefixed and MUST + NOT be imported across package boundaries. Every new module begins with + `from __future__ import annotations` and uses modern type syntax (`dict[str, Any]`, + `str | None`); legacy `Dict`/`List`/`Optional` forms are rejected. +- **Package directories use underscores; keys keep their canonical (often hyphenated) form** + (e.g. package `kiro_cli/`, `key = "kiro-cli"`). For CLI-backed integrations the `key` MUST + match the executable name so `shutil.which(key)` resolves. + +**Rationale:** A registry-plus-base-class architecture is what lets dozens of integrations, +extensions, and workflows coexist with minimal coupling. Drift here multiplies maintenance +cost and breaks the "add one subclass, register once, ship a test" contract. + +### II. Test-Backed Change (NON-NEGOTIABLE) + +Every behavioral change MUST be accompanied by automated tests, and the suite is a hard gate. + +- **Tests gate merges.** CI runs `pytest` across a matrix of ubuntu + windows × Python 3.11, + 3.12, and 3.13. Changes MUST pass on every cell of that matrix. +- **Parity invariants MUST hold.** Every integration MUST be present in the registry, have a + `CommandRegistrar` config entry where required, and ship a dedicated + `tests/integrations/test_integration_.py` (hyphens in the key become underscores in the + filename). These are enforced by parametrized tests (e.g. `test_registry.py`) and MUST NOT + be weakened. +- **Follow pytest conventions.** Test modules/classes/functions use the `test_*` / `Test*` + naming the project configures, run under `--strict-markers`, and isolate state with + `tmp_path`, `monkeypatch`, and the autouse auth-isolation fixture. Platform-specific tests + MUST be guarded (e.g. `@requires_bash`) rather than left to fail. +- **Security and idempotency tests are mandatory categories.** Path-traversal rejection, + manifest hash integrity/symlink safety, and no-overwrite idempotency are covered by existing + suites; changes touching file writes, path handling, or setup scripts MUST extend (never + reduce) that coverage. +- **Network is mocked.** No test may make a real outbound network call; HTTP MUST be stubbed + so the suite is deterministic and offline-runnable. + +**Rationale:** The breadth of supported agents and the offline/air-gapped guarantees can only +be sustained by exhaustive, parametrized tests. The parity and security suites are what stop a +single new integration from regressing the whole matrix. + +### III. CLI & User-Experience Consistency + +The CLI presents one coherent surface; every command group MUST feel like the others. + +- **Reuse the shared verb vocabulary.** Consumer-facing groups use the established verbs — + `list`, `add`/`install`, `remove`, `search`, `info`, `update`, plus `enable`/`disable` and + `set-priority` where relevant. New verbs MUST NOT be invented when an existing one fits, and + any genuinely new verb MUST be justified. +- **Mirror the catalog-stack model.** Catalog-backed groups MUST expose + ` catalog list|add|remove`, back it with a priority-ordered source stack (lower number + = higher precedence) plus per-source install policy (`install-allowed` vs `discovery-only`), + and fall back to a built-in default stack when no project config is present. +- **Register sub-apps the standard way.** Command groups are `typer.Typer(...)` instances + attached via `app.add_typer(child, name="...")`, preferably through a modular + `register(app)` function imported in `__init__.py`. Nesting MUST stay within ~2–3 levels. +- **Output is consistent and machine-friendly.** Human output uses the shared Rich + conventions (e.g. `[green]✓[/green]` success, `[red]Error:[/red]` + non-zero exit on + failure, actionable remediation in messages). Where a `--json` flag is offered, valid JSON + goes to stdout and all other logging is redirected to stderr. +- **Interactions are safe and idempotent.** Destructive actions show what will change before + confirming; "already installed / already present" outcomes succeed (exit 0) rather than + error. User-facing command groups MUST be documented under `docs/reference/`. + +**Rationale:** Predictability is the product. Users learn one set of verbs, one catalog model, +and one output grammar, then apply them to every group — including `specify bundle`. + +### IV. Offline-First Performance & Resource Discipline + +Spec Kit is a local CLI; responsiveness, offline operability, and graceful degradation are the +performance contract. + +- **`specify init` and core scaffolding MUST work fully offline** using bundled `core_pack` + assets. Asset resolution MUST prefer bundled assets, then a source checkout, before ever + reaching the network. +- **Network use is lazy, bounded, and degradable.** Network calls happen only on explicit + user commands, MUST set timeouts, MUST cache catalog results (1-hour TTL) and fall back to + stale cache on failure, and MUST surface offline/rate-limit conditions as clear messages + without crashing. +- **Keep startup cheap.** Avoid adding heavyweight work to import time. New optional + subsystems SHOULD prefer lazy loading over unconditional eager imports so that unrelated + commands (including `--help`) stay fast. +- **Filesystem writes are minimal and idempotent.** Installs MUST track files (SHA-256 + manifests), avoid clobbering user-modified content, only uninstall files whose hash still + matches, and never follow symlinks out of the project root. + +**Rationale:** Developers run this tool in air-gapped, enterprise, and flaky-network +environments. Offline-first behavior and idempotent, hash-tracked file operations are what +make it safe and fast to run repeatedly. + +### V. Minimal Dependencies & Safe, Idempotent File Operations + +The project guards its dependency surface and its on-disk footprint deliberately. + +- **Zero new runtime dependencies by default.** The runtime dependency set is intentionally + small and pinned to a minimum major version. Adding a dependency requires maintainer + agreement and a justification that existing deps (typer, click, rich, pyyaml, packaging, + platformdirs, pathspec, json5, readchar) cannot serve the need. New subsystems SHOULD reuse + existing primitive machinery in-process rather than re-implementing or re-shipping it. +- **All paths are validated.** Any project-relative path derived from user/manifest/catalog + input MUST be confined to the project root (`Path.relative_to` checks) and reject traversal + payloads; symlink escapes MUST be refused. +- **Errors are explicit and chained.** Validate inputs up front, raise with actionable context + (offending field/value plus a hint), and use `raise ... from exc` to preserve causes. I/O + that can legitimately fail MUST degrade gracefully rather than emit a raw traceback. +- **Versioning follows SemVer.** User-visible and packaged behavior changes follow + MAJOR.MINOR.PATCH semantics; backward-incompatible changes MUST be called out and justified. + +**Rationale:** A lean, pinned dependency set and hardened, idempotent file handling are what +keep the tool trustworthy in enterprise and air-gapped contexts and cheap to maintain. + +## Security & Cross-Platform Constraints + +- **Cross-platform parity is required.** Code MUST run on Linux, macOS, and Windows and on + Python 3.11–3.13. Windows specifics (UTF-8 stream reconfiguration, bash-dependent tests + auto-skipping) MUST be respected; do not introduce POSIX-only assumptions without a guarded + fallback. +- **Security tooling is a gate.** CodeQL and the project's security test suites + (path-traversal, manifest/symlink hardening) MUST remain green. Network access MUST default + to off in tests and be opt-in, timeout-bounded, and credential-isolated at runtime. +- **Formatting is enforced.** `.editorconfig` rules (LF endings, final newline, no trailing + whitespace, 4-space Python / 2-space YAML-JSON-Markdown), `ruff check src/`, and + `markdownlint-cli2` MUST pass. + +## Development Workflow & Quality Gates + +- **Branch naming** follows `/-` (or `/` with no + issue), with `` ∈ {feat, fix, docs, community, chore}. +- **PRs are focused** and MUST: pass `ruff`, `pytest` (full matrix), markdown lint, and CodeQL; + add/extend tests for new behavior; update user-facing docs (`README.md`, `docs/`, + `spec-driven.md`) when behavior changes; and disclose any AI assistance used. +- **Slash-command-affecting changes** MUST be manually exercised through a coding agent and the + results reported in the PR, per CONTRIBUTING.md. +- **Large or cross-cutting changes** (new templates, arguments, command groups) MUST be agreed + with maintainers before implementation. + +## Governance + +This constitution supersedes ad-hoc convention where they conflict; the existing codebase +patterns it codifies remain authoritative references. + +- **Authority.** Principles I–V are binding gates. The `## Constitution Check` section of the + plan template MUST be evaluated against these principles, and `/speckit.analyze` treats + conflicts with a MUST as CRITICAL. Violations are resolved by changing the spec, plan, or + tasks — not by diluting a principle. +- **Amendments.** Changes to this document require a PR with rationale, maintainer approval, + and a version bump per the policy below. Any amendment MUST propagate to dependent templates + and command guidance in the same change, recorded in the Sync Impact Report at the top of + this file. +- **Versioning policy (SemVer for governance).** MAJOR = backward-incompatible governance or + principle removal/redefinition; MINOR = a new principle/section or materially expanded + guidance; PATCH = clarifications and non-semantic refinements. +- **Compliance review.** Every PR and review MUST verify compliance with these principles. + Added complexity or any deviation MUST be justified in-PR (and, for plans, in the plan's + Complexity Tracking section). Unjustified violations block merge. + +**Version**: 1.0.0 | **Ratified**: 2026-06-19 | **Last Amended**: 2026-06-19 diff --git a/AGENTS.md b/AGENTS.md index d21db4f426..3d5ea32377 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ The toolkit supports multiple AI coding assistants, allowing teams to use their Each AI agent is a self-contained **integration subpackage** under `src/specify_cli/integrations//`. The subpackage exposes a single class that declares all metadata and inherits setup/teardown logic from a base class. Built-in integrations are then instantiated and added to the global `INTEGRATION_REGISTRY` by `src/specify_cli/integrations/__init__.py` via `_register_builtins()`. -``` +```text src/specify_cli/integrations/ ├── __init__.py # INTEGRATION_REGISTRY + _register_builtins() ├── base.py # IntegrationBase, MarkdownIntegration, TomlIntegration, YamlIntegration, SkillsIntegration @@ -340,18 +340,21 @@ Some agents require custom processing beyond the standard template transformatio ### Copilot Integration GitHub Copilot has unique requirements: + - Commands use `.agent.md` extension (not `.md`) - Each command gets a companion `.prompt.md` file in `.github/prompts/` - Installs `.vscode/settings.json` with prompt file recommendations - Context file lives at `.github/copilot-instructions.md` Implementation: Extends `IntegrationBase` with custom `setup()` method that: + 1. Processes templates with `process_template()` 2. Generates companion `.prompt.md` files 3. Merges VS Code settings **Skills mode (`--skills`):** Copilot also supports an alternative skills-based layout via `--integration-options="--skills"`. When enabled: + - Commands are scaffolded as `speckit-/SKILL.md` under `.github/skills/` - No companion `.prompt.md` files are generated - No `.vscode/settings.json` merge @@ -371,11 +374,13 @@ specify init my-project --integration copilot --integration-options="--skills" ### Forge Integration Forge has special frontmatter and argument requirements: + - Uses `{{parameters}}` instead of `$ARGUMENTS` - Strips `handoffs` frontmatter key (Forge-specific collaboration feature) - Injects `name` field into frontmatter when missing Implementation: Extends `MarkdownIntegration` with custom `setup()` method that: + 1. Inherits standard template processing from `MarkdownIntegration` 2. Adds extra `$ARGUMENTS` → `{{parameters}}` replacement after template processing 3. Applies Forge-specific transformations via `_apply_forge_transformations()` @@ -385,11 +390,13 @@ Implementation: Extends `MarkdownIntegration` with custom `setup()` method that: ### Goose Integration Goose is a YAML-format agent using Block's recipe system: + - Uses `.goose/recipes/` directory for YAML recipe files - Uses `{{args}}` argument placeholder - Produces YAML with `prompt: |` block scalar for command content Implementation: Extends `YamlIntegration` (parallel to `TomlIntegration`): + 1. Processes templates through the standard placeholder pipeline 2. Extracts title and description from frontmatter 3. Renders output as Goose recipe YAML (version, title, description, author, extensions, activities, prompt) @@ -400,7 +407,7 @@ Implementation: Extends `YamlIntegration` (parallel to `TomlIntegration`): Branches follow one of two patterns depending on whether an issue exists: -``` +```text /- # when an issue is created first / # when no issue exists (PR-only changes) ``` @@ -463,6 +470,7 @@ Disclosure is **continuous**, not a one-time event. A single AI-disclosure parag 3. **Incorrect `requires_cli` value**: Set to `True` only for agents that have a CLI tool; set to `False` for IDE-based agents. 4. **Wrong argument format**: Use `$ARGUMENTS` for Markdown agents, `{{args}}` for TOML agents. 5. **Skipping registration**: The import and `_register()` call in `_register_builtins()` must both be added. +6. **Running tests against the wrong environment**: Always run the suite inside this working tree's own virtualenv (`uv sync --extra test` then `.venv/bin/python -m pytest`, or activate the venv first). A bare `uv run pytest` can resolve to an ambient/global interpreter whose editable `.pth` points at a *different* worktree. The failure is sneaky: test collection still imports `specify_cli` successfully, but newly-added subpackages (e.g. a fresh `specify_cli/bundler/`) resolve as a stale namespace package and raise `ModuleNotFoundError`. If a brand-new subpackage imports under `python -c` but not under pytest, suspect environment contamination, not your code. --- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 12b095f5fc..899dae258c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -95,6 +95,24 @@ uv run python -m pytest tests/test_agent_config_consistency.py -q Run this when you change agent metadata, context update scripts, or integration wiring. +#### Running the full test suite + +Install the test dependencies into the project's own virtual environment and run +`pytest` through that interpreter: + +```bash +uv pip install -e ".[test]" +.venv/bin/python -m pytest tests -q # Windows: .venv\Scripts\python -m pytest tests -q +``` + +> **Note:** prefer `.venv/bin/python -m pytest` over a bare `uv run pytest`. +> If another Spec Kit checkout has an editable (`-e`) install registered in a +> shared/global environment, `uv run pytest` can resolve `specify_cli` to that +> *other* worktree, turning it into a partial namespace package that fails to +> import newly added subpackages. Running through the project `.venv` resolves +> `specify_cli` to this checkout's `src/`. This matches the gotcha documented in +> `AGENTS.md` (Common Pitfalls). + ### Manual testing #### Testing setup diff --git a/README.md b/README.md index afca9b15a5..34e1403324 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ - [🤖 Supported AI Coding Agent Integrations](#-supported-ai-coding-agent-integrations) - [🔧 Specify CLI Reference](#-specify-cli-reference) - [🧩 Making Spec Kit Your Own: Extensions & Presets](#-making-spec-kit-your-own-extensions--presets) +- [📦 Bundles: Role-Based Setups](#-bundles-role-based-setups) - [📚 Core Philosophy](#-core-philosophy) - [🌟 Development Phases](#-development-phases) - [🎯 Experimental Goals](#-experimental-goals) @@ -228,6 +229,56 @@ For example, presets could restructure spec templates to require regulatory trac See the [Presets reference](https://github.github.io/spec-kit/reference/presets.html) for the full command guide, including resolution order and priority stacking. +## 📦 Bundles: Role-Based Setups + +Extensions and presets are individual building blocks. A **bundle** packages a +curated set of them — extensions, presets, steps, and workflows — into a single, +versioned, role-oriented setup so a whole team persona (product manager, business +analyst, security researcher, developer, …) can be provisioned with one command. + +A bundle is described by a hand-written `bundle.yml` manifest. It pins each +component to a version and, optionally, targets a specific integration; a bundle +with no `integration` is **agnostic** and inherits whatever integration the +project already uses. + +```bash +# Discover bundles in the active catalog stack +specify bundle search [] + +# Inspect the exact component set a bundle will add (equals what install does) +specify bundle info + +# Install a bundle's full component set in one operation +specify bundle install + +# See what's installed, then update or remove non-destructively +specify bundle list +specify bundle update # or --all +specify bundle remove # removes only this bundle's components +``` + +Bundles resolve from a **priority-ordered catalog stack** (project > user > +built-in). Each source carries an install policy: `install-allowed` sources can +be installed from, while `discovery-only` sources are visible in `search`/`info` +but refuse installation. Manage the stack with `specify bundle catalog list|add|remove`. + +Authors validate and package bundles locally — there is no first-class publish; +distribution is hosting the built artifact and adding a catalog entry: + +```bash +specify bundle validate --path ./my-bundle # structural + reference checks +specify bundle build --path ./my-bundle # produce a versioned .zip artifact +``` + +Four ready-to-read example manifests live under +[`examples/bundles/`](examples/bundles/) (product manager, business analyst, +security researcher, developer). + +Key guarantees: `info` shows exactly what `install` adds (transparency); +installs are idempotent and confined to the project root; `remove` never touches +components another installed bundle still needs; and all consume/author commands +work **offline** against local or pinned sources. + ### When to Use Which | Goal | Use | @@ -237,6 +288,7 @@ See the [Presets reference](https://github.github.io/spec-kit/reference/presets. | Integrate an external tool or service | Extension | | Enforce organizational or regulatory standards | Preset | | Ship reusable domain-specific templates | Either — presets for template overrides, extensions for templates bundled with new commands | +| Provision a complete role-based setup in one command | Bundle | ## 📚 Core Philosophy diff --git a/docs/reference/bundles.md b/docs/reference/bundles.md new file mode 100644 index 0000000000..2a7384cf6b --- /dev/null +++ b/docs/reference/bundles.md @@ -0,0 +1,156 @@ +# Bundles + +Bundles compose existing Spec Kit components — extensions, presets, workflows, and steps — into a single, versioned, installable unit. Where extensions and presets are primitives, a bundle is a curated stack that declares everything a team or role needs and installs it in one step through each component's own machinery. Bundles add no new runtime behavior of their own: they are a distribution and composition layer over the primitives you already use. + +A bundle is described by a `bundle.yml` manifest and is discovered through the same catalog stack as other components. Installing a bundle resolves its declared components against pinned versions, checks for the single cross-bundle conflict point (the active integration), and applies each component idempotently with full provenance tracking so it can be cleanly removed or refreshed later. + +## Search Available Bundles + +```bash +specify bundle search [query] +``` + +| Option | Description | +| ----------- | ---------------------------- | +| `--offline` | Do not access the network | +| `--json` | Emit machine-readable JSON | + +Searches all active catalogs for bundles matching the query. Without a query, lists every available bundle with its version, role, source, and a trust indicator (`verified` for org-curated catalog entries, `community` otherwise) so you can judge trust before installing. + +## Bundle Info + +```bash +specify bundle info +``` + +| Option | Description | +| ------------ | --------------------------------- | +| `--offline` | Do not access the network | +| `--json` | Emit machine-readable JSON | + +Shows full metadata for a bundle along with the **fully expanded component set** it installs — every extension, preset, step, and workflow with its pinned version, plus preset priority and strategy. The output also includes a trust indicator (`verified` vs `community`) so you can judge trust before installing. This preview is the same plan `install` applies, so you can see exactly what will be added before committing. Foreseeable overlaps with components already provided by installed bundles are surfaced here as well. + +## Install a Bundle + +```bash +specify bundle install +``` + +| Option | Description | +| ---------------- | ------------------------------------------------------------------ | +| `--integration` | Override the integration used when initializing/installing | +| `--offline` | Do not access the network | + +Installs a bundle's full component set through each primitive's machinery. The argument may be a catalog bundle id, or a local path to a built `.zip` artifact, a bundle directory, or a `bundle.yml` file; local sources install directly without consulting the catalog stack. + +If the current directory is not yet a Spec Kit project, `install` initializes one first so a fresh checkout reaches a working state in a single command. `--integration` selects the integration when initializing a new project, and confirms the target when a bundle pins a specific integration but the project's active integration can't be determined (missing or unreadable `.specify/integration.json`). It does **not** override an already-initialized project's active integration: if a bundle targets a different integration than the project's, install aborts with no changes. Integration-agnostic bundles inherit the project's active integration. Installation is idempotent — components already present are skipped. On failure, no provenance record is written (a failed install records nothing), and the components installed during that run are removed on a best-effort basis — removal errors are swallowed, so partial on-disk state may remain. + +## Update Bundles + +```bash +specify bundle update [] +``` + +| Option | Description | +| ------------ | ------------------------------------ | +| `--all` | Update every installed bundle | +| `--offline` | Do not access the network | + +Re-resolves a bundle and **refreshes** its components through each primitive's update path, bringing already-installed components up to the bundle's newly pinned versions while preserving primitive-level overrides (such as preset priority). Provide a bundle id, or use `--all` to update everything installed. + +> **Pin enforcement is install-time only.** Idempotency checks are id-based, not version-aware: a component that is already present is skipped during `install` without comparing its on-disk version to the manifest pin. Version pins are therefore guaranteed to be applied only when the bundler actually installs a component for the first time or refreshes it. Run `specify bundle update` to re-apply every owned component at its pinned version. + +## Remove a Bundle + +```bash +specify bundle remove +``` + +Uninstalls only the components this bundle contributed, leaving any component that another installed bundle still needs in place (no collateral removals). + +## List Installed Bundles + +```bash +specify bundle list +``` + +| Option | Description | +| -------- | ---------------------------- | +| `--json` | Emit machine-readable JSON | + +Lists the bundles installed in the project with their versions, component counts, and install timestamps. + +## Initialize a Project with a Bundle + +```bash +specify bundle init [] +``` + +| Option | Description | +| ---------------- | ---------------------------------------- | +| `--integration` | Integration override | +| `--offline` | Do not access the network | + +Ensures the current directory is a Spec Kit project (initializing it idempotently if needed), then optionally installs the given bundle. Useful as an explicit one-step bootstrap for a new checkout. + +## Validate a Bundle + +```bash +specify bundle validate +``` + +| Option | Description | +| ------------ | ------------------------------------------------------------------- | +| `--path` | Bundle directory or `bundle.yml` (default: current directory) | +| `--offline` | Verify references against bundled/installed components only | + +Reports whether a `bundle.yml` is well-formed and whether every declared component reference resolves. References are checked against bundled components, the project's installed components, and — when online — the active catalogs. Validation fails only when a reference is definitively absent everywhere it could be checked: that is, when an active catalog is reachable and confirms the component is missing. References that cannot be verified — because validation is offline, or because a catalog is unreachable — are downgraded to warnings so authoring can continue, rather than failing the run. + +## Build a Bundle Artifact + +```bash +specify bundle build +``` + +| Option | Description | +| ----------- | ------------------------------------------------------- | +| `--path` | Bundle directory (default: current directory) | +| `--output` | Output directory for the artifact | + +Produces a single versioned, distributable `.zip` artifact from a bundle directory. The artifact embeds the manifest and can be installed directly with `specify bundle install `. + +## Manage Catalog Sources + +Bundles are discovered through a priority-ordered stack of catalog sources (project, user, and built-in scopes). + +### List the Catalog Stack + +```bash +specify bundle catalog list +``` + +Prints the active, priority-ordered catalog stack with each source's scope and install policy. + +### Add a Catalog Source + +```bash +specify bundle catalog add +``` + +| Option | Description | +| ------------- | ------------------------------------------------------- | +| `--policy` | `install-allowed` or `discovery-only` | +| `--priority` | Source priority (lower = higher precedence; default 10) | +| `--id` | Explicit source id | + +Registers a project-scoped catalog source and persists it. + +### Remove a Catalog Source + +```bash +specify bundle catalog remove +``` + +Removes a project-scoped catalog source. Built-in default sources cannot be deleted. + +> **Note:** `search` and `info` work anywhere — with no project they fall back to the built-in/user catalog stack. The remaining state-changing commands (`list`, `update`, `remove`, `catalog`) require a project already initialized with `specify init`. `install` and `init` will initialize a project on demand when run in an uninitialized directory. diff --git a/docs/reference/overview.md b/docs/reference/overview.md index 10fcdc3bca..162515772f 100644 --- a/docs/reference/overview.md +++ b/docs/reference/overview.md @@ -31,3 +31,9 @@ Presets customize how Spec Kit works — overriding command files, template file Workflows automate multi-step Spec-Driven Development processes into repeatable sequences. They chain commands, prompts, shell steps, and human checkpoints together, with support for conditional logic, loops, fan-out/fan-in, and the ability to pause and resume from the exact point of interruption. [Workflows reference →](workflows.md) + +## Bundles + +Bundles compose existing extensions, presets, workflows, and steps into a single, versioned, installable unit. Rather than adding new behavior, a bundle curates a stack of primitives — everything a team or role needs — and installs it in one step through each component's own machinery, with version pinning, conflict checks, and provenance tracking for clean updates and removal. + +[Bundles reference →](bundles.md) diff --git a/examples/bundles/business-analyst/README.md b/examples/bundles/business-analyst/README.md new file mode 100644 index 0000000000..9f63464640 --- /dev/null +++ b/examples/bundles/business-analyst/README.md @@ -0,0 +1,22 @@ +# Business Analyst bundle + +A role bundle for business analysts working in a Spec-Driven Development flow: +requirements elicitation, traceability, and acceptance criteria. + +## What it installs + +- **Extension** `agent-context` — keeps the agent context file in sync. +- **Preset** `requirements-elicitation` (priority 10, append) — elicitation and + analysis command set. +- **Steps** `capture-requirements`, `trace-acceptance-criteria`. +- **Workflow** `requirements-to-spec` — turns captured requirements into a spec. + +This bundle is **integration-agnostic**: it inherits the project's active +integration. + +## Usage + +```bash +specify bundle validate --path examples/bundles/business-analyst +specify bundle build --path examples/bundles/business-analyst --output dist/ +``` diff --git a/examples/bundles/business-analyst/bundle.yml b/examples/bundles/business-analyst/bundle.yml new file mode 100644 index 0000000000..b03875a22e --- /dev/null +++ b/examples/bundles/business-analyst/bundle.yml @@ -0,0 +1,33 @@ +schema_version: "1.0" + +bundle: + id: "business-analyst" + name: "Business Analyst" + version: "1.0.0" + role: "business-analyst" + description: "Spec-Driven Development setup for business analysts: requirements elicitation, traceability, and acceptance criteria." + author: "spec-kit-examples" + license: "MIT" + +requires: + speckit_version: ">=0.9.0" + tools: [] + mcp: [] + +provides: + extensions: + - id: "agent-context" + version: "1.0.0" + presets: + - id: "requirements-elicitation" + version: "1.0.0" + priority: 10 + strategy: "append" + steps: + - id: "capture-requirements" + - id: "trace-acceptance-criteria" + workflows: + - id: "requirements-to-spec" + version: "1.0.0" + +tags: ["requirements", "traceability", "analysis"] diff --git a/examples/bundles/developer/README.md b/examples/bundles/developer/README.md new file mode 100644 index 0000000000..c56d5071ca --- /dev/null +++ b/examples/bundles/developer/README.md @@ -0,0 +1,22 @@ +# Developer bundle + +A role bundle for developers practicing Spec-Driven Development: implementation +planning, task breakdown, and code review. + +## What it installs + +- **Extension** `agent-context` — keeps the agent context file in sync. +- **Preset** `implementation-planning` (priority 10, append) — implementation + planning command set. +- **Steps** `plan-implementation`, `break-down-tasks`. +- **Workflow** `spec-to-implementation` — drives a spec through to code. + +This bundle is **integration-agnostic**: it inherits the project's active +integration. + +## Usage + +```bash +specify bundle validate --path examples/bundles/developer +specify bundle build --path examples/bundles/developer --output dist/ +``` diff --git a/examples/bundles/developer/bundle.yml b/examples/bundles/developer/bundle.yml new file mode 100644 index 0000000000..3a365534e5 --- /dev/null +++ b/examples/bundles/developer/bundle.yml @@ -0,0 +1,33 @@ +schema_version: "1.0" + +bundle: + id: "developer" + name: "Developer" + version: "1.0.0" + role: "developer" + description: "Spec-Driven Development setup for developers: implementation planning, task breakdown, and code review." + author: "spec-kit-examples" + license: "MIT" + +requires: + speckit_version: ">=0.9.0" + tools: [] + mcp: [] + +provides: + extensions: + - id: "agent-context" + version: "1.0.0" + presets: + - id: "implementation-planning" + version: "1.0.0" + priority: 10 + strategy: "append" + steps: + - id: "plan-implementation" + - id: "break-down-tasks" + workflows: + - id: "spec-to-implementation" + version: "1.0.0" + +tags: ["development", "implementation", "code-review"] diff --git a/examples/bundles/product-manager/README.md b/examples/bundles/product-manager/README.md new file mode 100644 index 0000000000..c7b5c8b9f8 --- /dev/null +++ b/examples/bundles/product-manager/README.md @@ -0,0 +1,22 @@ +# Product Manager bundle + +A role bundle that prepares a Spec Kit project for product managers driving +Spec-Driven Development: discovery, specification, and roadmap planning. + +## What it installs + +- **Extension** `agent-context` — keeps the agent context file in sync. +- **Preset** `product-discovery` (priority 10, append) — discovery-oriented + command set. +- **Steps** `draft-spec`, `review-spec` — specification authoring steps. +- **Workflow** `spec-to-roadmap` — turns an approved spec into a roadmap. + +This bundle is **integration-agnostic**: it inherits whatever integration the +project already uses (e.g. `copilot`, `claude`). + +## Usage + +```bash +specify bundle validate --path examples/bundles/product-manager +specify bundle build --path examples/bundles/product-manager --output dist/ +``` diff --git a/examples/bundles/product-manager/bundle.yml b/examples/bundles/product-manager/bundle.yml new file mode 100644 index 0000000000..9abba40bd4 --- /dev/null +++ b/examples/bundles/product-manager/bundle.yml @@ -0,0 +1,35 @@ +schema_version: "1.0" + +bundle: + id: "product-manager" + name: "Product Manager" + version: "1.0.0" + role: "product-manager" + description: "Spec-Driven Development setup for product managers: discovery, specification, and roadmap workflows." + author: "spec-kit-examples" + license: "MIT" + +requires: + speckit_version: ">=0.9.0" + tools: [] + mcp: [] + +# Agnostic bundle: inherits the project's active integration. + +provides: + extensions: + - id: "agent-context" + version: "1.0.0" + presets: + - id: "product-discovery" + version: "1.0.0" + priority: 10 + strategy: "append" + steps: + - id: "draft-spec" + - id: "review-spec" + workflows: + - id: "spec-to-roadmap" + version: "1.0.0" + +tags: ["product", "discovery", "roadmap"] diff --git a/examples/bundles/security-researcher/README.md b/examples/bundles/security-researcher/README.md new file mode 100644 index 0000000000..417cf5aced --- /dev/null +++ b/examples/bundles/security-researcher/README.md @@ -0,0 +1,23 @@ +# Security Researcher bundle + +A role bundle for security researchers practicing Spec-Driven Development: +threat modeling, security review, and compliance. + +## What it installs + +- **Extension** `agent-context` — keeps the agent context file in sync. +- **Preset** `security-compliance` (priority 5, append) — security and + compliance command set; presets apply in ascending priority order, so this + low number (5) places it ahead of higher-numbered presets in the stack. +- **Steps** `threat-model`, `security-review`. +- **Workflow** `secure-sdd` — a security-first SDD workflow. + +This bundle is **integration-agnostic**: it inherits the project's active +integration. + +## Usage + +```bash +specify bundle validate --path examples/bundles/security-researcher +specify bundle build --path examples/bundles/security-researcher --output dist/ +``` diff --git a/examples/bundles/security-researcher/bundle.yml b/examples/bundles/security-researcher/bundle.yml new file mode 100644 index 0000000000..d0b289e872 --- /dev/null +++ b/examples/bundles/security-researcher/bundle.yml @@ -0,0 +1,33 @@ +schema_version: "1.0" + +bundle: + id: "security-researcher" + name: "Security Researcher" + version: "1.0.0" + role: "security-researcher" + description: "Spec-Driven Development setup for security researchers: threat modeling, security review, and compliance checks." + author: "spec-kit-examples" + license: "MIT" + +requires: + speckit_version: ">=0.9.0" + tools: [] + mcp: [] + +provides: + extensions: + - id: "agent-context" + version: "1.0.0" + presets: + - id: "security-compliance" + version: "1.0.0" + priority: 5 + strategy: "append" + steps: + - id: "threat-model" + - id: "security-review" + workflows: + - id: "secure-sdd" + version: "1.0.0" + +tags: ["security", "compliance", "threat-modeling"] diff --git a/src/specify_cli/__init__.py b/src/specify_cli/__init__.py index 3e4c15122f..9a05b7e518 100644 --- a/src/specify_cli/__init__.py +++ b/src/specify_cli/__init__.py @@ -609,6 +609,13 @@ def _require_specify_project() -> Path: _register_preset_cmds(app) +# ===== Bundle Commands ===== + +# Bundler subcommand group (specify bundle ...) — see commands/bundle/. +from .commands.bundle import register as _register_bundle_cmds # noqa: E402 +_register_bundle_cmds(app) + + # ===== Extension Commands ===== diff --git a/src/specify_cli/bundler/__init__.py b/src/specify_cli/bundler/__init__.py new file mode 100644 index 0000000000..dac5347b67 --- /dev/null +++ b/src/specify_cli/bundler/__init__.py @@ -0,0 +1,19 @@ +"""Spec Kit bundler — importable, Typer-free logic for the ``specify bundle`` group. + +This package holds the models, services, and helpers behind the ``specify bundle`` +subcommand. It is intentionally free of any Typer/CLI imports so the orchestration +logic can be unit-tested independently of the command surface (Constitution +Principle I). The CLI wiring lives in ``specify_cli.commands.bundle``. +""" +from __future__ import annotations + +__all__ = ["BundlerError"] + + +class BundlerError(Exception): + """Base class for all actionable bundler errors. + + Carrying a clean message lets the CLI layer print a single, user-facing line + on stderr and exit non-zero without leaking a traceback (Constitution + Principle V — explicit, actionable errors). + """ diff --git a/src/specify_cli/bundler/commands_impl/__init__.py b/src/specify_cli/bundler/commands_impl/__init__.py new file mode 100644 index 0000000000..ae91e9190a --- /dev/null +++ b/src/specify_cli/bundler/commands_impl/__init__.py @@ -0,0 +1,2 @@ +"""Bundler command-implementation helpers (kept thin; logic lives in services).""" +from __future__ import annotations diff --git a/src/specify_cli/bundler/commands_impl/catalog_config.py b/src/specify_cli/bundler/commands_impl/catalog_config.py new file mode 100644 index 0000000000..477099b7d7 --- /dev/null +++ b/src/specify_cli/bundler/commands_impl/catalog_config.py @@ -0,0 +1,191 @@ +"""Persistence for the project-scoped catalog config (``.specify/bundle-catalogs.yml``). + +Only project scope is writable; built-in defaults are never deleted (they can be +overridden by adding a same-id source). The on-disk shape mirrors +``bundle-catalog.schema.md``: ``{schema_version, catalogs: [{id,url,priority,install_policy}]}``. +""" +from __future__ import annotations + +from pathlib import Path +from urllib.parse import urlparse +import re + +from .. import BundlerError +from ..lib.yamlio import dump_yaml, ensure_within, load_yaml +from ..models.catalog import ( + CONFIG_FILENAME, + BUILTIN_DEFAULT_STACK, + CatalogSource, + InstallPolicy, + Scope, +) + +CONFIG_SCHEMA_VERSION = "1.0" + +_BUILTIN_IDS = {raw["id"] for raw in BUILTIN_DEFAULT_STACK} + +# Windows absolute paths like ``C:\catalog.json`` parse with a single-letter +# ``scheme`` under urlparse; treat them as local files rather than URLs. +_WINDOWS_DRIVE_RE = re.compile(r"^[A-Za-z]:[\\/]") + + +def _config_path(project_root: Path) -> Path: + return Path(project_root) / ".specify" / CONFIG_FILENAME + + +def _read(project_root: Path) -> list[dict]: + # Confine the read (parity with the write path's within= guard): refuse to + # follow a symlinked or traversal-escaping .specify that resolves outside + # project_root. + path = ensure_within(project_root, _config_path(project_root)) + if not path.exists(): + return [] + data = load_yaml(path) + if data is None: + return [] + if not isinstance(data, dict): + raise BundlerError( + f"Malformed catalog config at {path}: expected a mapping at the top " + f"level, got {type(data).__name__}." + ) + schema_version = data.get("schema_version") + if schema_version is not None and ( + str(schema_version).strip().split(".")[0] + != CONFIG_SCHEMA_VERSION.split(".")[0] + ): + raise BundlerError( + f"Unsupported catalog config schema version " + f"'{str(schema_version).strip()}' at {path}; this Spec Kit " + f"understands version {CONFIG_SCHEMA_VERSION}. The file may have been " + "written by a newer version or is corrupt." + ) + catalogs = data.get("catalogs") + if catalogs is None: + return [] + if not isinstance(catalogs, list): + raise BundlerError( + f"Malformed catalog config at {path}: 'catalogs' must be a list, " + f"got {type(catalogs).__name__}." + ) + for entry in catalogs: + if not isinstance(entry, dict): + raise BundlerError( + f"Malformed catalog config at {path}: each catalog entry must be " + f"a mapping, got {type(entry).__name__}." + ) + return list(catalogs) + + +def _write(project_root: Path, catalogs: list[dict]) -> None: + payload = {"schema_version": CONFIG_SCHEMA_VERSION, "catalogs": catalogs} + dump_yaml(_config_path(project_root), payload, within=project_root) + + +def _slug(value: str) -> str: + # Lowercase so derived ids are deterministic and case-insensitive across + # platforms (e.g. 'Team-A.json' and 'team-a.json' yield the same id), + # keeping the case-sensitive duplicate check from admitting logical dupes. + return "".join(ch if ch.isalnum() else "-" for ch in value.lower()).strip("-") + + +_REMOTE_SCHEMES = {"http", "https", "file", "builtin"} + + +def _is_local_path(url: str) -> bool: + """True when *url* denotes a local filesystem path rather than a URL.""" + if _WINDOWS_DRIVE_RE.match(url): + return True + scheme = urlparse(url).scheme.lower() + return scheme not in _REMOTE_SCHEMES + + +def _canonicalize_url(url: str) -> str: + """Make local file paths absolute so config is independent of the caller's cwd. + + Remote URLs (``http(s)://``, ``file://``, ``builtin://``) are returned + unchanged; only bare/relative local paths are resolved to an absolute path. + """ + if _is_local_path(url): + return str(Path(url).expanduser().resolve()) + return url + + +def _derive_id(url: str) -> str: + parsed = urlparse(url) + if parsed.netloc: + # Use .hostname (not netloc.split(':')) so credentials, ports, and IPv6 + # literals (e.g. https://[2001:db8::1]/x) are handled correctly. Use the + # full host (TLD included) so different domains sharing a second-level + # label (example.com vs example.net) don't collide. _slug() lowercases + # and turns separators into dashes, so 'Example.com' -> 'example-com'. + host = parsed.hostname or "" + path_stem = Path(parsed.path).stem if parsed.path else "" + parts = [p for p in (_slug(host), _slug(path_stem)) if p] + return "-".join(parts) or "catalog" + stem = Path(parsed.path or url).stem + return _slug(stem) or "catalog" + + +def add_source( + project_root: Path, + url: str, + *, + policy: str, + priority: int, + source_id: str | None = None, +) -> CatalogSource: + url = url.strip() + if not url: + raise BundlerError("A catalog url is required.") + parsed = urlparse(url) + if not (parsed.scheme or parsed.path): + raise BundlerError(f"Invalid catalog url: '{url}'.") + # Reject unsupported URL schemes (e.g. ssh://, ftp://) up front so they are + # never silently canonicalized as local filesystem paths. Local paths that + # merely contain a ':' but no '://' (e.g. Windows drives) are still allowed. + if "://" in url and parsed.scheme.lower() not in _REMOTE_SCHEMES: + raise BundlerError( + f"Unsupported catalog url scheme '{parsed.scheme}://' in '{url}'. " + "Use http(s)://, file://, builtin://, or a local path." + ) + + url = _canonicalize_url(url) + install_policy = InstallPolicy.parse(policy) + resolved_id = (source_id or _derive_id(url)).strip() + + catalogs = _read(project_root) + for existing in catalogs: + if existing.get("id") == resolved_id or existing.get("url") == url: + raise BundlerError( + f"Catalog source '{resolved_id}' (or url) already exists in this project." + ) + + entry = { + "id": resolved_id, + "url": url, + "priority": int(priority), + "install_policy": install_policy.value, + } + catalogs.append(entry) + _write(project_root, catalogs) + return CatalogSource.from_dict(entry, Scope.PROJECT) + + +def remove_source(project_root: Path, id_or_url: str) -> str: + target = id_or_url.strip() + if target in _BUILTIN_IDS: + raise BundlerError( + f"'{target}' is a built-in default source and cannot be deleted " + "(add a same-id source to override it instead)." + ) + + catalogs = _read(project_root) + remaining = [ + c for c in catalogs if c.get("id") != target and c.get("url") != target + ] + if len(remaining) == len(catalogs): + raise BundlerError( + f"No project-scoped catalog source matching '{target}' was found." + ) + _write(project_root, remaining) + return target diff --git a/src/specify_cli/bundler/lib/__init__.py b/src/specify_cli/bundler/lib/__init__.py new file mode 100644 index 0000000000..f0c89c4a0f --- /dev/null +++ b/src/specify_cli/bundler/lib/__init__.py @@ -0,0 +1,2 @@ +"""Shared, dependency-light helpers for the bundler (YAML/JSON IO, versioning, project detection).""" +from __future__ import annotations diff --git a/src/specify_cli/bundler/lib/project.py b/src/specify_cli/bundler/lib/project.py new file mode 100644 index 0000000000..66b8a1b27b --- /dev/null +++ b/src/specify_cli/bundler/lib/project.py @@ -0,0 +1,62 @@ +"""Spec Kit project detection and active-integration resolution.""" +from __future__ import annotations + +from pathlib import Path + +from .. import BundlerError +from .yamlio import ensure_within, load_json + +DEFAULT_INTEGRATION = "copilot" + + +def find_project_root(start: Path | None = None) -> Path | None: + """Return the nearest ancestor (incl. *start*) containing a ``.specify/`` dir, or None. + + A symlinked ``.specify`` is not accepted as a project root: following it + could read/write outside the intended tree, and other CLI surfaces refuse + it for the same reason. + """ + current = Path(start or Path.cwd()).resolve() + for candidate in (current, *current.parents): + marker = candidate / ".specify" + if marker.is_dir() and not marker.is_symlink(): + return candidate + return None + + +def require_project_root(start: Path | None = None) -> Path: + """Return the Spec Kit project root or raise an actionable error.""" + root = find_project_root(start) + if root is None: + raise BundlerError( + "Not a Spec Kit project (no .specify/ directory). " + "Run 'specify bundle init' or 'specify init' first." + ) + return root + + +def active_integration(project_root: Path) -> str | None: + """Return the project's active integration id, if recorded. + + Spec Kit records the chosen integration in ``.specify/integration.json`` + during init. Returns None when it cannot be determined (e.g. agnostic). + """ + marker = Path(project_root) / ".specify" / "integration.json" + # Confine the read (mirrors records/catalog IO): refuse to follow a + # symlinked or traversal-escaping .specify that resolves outside + # project_root. An escape is treated as "not determinable". + try: + marker = ensure_within(project_root, marker) + except BundlerError: + return None + if not marker.exists(): + return None + try: + data = load_json(marker) + except BundlerError: + return None + if isinstance(data, dict): + value = data.get("integration") or data.get("id") or data.get("active") + if isinstance(value, str) and value: + return value + return None diff --git a/src/specify_cli/bundler/lib/versioning.py b/src/specify_cli/bundler/lib/versioning.py new file mode 100644 index 0000000000..552f21950c --- /dev/null +++ b/src/specify_cli/bundler/lib/versioning.py @@ -0,0 +1,99 @@ +"""SemVer parsing and constraint evaluation, built on ``packaging`` (already a dependency).""" +from __future__ import annotations + +import re + +from packaging.specifiers import InvalidSpecifier, SpecifierSet +from packaging.version import InvalidVersion, Version + +from .. import BundlerError + +# Common SemVer prerelease spellings (``1.2.3-rc1``, ``1.2.3-alpha.1``) that +# PEP 440 / ``packaging`` rejects verbatim. Normalized to PEP 440 before +# parsing so prerelease versions validate consistently (mirrors +# ``specify_cli._version._normalize_tag``). +_PRERELEASE_PATTERN = re.compile( + r"^([0-9]+\.[0-9]+\.[0-9]+)[-.]?(alpha|beta|a|b|rc)[-.]?([0-9]+)(.*)$", + flags=re.IGNORECASE, +) + + +def _normalize_semver(value: str) -> str: + """Normalize common SemVer prerelease spellings into PEP 440 text.""" + text = str(value) + normalized = text[1:] if text[:1] in ("v", "V") else text + match = _PRERELEASE_PATTERN.match(normalized) + if match is None: + return normalized + base, label, number, rest = match.groups() + pep440_label = {"alpha": "a", "beta": "b"}.get(label.lower(), label.lower()) + return f"{base}{pep440_label}{number}{rest}" + + +def parse_version(value: str) -> Version: + """Parse a version string into a comparable :class:`Version`.""" + try: + return Version(_normalize_semver(value)) + except InvalidVersion as exc: + raise BundlerError(f"Invalid version '{value}': {exc}") from exc + + +_SPECIFIER_CLAUSE = re.compile(r"^\s*(===|==|~=|!=|<=|>=|<|>)?\s*(.*?)\s*$") + + +def _normalize_constraint(value: str) -> str: + """Normalize the version portion of each clause in a constraint string. + + ``packaging.SpecifierSet`` rejects SemVer prerelease spellings like + ``>=1.2.3-rc1`` verbatim, even though :func:`parse_version` accepts the same + spelling for installed versions. Normalize each comma-separated clause's + version so prerelease handling is consistent across versions and constraints. + """ + clauses = [] + for raw in str(value).split(","): + if not raw.strip(): + continue + match = _SPECIFIER_CLAUSE.match(raw) + operator, version = match.groups() + clauses.append(f"{operator or ''}{_normalize_semver(version)}") + return ",".join(clauses) + + +def parse_constraint(value: str) -> SpecifierSet: + """Parse a version constraint such as ``>=0.9.0`` into a :class:`SpecifierSet`.""" + try: + return SpecifierSet(_normalize_constraint(value)) + except InvalidSpecifier as exc: + raise BundlerError( + f"Invalid version constraint '{value}': {exc}" + ) from exc + + +def satisfies(installed: str, constraint: str) -> bool: + """Return True if *installed* satisfies *constraint* (e.g. ``">=0.9.0"``). + + Pre-releases are allowed so a dev/pre build of Spec Kit still counts. + """ + spec = parse_constraint(constraint) + version = parse_version(installed) + return spec.contains(version, prereleases=True) + + +_SEMVER_RE = re.compile( + r"^(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)" + r"(?:-(?:(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" + r"(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?" + r"(?:\+(?:[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$" +) + + +def is_semver(value: str) -> bool: + """Return True only for a full ``MAJOR.MINOR.PATCH`` SemVer string. + + Stricter than ``packaging.version.Version``, which also accepts partial + versions like ``"1"`` or ``"1.0"``. An optional leading ``v`` or ``V`` is + tolerated (mirrors ``_normalize_semver``). + """ + text = str(value) + core = text[1:] if text[:1] in ("v", "V") else text + return bool(_SEMVER_RE.match(core)) diff --git a/src/specify_cli/bundler/lib/yamlio.py b/src/specify_cli/bundler/lib/yamlio.py new file mode 100644 index 0000000000..e4cd538a09 --- /dev/null +++ b/src/specify_cli/bundler/lib/yamlio.py @@ -0,0 +1,119 @@ +"""YAML/JSON read-write helpers with path confinement (Constitution Principles IV & V). + +All reads/writes go through these functions so that: +- IO failures degrade into actionable :class:`~specify_cli.bundler.BundlerError`s + rather than raw tracebacks, and +- every path can be confined to an allowed root via :func:`ensure_within`. +""" +from __future__ import annotations + +import json +import os +import re +from pathlib import Path, PurePosixPath +from typing import Any + +import yaml + +from .. import BundlerError + + +def ensure_within(root: Path, candidate: Path) -> Path: + """Resolve *candidate* and guarantee it stays within *root*. + + Refuses path-traversal payloads and symlink escapes. Returns the resolved, + confined path. Raises :class:`BundlerError` if the path escapes *root*. + """ + root_resolved = Path(root).resolve() + # Resolve symlinks so a symlinked component cannot point outside the root. + candidate_resolved = Path(candidate).resolve() + try: + candidate_resolved.relative_to(root_resolved) + except ValueError as exc: + raise BundlerError( + f"Refusing path '{candidate}' — it escapes the allowed root '{root}'." + ) from exc + return candidate_resolved + + +def load_yaml(path: Path) -> Any: + """Parse a YAML file, returning ``{}`` for an empty document.""" + path = Path(path) + if not path.exists(): + raise BundlerError(f"File not found: {path}") + try: + with path.open("r", encoding="utf-8") as handle: + return yaml.safe_load(handle) or {} + except yaml.YAMLError as exc: + raise BundlerError(f"Invalid YAML in {path}: {exc}") from exc + except OSError as exc: + raise BundlerError(f"Could not read {path}: {exc}") from exc + + +def dump_yaml(path: Path, data: Any, *, within: Path | None = None) -> Path: + """Write *data* as YAML to *path* (optionally confined to *within*).""" + path = Path(path) + if within is not None: + path = ensure_within(within, path) + try: + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w", encoding="utf-8") as handle: + yaml.safe_dump(data, handle, sort_keys=False, default_flow_style=False) + except OSError as exc: + raise BundlerError(f"Could not write {path}: {exc}") from exc + return path + + +def load_json(path: Path) -> Any: + """Parse a JSON file.""" + path = Path(path) + if not path.exists(): + raise BundlerError(f"File not found: {path}") + try: + with path.open("r", encoding="utf-8") as handle: + return json.load(handle) + except json.JSONDecodeError as exc: + raise BundlerError(f"Invalid JSON in {path}: {exc}") from exc + except OSError as exc: + raise BundlerError(f"Could not read {path}: {exc}") from exc + + +def loads_json(text: str, *, origin: str = "") -> Any: + """Parse JSON from a string (used for catalog payloads fetched as text).""" + try: + return json.loads(text) + except json.JSONDecodeError as exc: + raise BundlerError(f"Invalid JSON from {origin}: {exc}") from exc + + +def dump_json(path: Path, data: Any, *, within: Path | None = None) -> Path: + """Write *data* as pretty JSON to *path* (optionally confined to *within*).""" + path = Path(path) + if within is not None: + path = ensure_within(within, path) + try: + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w", encoding="utf-8") as handle: + json.dump(data, handle, indent=2, sort_keys=False) + handle.write("\n") + except OSError as exc: + raise BundlerError(f"Could not write {path}: {exc}") from exc + return path + + +def is_safe_relpath(rel: str) -> bool: + """Return True if *rel* is a project-relative path with no traversal/absolute parts. + + Platform-independent: a POSIX-absolute path (``/abs``) or a Windows + drive-absolute path (``C:\\x``) is rejected on every OS, since these strings + can appear in untrusted catalog/manifest data regardless of the host. + """ + if not rel: + return False + normalized = rel.replace("\\", "/") + if os.path.isabs(rel) or normalized.startswith("/"): + return False + if re.match(r"^[A-Za-z]:", normalized): # Windows drive-absolute (C:/...) + return False + parts = PurePosixPath(normalized).parts + return ".." not in parts diff --git a/src/specify_cli/bundler/models/__init__.py b/src/specify_cli/bundler/models/__init__.py new file mode 100644 index 0000000000..2a5136287b --- /dev/null +++ b/src/specify_cli/bundler/models/__init__.py @@ -0,0 +1,2 @@ +"""Bundler data models (manifest, catalog, records).""" +from __future__ import annotations diff --git a/src/specify_cli/bundler/models/catalog.py b/src/specify_cli/bundler/models/catalog.py new file mode 100644 index 0000000000..dd069f5bc9 --- /dev/null +++ b/src/specify_cli/bundler/models/catalog.py @@ -0,0 +1,258 @@ +"""Catalog models: source stack (priority + install policy) and catalog entries. + +Mirrors ``contracts/bundle-catalog.schema.md``. The stack precedence is +project > user > built-in; install is permitted only from ``install-allowed`` +sources. +""" +from __future__ import annotations + +from dataclasses import dataclass, field +from enum import Enum +from pathlib import Path +from typing import Any + +from .. import BundlerError +from ..lib.yamlio import ensure_within, load_yaml + +CONFIG_FILENAME = "bundle-catalogs.yml" + + +class InstallPolicy(str, Enum): + INSTALL_ALLOWED = "install-allowed" + DISCOVERY_ONLY = "discovery-only" + + @classmethod + def parse(cls, value: Any) -> "InstallPolicy": + text = str(value or "").strip() + for policy in cls: + if policy.value == text: + return policy + raise BundlerError( + f"Invalid install_policy '{value}' " + f"(must be one of {[p.value for p in cls]})." + ) + + +class Scope(str, Enum): + PROJECT = "project" + USER = "user" + BUILTIN = "built-in" + + +# Built-in default stack (used when no project/user config overrides it). +BUILTIN_DEFAULT_STACK: tuple[dict[str, Any], ...] = ( + {"id": "default", "url": "builtin://default", "priority": 1, + "install_policy": InstallPolicy.INSTALL_ALLOWED.value}, + {"id": "community", "url": "builtin://community", "priority": 2, + "install_policy": InstallPolicy.DISCOVERY_ONLY.value}, +) + + +@dataclass(frozen=True) +class CatalogSource: + id: str + url: str + priority: int + install_policy: InstallPolicy + scope: Scope = Scope.PROJECT + + @property + def install_allowed(self) -> bool: + return self.install_policy is InstallPolicy.INSTALL_ALLOWED + + @classmethod + def from_dict(cls, data: Any, scope: Scope) -> "CatalogSource": + if not isinstance(data, dict): + raise BundlerError("Each catalog source must be a mapping.") + source_id = str(data.get("id", "")).strip() + url = str(data.get("url", "")).strip() + if not source_id: + raise BundlerError("A catalog source is missing its 'id'.") + if not url: + raise BundlerError(f"Catalog source '{source_id}' is missing its 'url'.") + priority = data.get("priority") + if priority is None: + raise BundlerError(f"Catalog source '{source_id}' is missing its 'priority'.") + if isinstance(priority, bool) or not isinstance(priority, (int, str)): + raise BundlerError( + f"Catalog source '{source_id}' has a non-integer priority: {priority!r}." + ) + try: + priority_int = int(priority) + except (TypeError, ValueError): + raise BundlerError( + f"Catalog source '{source_id}' has a non-integer priority: {priority!r}." + ) from None + return cls( + id=source_id, + url=url, + priority=priority_int, + install_policy=InstallPolicy.parse(data.get("install_policy")), + scope=scope, + ) + + def to_dict(self) -> dict[str, Any]: + return { + "id": self.id, + "url": self.url, + "priority": self.priority, + "install_policy": self.install_policy.value, + } + + +def _parse_tags(value: Any, entry_id: str) -> tuple[str, ...]: + """Coerce a catalog entry's ``tags`` into a tuple of strings. + + Catalogs are untrusted input: a bare string would otherwise be iterated + character-by-character, so reject anything that is not a list/tuple. + """ + if value is None: + return () + if isinstance(value, (str, bytes)) or not isinstance(value, (list, tuple)): + raise BundlerError( + f"Catalog entry '{entry_id}': 'tags' must be a list of strings." + ) + return tuple(str(t) for t in value) + + +def _parse_verified(value: Any, entry_id: str) -> bool: + """Validate a catalog entry's ``verified`` flag is a real boolean. + + ``bool("false")`` is truthy, so coercing arbitrary strings would silently + mark untrusted entries as verified; require an actual boolean instead. + """ + if isinstance(value, bool): + return value + raise BundlerError( + f"Catalog entry '{entry_id}': 'verified' must be a boolean (true/false)." + ) + + +@dataclass(frozen=True) +class CatalogEntry: + id: str + name: str + version: str + role: str + description: str + author: str + license: str + download_url: str + requires_speckit_version: str + provides: dict[str, int] = field(default_factory=dict) + repository: str | None = None + tags: tuple[str, ...] = () + verified: bool = False + # Resolution provenance (filled in by the catalog stack at lookup time): + source_id: str | None = None + source_policy: InstallPolicy | None = None + + @classmethod + def from_dict(cls, data: Any) -> "CatalogEntry": + if not isinstance(data, dict): + raise BundlerError("Each catalog entry must be a mapping.") + entry_id = str(data.get("id", "")).strip() + requires = data.get("requires") or {} + if not isinstance(requires, dict): + raise BundlerError( + f"Catalog entry '{entry_id or ''}': 'requires' must be a " + "mapping when present." + ) + provides_raw = data.get("provides") or {} + if not isinstance(provides_raw, dict): + raise BundlerError( + f"Catalog entry '{entry_id or ''}': 'provides' must be a " + "mapping when present." + ) + return cls( + id=entry_id, + name=str(data.get("name", "")).strip(), + version=str(data.get("version", "")).strip(), + role=str(data.get("role", "")).strip(), + description=str(data.get("description", "")).strip(), + author=str(data.get("author", "")).strip(), + license=str(data.get("license", "")).strip(), + download_url=str(data.get("download_url", "")).strip(), + requires_speckit_version=str(requires.get("speckit_version", "")).strip(), + provides=dict(provides_raw), + repository=(str(data["repository"]) if data.get("repository") else None), + tags=_parse_tags(data.get("tags"), entry_id), + verified=_parse_verified(data.get("verified", False), entry_id), + ) + + def with_provenance(self, source: CatalogSource) -> "CatalogEntry": + return CatalogEntry( + id=self.id, name=self.name, version=self.version, role=self.role, + description=self.description, author=self.author, license=self.license, + download_url=self.download_url, + requires_speckit_version=self.requires_speckit_version, + provides=self.provides, repository=self.repository, tags=self.tags, + verified=self.verified, source_id=source.id, + source_policy=source.install_policy, + ) + + +def load_catalog_payload(data: Any) -> dict[str, CatalogEntry]: + """Parse a catalog JSON payload into ``{bundle_id: CatalogEntry}``.""" + if not isinstance(data, dict): + raise BundlerError("Catalog payload must be a JSON object.") + bundles_raw = data.get("bundles") + if not isinstance(bundles_raw, dict): + raise BundlerError("Catalog payload is missing a 'bundles' object.") + entries: dict[str, CatalogEntry] = {} + for bundle_id, entry_raw in bundles_raw.items(): + key = str(bundle_id) + entry = CatalogEntry.from_dict(entry_raw) + # The enclosing key is the authoritative bundle id used by + # search/resolve/install. Reject entries whose own ``id`` is missing or + # disagrees with the key, so a malformed or malicious catalog can't list + # an id that resolves to a different (or no) bundle. + if not entry.id: + raise BundlerError( + f"Catalog entry for '{key}' is missing its 'id' field." + ) + if entry.id != key: + raise BundlerError( + f"Catalog entry id mismatch: key '{key}' != entry id " + f"'{entry.id}'." + ) + entries[key] = entry + return entries + + +def load_source_stack(project_root: Path, user_config_dir: Path | None = None) -> list[CatalogSource]: + """Build the effective, priority-sorted source stack (project > user > built-in). + + A source id present at a higher-precedence scope overrides the same id at a + lower scope. The built-in default stack is always the fallback. + """ + by_id: dict[str, CatalogSource] = {} + + # Lowest precedence first; later writes override earlier ones for the same id. + for raw in BUILTIN_DEFAULT_STACK: + src = CatalogSource.from_dict(raw, Scope.BUILTIN) + by_id[src.id] = src + + if user_config_dir is not None: + _merge_config(by_id, Path(user_config_dir) / CONFIG_FILENAME, Scope.USER) + + # Confine the project-scoped read: refuse a symlinked .specify/ that + # resolves outside the project root (consistent with other guarded reads). + project_config = Path(project_root) / ".specify" / CONFIG_FILENAME + if project_config.exists(): + ensure_within(project_root, project_config) + _merge_config(by_id, project_config, Scope.PROJECT) + + return sorted(by_id.values(), key=lambda s: (s.priority, s.id)) + + +def _merge_config(by_id: dict[str, CatalogSource], config_path: Path, scope: Scope) -> None: + if not config_path.exists(): + return + data = load_yaml(config_path) + catalogs = data.get("catalogs") if isinstance(data, dict) else None + if not catalogs: + return + for raw in catalogs: + src = CatalogSource.from_dict(raw, scope) + by_id[src.id] = src diff --git a/src/specify_cli/bundler/models/manifest.py b/src/specify_cli/bundler/models/manifest.py new file mode 100644 index 0000000000..4a903fbd18 --- /dev/null +++ b/src/specify_cli/bundler/models/manifest.py @@ -0,0 +1,263 @@ +"""Bundle manifest model (``bundle.yml``) — parsing and structural normalization. + +Mirrors ``contracts/bundle-manifest.schema.md``. Structural validation (shape, +required fields, enum/semver checks) lives here; *reference* resolution against a +catalog stack lives in the validator/resolver services. +""" +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +from .. import BundlerError +from ..lib.versioning import is_semver +from ..lib.yamlio import load_yaml + +SUPPORTED_SCHEMA_VERSIONS = {"1.0"} +PRESET_STRATEGIES = {"replace", "prepend", "append", "wrap"} + +COMPONENT_KINDS = ("extensions", "presets", "steps", "workflows") + +# A bundle id must be a filesystem-safe slug: it is interpolated into artifact +# filenames (e.g. ``-.zip``), so path separators or traversal +# segments must never appear. +_SAFE_BUNDLE_ID = re.compile(r"^[a-z0-9](?:[a-z0-9._-]*[a-z0-9])?$") + + +@dataclass(frozen=True) +class ComponentRef: + """A pointer to an existing Spec Kit primitive a bundle installs.""" + + kind: str # one of COMPONENT_KINDS (singularized concept), stored plural-of-origin + id: str + version: str | None = None + source: str | None = None + priority: int | None = None # presets only + strategy: str | None = None # presets only + + def label(self) -> str: + return f"{self.kind[:-1]}:{self.id}@{self.version or 'unpinned'}" + + +@dataclass(frozen=True) +class IntegrationRef: + id: str + + +@dataclass(frozen=True) +class Requires: + speckit_version: str + tools: tuple[str, ...] = () + mcp: tuple[str, ...] = () + + +@dataclass(frozen=True) +class BundleMeta: + id: str + name: str + version: str + role: str + description: str + author: str + license: str + + +@dataclass +class BundleManifest: + schema_version: str + bundle: BundleMeta + requires: Requires + integration: IntegrationRef | None = None + extensions: list[ComponentRef] = field(default_factory=list) + presets: list[ComponentRef] = field(default_factory=list) + steps: list[ComponentRef] = field(default_factory=list) + workflows: list[ComponentRef] = field(default_factory=list) + tags: tuple[str, ...] = () + source_path: Path | None = None + + @property + def components(self) -> list[ComponentRef]: + """All installable component references in deterministic order.""" + return [*self.extensions, *self.presets, *self.steps, *self.workflows] + + # -- construction --------------------------------------------------------- + + @classmethod + def from_file(cls, path: Path) -> "BundleManifest": + data = load_yaml(path) + manifest = cls.from_dict(data) + manifest.source_path = Path(path) + return manifest + + @classmethod + def from_dict(cls, data: Any) -> "BundleManifest": + if not isinstance(data, dict): + raise BundlerError("Manifest must be a YAML mapping at the top level.") + + schema_version = str(data.get("schema_version", "")).strip() + + bundle_raw = data.get("bundle") + if not isinstance(bundle_raw, dict): + raise BundlerError("Manifest is missing the required 'bundle' mapping.") + meta = BundleMeta( + id=str(bundle_raw.get("id", "")).strip(), + name=str(bundle_raw.get("name", "")).strip(), + version=str(bundle_raw.get("version", "")).strip(), + role=str(bundle_raw.get("role", "")).strip(), + description=str(bundle_raw.get("description", "")).strip(), + author=str(bundle_raw.get("author", "")).strip(), + license=str(bundle_raw.get("license", "")).strip(), + ) + + requires_raw = data.get("requires") or {} + if not isinstance(requires_raw, dict): + raise BundlerError("'requires' must be a mapping when present.") + requires = Requires( + speckit_version=str(requires_raw.get("speckit_version", "")).strip(), + tools=_parse_str_list(requires_raw.get("tools"), "requires.tools"), + mcp=_parse_str_list(requires_raw.get("mcp"), "requires.mcp"), + ) + + integration = None + integration_raw = data.get("integration") + if isinstance(integration_raw, dict) and integration_raw.get("id"): + integration = IntegrationRef(id=str(integration_raw["id"]).strip()) + + provides = data.get("provides") or {} + if not isinstance(provides, dict): + raise BundlerError("'provides' must be a mapping when present.") + + tags_raw = data.get("tags") + if tags_raw is None: + tags_raw = [] + else: + tags_raw = _parse_str_list(tags_raw, "tags") + + manifest = cls( + schema_version=schema_version, + bundle=meta, + requires=requires, + integration=integration, + extensions=_parse_refs("extensions", provides.get("extensions")), + presets=_parse_refs("presets", provides.get("presets")), + steps=_parse_refs("steps", provides.get("steps")), + workflows=_parse_refs("workflows", provides.get("workflows")), + tags=tuple(str(t) for t in tags_raw), + ) + return manifest + + # -- structural validation ------------------------------------------------ + + def structural_errors(self) -> list[str]: + """Return a list of human-readable structural problems (empty == valid).""" + errors: list[str] = [] + + if self.schema_version not in SUPPORTED_SCHEMA_VERSIONS: + errors.append( + f"schema_version '{self.schema_version or ''}' is not supported " + f"(supported: {sorted(SUPPORTED_SCHEMA_VERSIONS)})." + ) + + required = { + "bundle.id": self.bundle.id, + "bundle.name": self.bundle.name, + "bundle.version": self.bundle.version, + "bundle.role": self.bundle.role, + "bundle.description": self.bundle.description, + "bundle.author": self.bundle.author, + "bundle.license": self.bundle.license, + "requires.speckit_version": self.requires.speckit_version, + } + for field_path, value in required.items(): + if not value: + errors.append(f"Missing required field: {field_path}.") + + if self.bundle.version and not is_semver(self.bundle.version): + errors.append(f"bundle.version '{self.bundle.version}' is not valid semver.") + + if self.bundle.id and not _SAFE_BUNDLE_ID.match(self.bundle.id): + errors.append( + f"bundle.id '{self.bundle.id}' must be a slug " + "(lowercase letters, digits, '.', '_', '-'; no path separators)." + ) + + for ref in self.components: + if not ref.id: + errors.append(f"A {ref.kind[:-1]} entry is missing its 'id'.") + if ref.kind != "steps" and not ref.version: + errors.append( + f"{ref.kind[:-1]} '{ref.id or ''}' must be pinned to a 'version'." + ) + if ref.version and not is_semver(ref.version): + errors.append( + f"{ref.kind[:-1]} '{ref.id}' has invalid version '{ref.version}'." + ) + + for ref in self.presets: + if ref.priority is None: + errors.append(f"preset '{ref.id}' must declare an integer 'priority'.") + if ref.strategy is None or ref.strategy not in PRESET_STRATEGIES: + errors.append( + f"preset '{ref.id}' has invalid strategy '{ref.strategy}' " + f"(must be one of {sorted(PRESET_STRATEGIES)})." + ) + + return errors + + def is_agnostic(self) -> bool: + """True when the bundle declares no integration (inherits the active one).""" + return self.integration is None + + +def _parse_str_list(raw: Any, field_name: str) -> tuple[str, ...]: + """Coerce a manifest list-of-strings field into a tuple of strings. + + Rejects a bare string/bytes (which would otherwise be iterated + character-by-character) and any non-list/tuple, matching the manifest + contract (``string[]``). + """ + if raw is None: + return () + if isinstance(raw, (str, bytes)) or not isinstance(raw, (list, tuple)): + raise BundlerError(f"'{field_name}' must be a list of strings when present.") + return tuple(str(item) for item in raw) + + +def _parse_refs(kind: str, raw: Any) -> list[ComponentRef]: + if raw is None: + return [] + if not isinstance(raw, list): + raise BundlerError(f"provides.{kind} must be a list when present.") + refs: list[ComponentRef] = [] + for item in raw: + if not isinstance(item, dict): + raise BundlerError(f"Each provides.{kind} entry must be a mapping.") + priority = _parse_priority(kind, item.get("priority")) + refs.append( + ComponentRef( + kind=kind, + id=str(item.get("id", "")).strip(), + version=(str(item["version"]).strip() if item.get("version") else None), + source=(str(item["source"]).strip() if item.get("source") else None), + priority=priority, + strategy=(str(item["strategy"]).strip() if item.get("strategy") else None), + ) + ) + return refs + + +def _parse_priority(kind: str, raw: Any) -> int | None: + if raw is None: + return None + if isinstance(raw, bool) or not isinstance(raw, (int, str)): + raise BundlerError( + f"provides.{kind} priority must be an integer, got {raw!r}." + ) + try: + return int(raw) + except (TypeError, ValueError): + raise BundlerError( + f"provides.{kind} priority must be an integer, got {raw!r}." + ) from None diff --git a/src/specify_cli/bundler/models/records.py b/src/specify_cli/bundler/models/records.py new file mode 100644 index 0000000000..15c53523c3 --- /dev/null +++ b/src/specify_cli/bundler/models/records.py @@ -0,0 +1,229 @@ +"""Installed-bundle records — provenance for precise list/remove/update. + +Records are stored as JSON at ``.specify/bundle-records.json``. Each record +captures exactly which components a bundle contributed so removal touches only +that bundle's components and never collateral (FR-022, SC-004). +""" +from __future__ import annotations + +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +from .. import BundlerError +from ..lib.yamlio import dump_json, ensure_within, load_json +from .manifest import COMPONENT_KINDS, ComponentRef + +RECORDS_FILENAME = "bundle-records.json" +RECORDS_SCHEMA_VERSION = "1.0" + + +@dataclass(frozen=True) +class InstalledBundleRecord: + bundle_id: str + version: str + contributed_components: tuple[ComponentRef, ...] + installed_at: str + + @classmethod + def create( + cls, + bundle_id: str, + version: str, + components: list[ComponentRef], + installed_at: str | None = None, + ) -> "InstalledBundleRecord": + return cls( + bundle_id=bundle_id, + version=version, + contributed_components=tuple(components), + installed_at=installed_at or _utc_now(), + ) + + def to_dict(self) -> dict[str, Any]: + return { + "bundle_id": self.bundle_id, + "version": self.version, + "installed_at": self.installed_at, + "contributed_components": [ + _component_to_dict(c) for c in self.contributed_components + ], + } + + @classmethod + def from_dict(cls, data: Any) -> "InstalledBundleRecord": + if not isinstance(data, dict): + raise BundlerError("Each installed-bundle record must be a mapping.") + components_raw = data.get("contributed_components") or [] + if not isinstance(components_raw, list): + raise BundlerError( + "Corrupt record: 'contributed_components' must be a list." + ) + bundle_id = str(data.get("bundle_id", "")).strip() + version = str(data.get("version", "")).strip() + if not bundle_id: + raise BundlerError( + "Corrupt records file: an installed-bundle record is missing " + "its 'bundle_id'." + ) + if not version: + raise BundlerError( + f"Corrupt records file: record for bundle '{bundle_id}' is " + "missing its 'version'." + ) + return cls( + bundle_id=bundle_id, + version=version, + installed_at=str(data.get("installed_at", "")).strip(), + contributed_components=tuple( + _component_from_dict(c) for c in components_raw + ), + ) + + +def records_path(project_root: Path) -> Path: + return Path(project_root) / ".specify" / RECORDS_FILENAME + + +def _check_schema_version(value: Any, *, path: Path, required: bool) -> None: + """Reject a records file whose schema version we cannot safely parse. + + A future incompatible format (or a corrupted file) must fail fast with an + actionable error rather than being silently mis-parsed, which could lead to + incorrect bundle attribution or removal. Forward-compatible minor bumps that + keep the same major version are accepted. + """ + if value is None: + if required: + raise BundlerError( + f"Corrupt records file: {path} — missing 'schema_version'. " + f"Expected version {RECORDS_SCHEMA_VERSION}." + ) + return + seen = str(value).strip() + if seen.split(".")[0] != RECORDS_SCHEMA_VERSION.split(".")[0]: + raise BundlerError( + f"Unsupported records schema version '{seen}' at {path}; this " + f"Spec Kit understands version {RECORDS_SCHEMA_VERSION}. The file may " + "have been written by a newer version or is corrupt." + ) + + +def load_records(project_root: Path) -> list[InstalledBundleRecord]: + # Defense in depth (mirrors the write path's within= confinement): refuse to + # read through a symlinked or traversal-escaping ``.specify`` that resolves + # outside project_root. + path = ensure_within(project_root, records_path(project_root)) + if not path.exists(): + return [] + data = load_json(path) + if not isinstance(data, dict): + raise BundlerError(f"Corrupt records file: {path}") + _check_schema_version(data.get("schema_version"), path=path, required=True) + bundles = data.get("bundles") or [] + if not isinstance(bundles, list): + raise BundlerError( + f"Corrupt records file: {path} — 'bundles' must be a list." + ) + return [InstalledBundleRecord.from_dict(item) for item in bundles] + + +def save_records(project_root: Path, records: list[InstalledBundleRecord]) -> None: + payload = { + "schema_version": RECORDS_SCHEMA_VERSION, + "updated_at": _utc_now(), + "bundles": [r.to_dict() for r in records], + } + dump_json(records_path(project_root), payload, within=project_root) + + +def find_record( + records: list[InstalledBundleRecord], bundle_id: str +) -> InstalledBundleRecord | None: + for record in records: + if record.bundle_id == bundle_id: + return record + return None + + +def upsert_record( + records: list[InstalledBundleRecord], record: InstalledBundleRecord +) -> list[InstalledBundleRecord]: + """Return a new list with *record* replacing any same-id record (append otherwise).""" + updated = [r for r in records if r.bundle_id != record.bundle_id] + updated.append(record) + return updated + + +def remove_record( + records: list[InstalledBundleRecord], bundle_id: str +) -> list[InstalledBundleRecord]: + return [r for r in records if r.bundle_id != bundle_id] + + +def components_still_needed( + records: list[InstalledBundleRecord], exclude_bundle_id: str +) -> set[tuple[str, str]]: + """Set of ``(kind, id)`` component keys required by bundles other than the excluded one.""" + needed: set[tuple[str, str]] = set() + for record in records: + if record.bundle_id == exclude_bundle_id: + continue + for component in record.contributed_components: + needed.add((component.kind, component.id)) + return needed + + +def _component_to_dict(ref: ComponentRef) -> dict[str, Any]: + data: dict[str, Any] = {"kind": ref.kind, "id": ref.id} + if ref.version is not None: + data["version"] = ref.version + if ref.source is not None: + data["source"] = ref.source + if ref.priority is not None: + data["priority"] = ref.priority + if ref.strategy is not None: + data["strategy"] = ref.strategy + return data + + +def _component_from_dict(data: Any) -> ComponentRef: + if not isinstance(data, dict): + raise BundlerError("Each contributed component must be a mapping.") + kind = str(data.get("kind", "")).strip() + cid = str(data.get("id", "")).strip() + if kind not in COMPONENT_KINDS: + raise BundlerError( + f"Corrupt records file: component 'kind' must be one of " + f"{list(COMPONENT_KINDS)}, got {kind or ''!r}." + ) + if not cid: + raise BundlerError( + "Corrupt records file: a contributed component is missing its 'id'." + ) + return ComponentRef( + kind=kind, + id=cid, + version=(str(data["version"]) if data.get("version") else None), + source=(str(data["source"]) if data.get("source") else None), + priority=_parse_priority(data.get("priority")), + strategy=(str(data["strategy"]) if data.get("strategy") else None), + ) + + +def _parse_priority(raw: Any) -> int | None: + if raw is None: + return None + if isinstance(raw, bool) or not isinstance(raw, (int, str)): + raise BundlerError(f"Component priority must be an integer, got {raw!r}.") + try: + return int(raw) + except (TypeError, ValueError): + raise BundlerError( + f"Component priority must be an integer, got {raw!r}." + ) from None + + +def _utc_now() -> str: + return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") diff --git a/src/specify_cli/bundler/services/__init__.py b/src/specify_cli/bundler/services/__init__.py new file mode 100644 index 0000000000..1db5b56614 --- /dev/null +++ b/src/specify_cli/bundler/services/__init__.py @@ -0,0 +1,2 @@ +"""Bundler services (catalog stack, resolver, installer, conflict, validator, packager).""" +from __future__ import annotations diff --git a/src/specify_cli/bundler/services/adapters.py b/src/specify_cli/bundler/services/adapters.py new file mode 100644 index 0000000000..17c9d3324c --- /dev/null +++ b/src/specify_cli/bundler/services/adapters.py @@ -0,0 +1,193 @@ +"""Concrete adapters: catalog fetching and primitive installation. + +These wire the bundler's injectable seams to the real environment: + +* :func:`make_catalog_fetcher` returns an offline-first fetcher that reads + built-in catalogs and local/pinned file URLs without network, and falls back + to a timeout-bounded HTTP GET only for ``http(s)://`` sources. +* :class:`DefaultPrimitiveInstaller` dispatches component install/remove to the + existing Spec Kit primitive machinery in-process. +""" +from __future__ import annotations + +import re +from pathlib import Path +from urllib.parse import ParseResult, urlparse +from urllib.request import url2pathname + +from .. import BundlerError +from ..lib.yamlio import loads_json +from ..models.catalog import CatalogSource +from ..models.manifest import ComponentRef + +# Built-in catalog payloads ship empty by default; a host distribution can +# replace these with curated content. Keeping them here makes ``search``/``info`` +# work fully offline against the default stack. +_BUILTIN_CATALOGS: dict[str, dict] = { + "builtin://default": { + "schema_version": "1.0", + "catalog_url": "builtin://default", + "bundles": {}, + }, + "builtin://community": { + "schema_version": "1.0", + "catalog_url": "builtin://community", + "bundles": {}, + }, +} + +HTTP_TIMEOUT_SECONDS = 10 + +# Windows absolute paths like ``C:\catalog.json`` parse with a single-letter +# ``scheme`` under urlparse; treat them as local files rather than URLs. +_WINDOWS_DRIVE_RE = re.compile(r"^[A-Za-z]:[\\/]") + + +def _is_windows_drive_path(url: str) -> bool: + return bool(_WINDOWS_DRIVE_RE.match(url)) + + +def _file_url_to_path(parsed: ParseResult) -> Path: + """Convert a ``file://`` URL to a local path. + + Uses ``url2pathname`` for percent-decoding and OS-correct separators, and + preserves ``netloc`` so UNC paths (``file://server/share``) and Windows + drive URLs (``file:///C:/x``) resolve correctly instead of dropping host + or producing ``/C:/x``. + """ + netloc = parsed.netloc + if netloc and netloc.lower() != "localhost": + # UNC share: file://server/share/... -> \\server\share\... + return Path(url2pathname(f"//{netloc}{parsed.path}")) + return Path(url2pathname(parsed.path)) + + +def _validate_remote_url(source_id: str, url: str) -> None: + """Restrict remote catalogs to HTTPS (HTTP only for localhost) with a host. + + Mirrors ``specify_cli.catalogs`` URL validation to avoid MITM/downgrade + issues before any network call. + """ + parsed = urlparse(url) + is_localhost = parsed.hostname in ("localhost", "127.0.0.1", "::1") + if parsed.scheme != "https" and not (parsed.scheme == "http" and is_localhost): + raise BundlerError( + f"Catalog '{source_id}' URL must use HTTPS (got {parsed.scheme}://). " + "HTTP is only allowed for localhost." + ) + if not parsed.netloc: + raise BundlerError( + f"Catalog '{source_id}' URL must be a valid URL with a host: {url}" + ) + + +def make_catalog_fetcher(*, allow_network: bool = True): + """Return a fetcher callable suitable for :class:`CatalogStack`. + + When *allow_network* is False, ``http(s)://`` sources raise instead of + touching the network (used by offline tests and ``--offline`` flows). + """ + + def fetch(source: CatalogSource) -> dict: + url = source.url + parsed = urlparse(url) + scheme = parsed.scheme.lower() + + if scheme == "builtin": + payload = _BUILTIN_CATALOGS.get(url) + if payload is None: + raise BundlerError(f"Unknown built-in catalog '{url}'.") + return payload + + if scheme == "file": + path = _file_url_to_path(parsed) + if not path.exists(): + raise BundlerError(f"Catalog file not found: {path}") + return loads_json(path.read_text(encoding="utf-8"), origin=str(path)) + + if scheme == "" or _is_windows_drive_path(url): + path = Path(url) + if not path.exists(): + raise BundlerError(f"Catalog file not found: {path}") + return loads_json(path.read_text(encoding="utf-8"), origin=str(path)) + + if scheme in ("http", "https"): + if not allow_network: + raise BundlerError( + f"Network access disabled; cannot fetch catalog '{source.id}' " + f"from {url}." + ) + _validate_remote_url(source.id, url) + return _http_get_json(source.id, url) + + raise BundlerError(f"Unsupported catalog URL scheme: {url}") + + return fetch + + +def _http_get_json(source_id: str, url: str) -> dict: + """Fetch catalog JSON over HTTP(S) via the shared authenticated client. + + Routing through :func:`specify_cli.authentication.http.open_url` gives + ``auth.json`` token support and strips the ``Authorization`` header when a + redirect leaves the entry's trusted hosts or downgrades the scheme. We also + reject any redirect that leaves HTTPS (the ``redirect_validator`` runs + *before* each hop) and re-validate the final URL after redirects, so the + HTTPS/host guarantee from ``_validate_remote_url`` is preserved end to end + rather than only on the initial URL. + """ + from ...authentication.http import open_url + + def _validate_redirect(_old_url: str, new_url: str) -> None: + _validate_remote_url(source_id, new_url) + + try: + with open_url( + url, + timeout=HTTP_TIMEOUT_SECONDS, + redirect_validator=_validate_redirect, + ) as response: + final_url = response.geturl() + _validate_remote_url(source_id, final_url) + raw = response.read().decode("utf-8") + except BundlerError: + raise + except Exception as exc: # noqa: BLE001 + raise BundlerError(f"Failed to fetch catalog from {url}: {exc}") from exc + return loads_json(raw, origin=final_url) + + +class DefaultPrimitiveInstaller: + """Dispatch component install/remove to existing primitive machinery. + + This adapter is intentionally thin: it owns no install logic of its own, + delegating entirely to the per-primitive managers so the bundler honours + Principle I (no duplicated primitive logic). + + *allow_network* mirrors the bundle command's ``--offline`` flag: when False, + component kinds that can only be sourced from a remote catalog refuse rather + than touching the network. Bundled presets/extensions still install offline. + """ + + def __init__(self, *, allow_network: bool = True) -> None: + self._allow_network = allow_network + + def is_installed(self, project_root: Path, component: ComponentRef) -> bool: + manager = self._manager_for(component, project_root) + return manager.is_installed(component) + + def install(self, project_root: Path, component: ComponentRef) -> None: + manager = self._manager_for(component, project_root) + manager.install(component) + + def remove(self, project_root: Path, component: ComponentRef) -> None: + manager = self._manager_for(component, project_root) + manager.remove(component) + + def _manager_for(self, component: ComponentRef, project_root: Path): + # Lazy import to avoid import cycles and keep startup cheap (Principle IV). + from .primitives import primitive_manager + + return primitive_manager( + component.kind, project_root, allow_network=self._allow_network + ) diff --git a/src/specify_cli/bundler/services/catalog_stack.py b/src/specify_cli/bundler/services/catalog_stack.py new file mode 100644 index 0000000000..d1e7fddde9 --- /dev/null +++ b/src/specify_cli/bundler/services/catalog_stack.py @@ -0,0 +1,114 @@ +"""Catalog stack: aggregate bundle entries across sources with precedence + policy. + +Loads each source's catalog payload (via an injectable fetcher so tests stay +offline), then resolves a bundle id to the highest-precedence entry while +recording whether installation is permitted by that source's policy. +""" +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import Callable + +from .. import BundlerError +from ..models.catalog import ( + CatalogEntry, + CatalogSource, + load_catalog_payload, + load_source_stack, +) + +# A fetcher returns the raw JSON payload (a dict) for a given source. +CatalogFetcher = Callable[[CatalogSource], dict] + + +@dataclass +class ResolvedBundle: + entry: CatalogEntry + source: CatalogSource + + @property + def install_allowed(self) -> bool: + return self.source.install_allowed + + +class CatalogStack: + def __init__( + self, + sources: list[CatalogSource], + fetcher: CatalogFetcher, + ) -> None: + # Highest precedence (lowest priority number) first. + self._sources = sorted(sources, key=lambda s: (s.priority, s.id)) + self._fetcher = fetcher + self._payloads: dict[str, dict[str, CatalogEntry]] = {} + + @classmethod + def load( + cls, + project_root: Path, + fetcher: CatalogFetcher, + user_config_dir: Path | None = None, + ) -> "CatalogStack": + sources = load_source_stack(project_root, user_config_dir) + return cls(sources, fetcher) + + @property + def sources(self) -> list[CatalogSource]: + return list(self._sources) + + def _entries_for(self, source: CatalogSource) -> dict[str, CatalogEntry]: + if source.id not in self._payloads: + try: + raw = self._fetcher(source) + except BundlerError: + raise + except Exception as exc: # noqa: BLE001 - surface as chained BundlerError + raise BundlerError( + f"Failed to load catalog '{source.id}' ({source.url}): {exc}" + ) from exc + self._payloads[source.id] = load_catalog_payload(raw) + return self._payloads[source.id] + + def resolve(self, bundle_id: str) -> ResolvedBundle: + """Return the highest-precedence entry for *bundle_id* or raise.""" + for source in self._sources: + entries = self._entries_for(source) + entry = entries.get(bundle_id) + if entry is not None: + return ResolvedBundle(entry=entry.with_provenance(source), source=source) + raise BundlerError( + f"Bundle '{bundle_id}' was not found in any configured catalog." + ) + + def search(self, query: str = "") -> list[ResolvedBundle]: + """Return entries matching *query* (substring over id/name/role/tags/description). + + Each bundle id appears once, resolved at its highest-precedence source. + Results are sorted by bundle id for deterministic output. + """ + needle = query.strip().lower() + seen: dict[str, ResolvedBundle] = {} + for source in self._sources: + for bundle_id, entry in self._entries_for(source).items(): + if bundle_id in seen: + continue + if needle and not _matches(entry, needle): + continue + seen[bundle_id] = ResolvedBundle( + entry=entry.with_provenance(source), source=source + ) + return [seen[k] for k in sorted(seen)] + + +def _matches(entry: CatalogEntry, needle: str) -> bool: + haystack = " ".join( + [ + entry.id, + entry.name, + entry.role, + entry.description, + " ".join(entry.tags), + ] + ).lower() + return needle in haystack diff --git a/src/specify_cli/bundler/services/conflict.py b/src/specify_cli/bundler/services/conflict.py new file mode 100644 index 0000000000..e7cf356283 --- /dev/null +++ b/src/specify_cli/bundler/services/conflict.py @@ -0,0 +1,54 @@ +"""Conflict detection across the installed-bundle stack. + +The single cross-bundle conflict point is the active integration (FR-019). +Component-level overlaps (same preset id at different priorities, etc.) are +resolved by the existing primitive machinery's own precedence rules, so the +bundler only needs to guard the integration invariant and surface informational +overlaps. +""" +from __future__ import annotations + +from dataclasses import dataclass, field + +from ..models.manifest import BundleManifest +from ..models.records import InstalledBundleRecord + + +@dataclass +class ConflictReport: + integration_clash: str | None = None # message when a hard clash exists + overlaps: list[str] = field(default_factory=list) # components already provided + + @property + def has_blocking_conflict(self) -> bool: + return self.integration_clash is not None + + +def detect_conflicts( + manifest: BundleManifest, + active_integration: str | None, + installed: list[InstalledBundleRecord], +) -> ConflictReport: + report = ConflictReport() + + if manifest.integration is not None and active_integration: + if manifest.integration.id != active_integration: + report.integration_clash = ( + f"Bundle targets integration '{manifest.integration.id}' but the " + f"project's active integration is '{active_integration}'." + ) + + already: dict[tuple[str, str], str] = {} + for record in installed: + for component in record.contributed_components: + already[(component.kind, component.id)] = record.bundle_id + + for component in manifest.components: + owner = already.get((component.kind, component.id)) + if owner and owner != manifest.bundle.id: + report.overlaps.append( + f"{component.kind[:-1]} '{component.id}' is already provided by " + f"bundle '{owner}'." + ) + + return report diff --git a/src/specify_cli/bundler/services/installer.py b/src/specify_cli/bundler/services/installer.py new file mode 100644 index 0000000000..10f49ef94b --- /dev/null +++ b/src/specify_cli/bundler/services/installer.py @@ -0,0 +1,210 @@ +"""Installer: apply an :class:`InstallPlan` via existing primitive machinery. + +The actual component installation (extensions, presets, steps, workflows) is +delegated to a :class:`PrimitiveInstaller` so the bundler never re-implements +primitive logic (Principle I) and integration tests can inject a deterministic, +offline fake (Principle II/IV). The real adapter dispatches in-process to the +existing extension/preset/step/workflow machinery. + +Installation is idempotent and stops on first failure with no partial record +write (FR-018, SC partial-failure-stop). +""" +from __future__ import annotations + +from dataclasses import dataclass, field +from pathlib import Path +from typing import Protocol + +from .. import BundlerError +from ..models.manifest import BundleManifest, ComponentRef +from ..models.records import ( + InstalledBundleRecord, + components_still_needed, + find_record, + load_records, + remove_record, + save_records, + upsert_record, +) +from .conflict import detect_conflicts +from .resolver import InstallPlan + + +class PrimitiveInstaller(Protocol): + """Adapter over the existing Spec Kit primitive install/remove machinery.""" + + def is_installed(self, project_root: Path, component: ComponentRef) -> bool: ... + + def install(self, project_root: Path, component: ComponentRef) -> None: ... + + def remove(self, project_root: Path, component: ComponentRef) -> None: ... + + +@dataclass +class InstallResult: + bundle_id: str + installed: list[ComponentRef] = field(default_factory=list) + skipped: list[ComponentRef] = field(default_factory=list) + refreshed: list[ComponentRef] = field(default_factory=list) + uninstalled: list[ComponentRef] = field(default_factory=list) + + @property + def changed(self) -> bool: + return bool(self.installed or self.refreshed) + + +def install_bundle( + project_root: Path, + plan: InstallPlan, + installer: PrimitiveInstaller, + manifest: BundleManifest | None = None, + refresh: bool = False, +) -> InstallResult: + """Execute *plan*, recording provenance. Idempotent, with bounded rollback. + + Atomicity is scoped, not global: on failure only the components newly + installed during *this* call are rolled back, and the provenance record is + written solely on full success (a failure records nothing). Components that + were already installed beforehand — including those re-applied when *refresh* + is True — are never rolled back. + + When *refresh* is True (used by ``specify bundle update``), components that + are already installed are re-applied through the primitive machinery so they + are brought up to the plan's pinned versions, rather than skipped. Primitive + config (e.g. preset priority overrides) is preserved by the underlying + machinery. + + Version-pin enforcement is install-time only. The primitive ``is_installed`` + checks are id-based (they do not compare versions), so when a component is + already present and *refresh* is False it is skipped without verifying that + the on-disk version matches the manifest pin. Pins are therefore only + guaranteed to be applied when the bundler actually performs an install or a + refresh; running ``specify bundle update`` re-applies every owned component + at its pinned version. + """ + records = load_records(project_root) + + if manifest is not None: + report = detect_conflicts(manifest, plan.effective_integration, records) + if report.has_blocking_conflict: + raise BundlerError(report.integration_clash) + + result = InstallResult(bundle_id=plan.bundle_id) + existing = find_record(records, plan.bundle_id) + prior_ours = { + (c.kind, c.id) for c in existing.contributed_components + } if existing is not None else set() + # Components already attributed to a *different* installed bundle: these are + # legitimately shareable (refcounted on removal), so this bundle may also + # claim them. A component that is installed on disk but tracked by no bundle + # was installed independently and must NOT be attributed here — otherwise + # removing this bundle would uninstall it (collateral removal, FR-022). + other_tracked = { + (c.kind, c.id) + for r in records + if r.bundle_id != plan.bundle_id + for c in r.contributed_components + } + + contributed: list[ComponentRef] = [] + done: list[ComponentRef] = [] + try: + for component in plan.components: + key = (component.kind, component.id) + if installer.is_installed(project_root, component): + # A component is "ours" only when this bundle (or a sibling + # bundle) already owns it. Independently-installed components + # are never attributed and — crucially — never refreshed, so + # ``bundle update`` cannot make collateral changes to things it + # does not own (FR-022). + owned = key in prior_ours or key in other_tracked + if refresh and owned: + _refresh_component(project_root, installer, component) + result.refreshed.append(component) + else: + result.skipped.append(component) + if owned: + contributed.append(component) + continue + installer.install(project_root, component) + done.append(component) + result.installed.append(component) + contributed.append(component) + except BundlerError: + _rollback(project_root, installer, done) + raise + except Exception as exc: # noqa: BLE001 + _rollback(project_root, installer, done) + raise BundlerError( + f"Failed to install bundle '{plan.bundle_id}': {exc}. " + "No changes were recorded." + ) from exc + + record = InstalledBundleRecord.create( + bundle_id=plan.bundle_id, + version=plan.version, + components=contributed, + # Preserve the original install time across refresh/update so + # ``bundle list`` keeps reporting when the bundle was first installed. + installed_at=existing.installed_at if existing is not None else None, + ) + save_records(project_root, upsert_record(records, record)) + return result + + +def remove_bundle( + project_root: Path, + bundle_id: str, + installer: PrimitiveInstaller, +) -> InstallResult: + """Remove a bundle, uninstalling only components no other bundle still needs.""" + records = load_records(project_root) + target = next((r for r in records if r.bundle_id == bundle_id), None) + if target is None: + raise BundlerError(f"Bundle '{bundle_id}' is not installed.") + + still_needed = components_still_needed(records, exclude_bundle_id=bundle_id) + result = InstallResult(bundle_id=bundle_id) + + for component in target.contributed_components: + key = (component.kind, component.id) + if key in still_needed: + result.skipped.append(component) + continue + if installer.is_installed(project_root, component): + installer.remove(project_root, component) + result.uninstalled.append(component) + else: + result.skipped.append(component) + + save_records(project_root, remove_record(records, bundle_id)) + return result + + +def _refresh_component( + project_root: Path, + installer: PrimitiveInstaller, + component: ComponentRef, +) -> None: + """Re-apply an already-installed component to bring it up to its pinned version. + + Prefers a primitive-provided ``refresh`` hook when available; otherwise falls + back to a re-install through the existing idempotent install path. + """ + op = getattr(installer, "refresh", None) + if callable(op): + op(project_root, component) + else: + installer.install(project_root, component) + + +def _rollback( + project_root: Path, + installer: PrimitiveInstaller, + done: list[ComponentRef], +) -> None: + for component in reversed(done): + try: + installer.remove(project_root, component) + except Exception: # noqa: BLE001 - best-effort rollback + continue diff --git a/src/specify_cli/bundler/services/packager.py b/src/specify_cli/bundler/services/packager.py new file mode 100644 index 0000000000..481a0a2bb8 --- /dev/null +++ b/src/specify_cli/bundler/services/packager.py @@ -0,0 +1,145 @@ +"""Packager: produce a single versioned distributable artifact from a bundle dir. + +``specify bundle build`` zips the manifest, README, and any local assets into +``-.zip``. Build refuses on an invalid manifest, pointing the +author to ``validate``. All file reads are confined within the bundle source +directory (Principle V path confinement). +""" +from __future__ import annotations + +import os +import re +import zipfile +from dataclasses import dataclass +from pathlib import Path + +from .. import BundlerError +from ..lib.yamlio import ensure_within +from ..models.manifest import BundleManifest +from .validator import validate_manifest + +# Files/dirs never included in an artifact. +EXCLUDE_NAMES = {".git", "__pycache__", ".DS_Store"} + +# Fixed member timestamp (zip epoch) for reproducible, byte-stable artifacts. +_FIXED_TIMESTAMP = (1980, 1, 1, 0, 0, 0) + + +@dataclass +class BuildResult: + artifact_path: Path + file_count: int + + +def build_bundle( + bundle_dir: Path, + output_dir: Path | None = None, +) -> BuildResult: + bundle_dir = Path(bundle_dir).resolve() + manifest_path = bundle_dir / "bundle.yml" + if not manifest_path.exists(): + raise BundlerError(f"No bundle.yml found in '{bundle_dir}'.") + + # The artifact contract requires a human-facing README.md alongside the + # manifest; refuse early rather than publish a bundle with no description. + if not (bundle_dir / "README.md").exists(): + raise BundlerError( + f"No README.md found in '{bundle_dir}'. Every bundle must ship a " + "README.md describing it." + ) + + manifest = BundleManifest.from_file(manifest_path) + report = validate_manifest(manifest) + if not report.ok: + raise BundlerError( + "Refusing to build an invalid manifest. Run 'specify bundle validate' " + "and fix:\n - " + "\n - ".join(report.errors) + ) + + out_dir = Path(output_dir).resolve() if output_dir else bundle_dir + out_dir.mkdir(parents=True, exist_ok=True) + artifact_name = f"{manifest.bundle.id}-{manifest.bundle.version}.zip" + artifact_path = out_dir / artifact_name + # Defense in depth: even though validate_manifest() rejects unsafe ids, make + # sure a crafted id cannot push the artifact outside the output directory. + ensure_within(out_dir, artifact_path) + + # If the output dir lives inside the bundle, skip its whole subtree so + # previously-built artifacts are never re-packaged (keeps builds + # reproducible and bounded). + skip_dir = out_dir if out_dir != bundle_dir and _is_within(bundle_dir, out_dir) else None + # Also skip any prior build artifact for this bundle (e.g. an older + # -.zip sitting next to bundle.yml), not just the current one. + # Match only a semver-looking version segment so legitimate assets that + # merely start with the bundle id (e.g. -assets.zip) are still packaged. + artifact_re = re.compile( + rf"^{re.escape(manifest.bundle.id)}-" + r"\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?\.zip$" + ) + files = _collect_files( + bundle_dir, skip=artifact_path, skip_dir=skip_dir, artifact_re=artifact_re + ) + with zipfile.ZipFile(artifact_path, "w", zipfile.ZIP_DEFLATED) as archive: + for file_path in files: + # Confinement: every packaged file must live under bundle_dir. + ensure_within(bundle_dir, file_path) + arcname = file_path.relative_to(bundle_dir).as_posix() + # Fixed timestamp so identical inputs yield a byte-for-byte + # identical artifact (reproducible builds). + info = zipfile.ZipInfo(filename=arcname, date_time=_FIXED_TIMESTAMP) + info.compress_type = zipfile.ZIP_DEFLATED + # Reproducible, normalized permissions: preserve executability so + # bundled scripts (e.g. extension hook scripts) stay runnable after + # extraction, but collapse to two canonical modes (0755 when any + # execute bit is set on the source, otherwise 0644) so identical + # inputs yield a byte-for-byte identical artifact. + mode = 0o755 if file_path.stat().st_mode & 0o111 else 0o644 + info.external_attr = mode << 16 + archive.writestr(info, file_path.read_bytes()) + + return BuildResult(artifact_path=artifact_path, file_count=len(files)) + + +def _is_within(parent: Path, child: Path) -> bool: + try: + child.relative_to(parent) + return True + except ValueError: + return False + + +def _collect_files( + bundle_dir: Path, + skip: Path, + skip_dir: Path | None = None, + artifact_re: re.Pattern[str] | None = None, +) -> list[Path]: + collected: list[Path] = [] + # followlinks=False so a symlinked directory is never descended into, + # which would otherwise pull in out-of-tree files and then fail at + # ensure_within(). Symlinked dirs are pruned from traversal explicitly. + for root, dirnames, filenames in os.walk(bundle_dir, followlinks=False): + root_path = Path(root) + # Prune directories we must not descend into (in-place edit of dirnames). + dirnames[:] = [ + d + for d in dirnames + if d not in EXCLUDE_NAMES and not (root_path / d).is_symlink() + ] + if skip_dir is not None and _is_within(skip_dir, root_path): + dirnames[:] = [] + continue + for name in filenames: + path = root_path / name + if path == skip: + continue + if name in EXCLUDE_NAMES: + continue + if artifact_re is not None and artifact_re.match(name): + # A prior build artifact for this bundle — never re-package it. + continue + if path.is_symlink(): + # Skip symlinked files to avoid escaping the bundle directory. + continue + collected.append(path) + return sorted(collected) diff --git a/src/specify_cli/bundler/services/primitives.py b/src/specify_cli/bundler/services/primitives.py new file mode 100644 index 0000000000..bd0d8ddee0 --- /dev/null +++ b/src/specify_cli/bundler/services/primitives.py @@ -0,0 +1,345 @@ +"""Bridge from bundler component kinds to existing primitive managers. + +The bundler does not own install logic; it routes each component to the +existing Spec Kit primitive machinery so a bundle install behaves exactly as a +sequence of ``specify add`` calls would (Principle I: never +reimplement or fake primitive behaviour). + +Routing strategy per kind: + +* **presets** / **extensions** — wired through their reusable managers + (``install_from_directory`` / ``install_from_zip``). Bundled assets shipped + with Spec Kit install fully offline; catalog assets are fetched only when + network access is permitted. +* **workflows** / **steps** — their install/remove orchestration lives in the + CLI command layer rather than a reusable service method, so the bundler + delegates to those existing command callables in-process (with the project + root as the working directory) instead of duplicating their download and + validation logic. +""" +from __future__ import annotations + +import contextlib +import os +from pathlib import Path +from typing import Protocol + +from .. import BundlerError +from ..models.manifest import ComponentRef + +DEFAULT_PRIORITY = 10 + + +def _assert_pinned_version( + kind: str, component_id: str, pinned: str | None, advertised: object +) -> None: + """Refuse to install when the catalog version differs from the manifest pin. + + Bundle manifests pin component versions for reproducibility; installing + whatever the active catalog currently serves would silently violate the + pin. When the catalog advertises no version we cannot enforce the pin, so + installation proceeds (the catalog, not the bundler, owns that gap). + """ + if not pinned or advertised is None: + return + actual = str(advertised).strip() + if not actual: + return + from ..lib.versioning import parse_version + + try: + matches = parse_version(actual) == parse_version(pinned) + except BundlerError: + matches = actual == str(pinned).strip() + if not matches: + raise BundlerError( + f"{kind} '{component_id}' is pinned to version {pinned} in the bundle " + f"manifest, but the active catalog serves {actual}. Update the bundle's " + "pinned version or the catalog before installing." + ) + + +class _KindManager(Protocol): + def is_installed(self, component: ComponentRef) -> bool: ... + + def install(self, component: ComponentRef) -> None: ... + + def remove(self, component: ComponentRef) -> None: ... + + +def primitive_manager( + kind: str, project_root: Path, *, allow_network: bool = True +) -> _KindManager: + if kind == "presets": + return _PresetKindManager(project_root, allow_network) + if kind == "extensions": + return _ExtensionKindManager(project_root, allow_network) + if kind == "workflows": + return _WorkflowKindManager(project_root, allow_network) + if kind == "steps": + return _StepKindManager(project_root, allow_network) + raise BundlerError(f"Unknown component kind '{kind}'.") + + +@contextlib.contextmanager +def _chdir(path: Path): + """Temporarily switch the working directory. + + The delegated workflow/step command callables resolve the project via + ``Path.cwd()``; this makes that resolution land on *path*. + """ + previous = Path.cwd() + os.chdir(path) + try: + yield + finally: + os.chdir(previous) + + +def _delegate_command(action: str, label: str, call) -> None: + """Run a delegated CLI command callable, translating its exit into errors.""" + import typer + + try: + call() + except typer.Exit as exc: # raised by the delegated command on failure + code = getattr(exc, "exit_code", 0) or 0 + if code != 0: + raise BundlerError(f"Failed to {action} {label}.") from exc + except SystemExit as exc: # pragma: no cover - defensive + if exc.code not in (0, None): + raise BundlerError(f"Failed to {action} {label}.") from exc + + +class _PresetKindManager: + def __init__(self, project_root: Path, allow_network: bool) -> None: + from ...presets import PresetManager + + self._root = project_root + self._allow_network = allow_network + self._manager = PresetManager(project_root) + + def is_installed(self, component: ComponentRef) -> bool: + try: + return self._manager.get_pack(component.id) is not None + except Exception: # noqa: BLE001 + return False + + def install(self, component: ComponentRef) -> None: + from ... import get_speckit_version + from ..._assets import _locate_bundled_preset + + speckit_version = get_speckit_version() + priority = DEFAULT_PRIORITY if component.priority is None else component.priority + + bundled = _locate_bundled_preset(component.id) + if bundled is not None: + self._manager.install_from_directory(bundled, speckit_version, priority) + return + + if not self._allow_network: + raise BundlerError( + f"Preset '{component.id}' is not bundled and network access is " + f"disabled; re-run without --offline or install it first with " + f"'specify preset add {component.id}'." + ) + + from ...presets import PresetCatalog + + catalog = PresetCatalog(self._root) + info = catalog.get_pack_info(component.id) + if not info: + raise BundlerError(f"Preset '{component.id}' not found in any catalog.") + if not info.get("_install_allowed", True): + raise BundlerError( + f"Preset '{component.id}' is from a discovery-only catalog; " + "installation is not allowed." + ) + _assert_pinned_version( + "Preset", component.id, component.version, info.get("version") + ) + zip_path = catalog.download_pack(component.id) + try: + self._manager.install_from_zip(zip_path, speckit_version, priority) + finally: + with contextlib.suppress(Exception): + if zip_path.exists(): + zip_path.unlink() + + def remove(self, component: ComponentRef) -> None: + try: + self._manager.remove(component.id) + except Exception as exc: # noqa: BLE001 + raise BundlerError( + f"Failed to remove preset '{component.id}': {exc}" + ) from exc + + +class _ExtensionKindManager: + def __init__(self, project_root: Path, allow_network: bool) -> None: + from ...extensions import ExtensionManager + + self._root = project_root + self._allow_network = allow_network + self._manager = ExtensionManager(project_root) + + def is_installed(self, component: ComponentRef) -> bool: + try: + return self._manager.registry.is_installed(component.id) + except Exception: # noqa: BLE001 + return False + + def install(self, component: ComponentRef) -> None: + from ... import get_speckit_version + from ..._assets import _locate_bundled_extension + + speckit_version = get_speckit_version() + priority = DEFAULT_PRIORITY if component.priority is None else component.priority + + bundled = _locate_bundled_extension(component.id) + if bundled is not None: + self._manager.install_from_directory( + bundled, speckit_version, priority=priority + ) + return + + if not self._allow_network: + raise BundlerError( + f"Extension '{component.id}' is not bundled and network access is " + f"disabled; re-run without --offline or install it first with " + f"'specify extension add {component.id}'." + ) + + from ...extensions import ExtensionCatalog + + catalog = ExtensionCatalog(self._root) + info = catalog.get_extension_info(component.id) + if not info: + raise BundlerError( + f"Extension '{component.id}' not found in any catalog." + ) + if not info.get("_install_allowed", True): + raise BundlerError( + f"Extension '{component.id}' is from a discovery-only catalog; " + "installation is not allowed." + ) + _assert_pinned_version( + "Extension", component.id, component.version, info.get("version") + ) + zip_path = catalog.download_extension(component.id) + try: + self._manager.install_from_zip( + zip_path, speckit_version, priority=priority + ) + finally: + with contextlib.suppress(Exception): + if zip_path.exists(): + zip_path.unlink() + + def remove(self, component: ComponentRef) -> None: + try: + self._manager.remove(component.id) + except Exception as exc: # noqa: BLE001 + raise BundlerError( + f"Failed to remove extension '{component.id}': {exc}" + ) from exc + + +class _WorkflowKindManager: + def __init__(self, project_root: Path, allow_network: bool) -> None: + from ...workflows.catalog import WorkflowRegistry + + self._root = project_root + self._allow_network = allow_network + self._registry = WorkflowRegistry(project_root) + + def is_installed(self, component: ComponentRef) -> bool: + try: + return self._registry.is_installed(component.id) + except Exception: # noqa: BLE001 + return False + + def install(self, component: ComponentRef) -> None: + if not self._allow_network and not self._is_bundled(component.id): + raise BundlerError( + f"Workflow '{component.id}' installs from a catalog and network " + f"access is disabled; re-run without --offline or install it first " + f"with 'specify workflow add {component.id}'." + ) + self._assert_pinned_version(component) + from ... import workflow_add + + with _chdir(self._root): + _delegate_command( + "install", f"workflow '{component.id}'", + lambda: workflow_add(component.id), + ) + + def _assert_pinned_version(self, component: ComponentRef) -> None: + if not component.version: + return + try: + from ...workflows.catalog import WorkflowCatalog + + info = WorkflowCatalog(self._root).get_workflow_info(component.id) + except Exception: # noqa: BLE001 - catalog unreachable: cannot enforce + return + if info: + _assert_pinned_version( + "Workflow", component.id, component.version, info.get("version") + ) + + @staticmethod + def _is_bundled(workflow_id: str) -> bool: + # A workflow that ships with Spec Kit installs fully offline. + from ..._assets import _locate_bundled_workflow + + return _locate_bundled_workflow(workflow_id) is not None + + def remove(self, component: ComponentRef) -> None: + from ... import workflow_remove + + with _chdir(self._root): + _delegate_command( + "remove", f"workflow '{component.id}'", + lambda: workflow_remove(component.id), + ) + + +class _StepKindManager: + def __init__(self, project_root: Path, allow_network: bool) -> None: + from ...workflows.catalog import StepRegistry + + self._root = project_root + self._allow_network = allow_network + self._registry = StepRegistry(project_root) + + def is_installed(self, component: ComponentRef) -> bool: + try: + return self._registry.is_installed(component.id) + except Exception: # noqa: BLE001 + return False + + def install(self, component: ComponentRef) -> None: + if not self._allow_network: + raise BundlerError( + f"Step '{component.id}' installs from a catalog and network access " + f"is disabled; re-run without --offline or install it first with " + f"'specify workflow step add {component.id}'." + ) + from ... import workflow_step_add + + with _chdir(self._root): + _delegate_command( + "install", f"step '{component.id}'", + lambda: workflow_step_add(component.id), + ) + + def remove(self, component: ComponentRef) -> None: + from ... import workflow_step_remove + + with _chdir(self._root): + _delegate_command( + "remove", f"step '{component.id}'", + lambda: workflow_step_remove(component.id), + ) diff --git a/src/specify_cli/bundler/services/references.py b/src/specify_cli/bundler/services/references.py new file mode 100644 index 0000000000..3dd0f3d010 --- /dev/null +++ b/src/specify_cli/bundler/services/references.py @@ -0,0 +1,114 @@ +"""Resolve bundle component references against real, available components. + +Used by ``specify bundle validate`` (FR-005 / SC-007) to confirm that every +declared component points at something installable. Resolution is offline-first: +a reference resolves when the component is bundled with Spec Kit or already +installed in the project; catalog sources are consulted only when network access +is permitted. Offline runs that cannot confirm a reference downgrade to a +warning rather than a false failure, while definitively-unknown references +always error. +""" +from __future__ import annotations + +from pathlib import Path + +from ..models.manifest import ComponentRef + + +def _resolved_locally(root: Path, component: ComponentRef) -> bool: + kind = component.kind + try: + if kind == "presets": + from ..._assets import _locate_bundled_preset + from ...presets import PresetManager + + if _locate_bundled_preset(component.id) is not None: + return True + return PresetManager(root).get_pack(component.id) is not None + if kind == "extensions": + from ..._assets import _locate_bundled_extension + from ...extensions import ExtensionManager + + if _locate_bundled_extension(component.id) is not None: + return True + return ExtensionManager(root).registry.is_installed(component.id) + if kind == "workflows": + from ..._assets import _locate_bundled_workflow + from ...workflows.catalog import WorkflowRegistry + + if _locate_bundled_workflow(component.id) is not None: + return True + return WorkflowRegistry(root).is_installed(component.id) + if kind == "steps": + from ...workflows.catalog import StepRegistry + + return StepRegistry(root).is_installed(component.id) + except Exception: # noqa: BLE001 - resolution is best-effort + return False + return False + + +def _resolved_in_catalog(root: Path, component: ComponentRef) -> bool | None: + """Return True/False if a catalog could be consulted, or None on failure.""" + kind = component.kind + try: + if kind == "presets": + from ...presets import PresetCatalog + + return PresetCatalog(root).get_pack_info(component.id) is not None + if kind == "extensions": + from ...extensions import ExtensionCatalog + + return ExtensionCatalog(root).get_extension_info(component.id) is not None + if kind == "workflows": + from ...workflows.catalog import WorkflowCatalog + + return WorkflowCatalog(root).get_workflow_info(component.id) is not None + if kind == "steps": + from ...workflows.catalog import StepCatalog + + return StepCatalog(root).get_step_info(component.id) is not None + except Exception: # noqa: BLE001 - catalog may be unreachable/misconfigured + return None + return None + + +def make_reference_checker( + project_root: Path, + *, + allow_network: bool, + warnings: list[str], +): + """Build a ``ReferenceChecker`` for :func:`validate_manifest`. + + Returns an error string for a reference that is definitively unresolvable, + ``None`` otherwise. Unverifiable references (offline, or an unreachable + catalog) append a note to *warnings* and pass. + """ + + def check(component: ComponentRef) -> str | None: + if _resolved_locally(project_root, component): + return None + + if allow_network: + in_catalog = _resolved_in_catalog(project_root, component) + if in_catalog is True: + return None + if in_catalog is False: + return ( + f"{component.kind[:-1]} '{component.id}' is not bundled, " + "installed, or present in any active catalog." + ) + warnings.append( + f"Could not verify {component.kind[:-1]} '{component.id}' " + "(catalog unreachable); reference left unchecked." + ) + return None + + warnings.append( + f"Could not verify {component.kind[:-1]} '{component.id}' offline " + "(not bundled or installed); re-run validate online to check catalogs." + ) + return None + + return check diff --git a/src/specify_cli/bundler/services/resolver.py b/src/specify_cli/bundler/services/resolver.py new file mode 100644 index 0000000000..127fa683fd --- /dev/null +++ b/src/specify_cli/bundler/services/resolver.py @@ -0,0 +1,122 @@ +"""Resolver: expand a bundle manifest into a concrete, ordered install plan. + +The plan the resolver produces is the single source of truth shared by +``info`` (preview) and ``install`` (execution) so the two never diverge +(SC-002 transparency). Resolution also enforces the SpecKit version gate +(FR-016) and the integration-compatibility check (FR-019). +""" +from __future__ import annotations + +from dataclasses import dataclass, field +from pathlib import Path + +from .. import BundlerError +from ..lib.versioning import satisfies +from ..models.manifest import BundleManifest, ComponentRef + + +@dataclass +class InstallPlan: + bundle_id: str + version: str + role: str + effective_integration: str | None + components: list[ComponentRef] = field(default_factory=list) + warnings: list[str] = field(default_factory=list) + + @property + def component_count(self) -> int: + return len(self.components) + + def grouped(self) -> dict[str, list[ComponentRef]]: + groups: dict[str, list[ComponentRef]] = { + "extensions": [], + "presets": [], + "steps": [], + "workflows": [], + } + for component in self.components: + groups.setdefault(component.kind, []).append(component) + return groups + + +def resolve_install_plan( + manifest: BundleManifest, + *, + speckit_version: str, + active_integration: str | None, + integration_explicit: bool = False, + enforce_version: bool = True, +) -> InstallPlan: + """Expand *manifest* into an :class:`InstallPlan`, enforcing gates. + + Raises :class:`BundlerError` when a hard gate fails (version gate, + integration clash). Soft issues are collected in ``plan.warnings``. + + *integration_explicit* signals that ``active_integration`` came from an + explicit ``--integration`` override rather than project auto-detection. When + a bundle pins an integration but the project's active integration cannot be + determined (``active_integration is None``) and the caller did not supply an + explicit override, resolution fails instead of silently adopting the + bundle's required integration (FR-019 guard). + """ + structural = manifest.structural_errors() + if structural: + raise BundlerError( + "Cannot resolve an invalid manifest:\n - " + "\n - ".join(structural) + ) + + # FR-016: SpecKit version gate — refuse incompatible installs. + if enforce_version and manifest.requires.speckit_version: + if not satisfies(speckit_version, manifest.requires.speckit_version): + raise BundlerError( + f"Bundle '{manifest.bundle.id}' requires Spec Kit " + f"{manifest.requires.speckit_version}, but this project uses " + f"{speckit_version}. Update Spec Kit or choose a compatible bundle." + ) + + # FR-019: integration-compatibility — a bundle that pins a different + # integration than the project's active one halts (no silent change). + effective_integration = active_integration + if manifest.integration is not None: + required = manifest.integration.id + if active_integration and required != active_integration: + raise BundlerError( + f"Bundle '{manifest.bundle.id}' targets integration '{required}', " + f"but this project's active integration is '{active_integration}'. " + "Installing it would conflict; aborting with no changes." + ) + if active_integration is None and not integration_explicit: + raise BundlerError( + f"Bundle '{manifest.bundle.id}' targets integration '{required}', " + "but this project's active integration could not be determined " + "(missing or unreadable .specify/integration.json). Re-run with " + "'--integration' to confirm the target, or repair the project " + "before installing." + ) + effective_integration = required + + warnings: list[str] = [] + if manifest.requires.tools: + warnings.append( + "Requires external tools: " + ", ".join(manifest.requires.tools) + ) + if manifest.requires.mcp: + warnings.append("Requires MCP servers: " + ", ".join(manifest.requires.mcp)) + + return InstallPlan( + bundle_id=manifest.bundle.id, + version=manifest.bundle.version, + role=manifest.bundle.role, + effective_integration=effective_integration, + components=list(manifest.components), + warnings=warnings, + ) + + +def load_manifest_from_dir(bundle_dir: Path) -> BundleManifest: + """Load ``bundle.yml`` from a bundle directory.""" + manifest_path = Path(bundle_dir) / "bundle.yml" + if not manifest_path.exists(): + raise BundlerError(f"No bundle.yml found in '{bundle_dir}'.") + return BundleManifest.from_file(manifest_path) diff --git a/src/specify_cli/bundler/services/validator.py b/src/specify_cli/bundler/services/validator.py new file mode 100644 index 0000000000..a1b3ae6c93 --- /dev/null +++ b/src/specify_cli/bundler/services/validator.py @@ -0,0 +1,60 @@ +"""Validator: structural + reference validation for a bundle manifest. + +``specify bundle validate`` reports whether a manifest is well-formed and all +component references are resolvable. Structural checks come from the manifest +model; reference resolution is optional (requires a resolver callback) so the +command can run fully offline against pinned/local references. +""" +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Callable + +from .. import BundlerError +from ..lib.versioning import parse_constraint +from ..models.manifest import BundleManifest, ComponentRef + +# A reference checker returns None when resolvable, or an error string. +ReferenceChecker = Callable[[ComponentRef], str | None] + + +@dataclass +class ValidationReport: + errors: list[str] = field(default_factory=list) + warnings: list[str] = field(default_factory=list) + + @property + def ok(self) -> bool: + return not self.errors + + def merge(self, other: "ValidationReport") -> None: + self.errors.extend(other.errors) + self.warnings.extend(other.warnings) + + +def validate_manifest( + manifest: BundleManifest, + reference_checker: ReferenceChecker | None = None, +) -> ValidationReport: + report = ValidationReport() + + report.errors.extend(manifest.structural_errors()) + + if manifest.requires.speckit_version: + try: + parse_constraint(manifest.requires.speckit_version) + except BundlerError as exc: + report.errors.append( + f"requires.speckit_version '{manifest.requires.speckit_version}' " + f"is not a valid constraint: {exc}" + ) + + if reference_checker is not None: + for component in manifest.components: + problem = reference_checker(component) + if problem: + report.errors.append( + f"Unresolved reference {component.label()}: {problem}" + ) + + return report diff --git a/src/specify_cli/commands/bundle/__init__.py b/src/specify_cli/commands/bundle/__init__.py new file mode 100644 index 0000000000..185e00acf6 --- /dev/null +++ b/src/specify_cli/commands/bundle/__init__.py @@ -0,0 +1,834 @@ +"""``specify bundle`` command group — discover, install, author Spec Kit bundles. + +This module is the CLI/UX layer only (Principle I: thin commands over services). +Each command resolves a project, builds a catalog stack, delegates to a bundler +service, and renders Rich output. ``--json`` emits machine-readable data on +stdout; human logs go to stderr/console. +""" +from __future__ import annotations + +import json as _json +import re +from pathlib import Path + +import typer + +from ..._console import console +from ...bundler import BundlerError +from ...bundler.lib.project import ( + active_integration, + find_project_root, + require_project_root, +) +from ...bundler.models.records import load_records + +bundle_app = typer.Typer( + name="bundle", + help="Discover, install, and author Spec Kit bundles", + add_completion=False, +) + +bundle_catalog_app = typer.Typer( + name="catalog", + help="Manage bundle catalog sources", + add_completion=False, +) +bundle_app.add_typer(bundle_catalog_app, name="catalog") + + +# ===== helpers ===== + + +def _fail(message: str) -> None: + """Print an actionable error to stderr and exit non-zero.""" + console.print(f"[red]Error:[/red] {message}", style=None) + raise typer.Exit(code=1) + + +def _user_config_dir() -> Path: + # User-scope Spec Kit config lives under ~/.specify (same convention as + # auth.json, extension/preset catalogs). Passing this through to the source + # stack is what makes the documented project > user > built-in precedence + # reachable from the CLI. + return Path.home() / ".specify" + + +def _build_stack(project_root: Path, *, offline: bool): + from ...bundler.services.adapters import make_catalog_fetcher + from ...bundler.services.catalog_stack import CatalogStack + + fetcher = make_catalog_fetcher(allow_network=not offline) + return CatalogStack.load(project_root, fetcher, user_config_dir=_user_config_dir()) + + +def _speckit_version() -> str: + from ..._assets import get_speckit_version + + return get_speckit_version() + + +def _trust_level(verified: bool) -> str: + """Trust framing for a catalog entry (FR-010): org-curated vs community.""" + return "verified" if verified else "community" + + +def _trust_badge(verified: bool) -> str: + return ( + "[green]✔ verified[/green]" + if verified + else "[yellow]community[/yellow]" + ) + + +def _default_script_type() -> str: + """OS-appropriate default script flavor (FR-013).""" + import os + + return "ps" if os.name == "nt" else "sh" + + +def _run_init(integration: str, *, script_type: str, offline: bool = False) -> None: + """Idempotently scaffold a Spec Kit project here via the existing ``init`` machinery. + + Reuses the real ``specify init`` command callback in-process (Principle I) + with ``--here --force`` so it is non-interactive and merges into the current + directory. + """ + from ... import app + + init_cb = next( + c.callback + for c in app.registered_commands + if c.callback and c.callback.__name__ == "init" + ) + try: + init_cb( + project_name=None, + script_type=script_type, + ignore_agent_tools=True, + here=True, + force=True, + skip_tls=False, + debug=False, + github_token=None, + offline=offline, + preset=None, + integration=integration, + integration_options=None, + ) + except typer.Exit as exc: + if exc.exit_code: + raise BundlerError( + f"Failed to initialize a Spec Kit project (integration '{integration}')." + ) from exc + + +def _resolve_init_integration(override: str | None, manifest) -> str: + """Precedence (FR-013): explicit override → bundle-declared → default.""" + from ..._agent_config import DEFAULT_INIT_INTEGRATION + + if override: + return override + if manifest is not None and manifest.integration is not None: + return manifest.integration.id + return DEFAULT_INIT_INTEGRATION + + +# ===== Consume ===== + + +@bundle_app.command("search") +def bundle_search( + query: str = typer.Argument("", help="Optional text query"), + offline: bool = typer.Option(False, "--offline", help="Do not access the network"), + as_json: bool = typer.Option(False, "--json", help="Emit JSON to stdout"), +) -> None: + """List matching bundles across the active catalog stack.""" + try: + project_root = find_project_root() or Path.cwd() + stack = _build_stack(project_root, offline=offline) + results = stack.search(query) + except BundlerError as exc: + _fail(str(exc)) + return + + if as_json: + payload = [ + { + "id": r.entry.id, + "name": r.entry.name, + "role": r.entry.role, + "version": r.entry.version, + "description": r.entry.description, + "source": r.source.id, + "install_policy": r.source.install_policy.value, + "verified": r.entry.verified, + "trust": _trust_level(r.entry.verified), + } + for r in results + ] + print(_json.dumps(payload, indent=2)) + return + + if not results: + console.print("[yellow]No matching bundles found.[/yellow]") + return + + console.print("\n[bold cyan]Bundles:[/bold cyan]\n") + for r in results: + policy = ( + "[dim](discovery-only)[/dim]" + if not r.source.install_allowed + else "" + ) + console.print( + f" [bold]{r.entry.id}[/bold] v{r.entry.version} — {r.entry.name} " + f"[dim]({r.entry.role})[/dim] {_trust_badge(r.entry.verified)} {policy}" + ) + console.print(f" {r.entry.description}") + console.print(f" [dim]source: {r.source.id}[/dim]") + + +@bundle_app.command("info") +def bundle_info( + bundle_id: str = typer.Argument(..., help="Bundle id to inspect"), + offline: bool = typer.Option(False, "--offline", help="Do not access the network"), + as_json: bool = typer.Option(False, "--json", help="Emit JSON to stdout"), +) -> None: + """Show full metadata and the fully expanded component set (== what install adds).""" + try: + project_root = find_project_root() or Path.cwd() + stack = _build_stack(project_root, offline=offline) + resolved = stack.resolve(bundle_id) + # `info` must show the fully expanded component set that `install` would + # apply (contracts/cli-commands.md). Expansion happens regardless of + # install policy — discovery-only bundles stay inspectable; only + # `install` is refused. But if the manifest itself can't be resolved + # (e.g. --offline against an https:// download_url, or a download + # failure), fail loudly and exit non-zero rather than silently + # degrading to catalog `provides` counts, so users never mistake an + # unverifiable bundle for a known/installable one. + manifest = _download_manifest(resolved, offline=offline) + except BundlerError as exc: + _fail(str(exc)) + return + + overlaps = _bundle_overlaps(project_root, manifest, offline=offline) + components = _manifest_component_view(manifest) + + entry = resolved.entry + if as_json: + payload = { + "id": entry.id, + "name": entry.name, + "version": entry.version, + "role": entry.role, + "description": entry.description, + "author": entry.author, + "license": entry.license, + "source": resolved.source.id, + "install_policy": resolved.source.install_policy.value, + "provides": entry.provides, + "requires": {"speckit_version": entry.requires_speckit_version}, + "verified": entry.verified, + "trust": _trust_level(entry.verified), + "integration": (manifest.integration.id if manifest and manifest.integration else None), + "components": components, + "overlaps": overlaps, + } + print(_json.dumps(payload, indent=2)) + return + + console.print(f"\n[bold cyan]{entry.id}[/bold cyan] v{entry.version} — {entry.name}") + console.print(f" Role: {entry.role}") + console.print(f" {entry.description}") + console.print(f" Author: {entry.author} License: {entry.license}") + console.print(f" Source: {resolved.source.id} ({resolved.source.install_policy.value})") + console.print(f" Trust: {_trust_badge(entry.verified)}") + if entry.requires_speckit_version: + console.print(f" Requires Spec Kit: {entry.requires_speckit_version}") + if manifest and manifest.integration: + console.print(f" Integration: {manifest.integration.id}") + + if components: + console.print("\n [bold]Components[/bold] (added on install):") + for kind in ("extensions", "presets", "steps", "workflows"): + items = [c for c in components if c["kind"] == kind] + if not items: + continue + console.print(f" [bold]{kind}:[/bold]") + for item in items: + console.print(f" - {_format_component(item)}") + else: + console.print("\n [bold]Provides:[/bold]") + for kind in ("extensions", "presets", "steps", "workflows"): + count = entry.provides.get(kind, 0) + if count: + console.print(f" {kind}: {count}") + + if overlaps: + console.print("\n [yellow]Overlaps with already-installed bundles:[/yellow]") + for overlap in overlaps: + console.print(f" [yellow]-[/yellow] {overlap}") + + if not resolved.install_allowed: + console.print( + "\n [yellow]This source is discovery-only; the bundle cannot be " + "installed from here.[/yellow]" + ) + + +@bundle_app.command("list") +def bundle_list( + as_json: bool = typer.Option(False, "--json", help="Emit JSON to stdout"), +) -> None: + """List bundles currently installed in the project with versions.""" + try: + project_root = require_project_root() + records = load_records(project_root) + except BundlerError as exc: + _fail(str(exc)) + return + + if as_json: + print(_json.dumps([r.to_dict() for r in records], indent=2)) + return + + if not records: + console.print("[yellow]No bundles installed.[/yellow]") + console.print("\nInstall one with: [cyan]specify bundle install [/cyan]") + return + + console.print("\n[bold cyan]Installed bundles:[/bold cyan]\n") + for record in records: + console.print( + f" [bold]{record.bundle_id}[/bold] v{record.version} " + f"[dim]({len(record.contributed_components)} components, " + f"installed {record.installed_at})[/dim]" + ) + + +@bundle_app.command("install") +def bundle_install( + bundle_id: str = typer.Argument( + ..., + help="Bundle id (from the catalog stack) or a local path to a .zip " + "artifact, bundle directory, or bundle.yml", + ), + integration: str = typer.Option(None, "--integration", help="Override integration"), + offline: bool = typer.Option(False, "--offline", help="Do not access the network"), +) -> None: + """Install a bundle's full component set through each primitive's machinery. + + ``bundle_id`` may be a catalog bundle id, or a local path to a built + artifact (``.zip``), a bundle directory, or a ``bundle.yml`` file. Local + sources install directly without consulting the catalog stack. + """ + try: + from ...bundler.lib.project import find_project_root + from ...bundler.services.adapters import DefaultPrimitiveInstaller + from ...bundler.services.installer import install_bundle + from ...bundler.services.resolver import resolve_install_plan + + project_root = find_project_root() + + local_manifest = _local_manifest_source(bundle_id) + if local_manifest is not None: + manifest = local_manifest + else: + stack = _build_stack(project_root or Path.cwd(), offline=offline) + resolved = stack.resolve(bundle_id) + + if not resolved.install_allowed: + raise BundlerError( + f"Bundle '{bundle_id}' resolves only from a discovery-only source " + f"('{resolved.source.id}'); it cannot be installed from there." + ) + manifest = _download_manifest(resolved, offline=offline) + + if project_root is None: + init_integration = _resolve_init_integration(integration, manifest) + console.print( + f"[cyan]No Spec Kit project here; initializing with integration " + f"'{init_integration}'…[/cyan]" + ) + _run_init(init_integration, script_type=_default_script_type(), offline=offline) + project_root = require_project_root() + + for overlap in _bundle_overlaps(project_root, manifest, offline=offline): + console.print(f"[yellow]![/yellow] {overlap}") + + # For an already-initialized project, the project's recorded active + # integration is authoritative — an explicit --integration must not be + # able to bypass the FR-019 integration-clash guard. The override only + # selects the integration at init time (handled above) or confirms the + # target when the active integration cannot be determined. + detected = active_integration(project_root) + plan = resolve_install_plan( + manifest, + speckit_version=_speckit_version(), + active_integration=detected if detected is not None else integration, + integration_explicit=bool(integration) and detected is None, + ) + for warning in plan.warnings: + console.print(f"[yellow]![/yellow] {warning}") + + result = install_bundle( + project_root, + plan, + DefaultPrimitiveInstaller(allow_network=not offline), + manifest=manifest, + ) + except BundlerError as exc: + _fail(str(exc)) + return + + console.print( + f"[green]✓[/green] Installed '{result.bundle_id}' " + f"({len(result.installed)} added, {len(result.skipped)} already present)." + ) + + +@bundle_app.command("update") +def bundle_update( + bundle_id: str = typer.Argument(None, help="Bundle id, or omit with --all"), + all_bundles: bool = typer.Option(False, "--all", help="Update every installed bundle"), + integration: str = typer.Option(None, "--integration", help="Override integration"), + offline: bool = typer.Option(False, "--offline", help="Do not access the network"), +) -> None: + """Re-resolve and refresh a bundle's components via each primitive's update path.""" + try: + project_root = require_project_root() + records = load_records(project_root) + if not all_bundles and not bundle_id: + raise BundlerError("Specify a bundle id or use --all.") + targets = ( + [r.bundle_id for r in records] + if all_bundles + else [bundle_id] + ) + if not targets: + console.print("[yellow]No installed bundles to update.[/yellow]") + return + + stack = _build_stack(project_root, offline=offline) + from ...bundler.services.adapters import DefaultPrimitiveInstaller + from ...bundler.services.installer import install_bundle + from ...bundler.services.resolver import resolve_install_plan + + installer = DefaultPrimitiveInstaller(allow_network=not offline) + for target in targets: + if not any(r.bundle_id == target for r in records): + raise BundlerError(f"Bundle '{target}' is not installed.") + resolved = stack.resolve(target) + if not resolved.install_allowed: + raise BundlerError( + f"Bundle '{target}' resolves only from a discovery-only source " + f"('{resolved.source.id}'); it cannot be updated from there. " + "Update requires an install-allowed source (FR-025)." + ) + manifest = _download_manifest(resolved, offline=offline) + detected = active_integration(project_root) + plan = resolve_install_plan( + manifest, + speckit_version=_speckit_version(), + active_integration=detected if detected is not None else integration, + integration_explicit=bool(integration) and detected is None, + ) + install_bundle(project_root, plan, installer, manifest=manifest, refresh=True) + console.print(f"[green]✓[/green] Updated '{target}' to v{plan.version}.") + except BundlerError as exc: + _fail(str(exc)) + return + + +@bundle_app.command("remove") +def bundle_remove( + bundle_id: str = typer.Argument(..., help="Installed bundle id to remove"), +) -> None: + """Uninstall only the components this bundle contributed (no collateral removals).""" + try: + project_root = require_project_root() + from ...bundler.services.adapters import DefaultPrimitiveInstaller + from ...bundler.services.installer import remove_bundle + + result = remove_bundle(project_root, bundle_id, DefaultPrimitiveInstaller()) + except BundlerError as exc: + _fail(str(exc)) + return + + console.print( + f"[green]✓[/green] Removed '{result.bundle_id}' " + f"({len(result.uninstalled)} uninstalled, {len(result.skipped)} kept for other bundles)." + ) + + +# ===== Author ===== + + +@bundle_app.command("validate") +def bundle_validate( + path: Path = typer.Option( + None, "--path", help="Bundle directory or bundle.yml (default: cwd)" + ), + offline: bool = typer.Option( + False, + "--offline", + help="Do not access catalogs; verify references against bundled/installed only", + ), +) -> None: + """Report whether the manifest is well-formed and references resolve.""" + try: + manifest_path = _resolve_manifest_path(path) + from ...bundler.lib.project import find_project_root + from ...bundler.models.manifest import BundleManifest + from ...bundler.services.references import make_reference_checker + from ...bundler.services.validator import validate_manifest + + manifest = BundleManifest.from_file(manifest_path) + ref_root = find_project_root(manifest_path.parent) or Path.cwd() + ref_warnings: list[str] = [] + checker = make_reference_checker( + ref_root, allow_network=not offline, warnings=ref_warnings + ) + report = validate_manifest(manifest, reference_checker=checker) + report.warnings.extend(ref_warnings) + except BundlerError as exc: + _fail(str(exc)) + return + + for warning in report.warnings: + console.print(f"[yellow]![/yellow] {warning}") + if not report.ok: + console.print("[red]Manifest is invalid:[/red]") + for error in report.errors: + console.print(f" [red]-[/red] {error}") + raise typer.Exit(code=1) + console.print(f"[green]✓[/green] {manifest.bundle.id} is well-formed and valid.") + + +@bundle_app.command("build") +def bundle_build( + path: Path = typer.Option( + None, "--path", help="Bundle directory (default: cwd)" + ), + output: Path = typer.Option(None, "--output", help="Output directory for the artifact"), +) -> None: + """Produce a single versioned distributable artifact (.zip).""" + try: + bundle_dir = (path or Path.cwd()).resolve() + if bundle_dir.is_file(): + bundle_dir = bundle_dir.parent + from ...bundler.services.packager import build_bundle + + result = build_bundle(bundle_dir, output_dir=output) + except BundlerError as exc: + _fail(str(exc)) + return + + console.print( + f"[green]✓[/green] Built {result.artifact_path.name} " + f"({result.file_count} files) → {result.artifact_path}" + ) + + +@bundle_app.command("init") +def bundle_init( + bundle: str = typer.Argument(None, help="Optional bundle to install after init"), + integration: str = typer.Option(None, "--integration", help="Integration override"), + offline: bool = typer.Option(False, "--offline", help="Do not access the network"), +) -> None: + """Ensure the project is initialized (idempotent), then optionally install a bundle.""" + from ...bundler.lib.project import find_project_root + + try: + project_root = find_project_root() + if project_root is None: + init_integration = _resolve_init_integration(integration, None) + console.print( + f"[cyan]Initializing a Spec Kit project with integration " + f"'{init_integration}'…[/cyan]" + ) + _run_init(init_integration, script_type=_default_script_type(), offline=offline) + project_root = require_project_root() + except BundlerError as exc: + _fail(str(exc)) + return + + console.print(f"[green]✓[/green] Spec Kit project ready at {project_root}.") + if bundle: + bundle_install(bundle, integration=integration, offline=offline) + + +# ===== Catalog management ===== + + +@bundle_catalog_app.command("list") +def catalog_list() -> None: + """Print the active, priority-ordered catalog stack with scope and policy.""" + try: + project_root = require_project_root() + from ...bundler.models.catalog import Scope, load_source_stack + + sources = load_source_stack(project_root, user_config_dir=_user_config_dir()) + except BundlerError as exc: + _fail(str(exc)) + return + + console.print("\n[bold cyan]Catalog stack[/bold cyan] (highest precedence first):\n") + only_builtin = all(s.scope == Scope.BUILTIN for s in sources) + for source in sources: + console.print( + f" [bold]{source.id}[/bold] priority={source.priority} " + f"policy={source.install_policy.value} scope={source.scope.value}" + ) + console.print(f" [dim]{source.url}[/dim]") + if only_builtin: + console.print("\n[dim]Using the built-in default stack.[/dim]") + + +@bundle_catalog_app.command("add") +def catalog_add( + url: str = typer.Argument(..., help="Catalog URL"), + policy: str = typer.Option( + "install-allowed", "--policy", help="install-allowed | discovery-only" + ), + priority: int = typer.Option(10, "--priority", help="Source priority (lower = higher)"), + source_id: str = typer.Option(None, "--id", help="Explicit source id"), +) -> None: + """Register a project-scoped catalog source and persist it.""" + try: + project_root = require_project_root() + from ...bundler.commands_impl.catalog_config import add_source + + source = add_source(project_root, url, policy=policy, priority=priority, source_id=source_id) + except BundlerError as exc: + _fail(str(exc)) + return + + console.print( + f"[green]✓[/green] Added catalog '{source.id}' " + f"(priority {source.priority}, {source.install_policy.value})." + ) + + +@bundle_catalog_app.command("remove") +def catalog_remove( + id_or_url: str = typer.Argument(..., help="Source id or url to remove"), +) -> None: + """Remove a project-scoped catalog source (built-in defaults can't be deleted).""" + try: + project_root = require_project_root() + from ...bundler.commands_impl.catalog_config import remove_source + + removed = remove_source(project_root, id_or_url) + except BundlerError as exc: + _fail(str(exc)) + return + + console.print(f"[green]✓[/green] Removed catalog source '{removed}'.") + + +# ===== internal helpers ===== + + +def _manifest_component_view(manifest) -> list[dict]: + """Flatten a manifest's components to JSON-friendly dicts (id, version, ...).""" + if manifest is None: + return [] + view: list[dict] = [] + for component in manifest.components: + item = { + "kind": component.kind, + "id": component.id, + "version": component.version, + } + if component.priority is not None: + item["priority"] = component.priority + if component.strategy is not None: + item["strategy"] = component.strategy + view.append(item) + return view + + +def _format_component(item: dict) -> str: + label = f"{item['id']} v{item['version']}" if item.get("version") else item["id"] + extras = [] + if item.get("priority") is not None: + extras.append(f"priority={item['priority']}") + if item.get("strategy") is not None: + extras.append(f"strategy={item['strategy']}") + if extras: + label += f" ({', '.join(extras)})" + return label + + +def _bundle_overlaps(project_root: Path, manifest, *, offline: bool) -> list[str]: + """Return informational overlaps between *manifest* and installed bundles.""" + if manifest is None: + return [] + try: + from ...bundler.services.conflict import detect_conflicts + + report = detect_conflicts( + manifest, + active_integration(project_root), + load_records(project_root), + ) + return list(report.overlaps) + except BundlerError: + return [] + + +def _local_manifest_source(arg: str): + """Return a :class:`BundleManifest` if *arg* points at a local bundle. + + Supports a built ``.zip`` artifact, a bundle directory, or a ``bundle.yml`` + file. Returns ``None`` when *arg* is not an existing path, so callers fall + back to catalog-stack resolution by bundle id. + """ + from ...bundler.models.manifest import BundleManifest + + candidate = Path(arg).expanduser() + if not candidate.exists(): + return None + + if candidate.is_dir(): + manifest_path = candidate / "bundle.yml" + if not manifest_path.exists(): + raise BundlerError(f"No bundle.yml found in '{candidate}'.") + return BundleManifest.from_file(manifest_path) + + if candidate.suffix == ".zip": + import io + import zipfile + + import yaml as _yaml + + with zipfile.ZipFile(candidate) as archive: + try: + raw = archive.read("bundle.yml") + except KeyError as exc: + raise BundlerError( + f"Artifact '{candidate}' does not contain a bundle.yml." + ) from exc + data = _yaml.safe_load(io.BytesIO(raw)) + return BundleManifest.from_dict(data) + + if candidate.name == "bundle.yml" or candidate.suffix in (".yml", ".yaml"): + return BundleManifest.from_file(candidate) + + raise BundlerError( + f"'{candidate}' is not a recognised bundle source (.zip artifact, bundle " + "directory, or bundle.yml)." + ) + + +def _resolve_manifest_path(path: Path | None) -> Path: + target = (path or Path.cwd()).resolve() + if target.is_dir(): + target = target / "bundle.yml" + if not target.exists(): + raise BundlerError(f"No bundle.yml found at '{target}'.") + return target + + +def _download_manifest(resolved, *, offline: bool): + """Resolve a bundle's manifest from its catalog ``download_url``. + + Local/``file://`` URLs always work offline and may point at a ``.zip`` + artifact, a bundle directory, or a ``bundle.yml`` (handled by + :func:`_local_manifest_source`). Remote ``https://`` URLs are fetched with + the shared authenticated, redirect-validated HTTP client, and only when not + ``--offline``. + """ + from urllib.parse import urlparse + + url = resolved.entry.download_url + if not url: + raise BundlerError( + f"Catalog entry '{resolved.entry.id}' has no download_url; cannot resolve " + "its manifest." + ) + parsed = urlparse(url) + scheme = parsed.scheme.lower() + + # On Windows an absolute path like ``C:\bundle.yml`` parses with a + # single-letter ``scheme``; treat it as a local file, not a URL scheme. + if scheme in ("", "file") or re.match(r"^[A-Za-z]:[\\/]", url): + local = Path(parsed.path if scheme == "file" else url) + manifest = _local_manifest_source(str(local)) + if manifest is None: + raise BundlerError(f"Bundle manifest not found: {local}") + return manifest + + if scheme in ("http", "https"): + if offline: + raise BundlerError( + f"Network access disabled; cannot download bundle '{resolved.entry.id}' " + f"from {url}." + ) + return _download_remote_manifest(resolved.entry.id, url) + + raise BundlerError( + f"Unsupported download_url scheme for bundle '{resolved.entry.id}': {url}" + ) + + +def _require_https(label: str, url: str) -> None: + from urllib.parse import urlparse + + parsed = urlparse(url) + is_localhost = parsed.hostname in ("localhost", "127.0.0.1", "::1") + if parsed.scheme != "https" and not (parsed.scheme == "http" and is_localhost): + raise BundlerError( + f"Refusing to download {label} over non-HTTPS URL: {url}" + ) + if not parsed.hostname: + raise BundlerError(f"Refusing to download {label} from URL with no host: {url}") + + +def _download_remote_manifest(entry_id: str, url: str): + """Fetch a remote bundle artifact over HTTPS and extract its manifest.""" + import io + import tempfile + + from ...authentication.http import open_url + + def _validate_redirect(old_url: str, new_url: str) -> None: + _require_https(f"bundle '{entry_id}'", new_url) + + _require_https(f"bundle '{entry_id}'", url) + try: + with open_url(url, timeout=30, redirect_validator=_validate_redirect) as resp: + _require_https(f"bundle '{entry_id}'", resp.geturl()) + raw = resp.read() + except BundlerError: + raise + except Exception as exc: # noqa: BLE001 + raise BundlerError(f"Failed to download bundle '{entry_id}' from {url}: {exc}") from exc + + # A .zip artifact is written to a temp file and parsed via the local-source + # path (which extracts bundle.yml); any other payload is treated as YAML. + if url.lower().endswith(".zip"): + with tempfile.TemporaryDirectory() as tmp: + artifact = Path(tmp) / "bundle.zip" + artifact.write_bytes(raw) + manifest = _local_manifest_source(str(artifact)) + if manifest is None: + raise BundlerError( + f"Downloaded artifact for bundle '{entry_id}' is not a valid bundle." + ) + return manifest + + import yaml as _yaml + + from ...bundler.models.manifest import BundleManifest + + data = _yaml.safe_load(io.BytesIO(raw)) + return BundleManifest.from_dict(data) + + +def register(app: typer.Typer) -> None: + """Attach the bundle command group to the root Typer app.""" + app.add_typer(bundle_app, name="bundle") diff --git a/tests/bundler_helpers.py b/tests/bundler_helpers.py new file mode 100644 index 0000000000..0ebaf2f1c7 --- /dev/null +++ b/tests/bundler_helpers.py @@ -0,0 +1,125 @@ +"""Shared helpers and fakes for bundler tests. + +Kept out of ``tests/conftest.py`` so the existing root fixtures are untouched. +Import what you need explicitly, e.g.:: + + from tests.bundler_helpers import FakeInstaller, write_manifest +""" +from __future__ import annotations + +import json +from pathlib import Path + +import yaml + +from specify_cli.bundler.models.manifest import ComponentRef + + +def valid_manifest_dict(**overrides) -> dict: + """Return a structurally valid manifest dict; override any top-level key.""" + data = { + "schema_version": "1.0", + "bundle": { + "id": "demo-bundle", + "name": "Demo Bundle", + "version": "1.2.0", + "role": "developer", + "description": "A demo bundle for tests.", + "author": "Spec Kit", + "license": "MIT", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "extensions": [{"id": "ext-a", "version": "1.0.0"}], + "presets": [ + {"id": "preset-a", "version": "2.0.0", "priority": 10, "strategy": "append"} + ], + "steps": [{"id": "step-a"}], + "workflows": [{"id": "wf-a", "version": "0.3.0"}], + }, + "tags": ["demo", "test"], + } + data.update(overrides) + return data + + +def write_manifest(directory: Path, data: dict | None = None) -> Path: + directory.mkdir(parents=True, exist_ok=True) + manifest_path = directory / "bundle.yml" + manifest_path.write_text( + yaml.safe_dump(data if data is not None else valid_manifest_dict()), + encoding="utf-8", + ) + return manifest_path + + +def make_project(root: Path) -> Path: + """Create a minimal Spec Kit project skeleton under *root*.""" + (root / ".specify").mkdir(parents=True, exist_ok=True) + return root + + +def catalog_payload(bundles: dict | None = None) -> dict: + return { + "schema_version": "1.0", + "updated_at": "2026-06-19T00:00:00Z", + "catalog_url": "file://test", + "bundles": bundles or {}, + } + + +def catalog_entry_dict(bundle_id: str = "demo-bundle", **overrides) -> dict: + entry = { + "id": bundle_id, + "name": "Demo Bundle", + "version": "1.2.0", + "role": "developer", + "description": "A demo bundle.", + "author": "Spec Kit", + "license": "MIT", + "download_url": "", + "requires": {"speckit_version": ">=0.1.0"}, + "provides": {"extensions": 1, "presets": 1, "steps": 1, "workflows": 1}, + "verified": True, + } + entry.update(overrides) + return entry + + +def write_catalog_file(path: Path, bundles: dict) -> Path: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(catalog_payload(bundles)), encoding="utf-8") + return path + + +class FakeInstaller: + """Deterministic in-memory PrimitiveInstaller for offline integration tests.""" + + def __init__(self, *, fail_on: str | None = None) -> None: + self.installed: set[tuple[str, str]] = set() + self.install_calls: list[tuple[str, str]] = [] + self.remove_calls: list[tuple[str, str]] = [] + self.refresh_calls: list[tuple[str, str]] = [] + self._fail_on = fail_on + + def _key(self, component: ComponentRef) -> tuple[str, str]: + return (component.kind, component.id) + + def is_installed(self, project_root: Path, component: ComponentRef) -> bool: + return self._key(component) in self.installed + + def install(self, project_root: Path, component: ComponentRef) -> None: + from specify_cli.bundler import BundlerError + + self.install_calls.append(self._key(component)) + if self._fail_on is not None and component.id == self._fail_on: + raise BundlerError(f"Simulated failure installing {component.id}") + self.installed.add(self._key(component)) + + def remove(self, project_root: Path, component: ComponentRef) -> None: + self.remove_calls.append(self._key(component)) + self.installed.discard(self._key(component)) + + def refresh(self, project_root: Path, component: ComponentRef) -> None: + self.refresh_calls.append(self._key(component)) + self.installed.add(self._key(component)) diff --git a/tests/contract/test_bundle_cli.py b/tests/contract/test_bundle_cli.py new file mode 100644 index 0000000000..018b2bbec1 --- /dev/null +++ b/tests/contract/test_bundle_cli.py @@ -0,0 +1,391 @@ +"""Contract test for the `specify bundle` CLI surface (Typer integration). + +Exercises the wired commands end-to-end via CliRunner against a temp project, +asserting exit codes and the cross-cutting error guarantees from +contracts/cli-commands.md (offline, discovery-only refusal, not-a-project error). +""" +from __future__ import annotations + +import json +from pathlib import Path + +import pytest +import yaml +from typer.testing import CliRunner + +from specify_cli import app +from specify_cli.bundler.services.packager import build_bundle +from tests.bundler_helpers import ( + catalog_entry_dict, + valid_manifest_dict, + write_catalog_file, +) + +runner = CliRunner() + + +@pytest.fixture() +def project(tmp_path: Path, monkeypatch) -> Path: + (tmp_path / ".specify").mkdir() + monkeypatch.chdir(tmp_path) + return tmp_path + + +def test_bundle_help_lists_all_commands(): + result = runner.invoke(app, ["bundle", "--help"]) + assert result.exit_code == 0 + for cmd in ("search", "info", "list", "install", "update", "remove", + "validate", "build", "init", "catalog"): + assert cmd in result.output + + +def test_update_accepts_integration_override(): + # Update must expose --integration so integration-pinned bundles can be + # updated in projects where the active integration can't be auto-detected. + # Rich may insert ANSI escapes between the two leading dashes, so match the + # un-split option word rather than the literal "--integration". + result = runner.invoke(app, ["bundle", "update", "--help"]) + assert result.exit_code == 0 + assert "integration" in result.output + + +def test_list_empty_project(project: Path): + result = runner.invoke(app, ["bundle", "list"]) + assert result.exit_code == 0 + assert "No bundles installed" in result.output + + +def test_commands_outside_project_fail_with_guidance(tmp_path: Path, monkeypatch): + monkeypatch.chdir(tmp_path) # no .specify/ + result = runner.invoke(app, ["bundle", "list"]) + assert result.exit_code == 1 + assert "Spec Kit project" in result.output + + +def test_search_works_without_a_project(tmp_path: Path, monkeypatch): + # Discovery commands fall back to the built-in/user catalog stack and must + # not require a Spec Kit project (matches README/quickstart examples). + monkeypatch.chdir(tmp_path) # no .specify/ + result = runner.invoke(app, ["bundle", "search", "--offline", "--json"]) + assert result.exit_code == 0, result.output + assert result.output.strip().startswith("[") + + +def test_info_unknown_bundle_without_project_reports_not_found(tmp_path: Path, monkeypatch): + monkeypatch.chdir(tmp_path) # no .specify/ + result = runner.invoke(app, ["bundle", "info", "does-not-exist", "--offline"]) + # Reaches catalog resolution (not the project gate) and reports a clean miss. + assert result.exit_code == 1 + assert "Spec Kit project" not in result.output + + +def test_catalog_list_shows_builtin_defaults(project: Path): + result = runner.invoke(app, ["bundle", "catalog", "list"]) + assert result.exit_code == 0 + assert "default" in result.output + assert "community" in result.output + assert "built-in default stack" in result.output + + +def test_catalog_add_and_remove(project: Path): + catalog = project / "local-catalog.json" + write_catalog_file(catalog, {"demo": catalog_entry_dict("demo")}) + + added = runner.invoke( + app, ["bundle", "catalog", "add", str(catalog), "--id", "local"] + ) + assert added.exit_code == 0, added.output + + listed = runner.invoke(app, ["bundle", "catalog", "list"]) + assert "local" in listed.output + + removed = runner.invoke(app, ["bundle", "catalog", "remove", "local"]) + assert removed.exit_code == 0 + + +def test_catalog_remove_builtin_is_refused(project: Path): + result = runner.invoke(app, ["bundle", "catalog", "remove", "default"]) + assert result.exit_code == 1 + assert "built-in" in result.output + + +def test_validate_reports_invalid_manifest(project: Path): + data = valid_manifest_dict() + del data["bundle"]["license"] + (project / "bundle.yml").write_text(yaml.safe_dump(data), encoding="utf-8") + result = runner.invoke(app, ["bundle", "validate"]) + assert result.exit_code == 1 + assert "license" in result.output + + +def test_validate_accepts_valid_manifest(project: Path): + (project / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + # Offline mode does not fail on references it cannot verify (synthetic ids + # here); they surface as warnings while structure is confirmed valid. + result = runner.invoke(app, ["bundle", "validate", "--offline"]) + assert result.exit_code == 0, result.output + assert "valid" in result.output + + +def test_validate_rejects_broken_reference(project: Path): + # Synthetic component ids resolve to nothing in any catalog → hard failure. + (project / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + result = runner.invoke(app, ["bundle", "validate"]) + assert result.exit_code == 1 + assert "preset-a" in result.output or "ext-a" in result.output + + +def test_validate_accepts_bundled_reference(project: Path): + data = valid_manifest_dict() + data["provides"] = {"extensions": [{"id": "agent-context", "version": "1.0.0"}]} + (project / "bundle.yml").write_text(yaml.safe_dump(data), encoding="utf-8") + result = runner.invoke(app, ["bundle", "validate"]) + assert result.exit_code == 0, result.output + assert "valid" in result.output + + +def test_build_produces_artifact(project: Path): + (project / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + (project / "README.md").write_text("# Demo", encoding="utf-8") + result = runner.invoke(app, ["bundle", "build", "--output", str(project / "dist")]) + assert result.exit_code == 0, result.output + artifacts = list((project / "dist").glob("*.zip")) + assert len(artifacts) == 1 + + +def test_info_expands_full_component_set(project: Path): + bundle_dir = project / "src-bundle" + bundle_dir.mkdir() + (bundle_dir / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + catalog = project / "local-catalog.json" + entry = catalog_entry_dict( + "demo-bundle", download_url=str(bundle_dir / "bundle.yml") + ) + write_catalog_file(catalog, {"demo-bundle": entry}) + added = runner.invoke( + app, ["bundle", "catalog", "add", str(catalog), "--id", "local"] + ) + assert added.exit_code == 0, added.output + + result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--json", "--offline"]) + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + components = {(c["kind"], c["id"]): c for c in payload["components"]} + assert ("extensions", "ext-a") in components + preset = components[("presets", "preset-a")] + assert preset["version"] == "2.0.0" + assert preset["priority"] == 10 + assert preset["strategy"] == "append" + assert payload["trust"] == "verified" + + text = runner.invoke(app, ["bundle", "info", "demo-bundle", "--offline"]) + assert "preset-a v2.0.0" in text.output + assert "Trust" in text.output + + +def test_info_expands_discovery_only_bundle(project: Path): + # Discovery-only bundles must still be fully inspectable via `info`; + # only `install` is refused for them. + bundle_dir = project / "disc-bundle" + bundle_dir.mkdir() + (bundle_dir / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + catalog = project / "disc-catalog.json" + entry = catalog_entry_dict( + "demo-bundle", download_url=str(bundle_dir / "bundle.yml") + ) + write_catalog_file(catalog, {"demo-bundle": entry}) + config = { + "schema_version": "1.0", + "catalogs": [ + {"id": "disc", "url": str(catalog), "priority": 1, + "install_policy": "discovery-only"} + ], + } + (project / ".specify" / "bundle-catalogs.yml").write_text( + yaml.safe_dump(config), encoding="utf-8" + ) + result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--json", "--offline"]) + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + components = {(c["kind"], c["id"]) for c in payload["components"]} + assert ("extensions", "ext-a") in components + + +def test_info_resolves_local_zip_download_url(project: Path): + # A local .zip artifact as download_url is extracted to read bundle.yml. + bundle_dir = project / "zip-src" + bundle_dir.mkdir() + (bundle_dir / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + (bundle_dir / "README.md").write_text("# Demo", encoding="utf-8") + artifact = build_bundle(bundle_dir, output_dir=project / "dist").artifact_path + catalog = project / "zip-catalog.json" + write_catalog_file( + catalog, + {"demo-bundle": catalog_entry_dict("demo-bundle", download_url=str(artifact))}, + ) + added = runner.invoke( + app, ["bundle", "catalog", "add", str(catalog), "--id", "local"] + ) + assert added.exit_code == 0, added.output + result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--json", "--offline"]) + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + components = {(c["kind"], c["id"]) for c in payload["components"]} + assert ("extensions", "ext-a") in components + + +def test_install_refuses_discovery_only_source(project: Path, monkeypatch): + # Point a discovery-only catalog at a local payload containing the bundle. + catalog = project / "disc.json" + write_catalog_file(catalog, {"demo": catalog_entry_dict("demo")}) + config = { + "schema_version": "1.0", + "catalogs": [ + {"id": "disc", "url": str(catalog), "priority": 1, + "install_policy": "discovery-only"} + ], + } + (project / ".specify" / "bundle-catalogs.yml").write_text( + yaml.safe_dump(config), encoding="utf-8" + ) + result = runner.invoke(app, ["bundle", "install", "demo", "--offline"]) + assert result.exit_code == 1 + assert "discovery-only" in result.output + + +def test_update_refuses_discovery_only_source(project: Path): + # An installed bundle whose only resolvable source is discovery-only must + # not be updatable from there (FR-025), mirroring the install policy gate. + from specify_cli.bundler.models.manifest import ComponentRef + from specify_cli.bundler.models.records import ( + InstalledBundleRecord, + save_records, + ) + + save_records( + project, + [ + InstalledBundleRecord.create( + "demo", + "1.0.0", + [ComponentRef(kind="extensions", id="ext-a", version=None)], + ) + ], + ) + + catalog = project / "disc.json" + write_catalog_file(catalog, {"demo": catalog_entry_dict("demo")}) + config = { + "schema_version": "1.0", + "catalogs": [ + {"id": "disc", "url": str(catalog), "priority": 1, + "install_policy": "discovery-only"} + ], + } + (project / ".specify" / "bundle-catalogs.yml").write_text( + yaml.safe_dump(config), encoding="utf-8" + ) + + result = runner.invoke(app, ["bundle", "update", "demo", "--offline"]) + assert result.exit_code == 1 + assert "discovery-only" in result.output + + +def test_info_fails_loudly_when_manifest_unresolvable_offline(project: Path): + # `info` must expand the real component set; if the manifest can't be + # resolved (here: --offline against an https download_url), it should error + # and exit non-zero rather than silently degrading to `provides` counts. + catalog = project / "remote-catalog.json" + entry = catalog_entry_dict( + "demo-bundle", download_url="https://example.com/demo-bundle.zip" + ) + write_catalog_file(catalog, {"demo-bundle": entry}) + added = runner.invoke( + app, ["bundle", "catalog", "add", str(catalog), "--id", "remote"] + ) + assert added.exit_code == 0, added.output + + result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--offline"]) + assert result.exit_code == 1 + assert "Network access disabled" in result.output + + +def test_search_json_offline(project: Path): + catalog = project / "c.json" + write_catalog_file(catalog, {"demo": catalog_entry_dict("demo")}) + config = { + "schema_version": "1.0", + "catalogs": [ + {"id": "c", "url": str(catalog), "priority": 1, + "install_policy": "install-allowed"} + ], + } + (project / ".specify" / "bundle-catalogs.yml").write_text( + yaml.safe_dump(config), encoding="utf-8" + ) + result = runner.invoke(app, ["bundle", "search", "--offline", "--json"]) + assert result.exit_code == 0 + payload = json.loads(result.output) + assert payload[0]["id"] == "demo" + # Trust indicator is exposed on the discovery surface (FR-010 / FR-027). + assert payload[0]["verified"] is True + assert payload[0]["trust"] == "verified" + + +def test_search_text_shows_trust(project: Path): + catalog = project / "c.json" + write_catalog_file( + catalog, + { + "verified-one": catalog_entry_dict("verified-one", verified=True), + "community-one": catalog_entry_dict("community-one", verified=False), + }, + ) + config = { + "schema_version": "1.0", + "catalogs": [ + {"id": "c", "url": str(catalog), "priority": 1, + "install_policy": "install-allowed"} + ], + } + (project / ".specify" / "bundle-catalogs.yml").write_text( + yaml.safe_dump(config), encoding="utf-8" + ) + result = runner.invoke(app, ["bundle", "search", "--offline"]) + assert result.exit_code == 0, result.output + assert "verified" in result.output + assert "community" in result.output + + +def test_install_integration_override_cannot_bypass_clash_guard(project: Path): + # An initialized project's recorded active integration is authoritative: + # passing --integration must not let a differently-pinned bundle install. + import json + + (project / ".specify" / "integration.json").write_text( + json.dumps({"integration": "copilot"}), encoding="utf-8" + ) + bundle_dir = project / "claude-bundle" + bundle_dir.mkdir() + data = valid_manifest_dict(integration={"id": "claude"}) + (bundle_dir / "bundle.yml").write_text(yaml.safe_dump(data), encoding="utf-8") + (bundle_dir / "README.md").write_text("# Claude bundle", encoding="utf-8") + + result = runner.invoke( + app, + ["bundle", "install", str(bundle_dir), "--integration", "claude", "--offline"], + ) + assert result.exit_code == 1 + assert "claude" in result.output and "copilot" in result.output diff --git a/tests/contract/test_catalog_schema.py b/tests/contract/test_catalog_schema.py new file mode 100644 index 0000000000..2155a9f1fd --- /dev/null +++ b/tests/contract/test_catalog_schema.py @@ -0,0 +1,147 @@ +"""Contract tests for the catalog schema and source stack. + +Mirrors contracts/bundle-catalog.schema.md: source precedence project > user > +built-in, install policy gating, payload parsing. +""" +from __future__ import annotations + +from pathlib import Path + +import yaml + +from specify_cli.bundler.models.catalog import ( + BUILTIN_DEFAULT_STACK, + CatalogSource, + InstallPolicy, + Scope, + load_catalog_payload, + load_source_stack, +) +from specify_cli.bundler import BundlerError +import pytest +from tests.bundler_helpers import catalog_entry_dict, catalog_payload, make_project + + +def test_non_integer_source_priority_raises_actionable_error(): + with pytest.raises(BundlerError, match="non-integer priority"): + CatalogSource.from_dict( + {"id": "corp", "url": "https://corp/catalog.json", "priority": "high"}, + Scope.PROJECT, + ) + + +def test_builtin_default_stack_when_no_config(tmp_path: Path): + make_project(tmp_path) + sources = load_source_stack(tmp_path) + ids = [s.id for s in sources] + assert ids == ["default", "community"] + assert sources[0].install_policy is InstallPolicy.INSTALL_ALLOWED + assert sources[1].install_policy is InstallPolicy.DISCOVERY_ONLY + assert all(s.scope is Scope.BUILTIN for s in sources) + + +def test_project_config_overrides_same_id(tmp_path: Path): + make_project(tmp_path) + config = { + "schema_version": "1.0", + "catalogs": [ + {"id": "default", "url": "file://local", "priority": 1, + "install_policy": "install-allowed"}, + {"id": "corp", "url": "https://corp/catalog.json", "priority": 0, + "install_policy": "install-allowed"}, + ], + } + (tmp_path / ".specify" / "bundle-catalogs.yml").write_text( + yaml.safe_dump(config), encoding="utf-8" + ) + sources = load_source_stack(tmp_path) + by_id = {s.id: s for s in sources} + assert by_id["default"].scope is Scope.PROJECT + assert by_id["default"].url == "file://local" + # Highest precedence (lowest priority number) sorts first. + assert sources[0].id == "corp" + + +def test_user_scope_between_builtin_and_project(tmp_path: Path): + make_project(tmp_path) + user_dir = tmp_path / "userconf" + user_dir.mkdir() + (user_dir / "bundle-catalogs.yml").write_text( + yaml.safe_dump( + {"catalogs": [ + {"id": "community", "url": "https://u", "priority": 2, + "install_policy": "install-allowed"} + ]} + ), + encoding="utf-8", + ) + sources = load_source_stack(tmp_path, user_config_dir=user_dir) + by_id = {s.id: s for s in sources} + # User overrode the built-in community policy to install-allowed. + assert by_id["community"].scope is Scope.USER + assert by_id["community"].install_allowed is True + + +def test_load_payload_parses_entries(): + payload = catalog_payload({"demo-bundle": catalog_entry_dict()}) + entries = load_catalog_payload(payload) + assert "demo-bundle" in entries + assert entries["demo-bundle"].version == "1.2.0" + assert entries["demo-bundle"].provides["presets"] == 1 + + +def test_builtin_default_stack_constant_shape(): + ids = {raw["id"] for raw in BUILTIN_DEFAULT_STACK} + assert ids == {"default", "community"} + + +def test_catalog_entry_rejects_string_tags(): + from specify_cli.bundler.models.catalog import CatalogEntry + + data = catalog_entry_dict("demo") + data["tags"] = "not-a-list" + with pytest.raises(BundlerError, match="'tags' must be a list"): + CatalogEntry.from_dict(data) + + +def test_catalog_entry_rejects_non_boolean_verified(): + from specify_cli.bundler.models.catalog import CatalogEntry + + data = catalog_entry_dict("demo") + data["verified"] = "false" # truthy string must not mark the entry verified + with pytest.raises(BundlerError, match="'verified' must be a boolean"): + CatalogEntry.from_dict(data) + + +def test_load_payload_rejects_id_key_mismatch(): + # The enclosing key is authoritative; an entry whose own id disagrees with + # the key must be rejected so a catalog can't list a spoofed/unresolvable id. + payload = catalog_payload({"demo-bundle": catalog_entry_dict("other-id")}) + with pytest.raises(BundlerError, match="id mismatch"): + load_catalog_payload(payload) + + +def test_load_payload_rejects_missing_entry_id(): + entry = catalog_entry_dict("demo-bundle") + entry["id"] = "" + payload = catalog_payload({"demo-bundle": entry}) + with pytest.raises(BundlerError, match="missing its 'id'"): + load_catalog_payload(payload) + + +def test_catalog_entry_rejects_non_mapping_requires(): + from specify_cli.bundler.models.catalog import CatalogEntry + + data = catalog_entry_dict("demo") + data["requires"] = "speckit>=0.1" + with pytest.raises(BundlerError, match="'requires' must be a mapping"): + CatalogEntry.from_dict(data) + + +def test_catalog_entry_rejects_non_mapping_provides(): + from specify_cli.bundler.models.catalog import CatalogEntry + + data = catalog_entry_dict("demo") + data["provides"] = "extensions" + with pytest.raises(BundlerError, match="'provides' must be a mapping"): + CatalogEntry.from_dict(data) diff --git a/tests/contract/test_manifest_schema.py b/tests/contract/test_manifest_schema.py new file mode 100644 index 0000000000..4d0d95f608 --- /dev/null +++ b/tests/contract/test_manifest_schema.py @@ -0,0 +1,126 @@ +"""Contract tests for the bundle manifest schema (bundle.yml). + +Mirrors contracts/bundle-manifest.schema.md: required identity/metadata fields, +semver pinning of components, preset priority+strategy, integration optionality. +""" +from __future__ import annotations + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.manifest import BundleManifest +from tests.bundler_helpers import valid_manifest_dict + + +def test_valid_manifest_has_no_structural_errors(): + manifest = BundleManifest.from_dict(valid_manifest_dict()) + assert manifest.structural_errors() == [] + assert manifest.bundle.id == "demo-bundle" + assert manifest.is_agnostic() is True + + +def test_missing_required_field_is_reported_by_name(): + data = valid_manifest_dict() + del data["bundle"]["license"] + errors = BundleManifest.from_dict(data).structural_errors() + assert any("bundle.license" in e for e in errors) + + +def test_unsupported_schema_version_is_rejected(): + data = valid_manifest_dict(schema_version="9.9") + errors = BundleManifest.from_dict(data).structural_errors() + assert any("schema_version" in e for e in errors) + + +def test_non_semver_bundle_version_is_rejected(): + data = valid_manifest_dict() + data["bundle"]["version"] = "not-a-version" + errors = BundleManifest.from_dict(data).structural_errors() + assert any("semver" in e for e in errors) + + +def test_preset_requires_priority_and_strategy(): + data = valid_manifest_dict() + data["provides"]["presets"] = [{"id": "p", "version": "1.0.0"}] + errors = BundleManifest.from_dict(data).structural_errors() + assert any("priority" in e for e in errors) + assert any("strategy" in e for e in errors) + + +def test_invalid_preset_strategy_is_rejected(): + data = valid_manifest_dict() + data["provides"]["presets"][0]["strategy"] = "merge" + errors = BundleManifest.from_dict(data).structural_errors() + assert any("strategy" in e for e in errors) + + +def test_non_integer_priority_raises_actionable_error(): + data = valid_manifest_dict() + data["provides"]["presets"][0]["priority"] = "high" + with pytest.raises(BundlerError, match="priority must be an integer"): + BundleManifest.from_dict(data) + + +def test_non_step_components_must_be_pinned(): + data = valid_manifest_dict() + data["provides"]["extensions"] = [{"id": "ext-unpinned"}] + errors = BundleManifest.from_dict(data).structural_errors() + assert any("must be pinned" in e for e in errors) + + +def test_steps_may_be_unpinned(): + data = valid_manifest_dict() + data["provides"]["steps"] = [{"id": "step-x"}] + manifest = BundleManifest.from_dict(data) + assert manifest.structural_errors() == [] + + +def test_integration_makes_bundle_non_agnostic(): + data = valid_manifest_dict(integration={"id": "copilot"}) + manifest = BundleManifest.from_dict(data) + assert manifest.is_agnostic() is False + assert manifest.integration.id == "copilot" + + +def test_components_property_orders_by_kind(): + manifest = BundleManifest.from_dict(valid_manifest_dict()) + kinds = [c.kind for c in manifest.components] + assert kinds == ["extensions", "presets", "steps", "workflows"] + + +def test_string_tags_rejected_not_split_per_character(): + # A bare string would otherwise be iterated character-by-character; the + # schema requires a list of strings. + data = valid_manifest_dict() + data["tags"] = "security" + with pytest.raises(BundlerError, match="'tags' must be a list of strings"): + BundleManifest.from_dict(data) + + +def test_unsafe_bundle_id_flagged_by_structural_validation(): + data = valid_manifest_dict() + data["bundle"]["id"] = "../evil" + manifest = BundleManifest.from_dict(data) + errors = manifest.structural_errors() + assert any("bundle.id" in e and "slug" in e for e in errors) + + +def test_valid_slug_bundle_id_passes(): + data = valid_manifest_dict() + data["bundle"]["id"] = "team-a.bundle_1" + manifest = BundleManifest.from_dict(data) + assert not any("bundle.id" in e for e in manifest.structural_errors()) + + +def test_string_tools_rejected_not_split_per_character(): + data = valid_manifest_dict() + data["requires"]["tools"] = "docker" + with pytest.raises(BundlerError, match="'requires.tools' must be a list of strings"): + BundleManifest.from_dict(data) + + +def test_string_mcp_rejected_not_split_per_character(): + data = valid_manifest_dict() + data["requires"]["mcp"] = "github" + with pytest.raises(BundlerError, match="'requires.mcp' must be a list of strings"): + BundleManifest.from_dict(data) diff --git a/tests/integration/test_bundler_catalog_stack.py b/tests/integration/test_bundler_catalog_stack.py new file mode 100644 index 0000000000..55af49039b --- /dev/null +++ b/tests/integration/test_bundler_catalog_stack.py @@ -0,0 +1,79 @@ +"""Integration tests for the catalog stack: precedence, policy gating, search.""" +from __future__ import annotations + + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.catalog import CatalogSource, InstallPolicy, Scope +from specify_cli.bundler.services.catalog_stack import CatalogStack +from tests.bundler_helpers import catalog_entry_dict, catalog_payload + + +def _source(source_id, priority, policy, url="builtin://x"): + return CatalogSource( + id=source_id, url=url, priority=priority, + install_policy=InstallPolicy(policy), scope=Scope.PROJECT, + ) + + +def _stack(sources, payloads): + def fetcher(src): + return payloads[src.id] + return CatalogStack(sources, fetcher) + + +def test_resolve_prefers_highest_precedence_source(): + sources = [ + _source("low", 2, "install-allowed"), + _source("high", 1, "discovery-only"), + ] + payloads = { + "high": catalog_payload({"b": catalog_entry_dict("b", version="9.0.0")}), + "low": catalog_payload({"b": catalog_entry_dict("b", version="1.0.0")}), + } + resolved = _stack(sources, payloads).resolve("b") + assert resolved.source.id == "high" + assert resolved.entry.version == "9.0.0" + assert resolved.install_allowed is False + + +def test_resolve_unknown_bundle_errors(): + stack = _stack( + [_source("only", 1, "install-allowed")], + {"only": catalog_payload({})}, + ) + with pytest.raises(BundlerError, match="not found"): + stack.resolve("missing") + + +def test_search_dedupes_by_precedence_and_filters(): + sources = [_source("a", 1, "install-allowed"), _source("b", 2, "install-allowed")] + payloads = { + "a": catalog_payload({ + "alpha": catalog_entry_dict("alpha", role="developer"), + }), + "b": catalog_payload({ + "alpha": catalog_entry_dict("alpha", version="0.0.1"), + "beta": catalog_entry_dict("beta", role="qa"), + }), + } + stack = _stack(sources, payloads) + + all_results = stack.search() + ids = [r.entry.id for r in all_results] + assert ids == ["alpha", "beta"] + # alpha resolved from the higher-precedence source 'a'. + alpha = next(r for r in all_results if r.entry.id == "alpha") + assert alpha.source.id == "a" + + qa_only = stack.search("qa") + assert [r.entry.id for r in qa_only] == ["beta"] + + +def test_unreachable_source_raises_named_error(): + def fetcher(src): + raise RuntimeError("boom") + stack = CatalogStack([_source("bad", 1, "install-allowed")], fetcher) + with pytest.raises(BundlerError, match="bad"): + stack.resolve("anything") diff --git a/tests/integration/test_bundler_init_install.py b/tests/integration/test_bundler_init_install.py new file mode 100644 index 0000000000..c1e079ce27 --- /dev/null +++ b/tests/integration/test_bundler_init_install.py @@ -0,0 +1,92 @@ +"""Install-time initialization and integration precedence (T049, T050). + +``specify bundle install`` into an uninitialized directory must scaffold a Spec +Kit project first (FR-012), choosing the integration by precedence (FR-013): +explicit ``--integration`` override → bundle-declared integration → default. +The end-to-end test runs fully offline against bundled assets. +""" +from __future__ import annotations + +import json +import os +from pathlib import Path + +import yaml +from typer.testing import CliRunner + +from specify_cli import app +from specify_cli.bundler.models.manifest import BundleManifest +from specify_cli.commands.bundle import _resolve_init_integration +from specify_cli.bundler.services.packager import build_bundle +from tests.bundler_helpers import valid_manifest_dict + +runner = CliRunner() + + +def _manifest(**overrides): + data = valid_manifest_dict(**overrides) + return BundleManifest.from_dict(data) + + +def test_precedence_override_wins(): + manifest = _manifest(integration={"id": "claude"}) + assert _resolve_init_integration("gemini", manifest) == "gemini" + + +def test_precedence_bundle_declared_when_no_override(): + manifest = _manifest(integration={"id": "claude"}) + assert _resolve_init_integration(None, manifest) == "claude" + + +def test_precedence_default_when_unspecified(): + manifest = _manifest() + assert _resolve_init_integration(None, manifest) == "copilot" + assert _resolve_init_integration(None, None) == "copilot" + + +def _build_mini(tmp_path: Path) -> Path: + bundle = tmp_path / "mini" + bundle.mkdir() + (bundle / "bundle.yml").write_text( + yaml.safe_dump( + { + "schema_version": "1.0", + "bundle": { + "id": "mini", + "name": "Mini", + "version": "1.0.0", + "role": "developer", + "description": "minimal", + "author": "tests", + "license": "MIT", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": {"extensions": [{"id": "agent-context", "version": "1.0.0"}]}, + } + ), + encoding="utf-8", + ) + (bundle / "README.md").write_text("# Mini\n", encoding="utf-8") + return build_bundle(bundle).artifact_path + + +def test_install_initializes_uninitialized_project(tmp_path: Path): + project = tmp_path / "proj" + project.mkdir() + artifact = _build_mini(tmp_path) + + previous = Path.cwd() + os.chdir(project) + try: + result = runner.invoke( + app, ["bundle", "install", str(artifact), "--offline"] + ) + assert result.exit_code == 0, result.output + finally: + os.chdir(previous) + + assert (project / ".specify").is_dir() + marker = project / ".specify" / "integration.json" + assert marker.exists() + data = json.loads(marker.read_text(encoding="utf-8")) + assert "copilot" in json.dumps(data) diff --git a/tests/integration/test_bundler_install_flow.py b/tests/integration/test_bundler_install_flow.py new file mode 100644 index 0000000000..85dcc6c51b --- /dev/null +++ b/tests/integration/test_bundler_install_flow.py @@ -0,0 +1,222 @@ +"""Integration tests for the install → record → remove lifecycle (offline, fake installer). + +Uses :class:`FakeInstaller` so no network or real primitive machinery is touched +(Constitution Principle II network-mocking, Principle IV offline-first). +""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.manifest import BundleManifest +from specify_cli.bundler.models.records import load_records +from specify_cli.bundler.services.installer import install_bundle, remove_bundle +from specify_cli.bundler.services.resolver import resolve_install_plan +from tests.bundler_helpers import FakeInstaller, make_project, valid_manifest_dict + + +def _plan(manifest): + return resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration="copilot" + ) + + +def test_install_records_and_invokes_primitives(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + + result = install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + assert len(result.installed) == 4 + assert len(installer.install_calls) == 4 + records = load_records(tmp_path) + assert len(records) == 1 + assert records[0].bundle_id == "demo-bundle" + + +def test_install_is_idempotent(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + second = install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + # Second install adds nothing and does not duplicate the record. + assert second.installed == [] + assert len(second.skipped) == 4 + assert len(load_records(tmp_path)) == 1 + + +def test_partial_failure_rolls_back_and_records_nothing(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller(fail_on="preset-a") + + with pytest.raises(BundlerError): + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + # ext-a was installed first, then rolled back; no record persisted. + assert installer.installed == set() + assert load_records(tmp_path) == [] + + +def test_remove_is_non_collateral(tmp_path: Path): + make_project(tmp_path) + installer = FakeInstaller() + + # Bundle A provides a shared preset; Bundle B also provides it. + data_a = valid_manifest_dict() + data_a["bundle"]["id"] = "a" + data_b = valid_manifest_dict() + data_b["bundle"]["id"] = "b" + data_b["provides"] = {"presets": [ + {"id": "preset-a", "version": "2.0.0", "priority": 10, "strategy": "append"} + ]} + + man_a = BundleManifest.from_dict(data_a) + man_b = BundleManifest.from_dict(data_b) + install_bundle(tmp_path, _plan(man_a), installer, manifest=man_a) + install_bundle(tmp_path, _plan(man_b), installer, manifest=man_b) + + # Removing B must NOT uninstall preset-a (still needed by A). + result = remove_bundle(tmp_path, "b", installer) + assert ("presets", "preset-a") in {(c.kind, c.id) for c in result.skipped} + assert installer.is_installed(tmp_path, man_a.presets[0]) is True + + remaining = {r.bundle_id for r in load_records(tmp_path)} + assert remaining == {"a"} + + +def test_remove_unknown_bundle_errors(tmp_path: Path): + make_project(tmp_path) + with pytest.raises(BundlerError, match="not installed"): + remove_bundle(tmp_path, "ghost", FakeInstaller()) + + +def test_remove_reports_uninstalled_not_installed(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + result = remove_bundle(tmp_path, "demo-bundle", installer) + + # Removal flows populate the dedicated ``uninstalled`` list; ``installed`` + # stays empty so the result type is never ambiguous for callers. + assert result.installed == [] + assert len(result.uninstalled) == 4 + assert installer.installed == set() + + +def test_remove_counts_only_components_actually_removed(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + # Simulate one contributed component already gone from disk (e.g. removed + # out of band). It must not be reported as uninstalled and remove() must + # not be called for it. + gone = manifest.components[0] + installer.installed.discard((gone.kind, gone.id)) + + result = remove_bundle(tmp_path, "demo-bundle", installer) + + assert len(result.uninstalled) == 3 + assert (gone.kind, gone.id) not in installer.remove_calls + assert gone in result.skipped + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + result = install_bundle( + tmp_path, _plan(manifest), installer, manifest=manifest, refresh=True + ) + + # With refresh, already-installed components are re-applied, not skipped. + assert result.skipped == [] + assert len(result.refreshed) == 4 + assert len(installer.refresh_calls) == 4 + assert result.changed is True + + +def test_refresh_falls_back_to_install_without_hook(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + + class NoRefreshInstaller(FakeInstaller): + refresh = None # type: ignore[assignment] + + installer = NoRefreshInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + before = len(installer.install_calls) + result = install_bundle( + tmp_path, _plan(manifest), installer, manifest=manifest, refresh=True + ) + + # No refresh hook → re-install path keeps components current. + assert len(result.refreshed) == 4 + assert len(installer.install_calls) == before + 4 + + +def test_update_preserves_original_installed_at(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + original = load_records(tmp_path)[0].installed_at + + # A refresh (bundle update) must not rewrite the original install timestamp. + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest, refresh=True) + + assert load_records(tmp_path)[0].installed_at == original + + +def test_refresh_does_not_touch_independently_installed_component(tmp_path: Path): + # bundle update (refresh) must not re-apply a component installed + # independently and tracked by no bundle — refreshing it would be a + # collateral change to something the bundle does not own (FR-022). + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + installer.installed.add(("extensions", "ext-a")) + + result = install_bundle( + tmp_path, _plan(manifest), installer, manifest=manifest, refresh=True + ) + + # ext-a is skipped (not refreshed) and never attributed to the bundle. + assert ("extensions", "ext-a") not in installer.refresh_calls + assert ("extensions", "ext-a") in {(c.kind, c.id) for c in result.skipped} + assert ("extensions", "ext-a") not in {(c.kind, c.id) for c in result.refreshed} + contributed = { + (c.kind, c.id) for c in load_records(tmp_path)[0].contributed_components + } + assert ("extensions", "ext-a") not in contributed + + +def test_pre_existing_component_is_not_attributed_or_removed(tmp_path: Path): + # A component installed independently (before any bundle) must not be + # attributed to the bundle, so removing the bundle never uninstalls it + # (FR-022, no collateral removal). + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + # Pre-install ext-a independently — no bundle record references it yet. + installer.installed.add(("extensions", "ext-a")) + + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + contributed = { + (c.kind, c.id) for c in load_records(tmp_path)[0].contributed_components + } + assert ("extensions", "ext-a") not in contributed + + remove_bundle(tmp_path, "demo-bundle", installer) + assert ("extensions", "ext-a") in installer.installed diff --git a/tests/integration/test_bundler_local_install.py b/tests/integration/test_bundler_local_install.py new file mode 100644 index 0000000000..a150ffce04 --- /dev/null +++ b/tests/integration/test_bundler_local_install.py @@ -0,0 +1,114 @@ +"""Tests for installing a bundle from a local artifact/path (T045). + +The resolution-level tests are pure; the end-to-end test installs the bundled +``agent-context`` extension fully offline from a built ``.zip`` artifact, +proving the real in-process primitive dispatch (T044) works without a network. +""" +from __future__ import annotations + +import os +from pathlib import Path + +import pytest +import yaml +from typer.testing import CliRunner + +from specify_cli import app +from specify_cli.bundler import BundlerError +from specify_cli.commands.bundle import _local_manifest_source +from tests.bundler_helpers import make_project, valid_manifest_dict, write_manifest + + +def test_local_source_none_for_non_path(): + assert _local_manifest_source("some-catalog-bundle-id") is None + + +def test_local_source_from_directory(tmp_path: Path): + write_manifest(tmp_path, valid_manifest_dict()) + manifest = _local_manifest_source(str(tmp_path)) + assert manifest is not None + assert manifest.bundle.id == "demo-bundle" + + +def test_local_source_from_bundle_yml(tmp_path: Path): + path = write_manifest(tmp_path, valid_manifest_dict()) + manifest = _local_manifest_source(str(path)) + assert manifest is not None + assert manifest.bundle.id == "demo-bundle" + + +def test_local_source_from_zip_artifact(tmp_path: Path): + bundle_dir = tmp_path / "bundle" + bundle_dir.mkdir() + write_manifest(bundle_dir, valid_manifest_dict()) + (bundle_dir / "README.md").write_text("# demo\n", encoding="utf-8") + + runner = CliRunner() + result = runner.invoke(app, ["bundle", "build", "--path", str(bundle_dir)]) + assert result.exit_code == 0, result.output + artifact = next(bundle_dir.glob("*.zip")) + + manifest = _local_manifest_source(str(artifact)) + assert manifest is not None + assert manifest.bundle.id == "demo-bundle" + + +def test_local_source_rejects_unknown_file(tmp_path: Path): + weird = tmp_path / "thing.txt" + weird.write_text("nope", encoding="utf-8") + with pytest.raises(BundlerError, match="not a recognised bundle source"): + _local_manifest_source(str(weird)) + + +def test_install_bundled_extension_from_zip_offline(tmp_path: Path): + """End-to-end: build → install (offline, local .zip) → list → remove.""" + project = make_project(tmp_path / "proj") + + bundle_dir = tmp_path / "mini" + bundle_dir.mkdir() + (bundle_dir / "bundle.yml").write_text( + yaml.safe_dump( + { + "schema_version": "1.0", + "bundle": { + "id": "mini", + "name": "Mini", + "version": "1.0.0", + "role": "developer", + "description": "minimal", + "author": "tests", + "license": "MIT", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "extensions": [{"id": "agent-context", "version": "1.0.0"}] + }, + } + ), + encoding="utf-8", + ) + (bundle_dir / "README.md").write_text("# Mini\n", encoding="utf-8") + + runner = CliRunner() + previous = Path.cwd() + os.chdir(project) + try: + build = runner.invoke(app, ["bundle", "build", "--path", str(bundle_dir)]) + assert build.exit_code == 0, build.output + artifact = next(bundle_dir.glob("*.zip")) + + install = runner.invoke(app, ["bundle", "install", str(artifact), "--offline"]) + assert install.exit_code == 0, install.output + + from specify_cli.extensions import ExtensionManager + + assert ExtensionManager(project).registry.is_installed("agent-context") + + listing = runner.invoke(app, ["bundle", "list"]) + assert "mini" in listing.output + + remove = runner.invoke(app, ["bundle", "remove", "mini"]) + assert remove.exit_code == 0, remove.output + assert not ExtensionManager(project).registry.is_installed("agent-context") + finally: + os.chdir(previous) diff --git a/tests/integration/test_bundler_offline.py b/tests/integration/test_bundler_offline.py new file mode 100644 index 0000000000..582f69cef0 --- /dev/null +++ b/tests/integration/test_bundler_offline.py @@ -0,0 +1,78 @@ +"""Offline-first tests (Constitution Principle IV). + +Assert that consume/author flows work with no network access: built-in catalogs +resolve offline, file:// catalogs resolve offline, and http(s) sources are +refused (never silently attempted) when network is disabled. +""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.catalog import CatalogSource, InstallPolicy, Scope +from specify_cli.bundler.services.adapters import make_catalog_fetcher +from specify_cli.bundler.services.catalog_stack import CatalogStack +from tests.bundler_helpers import catalog_entry_dict, write_catalog_file + + +def _src(source_id, url, priority=1, policy="install-allowed"): + return CatalogSource( + id=source_id, url=url, priority=priority, + install_policy=InstallPolicy(policy), scope=Scope.PROJECT, + ) + + +def test_builtin_catalog_resolves_offline(): + fetcher = make_catalog_fetcher(allow_network=False) + stack = CatalogStack([_src("default", "builtin://default")], fetcher) + # Built-in default ships empty; search works without network and returns []. + assert stack.search() == [] + + +def test_file_catalog_resolves_offline(tmp_path: Path): + catalog_path = tmp_path / "catalog.json" + write_catalog_file(catalog_path, {"demo": catalog_entry_dict("demo")}) + fetcher = make_catalog_fetcher(allow_network=False) + stack = CatalogStack([_src("local", str(catalog_path))], fetcher) + resolved = stack.resolve("demo") + assert resolved.entry.id == "demo" + + +def test_http_source_refused_when_offline(): + fetcher = make_catalog_fetcher(allow_network=False) + stack = CatalogStack([_src("remote", "https://example.com/catalog.json")], fetcher) + with pytest.raises(BundlerError, match="Network access disabled"): + stack.resolve("anything") + + +def test_missing_file_catalog_errors_offline(tmp_path: Path): + fetcher = make_catalog_fetcher(allow_network=False) + stack = CatalogStack([_src("local", str(tmp_path / "nope.json"))], fetcher) + with pytest.raises(BundlerError): + stack.resolve("anything") + + +def test_file_url_catalog_resolves_offline(tmp_path: Path): + catalog_path = tmp_path / "catalog.json" + write_catalog_file(catalog_path, {"demo": catalog_entry_dict("demo")}) + fetcher = make_catalog_fetcher(allow_network=False) + stack = CatalogStack([_src("local", catalog_path.as_uri())], fetcher) + resolved = stack.resolve("demo") + assert resolved.entry.id == "demo" + + +def test_plain_http_remote_rejected_before_network(): + # HTTPS is required for non-localhost catalogs; reject http:// up front. + fetcher = make_catalog_fetcher(allow_network=True) + stack = CatalogStack([_src("remote", "http://example.com/catalog.json")], fetcher) + with pytest.raises(BundlerError, match="must use HTTPS"): + stack.resolve("anything") + + +def test_remote_url_without_host_rejected(): + fetcher = make_catalog_fetcher(allow_network=True) + stack = CatalogStack([_src("remote", "https:///catalog.json")], fetcher) + with pytest.raises(BundlerError, match="valid URL with a host"): + stack.resolve("anything") diff --git a/tests/integration/test_bundler_security_paths.py b/tests/integration/test_bundler_security_paths.py new file mode 100644 index 0000000000..85c64919cf --- /dev/null +++ b/tests/integration/test_bundler_security_paths.py @@ -0,0 +1,173 @@ +"""Security tests: path-traversal / symlink confinement (Constitution Principle V). + +These assert the bundler refuses to read or write outside an allowed root, so a +malicious manifest or artifact path cannot escape the project/bundle directory. +""" +from __future__ import annotations + +import os +from pathlib import Path + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.lib.yamlio import ensure_within, is_safe_relpath + + +def test_ensure_within_allows_child(tmp_path: Path): + root = tmp_path / "bundle" + root.mkdir() + child = root / "sub" / "file.txt" + assert ensure_within(root, child) == child.resolve() + + +def test_ensure_within_rejects_parent_traversal(tmp_path: Path): + root = tmp_path / "bundle" + root.mkdir() + escape = root / ".." / "secret.txt" + with pytest.raises(BundlerError, match="escapes"): + ensure_within(root, escape) + + +def test_ensure_within_rejects_absolute_outside(tmp_path: Path): + root = tmp_path / "bundle" + root.mkdir() + with pytest.raises(BundlerError): + ensure_within(root, Path("/etc/passwd")) + + +@pytest.mark.skipif(os.name == "nt", reason="symlink semantics differ on Windows") +def test_ensure_within_rejects_symlink_escape(tmp_path: Path): + root = tmp_path / "bundle" + root.mkdir() + outside = tmp_path / "outside.txt" + outside.write_text("secret", encoding="utf-8") + link = root / "link.txt" + link.symlink_to(outside) + with pytest.raises(BundlerError, match="escapes"): + ensure_within(root, link) + + +@pytest.mark.parametrize("rel,safe", [ + ("a/b.txt", True), + ("./a.txt", True), + ("../escape", False), + ("a/../../escape", False), + ("/abs", False), + ("C:/abs", False), + ("C:\\abs", False), + ("\\\\server\\share", False), + ("", False), +]) +def test_is_safe_relpath(rel, safe): + assert is_safe_relpath(rel) is safe + + +def test_build_skips_symlinks(tmp_path: Path): + """Packager must not follow symlinks out of the bundle dir.""" + import yaml + + from specify_cli.bundler.services.packager import build_bundle + from tests.bundler_helpers import valid_manifest_dict + + bundle = tmp_path / "bundle" + bundle.mkdir() + (bundle / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + (bundle / "README.md").write_text("# Demo", encoding="utf-8") + + if os.name != "nt": + secret = tmp_path / "secret.txt" + secret.write_text("top secret", encoding="utf-8") + (bundle / "leak.txt").symlink_to(secret) + + result = build_bundle(bundle, output_dir=tmp_path / "out") + import zipfile + + with zipfile.ZipFile(result.artifact_path) as archive: + names = archive.namelist() + assert "leak.txt" not in names + assert "bundle.yml" in names + + +def test_load_records_refuses_symlinked_specify_escape(tmp_path: Path): + # Reading bundle-records.json must honour the same confinement as writes: + # a symlinked .specify pointing outside project_root is refused. + from specify_cli.bundler.models.records import load_records + + project = tmp_path / "proj" + project.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (outside / "bundle-records.json").write_text( + '{"schema_version": "1.0", "bundles": []}', encoding="utf-8" + ) + (project / ".specify").symlink_to(outside, target_is_directory=True) + + with pytest.raises(BundlerError, match="escapes the allowed root"): + load_records(project) + + +def test_active_integration_refuses_symlinked_specify_escape(tmp_path: Path): + # Reading the integration marker must not follow a .specify symlink that + # resolves outside project_root; an escape is treated as "not determinable". + from specify_cli.bundler.lib.project import active_integration + + project = tmp_path / "proj" + project.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (outside / "integration.json").write_text( + '{"integration": "leaked"}', encoding="utf-8" + ) + (project / ".specify").symlink_to(outside, target_is_directory=True) + + assert active_integration(project) is None + + +def test_read_catalog_config_refuses_symlinked_specify_escape(tmp_path: Path): + from specify_cli.bundler.commands_impl import catalog_config as cc + + project = tmp_path / "proj" + project.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (outside / "bundle-catalogs.yml").write_text( + "schema_version: '1.0'\ncatalogs: []\n", encoding="utf-8" + ) + (project / ".specify").symlink_to(outside, target_is_directory=True) + + with pytest.raises(BundlerError, match="escapes the allowed root"): + cc._read(project) + + +def test_load_source_stack_refuses_symlinked_specify_dir(tmp_path: Path): + from specify_cli.bundler.models.catalog import load_source_stack + + project = tmp_path / "project" + project.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (outside / "bundle-catalogs.yml").write_text("catalogs: []\n", encoding="utf-8") + try: + (project / ".specify").symlink_to(outside, target_is_directory=True) + except (OSError, NotImplementedError): + pytest.skip("symlinks not supported on this platform") + with pytest.raises(BundlerError, match="escapes the allowed root"): + load_source_stack(project) + + +def test_find_project_root_ignores_symlinked_specify(tmp_path: Path): + from specify_cli.bundler.lib.project import find_project_root + + real = tmp_path / "real-specify" + real.mkdir() + project = tmp_path / "project" + project.mkdir() + try: + (project / ".specify").symlink_to(real, target_is_directory=True) + except (OSError, NotImplementedError): + pytest.skip("symlinks not supported on this platform") + # A symlinked .specify must not be accepted as a project root. + assert find_project_root(project) is None diff --git a/tests/unit/test_bundler_adapters.py b/tests/unit/test_bundler_adapters.py new file mode 100644 index 0000000000..4a6b2cb808 --- /dev/null +++ b/tests/unit/test_bundler_adapters.py @@ -0,0 +1,71 @@ +"""Unit tests for catalog-fetch adapters (auth + redirect safety).""" +from __future__ import annotations + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.catalog import CatalogSource, InstallPolicy +from specify_cli.bundler.services import adapters + + +def _source(url: str) -> CatalogSource: + return CatalogSource( + id="team", + url=url, + priority=10, + install_policy=InstallPolicy.INSTALL_ALLOWED, + ) + + +class _FakeResponse: + def __init__(self, body: bytes, final_url: str) -> None: + self._body = body + self._final_url = final_url + + def __enter__(self) -> "_FakeResponse": + return self + + def __exit__(self, *exc) -> bool: + return False + + def geturl(self) -> str: + return self._final_url + + def read(self) -> bytes: + return self._body + + +def test_http_fetch_uses_shared_client_and_rejects_redirect_downgrade(monkeypatch): + captured: dict = {} + + def fake_open_url(url, timeout=10, extra_headers=None, redirect_validator=None): + captured["url"] = url + captured["validator"] = redirect_validator + return _FakeResponse(b'{"schema_version": "1.0"}', url) + + monkeypatch.setattr("specify_cli.authentication.http.open_url", fake_open_url) + + fetcher = adapters.make_catalog_fetcher(allow_network=True) + result = fetcher(_source("https://example.com/c.json")) + assert result == {"schema_version": "1.0"} + assert captured["url"] == "https://example.com/c.json" + + # The validator handed to open_url must reject an HTTP downgrade redirect. + validator = captured["validator"] + assert validator is not None + with pytest.raises(BundlerError, match="must use HTTPS"): + validator("https://example.com/c.json", "http://evil.example/c.json") + # And a same-scheme HTTPS redirect is allowed (no raise). + validator("https://example.com/c.json", "https://cdn.example/c.json") + + +def test_http_fetch_rejects_non_https_final_url(monkeypatch): + def fake_open_url(url, timeout=10, extra_headers=None, redirect_validator=None): + # Simulate a response whose final URL silently downgraded to HTTP. + return _FakeResponse(b"{}", "http://evil.example/c.json") + + monkeypatch.setattr("specify_cli.authentication.http.open_url", fake_open_url) + + fetcher = adapters.make_catalog_fetcher(allow_network=True) + with pytest.raises(BundlerError, match="must use HTTPS"): + fetcher(_source("https://example.com/c.json")) diff --git a/tests/unit/test_bundler_catalog_config.py b/tests/unit/test_bundler_catalog_config.py new file mode 100644 index 0000000000..0ccb219a53 --- /dev/null +++ b/tests/unit/test_bundler_catalog_config.py @@ -0,0 +1,181 @@ +"""Unit tests for project catalog-config id derivation and url canonicalization.""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.commands_impl import catalog_config as cc + + +def test_derive_id_incorporates_path_stem_for_same_host(): + # Two catalogs on the same host must not collide on the derived id. + a = cc._derive_id("https://example.com/team-a.json") + b = cc._derive_id("https://example.com/team-b.json") + assert a == "example-com-team-a" + assert b == "example-com-team-b" + assert a != b + + +def test_derive_id_distinguishes_tlds(): + # Different TLDs sharing a second-level label must not collide. + com = cc._derive_id("https://example.com/team-a.json") + net = cc._derive_id("https://example.net/team-a.json") + assert com == "example-com-team-a" + assert net == "example-net-team-a" + assert com != net + + +def test_derive_id_falls_back_to_host_when_no_path(): + assert cc._derive_id("https://example.com/") == "example-com" + + +def test_derive_id_for_local_path_uses_stem(): + assert cc._derive_id("./catalogs/my-catalog.json") == "my-catalog" + + +def test_canonicalize_makes_relative_local_path_absolute(tmp_path: Path, monkeypatch): + monkeypatch.chdir(tmp_path) + (tmp_path / "local.json").write_text("{}", encoding="utf-8") + + result = cc._canonicalize_url("local.json") + + assert Path(result).is_absolute() + assert Path(result) == (tmp_path / "local.json").resolve() + + +def test_canonicalize_leaves_remote_urls_untouched(): + for url in ( + "https://example.com/c.json", + "http://localhost:8080/c.json", + "file:///tmp/c.json", + "builtin://default", + ): + assert cc._canonicalize_url(url) == url + + +def test_add_source_persists_absolute_local_path(tmp_path: Path, monkeypatch): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + catalog = project / "sub" / "cat.json" + catalog.parent.mkdir() + catalog.write_text("{}", encoding="utf-8") + + monkeypatch.chdir(project) + source = cc.add_source(project, "sub/cat.json", policy="install-allowed", priority=50) + + assert Path(source.url).is_absolute() + assert Path(source.url) == catalog.resolve() + + +def test_add_source_refuses_symlinked_specify_escape(tmp_path: Path): + project = tmp_path / "proj" + project.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (project / ".specify").symlink_to(outside, target_is_directory=True) + + with pytest.raises(BundlerError, match="escapes the allowed root"): + cc.add_source(project, "https://example.com/c.json", policy="install-allowed", priority=50) + + +def test_read_rejects_non_list_catalogs(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + cc._config_path(project).write_text( + "schema_version: '1.0'\ncatalogs: not-a-list\n", encoding="utf-8" + ) + + with pytest.raises(BundlerError, match="'catalogs' must be a list"): + cc._read(project) + + +def test_read_rejects_non_mapping_catalog_entry(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + cc._config_path(project).write_text( + "schema_version: '1.0'\ncatalogs:\n - just-a-string\n", encoding="utf-8" + ) + + with pytest.raises(BundlerError, match="each catalog entry must be a mapping"): + cc._read(project) + + +def test_read_rejects_non_mapping_top_level(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + cc._config_path(project).write_text("- a\n- b\n", encoding="utf-8") + + with pytest.raises(BundlerError, match="expected a mapping at the top level"): + cc._read(project) + + +def test_read_rejects_unknown_schema_version(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + cc._config_path(project).write_text( + "schema_version: '2.0'\ncatalogs: []\n", encoding="utf-8" + ) + + with pytest.raises(BundlerError, match="Unsupported catalog config schema version"): + cc._read(project) + + +def test_read_accepts_forward_compatible_minor_schema(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + cc._config_path(project).write_text( + "schema_version: '1.5'\ncatalogs: []\n", encoding="utf-8" + ) + assert cc._read(project) == [] + + +def test_read_tolerates_missing_schema_version(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + cc._config_path(project).write_text("catalogs: []\n", encoding="utf-8") + assert cc._read(project) == [] + + +def test_read_returns_empty_for_missing_or_empty_config(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + assert cc._read(project) == [] + + cc._config_path(project).write_text("schema_version: '1.0'\n", encoding="utf-8") + assert cc._read(project) == [] + + +def test_slug_lowercases_for_deterministic_ids(): + # Mixed-case local filenames must derive the same id regardless of case so + # the case-sensitive duplicate check cannot admit logical duplicates. + assert cc._slug("Team-A") == "team-a" + assert cc._derive_id("./catalogs/Team-A.json") == "team-a" + assert cc._derive_id("https://Example.com/Team-A.json") == "example-com-team-a" + + +def test_derive_id_handles_ipv6_literal(): + # An IPv6 host must not be truncated at the first colon. + derived = cc._derive_id("https://[2001:db8::1]/catalog.json") + assert derived == "2001-db8--1-catalog" + + +def test_derive_id_ignores_credentials_and_port(): + assert cc._derive_id("https://user:pw@example.com:8443/c.json") == "example-com-c" + + +def test_add_source_rejects_unsupported_scheme(tmp_path: Path): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + with pytest.raises(BundlerError, match="Unsupported catalog url scheme"): + cc.add_source(project, "ssh://host/catalog.json", policy="install-allowed", priority=50) + + +def test_add_source_allows_local_path_with_colon(tmp_path: Path, monkeypatch): + project = tmp_path / "proj" + (project / ".specify").mkdir(parents=True) + monkeypatch.chdir(project) + # A relative path containing ':' but no '://' is still a local path. + source = cc.add_source(project, "weird:name.json", policy="install-allowed", priority=50) + assert source.url.endswith("weird:name.json") or "weird" in source.url diff --git a/tests/unit/test_bundler_conflict.py b/tests/unit/test_bundler_conflict.py new file mode 100644 index 0000000000..5dbcb3dba1 --- /dev/null +++ b/tests/unit/test_bundler_conflict.py @@ -0,0 +1,54 @@ +"""Unit tests for conflict detection (T034): integration clash and overlap precedence.""" +from __future__ import annotations + +from specify_cli.bundler.models.manifest import BundleManifest, ComponentRef +from specify_cli.bundler.models.records import InstalledBundleRecord +from specify_cli.bundler.services.conflict import detect_conflicts +from tests.bundler_helpers import valid_manifest_dict + + +def _manifest(**overrides) -> BundleManifest: + return BundleManifest.from_dict(valid_manifest_dict(**overrides)) + + +def test_integration_clash_is_blocking(): + manifest = _manifest(integration={"id": "claude"}) + report = detect_conflicts(manifest, active_integration="copilot", installed=[]) + assert report.has_blocking_conflict is True + assert "claude" in report.integration_clash + assert "copilot" in report.integration_clash + + +def test_matching_integration_no_clash(): + manifest = _manifest(integration={"id": "copilot"}) + report = detect_conflicts(manifest, active_integration="copilot", installed=[]) + assert report.has_blocking_conflict is False + + +def test_agnostic_bundle_never_clashes(): + manifest = _manifest() # no integration + report = detect_conflicts(manifest, active_integration="copilot", installed=[]) + assert report.has_blocking_conflict is False + + +def test_overlap_with_other_bundle_is_reported(): + manifest = _manifest() + other = InstalledBundleRecord.create( + bundle_id="other", + version="1.0.0", + components=[ComponentRef(kind="presets", id="preset-a")], + ) + report = detect_conflicts(manifest, active_integration="copilot", installed=[other]) + assert any("preset-a" in o and "other" in o for o in report.overlaps) + assert report.has_blocking_conflict is False + + +def test_same_bundle_reinstall_is_not_overlap(): + manifest = _manifest() + same = InstalledBundleRecord.create( + bundle_id="demo-bundle", + version="1.2.0", + components=[ComponentRef(kind="presets", id="preset-a")], + ) + report = detect_conflicts(manifest, active_integration="copilot", installed=[same]) + assert report.overlaps == [] diff --git a/tests/unit/test_bundler_packager.py b/tests/unit/test_bundler_packager.py new file mode 100644 index 0000000000..5835047f7d --- /dev/null +++ b/tests/unit/test_bundler_packager.py @@ -0,0 +1,193 @@ +"""Unit tests for the artifact packager (T023): contents, versioning, determinism.""" +from __future__ import annotations + +import os +import zipfile +from pathlib import Path + +import pytest +import yaml + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.services.packager import build_bundle +from tests.bundler_helpers import valid_manifest_dict + + +def _make_bundle(directory: Path, *, extra_files: dict | None = None) -> Path: + directory.mkdir(parents=True, exist_ok=True) + (directory / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + (directory / "README.md").write_text("# Demo bundle", encoding="utf-8") + for rel, content in (extra_files or {}).items(): + target = directory / rel + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(content, encoding="utf-8") + return directory + + +def test_artifact_named_by_id_and_version(tmp_path: Path): + bundle = _make_bundle(tmp_path / "b") + result = build_bundle(bundle, output_dir=tmp_path / "out") + assert result.artifact_path.name == "demo-bundle-1.2.0.zip" + + +def test_artifact_contains_manifest_and_assets(tmp_path: Path): + bundle = _make_bundle(tmp_path / "b", extra_files={"assets/logo.txt": "logo"}) + result = build_bundle(bundle, output_dir=tmp_path / "out") + with zipfile.ZipFile(result.artifact_path) as archive: + names = set(archive.namelist()) + assert "bundle.yml" in names + assert "README.md" in names + assert "assets/logo.txt" in names + + +def test_build_refuses_invalid_manifest(tmp_path: Path): + bundle = tmp_path / "b" + bundle.mkdir() + data = valid_manifest_dict() + del data["bundle"]["license"] + (bundle / "bundle.yml").write_text(yaml.safe_dump(data), encoding="utf-8") + (bundle / "README.md").write_text("# x", encoding="utf-8") + with pytest.raises(BundlerError, match="validate"): + build_bundle(bundle, output_dir=tmp_path / "out") + + +def test_build_missing_manifest_errors(tmp_path: Path): + with pytest.raises(BundlerError, match="No bundle.yml"): + build_bundle(tmp_path, output_dir=tmp_path / "out") + + +def test_build_is_deterministic(tmp_path: Path): + bundle = _make_bundle(tmp_path / "b", extra_files={"a.txt": "a", "z.txt": "z"}) + first = build_bundle(bundle, output_dir=tmp_path / "out1") + second = build_bundle(bundle, output_dir=tmp_path / "out2") + with zipfile.ZipFile(first.artifact_path) as a, zipfile.ZipFile(second.artifact_path) as b: + # Same files, same order (sorted). + assert a.namelist() == b.namelist() + # Fixed timestamps + permissions make each member byte-identical. + for left, right in zip(a.infolist(), b.infolist()): + assert left.date_time == right.date_time + assert left.external_attr == right.external_attr + # The whole artifact is byte-for-byte reproducible. + assert first.artifact_path.read_bytes() == second.artifact_path.read_bytes() + + +def test_output_dir_inside_bundle_excludes_prior_artifacts(tmp_path: Path): + bundle = _make_bundle(tmp_path / "b", extra_files={"a.txt": "a"}) + out_dir = bundle / "dist" + # Build twice into a dir nested in the bundle; the second build must not + # re-package the first artifact, so contents stay identical and bounded. + first = build_bundle(bundle, output_dir=out_dir) + second = build_bundle(bundle, output_dir=out_dir) + with zipfile.ZipFile(second.artifact_path) as archive: + names = archive.namelist() + assert not any(name.startswith("dist/") for name in names) + assert not any(name.endswith(".zip") for name in names) + assert first.file_count == second.file_count + + +def test_prior_version_artifact_not_repackaged(tmp_path: Path): + # An older artifact sitting next to bundle.yml must not be packaged. + bundle = _make_bundle(tmp_path / "b", extra_files={"a.txt": "a"}) + (bundle / "demo-bundle-0.9.0.zip").write_bytes(b"PK\x03\x04 old artifact") + result = build_bundle(bundle, output_dir=bundle) + with zipfile.ZipFile(result.artifact_path) as archive: + names = archive.namelist() + assert not any(name.endswith(".zip") for name in names) + assert "demo-bundle-0.9.0.zip" not in names + + +def test_symlinked_directory_is_not_followed(tmp_path: Path): + outside = tmp_path / "outside" + outside.mkdir() + (outside / "secret.txt").write_text("secret", encoding="utf-8") + bundle = _make_bundle(tmp_path / "b", extra_files={"a.txt": "a"}) + link = bundle / "linkdir" + try: + link.symlink_to(outside, target_is_directory=True) + except (OSError, NotImplementedError): + pytest.skip("symlinks not supported on this platform") + # Build must succeed (no ensure_within failure) and must not pull in the + # out-of-tree file behind the symlinked directory. + result = build_bundle(bundle, output_dir=tmp_path / "out") + with zipfile.ZipFile(result.artifact_path) as archive: + names = archive.namelist() + assert "linkdir/secret.txt" not in names + assert not any("secret" in name for name in names) + + +def test_unsafe_bundle_id_is_rejected_before_build(tmp_path: Path): + data = valid_manifest_dict() + data["bundle"]["id"] = "../evil" + bundle = tmp_path / "b" + bundle.mkdir(parents=True) + (bundle / "bundle.yml").write_text(yaml.safe_dump(data), encoding="utf-8") + (bundle / "README.md").write_text("# x", encoding="utf-8") + with pytest.raises(BundlerError): + build_bundle(bundle, output_dir=tmp_path / "out") + # The traversal target must not have been written outside out_dir. + assert not (tmp_path / "evil-1.2.0.zip").exists() + + +def test_build_refuses_missing_readme(tmp_path: Path): + bundle = tmp_path / "b" + bundle.mkdir() + (bundle / "bundle.yml").write_text( + yaml.safe_dump(valid_manifest_dict()), encoding="utf-8" + ) + with pytest.raises(BundlerError, match="README.md"): + build_bundle(bundle, output_dir=tmp_path / "out") + + +def test_asset_zip_starting_with_bundle_id_is_packaged(tmp_path: Path): + # A non-artifact asset whose name merely starts with the bundle id (but is + # not a semver-named build artifact) must still be included. + bundle = _make_bundle(tmp_path / "b", extra_files={"demo-bundle-assets.zip": "data"}) + result = build_bundle(bundle, output_dir=tmp_path / "out") + with zipfile.ZipFile(result.artifact_path) as archive: + names = set(archive.namelist()) + assert "demo-bundle-assets.zip" in names + + +def test_prior_semver_artifact_is_excluded(tmp_path: Path): + bundle = _make_bundle(tmp_path / "b", extra_files={"demo-bundle-0.9.0.zip": "old"}) + result = build_bundle(bundle, output_dir=bundle) + with zipfile.ZipFile(result.artifact_path) as archive: + names = set(archive.namelist()) + assert "demo-bundle-0.9.0.zip" not in names + + +def test_prior_artifact_with_prerelease_and_build_is_excluded(tmp_path: Path): + # A semver artifact carrying both prerelease and build metadata must still + # be recognized as a prior build artifact and excluded. + bundle = _make_bundle( + tmp_path / "b", extra_files={"demo-bundle-1.0.0-rc1+build5.zip": "old"} + ) + result = build_bundle(bundle, output_dir=bundle) + with zipfile.ZipFile(result.artifact_path) as archive: + names = set(archive.namelist()) + assert "demo-bundle-1.0.0-rc1+build5.zip" not in names + + +@pytest.mark.skipif( + os.name == "nt", + reason="Windows filesystems do not carry Unix execute bits, so chmod(0o755) " + "is a no-op and there is no executability to preserve.", +) +def test_executable_bit_preserved_in_artifact(tmp_path: Path): + bundle = _make_bundle(tmp_path / "bundle") + script = bundle / "scripts" / "hook.sh" + script.parent.mkdir(parents=True, exist_ok=True) + script.write_text("#!/bin/sh\necho hi\n", encoding="utf-8") + script.chmod(0o755) + + result = build_bundle(bundle, output_dir=tmp_path / "out") + with zipfile.ZipFile(result.artifact_path) as archive: + modes = { + info.filename: (info.external_attr >> 16) & 0o777 + for info in archive.infolist() + } + # Executable source -> 0755; plain text files -> 0644. + assert modes["scripts/hook.sh"] == 0o755 + assert modes["README.md"] == 0o644 diff --git a/tests/unit/test_bundler_primitives.py b/tests/unit/test_bundler_primitives.py new file mode 100644 index 0000000000..f662d22fc9 --- /dev/null +++ b/tests/unit/test_bundler_primitives.py @@ -0,0 +1,133 @@ +"""Unit tests for the primitive-dispatch bridge (T044). + +Covers routing, offline gating, and the network-aware ``DefaultPrimitiveInstaller`` +seam — without touching real catalogs or the network (Constitution Principle II, +offline-first). +""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.manifest import ComponentRef +from specify_cli.bundler.services.adapters import DefaultPrimitiveInstaller +from specify_cli.bundler.services.primitives import ( + _ExtensionKindManager, + _PresetKindManager, + _StepKindManager, + _WorkflowKindManager, + primitive_manager, +) + + +def _component(kind: str, cid: str = "x") -> ComponentRef: + return ComponentRef(kind=kind, id=cid) + + +def test_primitive_manager_routes_each_kind(tmp_path: Path): + assert isinstance(primitive_manager("presets", tmp_path), _PresetKindManager) + assert isinstance(primitive_manager("extensions", tmp_path), _ExtensionKindManager) + assert isinstance(primitive_manager("workflows", tmp_path), _WorkflowKindManager) + assert isinstance(primitive_manager("steps", tmp_path), _StepKindManager) + + +def test_primitive_manager_rejects_unknown_kind(tmp_path: Path): + with pytest.raises(BundlerError, match="Unknown component kind"): + primitive_manager("bogus", tmp_path) + + +def test_offline_preset_not_bundled_refuses(tmp_path: Path): + manager = primitive_manager("presets", tmp_path, allow_network=False) + with pytest.raises(BundlerError, match="network access is disabled"): + manager.install(_component("presets", "definitely-not-bundled")) + + +def test_offline_extension_not_bundled_refuses(tmp_path: Path): + manager = primitive_manager("extensions", tmp_path, allow_network=False) + with pytest.raises(BundlerError, match="network access is disabled"): + manager.install(_component("extensions", "definitely-not-bundled")) + + +def test_offline_workflow_refuses_without_network(tmp_path: Path): + manager = primitive_manager("workflows", tmp_path, allow_network=False) + with pytest.raises(BundlerError, match="network access is disabled"): + manager.install(_component("workflows")) + + +def test_offline_step_refuses_without_network(tmp_path: Path): + manager = primitive_manager("steps", tmp_path, allow_network=False) + with pytest.raises(BundlerError, match="network access is disabled"): + manager.install(_component("steps")) + + +def test_default_installer_threads_allow_network(tmp_path: Path): + installer = DefaultPrimitiveInstaller(allow_network=False) + with pytest.raises(BundlerError, match="network access is disabled"): + installer.install(tmp_path, _component("workflows")) + + +def test_offline_workflow_allows_bundled(tmp_path: Path, monkeypatch): + # A workflow that ships with Spec Kit must install even with --offline. + import specify_cli + import specify_cli._assets as assets + + monkeypatch.setattr( + assets, "_locate_bundled_workflow", lambda wid: tmp_path / "wf" + ) + calls: list[str] = [] + monkeypatch.setattr(specify_cli, "workflow_add", lambda wid: calls.append(wid)) + + manager = primitive_manager("workflows", tmp_path, allow_network=False) + manager.install(_component("workflows", "bundled-wf")) + + assert calls == ["bundled-wf"] + + +def test_assert_pinned_version_matches_passes(): + from specify_cli.bundler.services.primitives import _assert_pinned_version + + # Equal (including v-prefix/normalization) is accepted; no version pins are no-ops. + _assert_pinned_version("Preset", "p", "2.0.0", "2.0.0") + _assert_pinned_version("Preset", "p", "2.0.0", "v2.0.0") + _assert_pinned_version("Preset", "p", None, "9.9.9") + _assert_pinned_version("Preset", "p", "2.0.0", None) + + +def test_assert_pinned_version_mismatch_raises(): + from specify_cli.bundler.services.primitives import _assert_pinned_version + + with pytest.raises(BundlerError, match="pinned to version 2.0.0"): + _assert_pinned_version("Preset", "preset-a", "2.0.0", "3.1.0") + + +def test_workflow_version_mismatch_refuses(tmp_path: Path, monkeypatch): + from specify_cli.workflows.catalog import WorkflowCatalog + + monkeypatch.setattr( + WorkflowCatalog, "get_workflow_info", lambda self, wid: {"version": "9.9.9"} + ) + manager = primitive_manager("workflows", tmp_path, allow_network=True) + component = ComponentRef(kind="workflows", id="wf-a", version="0.3.0") + with pytest.raises(BundlerError, match="pinned to version 0.3.0"): + manager.install(component) + + +def test_preset_install_preserves_explicit_zero_priority(tmp_path: Path, monkeypatch): + import specify_cli._assets as assets + + calls = {} + + class _FakeManager: + def install_from_directory(self, directory, speckit_version, priority): + calls["priority"] = priority + + monkeypatch.setattr(assets, "_locate_bundled_preset", lambda cid: tmp_path) + + manager = primitive_manager("presets", tmp_path, allow_network=False) + manager._manager = _FakeManager() + manager.install(ComponentRef(kind="presets", id="p", priority=0)) + + # An explicit priority of 0 must be passed through, not replaced by default. + assert calls["priority"] == 0 diff --git a/tests/unit/test_bundler_records.py b/tests/unit/test_bundler_records.py new file mode 100644 index 0000000000..21771adb0f --- /dev/null +++ b/tests/unit/test_bundler_records.py @@ -0,0 +1,190 @@ +"""Unit tests for installed-bundle records and collateral-protection logic.""" +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.manifest import ComponentRef +from specify_cli.bundler.models.records import ( + InstalledBundleRecord, + components_still_needed, + load_records, + records_path, + remove_record, + save_records, + upsert_record, +) + + +def _record(bundle_id: str, comps) -> InstalledBundleRecord: + return InstalledBundleRecord.create( + bundle_id=bundle_id, + version="1.0.0", + components=[ComponentRef(kind=k, id=i) for k, i in comps], + ) + + +def test_save_and_load_roundtrip(tmp_path: Path): + (tmp_path / ".specify").mkdir() + rec = _record("a", [("presets", "p1"), ("steps", "s1")]) + save_records(tmp_path, [rec]) + loaded = load_records(tmp_path) + assert len(loaded) == 1 + assert loaded[0].bundle_id == "a" + assert {(c.kind, c.id) for c in loaded[0].contributed_components} == { + ("presets", "p1"), + ("steps", "s1"), + } + + +def test_load_missing_file_returns_empty(tmp_path: Path): + (tmp_path / ".specify").mkdir() + assert load_records(tmp_path) == [] + + +def test_corrupt_priority_raises_actionable_error(tmp_path: Path): + (tmp_path / ".specify").mkdir() + rec = _record("a", [("presets", "p1")]) + save_records(tmp_path, [rec]) + path = records_path(tmp_path) + data = json.loads(path.read_text(encoding="utf-8")) + data["bundles"][0]["contributed_components"][0]["priority"] = "high" + path.write_text(json.dumps(data), encoding="utf-8") + with pytest.raises(BundlerError, match="priority must be an integer"): + load_records(tmp_path) + + +def test_upsert_replaces_same_id(): + rec1 = _record("a", [("presets", "p1")]) + rec2 = _record("a", [("presets", "p2")]) + result = upsert_record([rec1], rec2) + assert len(result) == 1 + assert result[0].contributed_components[0].id == "p2" + + +def test_remove_record_drops_target(): + recs = [_record("a", [("presets", "p1")]), _record("b", [("steps", "s1")])] + result = remove_record(recs, "a") + assert [r.bundle_id for r in result] == ["b"] + + +def test_components_still_needed_excludes_target(): + recs = [ + _record("a", [("presets", "shared"), ("steps", "only-a")]), + _record("b", [("presets", "shared")]), + ] + needed = components_still_needed(recs, exclude_bundle_id="a") + assert ("presets", "shared") in needed + assert ("steps", "only-a") not in needed + + +def test_save_records_refuses_symlinked_specify_escape(tmp_path: Path): + # Defense-in-depth: a symlinked .specify pointing outside the project must + # not let records be written outside project_root. + project = tmp_path / "proj" + project.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (project / ".specify").symlink_to(outside, target_is_directory=True) + + with pytest.raises(BundlerError, match="escapes the allowed root"): + save_records(project, [_record("a", [("presets", "p1")])]) + + +def test_load_records_rejects_non_list_bundles(tmp_path: Path): + (tmp_path / ".specify").mkdir() + path = records_path(tmp_path) + path.write_text(json.dumps({"schema_version": "1.0", "bundles": "oops"}), encoding="utf-8") + with pytest.raises(BundlerError, match="'bundles' must be a list"): + load_records(tmp_path) + + +def test_load_records_rejects_non_list_contributed_components(tmp_path: Path): + (tmp_path / ".specify").mkdir() + path = records_path(tmp_path) + payload = { + "schema_version": "1.0", + "bundles": [ + {"bundle_id": "a", "version": "1.0.0", "contributed_components": "oops"} + ], + } + path.write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(BundlerError, match="'contributed_components' must be a list"): + load_records(tmp_path) + + +def test_load_records_rejects_unknown_component_kind(tmp_path: Path): + (tmp_path / ".specify").mkdir() + path = records_path(tmp_path) + payload = { + "schema_version": "1.0", + "bundles": [ + { + "bundle_id": "a", + "version": "1.0.0", + "contributed_components": [{"kind": "bogus", "id": "x"}], + } + ], + } + path.write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(BundlerError, match="must be one of"): + load_records(tmp_path) + + +def test_load_records_rejects_component_missing_id(tmp_path: Path): + (tmp_path / ".specify").mkdir() + path = records_path(tmp_path) + payload = { + "schema_version": "1.0", + "bundles": [ + { + "bundle_id": "a", + "version": "1.0.0", + "contributed_components": [{"kind": "presets", "id": ""}], + } + ], + } + path.write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(BundlerError, match="missing its 'id'"): + load_records(tmp_path) + + +def test_load_records_rejects_missing_schema_version(tmp_path: Path): + (tmp_path / ".specify").mkdir() + records_path(tmp_path).write_text(json.dumps({"bundles": []}), encoding="utf-8") + with pytest.raises(BundlerError, match="missing 'schema_version'"): + load_records(tmp_path) + + +def test_load_records_rejects_unknown_schema_version(tmp_path: Path): + (tmp_path / ".specify").mkdir() + payload = {"schema_version": "2.0", "bundles": []} + records_path(tmp_path).write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(BundlerError, match="Unsupported records schema version"): + load_records(tmp_path) + + +def test_load_records_rejects_record_missing_bundle_id(tmp_path: Path): + (tmp_path / ".specify").mkdir() + payload = {"schema_version": "1.0", "bundles": [{"version": "1.0.0"}]} + records_path(tmp_path).write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(BundlerError, match="missing its 'bundle_id'"): + load_records(tmp_path) + + +def test_load_records_rejects_record_missing_version(tmp_path: Path): + (tmp_path / ".specify").mkdir() + payload = {"schema_version": "1.0", "bundles": [{"bundle_id": "a"}]} + records_path(tmp_path).write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(BundlerError, match="missing its 'version'"): + load_records(tmp_path) + + +def test_load_records_accepts_forward_compatible_minor_schema(tmp_path: Path): + (tmp_path / ".specify").mkdir() + payload = {"schema_version": "1.5", "bundles": []} + records_path(tmp_path).write_text(json.dumps(payload), encoding="utf-8") + assert load_records(tmp_path) == [] diff --git a/tests/unit/test_bundler_references.py b/tests/unit/test_bundler_references.py new file mode 100644 index 0000000000..1291ba08bd --- /dev/null +++ b/tests/unit/test_bundler_references.py @@ -0,0 +1,41 @@ +"""Unit tests for the bundle reference checker (T047 / FR-005 / SC-007). + +Resolution is offline-first: bundled and installed components resolve without a +network; unknown ids fail online and downgrade to warnings offline. +""" +from __future__ import annotations + +from pathlib import Path + +from specify_cli.bundler.models.manifest import ComponentRef +from specify_cli.bundler.services.references import make_reference_checker +from tests.bundler_helpers import make_project + + +def _ref(kind: str, id_: str) -> ComponentRef: + return ComponentRef(kind=kind, id=id_, version="1.0.0") + + +def test_bundled_extension_resolves(tmp_path: Path): + root = make_project(tmp_path) + warnings: list[str] = [] + check = make_reference_checker(root, allow_network=True, warnings=warnings) + assert check(_ref("extensions", "agent-context")) is None + assert warnings == [] + + +def test_unknown_reference_errors_online(tmp_path: Path): + root = make_project(tmp_path) + warnings: list[str] = [] + check = make_reference_checker(root, allow_network=True, warnings=warnings) + problem = check(_ref("presets", "does-not-exist")) + assert problem is not None + assert "does-not-exist" in problem + + +def test_unknown_reference_warns_offline(tmp_path: Path): + root = make_project(tmp_path) + warnings: list[str] = [] + check = make_reference_checker(root, allow_network=False, warnings=warnings) + assert check(_ref("presets", "does-not-exist")) is None + assert any("does-not-exist" in w for w in warnings) diff --git a/tests/unit/test_bundler_resolver.py b/tests/unit/test_bundler_resolver.py new file mode 100644 index 0000000000..7068a4813e --- /dev/null +++ b/tests/unit/test_bundler_resolver.py @@ -0,0 +1,81 @@ +"""Unit tests for the resolver: version gate and integration compatibility.""" +from __future__ import annotations + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.models.manifest import BundleManifest +from specify_cli.bundler.services.resolver import resolve_install_plan +from tests.bundler_helpers import valid_manifest_dict + + +def _manifest(**overrides) -> BundleManifest: + return BundleManifest.from_dict(valid_manifest_dict(**overrides)) + + +def test_plan_expands_all_components(): + plan = resolve_install_plan( + _manifest(), speckit_version="0.11.2", active_integration="copilot" + ) + assert plan.component_count == 4 + assert plan.bundle_id == "demo-bundle" + + +def test_version_gate_refuses_incompatible(): + manifest = _manifest(requires={"speckit_version": ">=99.0.0"}) + with pytest.raises(BundlerError, match="requires Spec Kit"): + resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration="copilot" + ) + + +def test_integration_clash_halts(): + manifest = _manifest(integration={"id": "claude"}) + with pytest.raises(BundlerError, match="active integration"): + resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration="copilot" + ) + + +def test_agnostic_inherits_active_integration(): + plan = resolve_install_plan( + _manifest(), speckit_version="0.11.2", active_integration="copilot" + ) + assert plan.effective_integration == "copilot" + + +def test_matching_integration_is_allowed(): + manifest = _manifest(integration={"id": "copilot"}) + plan = resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration="copilot" + ) + assert plan.effective_integration == "copilot" + + +def test_pinned_integration_with_indeterminate_active_fails(): + # FR-019 guard: a bundle that pins an integration must not silently adopt it + # when the project's active integration cannot be determined. + manifest = _manifest(integration={"id": "claude"}) + with pytest.raises(BundlerError, match="could not be determined"): + resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration=None + ) + + +def test_pinned_integration_with_indeterminate_active_allows_explicit_override(): + manifest = _manifest(integration={"id": "claude"}) + plan = resolve_install_plan( + manifest, + speckit_version="0.11.2", + active_integration="claude", + integration_explicit=True, + ) + assert plan.effective_integration == "claude" + + +def test_tool_requirements_become_warnings(): + manifest = _manifest(requires={"speckit_version": ">=0.1.0", "tools": ["docker"]}) + plan = resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration="copilot" + ) + assert any("docker" in w for w in plan.warnings) diff --git a/tests/unit/test_bundler_validator.py b/tests/unit/test_bundler_validator.py new file mode 100644 index 0000000000..d69c6535e5 --- /dev/null +++ b/tests/unit/test_bundler_validator.py @@ -0,0 +1,32 @@ +"""Unit tests for the bundle manifest validator service.""" +from __future__ import annotations + +import pytest + +from specify_cli.bundler.models.manifest import BundleManifest +from specify_cli.bundler.services import validator as validator_mod +from specify_cli.bundler.services.validator import validate_manifest +from tests.bundler_helpers import valid_manifest_dict + + +def _manifest(**overrides) -> BundleManifest: + return BundleManifest.from_dict(valid_manifest_dict(**overrides)) + + +def test_invalid_speckit_constraint_reported_as_error(): + manifest = _manifest(requires={"speckit_version": ">>bad"}) + report = validate_manifest(manifest) + assert not report.ok + assert any("speckit_version" in e for e in report.errors) + + +def test_non_bundler_error_not_swallowed(monkeypatch): + # A programming error inside constraint parsing must propagate, not be + # masked behind an "invalid constraint" validation message. + def boom(_value): + raise RuntimeError("unexpected bug") + + monkeypatch.setattr(validator_mod, "parse_constraint", boom) + manifest = _manifest(requires={"speckit_version": ">=1.0.0"}) + with pytest.raises(RuntimeError, match="unexpected bug"): + validate_manifest(manifest) diff --git a/tests/unit/test_bundler_versioning.py b/tests/unit/test_bundler_versioning.py new file mode 100644 index 0000000000..15c42ea673 --- /dev/null +++ b/tests/unit/test_bundler_versioning.py @@ -0,0 +1,68 @@ +"""Unit tests for version parsing and constraint satisfaction (FR-016 gate).""" +from __future__ import annotations + +import pytest + +from specify_cli.bundler import BundlerError +from specify_cli.bundler.lib.versioning import is_semver, satisfies + + +@pytest.mark.parametrize("value,expected", [ + ("1.0.0", True), + ("0.11.2", True), + ("1.2.3-rc1", True), + ("1.2.3-alpha1", True), + ("1.2.3-beta2", True), + ("v1.2.3", True), + ("not-a-version", False), + ("", False), + # packaging.version.Version accepts these partial versions; SemVer must not. + ("1", False), + ("1.0", False), + ("1.2.3.4", False), +]) +def test_is_semver(value, expected): + assert is_semver(value) is expected + + +@pytest.mark.parametrize("installed,constraint,ok", [ + ("0.11.2", ">=0.1.0", True), + ("0.11.2", ">=1.0.0", False), + ("1.0.0", ">=1.0.0,<2.0.0", True), + ("2.0.0", ">=1.0.0,<2.0.0", False), + ("1.5.0", "", True), # empty constraint is permissive + # Prerelease spellings normalize consistently for constraint checks. + ("1.2.3-rc1", ">=1.2.0", True), + ("1.2.3-alpha1", ">=2.0.0", False), +]) +def test_satisfies(installed, constraint, ok): + assert satisfies(installed, constraint) is ok + + +def test_invalid_constraint_raises(): + with pytest.raises(BundlerError): + satisfies("1.0.0", ">>bad") + + +def test_uppercase_v_prefix_tolerated(): + # Mirrors specify_cli._version tag normalization (V -> v). + assert is_semver("V1.2.3") is True + assert satisfies("V1.2.3", ">=1.2.0") is True + + +@pytest.mark.parametrize("installed,constraint,ok", [ + # Prerelease spellings are now normalized inside constraints too, so a + # constraint like ">=1.2.3-rc1" parses (previously raised InvalidSpecifier). + ("1.2.3-rc2", ">=1.2.3-rc1", True), + ("1.2.2", ">=1.2.3-rc1", False), + ("1.5.0", ">=1.2.3-rc1,<2.0.0", True), + ("1.2.3-beta.1", ">=1.2.3-alpha1", True), +]) +def test_satisfies_prerelease_in_constraint(installed, constraint, ok): + assert satisfies(installed, constraint) is ok + + +def test_parse_constraint_empty_is_permissive(): + from specify_cli.bundler.lib.versioning import parse_constraint + + assert str(parse_constraint("")) == ""