From a7a78169251b6e3ee0cb04cc9a2e915bce9c7fed Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 24 Oct 2026 07:00:50 +0000 Subject: [PATCH] Move the format crates from plugins/ to crates/formats/ Since #437 the server links in every format, so plugins/ was only a second directory of workspace crates. NuGet and raw now live in crates/formats//, with the same crate names and code, and plugins/README.md is crates/formats/README.md. The workspace members, the path dependencies on the format crates and theirs on forge-plugin-api, and the NuGet fixture path follow the new depth. The CI change filter, CodeQL's language filter and its comment no longer name plugins/ (crates/ covers the new place), and .dockerignore drops the target directory of a crate built on its own, which no crate has now. AGENTS.md's rule on formats, README.md, DEVELOPMENT.md, the format docs, the module docs and the expect messages name crates/formats/. Fixes #443 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01CxwRK8xYPsMDBiMkySxcHf --- .dockerignore | 1 - .github/codeql/codeql-config.yml | 12 +++--- .github/scripts/ci-changes.sh | 2 +- .github/workflows/codeql.yml | 2 +- AGENTS.md | 22 +++++----- Cargo.toml | 4 +- DEVELOPMENT.md | 29 +++++++------- README.md | 10 ++--- crates/core/Cargo.toml | 2 +- crates/core/src/lib.rs | 2 +- crates/core/tests/group_read.rs | 2 +- crates/core/tests/hosted_read.rs | 2 +- crates/core/tests/hosted_write.rs | 2 +- crates/core/tests/proxy_read.rs | 2 +- crates/core/tests/retention_worker.rs | 2 +- crates/e2e/Cargo.toml | 2 +- {plugins => crates/formats}/README.md | 8 ++-- {plugins => crates/formats}/nuget/Cargo.toml | 4 +- {plugins => crates/formats}/nuget/plugin.toml | 2 +- {plugins => crates/formats}/nuget/src/lib.rs | 0 .../formats}/nuget/src/model.rs | 0 .../formats}/nuget/src/nuspec.rs | 0 .../formats}/nuget/src/protocol.rs | 0 .../formats}/nuget/src/upstream.rs | 0 .../formats}/nuget/src/version.rs | 0 .../formats}/nuget/tests/parse.rs | 2 +- .../formats}/nuget/tests/protocol.rs | 0 .../formats}/nuget/tests/upstream.rs | 0 {plugins => crates/formats}/raw/Cargo.toml | 4 +- {plugins => crates/formats}/raw/plugin.toml | 2 +- {plugins => crates/formats}/raw/src/form.rs | 0 {plugins => crates/formats}/raw/src/lib.rs | 0 .../formats}/raw/src/listing.rs | 0 .../formats}/raw/src/sidecar.rs | 0 .../formats}/raw/tests/protocol.rs | 0 crates/plugin-api/src/lib.rs | 6 +-- crates/plugin/src/lib.rs | 10 ++--- crates/plugin/src/manifest.rs | 2 +- crates/server/Cargo.toml | 9 +++-- crates/server/src/files/mod.rs | 5 ++- crates/server/src/plugins.rs | 10 ++--- crates/server/tests/nuget_publish.rs | 6 +-- docs/format-plugins.md | 40 ++++++++++--------- docs/raw.md | 5 ++- fixtures/nuget/make_fixtures.py | 2 +- 45 files changed, 111 insertions(+), 104 deletions(-) rename {plugins => crates/formats}/README.md (78%) rename {plugins => crates/formats}/nuget/Cargo.toml (84%) rename {plugins => crates/formats}/nuget/plugin.toml (88%) rename {plugins => crates/formats}/nuget/src/lib.rs (100%) rename {plugins => crates/formats}/nuget/src/model.rs (100%) rename {plugins => crates/formats}/nuget/src/nuspec.rs (100%) rename {plugins => crates/formats}/nuget/src/protocol.rs (100%) rename {plugins => crates/formats}/nuget/src/upstream.rs (100%) rename {plugins => crates/formats}/nuget/src/version.rs (100%) rename {plugins => crates/formats}/nuget/tests/parse.rs (99%) rename {plugins => crates/formats}/nuget/tests/protocol.rs (100%) rename {plugins => crates/formats}/nuget/tests/upstream.rs (100%) rename {plugins => crates/formats}/raw/Cargo.toml (82%) rename {plugins => crates/formats}/raw/plugin.toml (86%) rename {plugins => crates/formats}/raw/src/form.rs (100%) rename {plugins => crates/formats}/raw/src/lib.rs (100%) rename {plugins => crates/formats}/raw/src/listing.rs (100%) rename {plugins => crates/formats}/raw/src/sidecar.rs (100%) rename {plugins => crates/formats}/raw/tests/protocol.rs (100%) diff --git a/.dockerignore b/.dockerignore index 4acb995d..5fa61f0f 100644 --- a/.dockerignore +++ b/.dockerignore @@ -4,7 +4,6 @@ # Rust build artifacts (the builder stage runs its own `cargo build`). /target -/plugins/*/target **/*.rs.bk # Node / pnpm (the web build stage runs `pnpm install` fresh from the lockfile). diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml index 8f6c02cf..e9febddb 100644 --- a/.github/codeql/codeql-config.yml +++ b/.github/codeql/codeql-config.yml @@ -7,12 +7,12 @@ name: Forge paths-ignore: - # Rust: every crate's integration tests (crates/*/tests, plugins/*/tests, - # xtask/tests), the end-to-end crate with its mock IdPs and upstreams, and - # the test helpers behind the `testing` feature. The unit tests inside the - # source files (`#[cfg(test)] mod tests`) cannot be left out by path: the - # workflow extracts the Rust code with `cfg(test)` off instead, as a release - # build compiles it. + # Rust: every crate's integration tests (crates/*/tests, + # crates/formats/*/tests, xtask/tests), the end-to-end crate with its mock + # IdPs and upstreams, and the test helpers behind the `testing` feature. The + # unit tests inside the source files (`#[cfg(test)] mod tests`) cannot be + # left out by path: the workflow extracts the Rust code with `cfg(test)` off + # instead, as a release build compiles it. - "**/tests" - crates/e2e - crates/plugin/src/testing.rs diff --git a/.github/scripts/ci-changes.sh b/.github/scripts/ci-changes.sh index cd6d52de..40dcaf2d 100755 --- a/.github/scripts/ci-changes.sh +++ b/.github/scripts/ci-changes.sh @@ -51,7 +51,7 @@ declare -A CI_JOB_GROUP=( # reads. RUST=( Cargo.toml Cargo.lock rust-toolchain.toml .cargo ':(exclude).cargo/audit.toml' - crates plugins xtask migrations .sqlx fixtures + crates xtask migrations .sqlx fixtures web/index.html docker/forge.docker.toml .github/scripts/test-parallel.sh .github/scripts/test-timings-runner.sh .github/scripts/test-timings-summary.sh .github/scripts/oci-e2e.sh diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index e9b93bd4..3c2fb0c6 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -84,7 +84,7 @@ jobs: if touches web ':(glob)**/*.js' ':(glob)**/*.cjs' ':(glob)**/*.mjs' ':(glob)**/*.ts' ':(glob)**/*.tsx'; then languages+=(javascript-typescript) fi - if touches Cargo.toml Cargo.lock rust-toolchain.toml .cargo .sqlx crates plugins xtask; then + if touches Cargo.toml Cargo.lock rust-toolchain.toml .cargo .sqlx crates xtask; then languages+=(rust) fi diff --git a/AGENTS.md b/AGENTS.md index 28a00766..31139854 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -50,17 +50,17 @@ request: administrators run Forge from the browser, and a setting that only (`/manage/realms`) offers the same. When a feature ships without its screen, its pull request's description says why. -## Formats: every format plugin lives under plugins/ - -A package format is a crate in `plugins//`, the directory named after its -format id, linked into the server like `plugins/nuget` (Forge loads no -WebAssembly plugins). Never add a format as a crate under `crates/` (no -`crates/format-`) or as a module of the server: it implements -`FormatPlugin` with the layout [plugins/README.md](plugins/README.md) -describes. Docker / OCI (`crates/format-oci`) and raw -(`crates/server/src/raw/`) are not there yet and are to move, to -`plugins/docker` and `plugins/raw`, each in a pull request of its own; until -then they are not the model for a new format. +## Formats: every format lives under crates/formats/ + +A package format is a crate in `crates/formats//`, the directory named +after its format id, linked into the server like `crates/formats/nuget` +(Forge loads no WebAssembly plugins). Never add a format as a crate elsewhere +under `crates/` (no `crates/format-`) or as a module of the server: it +implements `FormatPlugin` with the layout +[crates/formats/README.md](crates/formats/README.md) describes. Docker / OCI +(`crates/format-oci`) is not there yet and is to move to +`crates/formats/docker`, in a pull request of its own; until then it is not +the model for a new format. ## Changelog: a fragment per change, never CHANGELOG.md diff --git a/Cargo.toml b/Cargo.toml index 8a363f31..1e850dec 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -11,8 +11,8 @@ members = [ "crates/format-oci", "crates/server", "crates/e2e", - "plugins/nuget", - "plugins/raw", + "crates/formats/nuget", + "crates/formats/raw", "xtask", ] diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 95a9c84e..3d117050 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -241,8 +241,8 @@ password: `docker login localhost:8421 -u admin@example.com`, then `docker push localhost:8421/docker-hosted/team/app:1`. See [docs/docker.md](docs/docker.md). -Raw repositories (`"format":"raw"`, the built-in plugin `plugins/raw`) hold -files under arbitrary paths, served under `/raw/`: `curl -u +Raw repositories (`"format":"raw"`, the built-in plugin `crates/formats/raw`) +hold files under arbitrary paths, served under `/raw/`: `curl -u admin@example.com: --upload-file x.tgz localhost:8421/raw/binaries/tools/x.tgz`, a proxy of any HTTP(S) base URL (`"remote_url":"https://github.com///releases/download/"`), and @@ -250,16 +250,17 @@ groups. See [docs/raw.md](docs/raw.md). ### Repositories of other formats (format plugins) -Every package format is a format plugin, a crate in [`plugins/`](./plugins) -linked into the server (see [docs/format-plugins.md](./docs/format-plugins.md)); -NuGet is the one there is. A new format goes in `plugins/` too, never in a -`crates/format-` crate or the server. Docker / OCI (`crates/format-oci`) -and raw (`crates/server/src/raw/`) are not there yet and are to move to -`plugins/docker` and `plugins/raw`. A repository of a plugin format is created -with `"format": ""` (hosted, proxy or group; a plugin format has proxies -when it implements the upstream side) and served under `/repository//` -unless the plugin names another mount (NuGet's is `/v3`). Helm charts are -pushed to docker repositories, see [docs/docker.md](docs/docker.md#helm-charts). +Every package format is a format plugin, a crate in +[`crates/formats/`](./crates/formats) linked into the server (see +[docs/format-plugins.md](./docs/format-plugins.md)): NuGet and raw. A new +format goes in `crates/formats/` too, never in a `crates/format-` +crate or the server. Docker / OCI (`crates/format-oci`) is not there yet and +is to move to `crates/formats/docker`. A repository of a plugin format is +created with `"format": ""` (hosted, proxy or group; a plugin format has +proxies when it implements the upstream side) and served under +`/repository//` unless the plugin names another mount (NuGet's is +`/v3`). Helm charts are pushed to docker repositories, see +[docs/docker.md](docs/docker.md#helm-charts). ## Configuration @@ -843,7 +844,7 @@ It sorts the change into groups of paths: | Group | Paths | Jobs | |---|---|---| -| Rust | `crates/`, `plugins/`, `xtask/`, `Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml`, `.cargo/` (but `audit.toml`), `migrations/`, `.sqlx/`, `fixtures/`; the files the code or its tests read from outside the crates: `web/index.html`, `docker/forge.docker.toml`; and the scripts the jobs run: `test-parallel.sh`, `test-timings-runner.sh`, `test-timings-summary.sh`, `oci-e2e.sh`, `raw-e2e.sh` | `fmt`, `clippy`, `plugins`, `test`, `oci-e2e`, `raw-e2e`, `e2e`, `web-e2e` | +| Rust | `crates/`, `xtask/`, `Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml`, `.cargo/` (but `audit.toml`), `migrations/`, `.sqlx/`, `fixtures/`; the files the code or its tests read from outside the crates: `web/index.html`, `docker/forge.docker.toml`; and the scripts the jobs run: `test-parallel.sh`, `test-timings-runner.sh`, `test-timings-summary.sh`, `oci-e2e.sh`, `raw-e2e.sh` | `fmt`, `clippy`, `test`, `oci-e2e`, `raw-e2e`, `e2e`, `web-e2e` | | Changelog | `docs/changes/` | `fmt` | | Release | what xtask's tests read from outside `xtask/`: the version lines `cargo xtask release` sets (`web/package.json`, `deploy/kubernetes/base/kustomization.yaml`, `deploy/helm/forge/Chart.yaml`) and `release-channels.sh` | `fmt` (which runs xtask's tests) | | Audit | `Cargo.lock`, `.cargo/audit.toml` | `audit` | @@ -995,7 +996,7 @@ only when a change touches its code: | Language | Paths | |---|---| -| `rust` | `crates/`, `plugins/`, `xtask/`, `Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml`, `.cargo/`, `.sqlx/` | +| `rust` | `crates/`, `xtask/`, `Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml`, `.cargo/`, `.sqlx/` | | `javascript-typescript` | `web/`, and every `.js`, `.cjs`, `.mjs`, `.ts` and `.tsx` file | | `actions` | `.github/workflows/`, and every `action.yml` and `action.yaml` | diff --git a/README.md b/README.md index b2e02064..d8d56b24 100644 --- a/README.md +++ b/README.md @@ -53,16 +53,16 @@ A Cargo workspace of small crates plus the web SPA: | `format-oci` | `forge-oci` | Docker / OCI registry: the distribution-spec `/v2` API, manifests, registry tokens, hosted/proxy/group resolution. | | `plugin-api` | `forge-plugin-api` | The format-plugin contract: the `FormatPlugin` trait and its types. | | `plugin` | `forge-plugin` | The host side of format plugins: an installed plugin and its adapter to the repository machinery. | +| `formats/` | `forge-format-*` | Format plugins, built into the server: `nuget`, `raw`. | | `server` | `forge-server` | Axum HTTP server: the format plugins' feeds (NuGet at `/v3`, raw files at `/raw`), the file store of files formats, the Docker registry, `/admin` + `/auth` REST, background workers, static SPA serving. | | `e2e` | `forge-e2e` | End-to-end suite (testcontainers + mock upstream/IdP). | | `xtask` | — | The task runner (`cargo xtask …`); replaces Make/Just. | -| `plugins/` | `forge-format-*` | Format plugins, built into the server: `nuget`, `raw`. | | `web/` | — | TanStack Router SPA (pnpm + Vite), built into `web/dist/`. | -**Every format is a plugin in `plugins/`.** A package format is a crate in -`plugins//` implementing `FormatPlugin`, linked into the server. Docker / -OCI (`crates/format-oci`) and raw (in `forge-server`) are not there yet and -are to move to `plugins/docker` and `plugins/raw`. +**Every format is a plugin in `crates/formats/`.** A package format is a +crate in `crates/formats//` implementing `FormatPlugin`, linked into the +server. Docker / OCI (`crates/format-oci`) is not there yet and is to move to +`crates/formats/docker`. **Single-origin serving.** `forge-server` mounts each format plugin's feed (NuGet at `/v3`, others at `/repository`), the Docker / OCI registry (`/v2`), diff --git a/crates/core/Cargo.toml b/crates/core/Cargo.toml index e7c07649..4671f1d8 100644 --- a/crates/core/Cargo.toml +++ b/crates/core/Cargo.toml @@ -69,7 +69,7 @@ tokio = { workspace = true, features = ["rt-multi-thread", "macros", "fs", "io-u forge-db = { path = "../db" } # NuGet, the format most integration tests use: the built-in plugin and its # Format (a dev-only cycle through forge-plugin, which Cargo permits). -forge-format-nuget = { path = "../../plugins/nuget" } +forge-format-nuget = { path = "../formats/nuget" } forge-plugin = { path = "../plugin" } argon2 = { workspace = true, features = ["password-hash"] } serde_json = { workspace = true } diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index 9724c81a..0144aa0d 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -1,7 +1,7 @@ //! `forge-core`: format-agnostic domain logic for Forge. //! //! This crate sits above [`forge_db`] and below the format plugins -//! (`plugins/*`) and the server. It defines: +//! (`crates/formats/*`) and the server. It defines: //! //! * [`domain`] — neutral types ([`domain::ParsedUpload`]) produced by parsing //! an uploaded package, the boundary between a concrete format and the rest diff --git a/crates/core/tests/group_read.rs b/crates/core/tests/group_read.rs index cecbee89..0a8f1e57 100644 --- a/crates/core/tests/group_read.rs +++ b/crates/core/tests/group_read.rs @@ -44,7 +44,7 @@ fn nuget() -> forge_plugin::PluginFormat { forge_format_nuget::PLUGIN_TOML, forge_format_nuget::Nuget::default(), ) - .expect("plugins/nuget/plugin.toml is valid"); + .expect("crates/formats/nuget/plugin.toml is valid"); forge_plugin::PluginFormat::new(&plugin) } diff --git a/crates/core/tests/hosted_read.rs b/crates/core/tests/hosted_read.rs index fbb20c1a..f8c3cc80 100644 --- a/crates/core/tests/hosted_read.rs +++ b/crates/core/tests/hosted_read.rs @@ -25,7 +25,7 @@ fn nuget() -> forge_plugin::PluginFormat { forge_format_nuget::PLUGIN_TOML, forge_format_nuget::Nuget::default(), ) - .expect("plugins/nuget/plugin.toml is valid"); + .expect("crates/formats/nuget/plugin.toml is valid"); forge_plugin::PluginFormat::new(&plugin) } diff --git a/crates/core/tests/hosted_write.rs b/crates/core/tests/hosted_write.rs index 5cb6081a..976b5e83 100644 --- a/crates/core/tests/hosted_write.rs +++ b/crates/core/tests/hosted_write.rs @@ -31,7 +31,7 @@ fn nuget() -> forge_plugin::PluginFormat { forge_format_nuget::PLUGIN_TOML, forge_format_nuget::Nuget::default(), ) - .expect("plugins/nuget/plugin.toml is valid"); + .expect("crates/formats/nuget/plugin.toml is valid"); forge_plugin::PluginFormat::new(&plugin) } diff --git a/crates/core/tests/proxy_read.rs b/crates/core/tests/proxy_read.rs index 718c0fca..6bc924b4 100644 --- a/crates/core/tests/proxy_read.rs +++ b/crates/core/tests/proxy_read.rs @@ -61,7 +61,7 @@ fn nuget() -> forge_plugin::PluginFormat { forge_format_nuget::PLUGIN_TOML, forge_format_nuget::Nuget::default(), ) - .expect("plugins/nuget/plugin.toml is valid"); + .expect("crates/formats/nuget/plugin.toml is valid"); forge_plugin::PluginFormat::new(&plugin) } diff --git a/crates/core/tests/retention_worker.rs b/crates/core/tests/retention_worker.rs index a02b7fb1..cdfcc80a 100644 --- a/crates/core/tests/retention_worker.rs +++ b/crates/core/tests/retention_worker.rs @@ -40,7 +40,7 @@ fn nuget() -> forge_plugin::PluginFormat { forge_format_nuget::PLUGIN_TOML, forge_format_nuget::Nuget::default(), ) - .expect("plugins/nuget/plugin.toml is valid"); + .expect("crates/formats/nuget/plugin.toml is valid"); forge_plugin::PluginFormat::new(&plugin) } diff --git a/crates/e2e/Cargo.toml b/crates/e2e/Cargo.toml index 32a23af5..8537d4d2 100644 --- a/crates/e2e/Cargo.toml +++ b/crates/e2e/Cargo.toml @@ -22,7 +22,7 @@ forge-db = { path = "../db" } forge-storage = { path = "../storage" } forge-cache = { path = "../cache" } forge-core = { path = "../core" } -forge-format-nuget = { path = "../../plugins/nuget" } +forge-format-nuget = { path = "../formats/nuget" } forge-server = { path = "../server" } tokio = { workspace = true, features = ["rt-multi-thread", "macros", "fs", "io-util", "time"] } diff --git a/plugins/README.md b/crates/formats/README.md similarity index 78% rename from plugins/README.md rename to crates/formats/README.md index d0b2a277..f545f4d9 100644 --- a/plugins/README.md +++ b/crates/formats/README.md @@ -1,14 +1,14 @@ # Format plugins Package formats for Forge, one crate per directory. Each implements -`FormatPlugin` from [`forge-plugin-api`](../crates/plugin-api) and is linked -into the server. See [docs/format-plugins.md](../docs/format-plugins.md) for +`FormatPlugin` from [`forge-plugin-api`](../plugin-api) and is linked +into the server. See [docs/format-plugins.md](../../docs/format-plugins.md) for the design, the contract and how to write one. | Plugin | Format id | What it serves | | --- | --- | --- | | [`nuget`](nuget) | `nuget` | NuGet V3 feeds at `/v3`: push, restore, search, unlist and relist; hosted, proxy and group repositories | -| [`raw`](raw) | `raw` | Raw files at `/raw`: files under paths, uploads with `PUT` or a form, ranges, checksum sidecars, listings; hosted, proxy and group repositories ([docs/raw.md](../docs/raw.md)) | +| [`raw`](raw) | `raw` | Raw files at `/raw`: files under paths, uploads with `PUT` or a form, ranges, checksum sidecars, listings; hosted, proxy and group repositories ([docs/raw.md](../../docs/raw.md)) | A plugin directory holds: @@ -22,5 +22,5 @@ A plugin directory holds: forge_plugin_api::plugin_toml!();`. The server installs it in `plugins::builtin()` -([`crates/server/src/plugins.rs`](../crates/server/src/plugins.rs)). The +([`crates/server/src/plugins.rs`](../server/src/plugins.rs)). The workspace builds, lints and tests the plugins like every other crate. diff --git a/plugins/nuget/Cargo.toml b/crates/formats/nuget/Cargo.toml similarity index 84% rename from plugins/nuget/Cargo.toml rename to crates/formats/nuget/Cargo.toml index 5a3ae9a8..c91b5f52 100644 --- a/plugins/nuget/Cargo.toml +++ b/crates/formats/nuget/Cargo.toml @@ -9,7 +9,7 @@ publish.workspace = true description = "NuGet V3 feeds for Forge, as a format plugin" [dependencies] -forge-plugin-api = { path = "../../crates/plugin-api" } +forge-plugin-api = { path = "../../plugin-api" } thiserror = { workspace = true } serde = { workspace = true } @@ -20,5 +20,5 @@ zip = { version = "9.0", default-features = false, features = ["deflate"] } quick-xml = { version = "0.42" } [dev-dependencies] -forge-plugin-api = { path = "../../crates/plugin-api", features = ["testing"] } +forge-plugin-api = { path = "../../plugin-api", features = ["testing"] } zip = { version = "9.0", default-features = false, features = ["deflate"] } diff --git a/plugins/nuget/plugin.toml b/crates/formats/nuget/plugin.toml similarity index 88% rename from plugins/nuget/plugin.toml rename to crates/formats/nuget/plugin.toml index 99cb0a5d..8c01953d 100644 --- a/plugins/nuget/plugin.toml +++ b/crates/formats/nuget/plugin.toml @@ -1,4 +1,4 @@ -# A Forge format plugin: NuGet V3 feeds (see plugins/README.md). +# A Forge format plugin: NuGet V3 feeds (see crates/formats/README.md). [plugin] id = "nuget" name = "NuGet" diff --git a/plugins/nuget/src/lib.rs b/crates/formats/nuget/src/lib.rs similarity index 100% rename from plugins/nuget/src/lib.rs rename to crates/formats/nuget/src/lib.rs diff --git a/plugins/nuget/src/model.rs b/crates/formats/nuget/src/model.rs similarity index 100% rename from plugins/nuget/src/model.rs rename to crates/formats/nuget/src/model.rs diff --git a/plugins/nuget/src/nuspec.rs b/crates/formats/nuget/src/nuspec.rs similarity index 100% rename from plugins/nuget/src/nuspec.rs rename to crates/formats/nuget/src/nuspec.rs diff --git a/plugins/nuget/src/protocol.rs b/crates/formats/nuget/src/protocol.rs similarity index 100% rename from plugins/nuget/src/protocol.rs rename to crates/formats/nuget/src/protocol.rs diff --git a/plugins/nuget/src/upstream.rs b/crates/formats/nuget/src/upstream.rs similarity index 100% rename from plugins/nuget/src/upstream.rs rename to crates/formats/nuget/src/upstream.rs diff --git a/plugins/nuget/src/version.rs b/crates/formats/nuget/src/version.rs similarity index 100% rename from plugins/nuget/src/version.rs rename to crates/formats/nuget/src/version.rs diff --git a/plugins/nuget/tests/parse.rs b/crates/formats/nuget/tests/parse.rs similarity index 99% rename from plugins/nuget/tests/parse.rs rename to crates/formats/nuget/tests/parse.rs index b7f73d58..fc01f597 100644 --- a/plugins/nuget/tests/parse.rs +++ b/crates/formats/nuget/tests/parse.rs @@ -10,7 +10,7 @@ use forge_plugin_api::{FormatError, FormatPlugin}; fn fixture(name: &str) -> File { let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); - p.push("../../fixtures/nuget"); + p.push("../../../fixtures/nuget"); p.push(name); File::open(&p).unwrap_or_else(|e| panic!("open fixture {}: {e}", p.display())) } diff --git a/plugins/nuget/tests/protocol.rs b/crates/formats/nuget/tests/protocol.rs similarity index 100% rename from plugins/nuget/tests/protocol.rs rename to crates/formats/nuget/tests/protocol.rs diff --git a/plugins/nuget/tests/upstream.rs b/crates/formats/nuget/tests/upstream.rs similarity index 100% rename from plugins/nuget/tests/upstream.rs rename to crates/formats/nuget/tests/upstream.rs diff --git a/plugins/raw/Cargo.toml b/crates/formats/raw/Cargo.toml similarity index 82% rename from plugins/raw/Cargo.toml rename to crates/formats/raw/Cargo.toml index a8935ec1..948b6082 100644 --- a/plugins/raw/Cargo.toml +++ b/crates/formats/raw/Cargo.toml @@ -9,7 +9,7 @@ publish.workspace = true description = "Raw file repositories for Forge, as a format plugin" [dependencies] -forge-plugin-api = { path = "../../crates/plugin-api" } +forge-plugin-api = { path = "../../plugin-api" } # The JSON listing. serde = { workspace = true } serde_json = { workspace = true } @@ -21,4 +21,4 @@ percent-encoding = "2.3" base64 = { workspace = true } [dev-dependencies] -forge-plugin-api = { path = "../../crates/plugin-api", features = ["testing"] } +forge-plugin-api = { path = "../../plugin-api", features = ["testing"] } diff --git a/plugins/raw/plugin.toml b/crates/formats/raw/plugin.toml similarity index 86% rename from plugins/raw/plugin.toml rename to crates/formats/raw/plugin.toml index be343f08..b26655d9 100644 --- a/plugins/raw/plugin.toml +++ b/crates/formats/raw/plugin.toml @@ -1,4 +1,4 @@ -# A Forge format plugin: raw file repositories (see plugins/README.md). +# A Forge format plugin: raw file repositories (see crates/formats/README.md). [plugin] id = "raw" name = "Raw files" diff --git a/plugins/raw/src/form.rs b/crates/formats/raw/src/form.rs similarity index 100% rename from plugins/raw/src/form.rs rename to crates/formats/raw/src/form.rs diff --git a/plugins/raw/src/lib.rs b/crates/formats/raw/src/lib.rs similarity index 100% rename from plugins/raw/src/lib.rs rename to crates/formats/raw/src/lib.rs diff --git a/plugins/raw/src/listing.rs b/crates/formats/raw/src/listing.rs similarity index 100% rename from plugins/raw/src/listing.rs rename to crates/formats/raw/src/listing.rs diff --git a/plugins/raw/src/sidecar.rs b/crates/formats/raw/src/sidecar.rs similarity index 100% rename from plugins/raw/src/sidecar.rs rename to crates/formats/raw/src/sidecar.rs diff --git a/plugins/raw/tests/protocol.rs b/crates/formats/raw/tests/protocol.rs similarity index 100% rename from plugins/raw/tests/protocol.rs rename to crates/formats/raw/tests/protocol.rs diff --git a/crates/plugin-api/src/lib.rs b/crates/plugin-api/src/lib.rs index 1e4921aa..563dba70 100644 --- a/crates/plugin-api/src/lib.rs +++ b/crates/plugin-api/src/lib.rs @@ -13,9 +13,9 @@ //! implements the protocol over the host's file store ([`Query::File`] … //! [`Query::Remove`], [`Action::ServeFile`]). //! -//! A plugin is a crate under `plugins//` implementing [`FormatPlugin`], -//! with a `plugin.toml` naming it, linked into the server and called -//! directly (NuGet, raw). +//! A plugin is a crate under `crates/formats//` implementing +//! [`FormatPlugin`], with a `plugin.toml` naming it, linked into the server +//! and called directly (NuGet, raw). pub mod files; mod types; diff --git a/crates/plugin/src/lib.rs b/crates/plugin/src/lib.rs index 5e67b9b4..ba100537 100644 --- a/crates/plugin/src/lib.rs +++ b/crates/plugin/src/lib.rs @@ -6,11 +6,11 @@ //! storage, metadata, authentication and authorization, hosted/group //! semantics, retention, the streaming of package bytes. //! -//! A [`Plugin`] is one installed plugin: a crate under `plugins//` linked -//! into the server ([`Plugin::builtin`]) and called directly. The server holds -//! it as an `Arc`; [`PluginFormat`] adapts it to -//! [`forge_core::Format`], so hosted publishing, reads, groups and retention -//! work for it unchanged. +//! A [`Plugin`] is one installed plugin: a crate under `crates/formats//` +//! linked into the server ([`Plugin::builtin`]) and called directly. The +//! server holds it as an `Arc`; [`PluginFormat`] adapts it +//! to [`forge_core::Format`], so hosted publishing, reads, groups and +//! retention work for it unchanged. mod format; pub mod manifest; diff --git a/crates/plugin/src/manifest.rs b/crates/plugin/src/manifest.rs index 1c3a0997..8df5cba4 100644 --- a/crates/plugin/src/manifest.rs +++ b/crates/plugin/src/manifest.rs @@ -1,5 +1,5 @@ //! `plugin.toml`: what a plugin declares about itself, next to its crate's -//! `Cargo.toml` (`plugins//plugin.toml`). +//! `Cargo.toml` (`crates/formats//plugin.toml`). //! //! ```toml //! [plugin] diff --git a/crates/server/Cargo.toml b/crates/server/Cargo.toml index a156baf2..9a857ed9 100644 --- a/crates/server/Cargo.toml +++ b/crates/server/Cargo.toml @@ -22,9 +22,10 @@ forge-core = { path = "../core" } forge-crypto = { path = "../crypto" } forge-plugin = { path = "../plugin" } forge-oci = { path = "../format-oci" } -# The built-in format plugins (plugins/nuget, plugins/raw), linked in. -forge-format-nuget = { path = "../../plugins/nuget" } -forge-format-raw = { path = "../../plugins/raw" } +# The built-in format plugins (crates/formats/nuget, crates/formats/raw), +# linked in. +forge-format-nuget = { path = "../formats/nuget" } +forge-format-raw = { path = "../formats/raw" } tokio = { workspace = true, features = ["rt-multi-thread", "macros", "fs", "io-util", "time", "signal"] } axum = { workspace = true, features = ["multipart"] } @@ -89,7 +90,7 @@ tokio = { workspace = true, features = ["rt-multi-thread", "macros", "fs", "io-u testcontainers = "0.27" testcontainers-modules = { version = "0.15", features = ["postgres"] } reqwest = { workspace = true, features = ["multipart", "query"] } -forge-format-nuget = { path = "../../plugins/nuget" } +forge-format-nuget = { path = "../formats/nuget" } http-body-util = "0.1" tower = { version = "0.5", features = ["util"] } figment = { version = "0.10", features = ["toml", "env", "test"] } diff --git a/crates/server/src/files/mod.rs b/crates/server/src/files/mod.rs index 9debdcc2..406fa7ff 100644 --- a/crates/server/src/files/mod.rs +++ b/crates/server/src/files/mod.rs @@ -1,8 +1,9 @@ //! The file store: the repositories of a format whose `plugin.toml` says //! `storage = "files"` (raw) hold files under paths, like Nexus' raw-hosted, //! raw-proxy and raw-group repositories. The format's plugin owns the -//! protocol (`plugins/raw`, served under its mount by [`crate::plugins`]); -//! this module answers the file queries and actions it asks for: +//! protocol (`crates/formats/raw`, served under its mount by +//! [`crate::plugins`]); this module answers the file queries and actions it +//! asks for: //! //! * `file` and `fetch-file`: [`Files::locate`] — the stored file; a proxy's //! copy, re-validated with its upstream once it is due, or fetched from it diff --git a/crates/server/src/plugins.rs b/crates/server/src/plugins.rs index abff7ba0..fca9d6e5 100644 --- a/crates/server/src/plugins.rs +++ b/crates/server/src/plugins.rs @@ -105,18 +105,18 @@ pub const PASSED_RESPONSE_HEADERS: &[&str] = &[ "vary", ]; -/// The format plugins built into this server: crates under `plugins//`, -/// linked in and always installed (NuGet, raw). NuGet reads archives under -/// `nuget_limits` (`[nuget]`). +/// The format plugins built into this server: crates under +/// `crates/formats//`, linked in and always installed (NuGet, raw). NuGet +/// reads archives under `nuget_limits` (`[nuget]`). pub fn builtin(nuget_limits: forge_format_nuget::NupkgLimits) -> Vec> { vec![ Plugin::builtin( forge_format_nuget::PLUGIN_TOML, forge_format_nuget::Nuget::with_limits(nuget_limits), ) - .expect("plugins/nuget/plugin.toml is valid"), + .expect("crates/formats/nuget/plugin.toml is valid"), Plugin::builtin(forge_format_raw::PLUGIN_TOML, forge_format_raw::Raw) - .expect("plugins/raw/plugin.toml is valid"), + .expect("crates/formats/raw/plugin.toml is valid"), ] } diff --git a/crates/server/tests/nuget_publish.rs b/crates/server/tests/nuget_publish.rs index 6ed3d34a..87b82603 100644 --- a/crates/server/tests/nuget_publish.rs +++ b/crates/server/tests/nuget_publish.rs @@ -1,7 +1,7 @@ //! NuGet's publish endpoints end to end, now that the NuGet plugin -//! (`plugins/nuget`) routes them and the host's plugin dispatcher streams the -//! upload: multipart framing, the upload limit, unlist and relist, and the -//! answers for a method a path does not take. +//! (`crates/formats/nuget`) routes them and the host's plugin dispatcher +//! streams the upload: multipart framing, the upload limit, unlist and +//! relist, and the answers for a method a path does not take. mod common; diff --git a/docs/format-plugins.md b/docs/format-plugins.md index 2ffe6166..4b00ef53 100644 --- a/docs/format-plugins.md +++ b/docs/format-plugins.md @@ -9,10 +9,11 @@ stays Forge's own. That covers storage, metadata, authentication, RBAC and package selectors, hosted, proxy and group repositories, retention, and the streaming of package bytes. -Every plugin is built in: a crate under `plugins//` linked into the -server and called directly, like any other Rust code. Plugins need no -configuration. Forge ships two: **NuGet** ([`plugins/nuget`](../plugins/nuget)) -and **raw files** ([`plugins/raw`](../plugins/raw), +Every plugin is built in: a crate under `crates/formats//` linked into +the server and called directly, like any other Rust code. Plugins need no +configuration. Forge ships two: **NuGet** +([`crates/formats/nuget`](../crates/formats/nuget)) and **raw files** +([`crates/formats/raw`](../crates/formats/raw), [Files formats](#files-formats)). Helm charts are served by the registry ([docker.md](docker.md#helm-charts)). @@ -38,7 +39,7 @@ and **raw files** ([`plugins/raw`](../plugins/raw), A plugin is a workspace crate with a `plugin.toml`: ``` -plugins/nuget/ +crates/formats/nuget/ ├── Cargo.toml ├── plugin.toml # id, name, version, contract version; mount "/v3", kinds hosted/proxy/group ├── src/lib.rs # impl FormatPlugin for Nuget @@ -46,7 +47,7 @@ plugins/nuget/ ├── src/upstream.rs # proxies: reading another V3 feed (upstream) └── tests/ # the protocol against canned answers (forge-plugin-api `testing`) -plugins/raw/ +crates/formats/raw/ ├── Cargo.toml ├── plugin.toml # mount "/raw", storage "files", kinds hosted/proxy/group ├── src/lib.rs # impl FormatPlugin for Raw: paths, methods, upstream URLs @@ -61,7 +62,7 @@ The pieces: | Crate | What it holds | | --- | --- | | [`forge-plugin-api`](../crates/plugin-api) | The contract: the `FormatPlugin` trait and its types, and, behind its `testing` feature, helpers that run a plugin's protocol against canned answers. Depends on nothing but `std`. | -| [`plugins/`](../plugins) | One crate per plugin, implementing `FormatPlugin`. | +| [`crates/formats/`](../crates/formats) | One crate per plugin, implementing `FormatPlugin`. | | [`forge-plugin`](../crates/plugin) | The host side: `Plugin` (an installed plugin: its `plugin.toml` and its implementation) and `PluginFormat`, which adapts it to Forge's `Format` seam. | | [`forge-server`](../crates/server/src/plugins.rs) | The plugins it links in (`plugins::builtin`), the `{mount}/{repo}/...` dispatcher, and the file store files formats keep their files in ([`crates/server/src/files`](../crates/server/src/files)). | @@ -187,8 +188,9 @@ the sidecar rendered from the file's digests, else `fetch-file` in a proxy or group, else a `directory` probe for a redirect; `GET …//` is a `directory` page rendered as HTML or JSON; `PUT` is a `store` of the body; `POST …//` is a `form`, then a `store` of its parts with Nexus' field -names applied; `DELETE` is a `remove`. [plugins/raw](../plugins/raw) is the -whole protocol, in a few hundred lines; the file store is +names applied; `DELETE` is a `remove`. +[crates/formats/raw](../crates/formats/raw) is the whole protocol, in a few +hundred lines; the file store is [`crates/server/src/files`](../crates/server/src/files). ## Proxy repositories @@ -345,9 +347,9 @@ policy (an outbound proxy, a Kubernetes `NetworkPolicy`). ## Writing a plugin -A plugin is a crate in `plugins//` with a `plugin.toml`, a member of the -workspace, depending on `forge-plugin-api` and implementing its trait -([`plugins/nuget`](../plugins/nuget) is a complete one): +A plugin is a crate in `crates/formats//` with a `plugin.toml`, a member +of the workspace, depending on `forge-plugin-api` and implementing its trait +([`crates/formats/nuget`](../crates/formats/nuget) is a complete one): ```rust use forge_plugin_api::{Action, FormatPlugin, Loaded, Repository, Request /* … */}; @@ -408,9 +410,11 @@ rules hold: resolved by the time data reaches you. - **Test the protocol without a server.** `forge-plugin-api`'s `testing` feature runs `handle` and `render` against canned answers to the queries - (`testing::serve`), as [`plugins/nuget/tests/protocol.rs`](../plugins/nuget/tests/protocol.rs) + (`testing::serve`), as + [`crates/formats/nuget/tests/protocol.rs`](../crates/formats/nuget/tests/protocol.rs) does; `testing::converse` runs `upstream` against canned upstream - documents ([`plugins/nuget/tests/upstream.rs`](../plugins/nuget/tests/upstream.rs)). + documents + ([`crates/formats/nuget/tests/upstream.rs`](../crates/formats/nuget/tests/upstream.rs)). - **Test it in a server.** The crate's unit tests run with the workspace. [`crates/server/tests/plugin_contract.rs`](../crates/server/tests/plugin_contract.rs) runs a server with a small plugin of its own installed; copy it for a new @@ -453,7 +457,7 @@ to lift them: ### NuGet as a plugin -NuGet is a built-in plugin: [`plugins/nuget`](../plugins/nuget) +NuGet is a built-in plugin: [`crates/formats/nuget`](../crates/formats/nuget) implements `FormatPlugin`, and the dispatcher serves it at `/v3/{repo}/...` with the same paths, status codes, messages and JSON as before. The pieces it needed from the contract are general ones: a mount, its own asset names, @@ -461,7 +465,7 @@ ranged downloads, paged registrations (`packages-of`), the search options of the V3 search service, and publishing the first file of a multipart body. Its proxies read another V3 feed through the plugin's upstream side -([`src/upstream.rs`](../plugins/nuget/src/upstream.rs)): the service +([`src/upstream.rs`](../crates/formats/nuget/src/upstream.rs)): the service index's resources, the flat container, the registration (bounded to the newest pages a request needs), catalog entries for the package delay of unlisted versions, and the search service. @@ -490,5 +494,5 @@ list directories, take several files in one form, always answer ranges and conditional requests, and their proxies follow redirects and re-validate with `If-None-Match`. 0.4.0 gave the contract a second storage model, files ([Files formats](#files-formats)), so raw is a plugin now -([`plugins/raw`](../plugins/raw)) like NuGet, and what was the raw server -module is the host's file store. +([`crates/formats/raw`](../crates/formats/raw)) like NuGet, and what was the +raw server module is the host's file store. diff --git a/docs/raw.md b/docs/raw.md index e9fc880b..b0ba44e9 100644 --- a/docs/raw.md +++ b/docs/raw.md @@ -7,8 +7,9 @@ have no package manager of their own (the example), installers, firmware images, test fixtures, tool downloads. Files live in repositories of `format: "raw"`, a format plugin built into -Forge like NuGet ([`plugins/raw`](../plugins/raw); `GET /admin/formats` -lists it), served at `https:///raw//`. The plugin +Forge like NuGet ([`crates/formats/raw`](../crates/formats/raw); +`GET /admin/formats` lists it), served at +`https:///raw//`. The plugin speaks the protocol below; the host's file store keeps the files ([format-plugins.md](format-plugins.md#files-formats)). They come in the usual three kinds: diff --git a/fixtures/nuget/make_fixtures.py b/fixtures/nuget/make_fixtures.py index 5bd3e97e..8db942f8 100644 --- a/fixtures/nuget/make_fixtures.py +++ b/fixtures/nuget/make_fixtures.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Generate the NuGet `.nupkg` test fixtures used by `plugins/nuget`. +"""Generate the NuGet `.nupkg` test fixtures of `crates/formats/nuget`. A `.nupkg` is just an OPC zip whose root contains a `.nuspec` manifest. We build small, deterministic archives here so the parsing tests have real (not