From 581a0ca0a4e3e05e5f64905f2220301d065c6b2b Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 18:44:52 -0400 Subject: [PATCH 01/14] docs: plan for page metadata in markfluence.yaml Design for #139: a pages: key mapping a root-relative path to that file's page metadata, so a .md can be published while staying pristine. Lands on #100's loader and makes #29's Action usable -- one step, a glob, no per-file inputs. The core move is making #139's "an entry is a whole frontmatter block that lives elsewhere" literal: an entry carries the same two maps frontmatter.MarkdownFile does. That is not just tidy, it is the only shape that works -- internal/project cannot import internal/pagewidth or internal/labels, both of which reach internal/client, which holds a *project.Cache, so a typed validated Entry would need a broken cycle or a second copy of every field's rules. With the two-map shape, labels.Declared(e.Lists, e.Fields) and pagewidth.Declared(e.Fields) work unchanged in the commands that already call them. Facts from the code that shaped it. linkindex.Build already takes the root and already keys idx.pages by the exact root-relative slash path pages: uses, so the merge needs no signature change. The dialect refuses nesting and must keep refusing it for frontmatter, so nesting becomes a MaxDepth on Dialect rather than a key-name list. create writes each file's frontmatter as that page publishes, inside the loop, so a crash leaves every already-created page recorded -- the manifest writer has to keep that. Landing as two PRs: read and consume first (the whole CI use case), then the nested surgical writer plus create and fix. The writer is the part most likely to need a second pass, and holding the read half hostage to it would mean re-reviewing all of it. Also records a considered-and-rejected alternative -- pages: as a nested hierarchy -- because the question will come back. It cannot replace parent: (which is null, an opaque page or folder id, or a path, and nesting expresses only the third), every consumer looks up by path so a hierarchy is flattened on load anyway, and the correctness argument does not survive: cycles are already rejected in create's topological sort. Two follow-ups came out of the discussion and are filed: #148 (a status command) and #149 (update silently overwriting a Confluence-side edit). --- _plans/039_project-file-pages.md | 614 +++++++++++++++++++++++++++++++ 1 file changed, 614 insertions(+) create mode 100644 _plans/039_project-file-pages.md diff --git a/_plans/039_project-file-pages.md b/_plans/039_project-file-pages.md new file mode 100644 index 0000000..0393fd2 --- /dev/null +++ b/_plans/039_project-file-pages.md @@ -0,0 +1,614 @@ +# Plan: page metadata in `markfluence.yaml` + +Add a `pages:` key mapping a root-relative path to that file's page metadata, so +a `.md` can be published to Confluence while staying pristine. Implements #139, +on top of #100's loader. Makes #29's GitHub Action usable: one step, a glob, no +per-file inputs. + +The motivation is precise about why flags do not cover this. `update README.md +--page-id 12345 --title X` already works, but **flags scale with fields while +the problem scales with files**: `--title`/`--page-id` are single-FILE-only, so +`docs/**/*.md` cannot be expressed at all, and every new frontmatter field +(`labels`, `layout`) would grow its own flag to stay usable from CI. + +## The shape + +A repository whose markdown carries no frontmatter at all: + +``` +myrepo/ +├── markfluence.yaml +├── assets/ +│ └── architecture.png shared: referenced from more than one page +└── docs/ + ├── engineering-docs.md + ├── deploy-runbook.md + ├── on-call-handbook.md + └── on-call-handbook/ + └── pager-flow.png page-specific +``` + +`docs/deploy-runbook.md`, in full — no frontmatter, and nothing markfluence-shaped +in the body either: + +```markdown +# Deploy Runbook + +![System architecture](../assets/architecture.png) + +Escalation is covered in [the on-call handbook](on-call-handbook.md). +``` + +`docs/on-call-handbook.md` references the page-specific image the same way, with +a path that happens not to leave its own directory: +`![](on-call-handbook/pager-flow.png)`. The two images differ in nothing +markfluence sees — both resolve page-relative against the root — which is why +there is one rule and not two. + +The directory is named for its page rather than arbitrarily, and that matters +for the round trip rather than for tidiness. **The convention is `.md` +beside `/`**: `export` writes a page as `.md` plus a `/` +directory holding its children *and* any attachment with no recorded source +path, so a page-scoped image belongs beside its page under the page's own name +(`cmd/export/layout`, `pagedoc.AttachmentDirFor`). A tree that follows it is a +tree `export` could have produced, which is what lets `export` and `update` be +used on the same tree rather than on two shapes of it. + +`export` derives each stem from the *title*'s slug, which is why every file +above is spelled the way it is — `on-call-handbook.md` rather than `oncall.md`, +`engineering-docs.md` rather than `index.md`. So this tree is not merely +export-shaped by coincidence: it is exactly what `export` would write for these +three pages, which is the strongest version of the round-trip property and the +one worth holding the example to. + +A hand-authored tree is not *required* to follow it — nothing resolves a path +through a title, and the only relationship markfluence needs is the attachment +directory tracking its page's stem. What a divergent stem costs is narrower than +a rename, and worth stating precisely: `export` never renames anything, it +writes the path it computed. So exporting a page whose file is called +`oncall.md` produces `on-call-handbook.md` *beside* it — a second copy, not an +update of the first, since the skip-if-present check tests the path `export` +computed and nothing told it the two are the same page. + +The project file is where every markfluence-shaped thing about this tree lives: + +```yaml +# Marks the root of a markfluence project. Image and link paths are recorded +# relative to this directory. https://github.com/mozilla/markfluence + +# Project-wide defaults (#100). +space: ENG +page_width: max + +# Per-file page metadata (this issue). Keys are root-relative paths. +pages: + docs/engineering-docs.md: + title: Engineering Docs + page_id: 12345 + labels: [howto] + + docs/deploy-runbook.md: + title: Deploy Runbook + parent: docs/engineering-docs.md + page_id: 12346 + labels: [runbook, ci/cd] + + docs/on-call-handbook.md: + title: On-call Handbook + parent: docs/engineering-docs.md + page_id: 12347 + page_width: wide +``` + +Six things to read off it, each of which a decision below has to hold up: + +- **An entry is a frontmatter block, moved.** The same field names, the same + value domains, the same canonical order (`title`, `space`, `parent`, + `page_id`, then the rest alphabetically — `fieldOrder`, fact 7). There is + deliberately **no `path: ` shorthand**: one form, not two. +- **A field an entry omits falls back to the project-wide setting.** + `docs/engineering-docs.md` has no `space:` and no `page_width:`, so it takes `ENG` and + `max` from the keys above it. An entry sits exactly where frontmatter sits in + #100's chain, since it *is* frontmatter that lives elsewhere — but "sits where + frontmatter sits" is not a four-level precedence, and writing it as one is + wrong. Frontmatter and an entry are **two spellings of one level**, so when + both speak the rule is not "the higher wins" but D6's grading: a coordinate + disagreement is an error and a soft one is a warning. Only *below* them is + there precedence, and only for a field neither declares. +- **`parent` is a relative `.md` path here**, exactly as in frontmatter, and is + resolved the same way. The `page_id` form works too; nothing new. +- **Keys are root-relative and lexical** (D3), which is the same key space + `linkindex` already uses (fact 1) — so `docs/deploy-runbook.md` linking to + `engineering-docs.md` resolves through the manifest with no new path vocabulary. +- **The `.md` files stay pristine.** That is the point: a README or a docs tree + with other readers keeps no markfluence keys, and CI publishes it anyway. +- **Images need nothing from the manifest, and must not get anything.** + `../assets/architecture.png` is resolved page-relative against the root and + published as the attachment `architecture.png`, with the root-relative + `assets/architecture.png` recorded in its comment as the `Source` + (`images.go:148`, `AttachmentFilename`, **L3**). None of that reads an entry, + and there is deliberately no `attachments:` key: attachments are *discovered + from the body*, never declared, and a key that let the two disagree would be a + new way to publish an image the document does not reference. + +Then the whole GitHub Action is one step with no matrix and no per-file inputs: + +```yaml +- uses: mozilla/markfluence@v1 + with: { command: update, files: "docs/**/*.md", url: …, username: … } + env: { CONFLUENCE_TOKEN: ${{ secrets.CONFLUENCE_TOKEN }} } +``` + +Two things about the image are worth noticing before the decisions, because both +fall out rather than needing designing. + +**The manifest's key space and an attachment's `Source` are the same space** — +root-relative, slash-form, anchored on the directory holding `markfluence.yaml`. +So `docs/engineering-docs.md` as a key and `assets/architecture.png` in an +attachment comment are relative to the same thing, and a reader of the project file never +has to ask which. That is also why the manifest needs no path vocabulary of its +own (D3, fact 1). + +**A shared asset above a page works here precisely because a manifest project +always has a project file.** `_plans/025` recorded that the shared-parent layout +is "repaired by this model *only when a project file exists*" — with none, a +page's root is its own directory and `../assets/architecture.png` publishes as +`IMAGE BROKEN`. A project using `pages:` has one by construction, so the layout +the README endorses is always available to it. Nothing to implement; worth +knowing before someone reads the `../` above and worries. + +A mixed tree is equally legal and is what makes migration incremental (D6): a +file may carry frontmatter, or have an entry, or both with the same values. + +## The core move: an entry *is* a parsed frontmatter block + +#139 says an entry is "a whole frontmatter block that lives elsewhere — the same +keys, the same values, the same validation, the same canonical order." The way to +make that literal rather than aspirational is to give an entry the **same two +maps `frontmatter.MarkdownFile` carries**: + +```go +type Entry struct { + Fields map[string]string // like MarkdownFile.Frontmatter + Lists map[string][]string // like MarkdownFile.Lists +} +``` + +Everything follows from that, including the thing that would otherwise sink the +design. `internal/project` **cannot** import `internal/pagewidth` or +`internal/labels` — both reach `internal/client`, which holds a `*project.Cache` +— so a validated, typed `Entry` would need either a broken cycle or a second +copy of every field's rules. With the two-map shape it needs neither: +`labels.Declared(e.Lists, e.Fields)`, `pagewidth.Declared(e.Fields)` and +`pageref.IsDigits(e.Fields["page_id"])` all work on an entry unchanged, in the +commands that already call them. + +## What the code says — read 2026-09-12 + +1. **`linkindex.Build` already takes the root.** `Build(root *project.Root)` + (`linkindex.go:53`) reads each sibling's frontmatter for `page_id`/`title` + (`linkindex.go:68-76`) and keys `idx.pages` by the **root-relative, + slash-separated path** — the identical key space `pages:` uses. So the merge + needs no signature change: `root.Config` is already in hand. This is #139's + "sharp one" and it is sharp in consequence, not in plumbing: miss it and every + cross-document link in a manifest project degrades to the "exists on disk, not + published yet" warning and republishes as plain text. + +2. **The dialect refuses nesting, and must keep refusing it for frontmatter.** + `plainScalar` rejects a `*ast.MappingNode`, pinned by + `TestParseErrorWordingIsExact`'s `nested value` case. `pages:` needs two + levels (`pages:` → path → fields), so nesting has to be opt-in per `Dialect`. + +3. **Both write paths are flat and identical in shape.** `fix` (`fix.go:198-208`) + and `create` (`create.go:988-992`) each call + `frontmatter.UpdateField`/`UpdateListField` then `Normalize`, on the file's own + content. Writing an entry needs the nested analogue, and there is none. + +4. **`create` writes each file's frontmatter as that page is published** + (`create.go:556`, inside the per-record loop), not once at the end. A crash + therefore leaves every already-created page recorded. The manifest must keep + that property (D10). + +5. **`updateResult` already has a `skipped` status** and `createResult` has + `not_created`; `update`'s required field list has 16 entries and no + `omitempty` anywhere. So `metadata_source` is a new **required** field on both, + and #139's "unmanaged file is skipped" reuses an existing status value. + +6. **`update`'s three doomed flags are load-bearing in four places**: + `titleFlag`/`pageIDFlag`/`pageWidthFlag` declarations (`update.go:33-35`), + `init` (85-90), `overrideNeedsSingleFile` (96, 369) and the two resolvers + (183, 216). Removing them deletes `overrideNeedsSingleFile` outright. + +7. **The canonical field order is already shared.** `fieldOrder` + + `keyLess`/`fieldRank` (`frontmatter.go:63-110`) is "the single source of + frontmatter field order across all commands", so an entry reuses it and + #139's "the same canonical order" costs nothing. + +## Decisions + +**D1 — An entry is `Entry{Fields, Lists}`,** for the reason above. `Config` +gains `Pages map[string]Entry`, keyed by normalized path. + +**D2 — The reader gains *depth*, not key names.** `Dialect` gains +`MaxDepth int` (0 = flat, which is what `blockReader` keeps, so every +frontmatter message and refusal is untouched — the property the #100 review +caught me breaking once already). `Item` gains `Map []Item`, populated for a +mapping value within the allowed depth. The project dialect uses 2. + +Nesting support does not loosen any per-key rule: a mapping where the `settings` +table says scalar is refused by the table, with the table's own message. Depth +rather than a name list also keeps the dialect from learning about `pages`, +matching the existing rule that the parser learns a kind, not a name. + +**D3 — Path keys are lexical, root-relative, slash-form, byte-compared,** and an +argument is normalized the same way before lookup. A mismatch now means a silent +*skip* (D7), so the rule has to be exact. + +- **Lexical only, no `EvalSymlinks`,** matching `withinRoot`/`attachfile.Resolve`/ + `destPath`. A symlinked `docs/` is legitimate, and resolving it would make a key + depend on the checkout's layout, which **L2** forbids. +- A key **escaping the root** (`../elsewhere.md`) is a **load-time** error: the + project file declares the project's boundary. +- **Two keys normalizing to one path** (`docs/a.md` and `./docs/a.md`) is a + **load-time** error naming both; YAML only catches literal duplicates. +- **No case folding.** Known limit, documented beside `pageslug`'s NFD/NFC note: + on a case-insensitive filesystem `Docs/a.md` opens the file but matches no + `docs/a.md` key, so it reads as unmanaged and is skipped. + +Both load-time errors are deliberately *not* per-file: an escaping or duplicate +key means the manifest's structure is wrong, not that one entry is bad. + +**D4 — Load validates structure; a value is validated where it is consumed, and +only for a file the invocation named.** This is #100's structure/vocabulary line +plus #139's scoping constraint, and they agree. At load: `pages:` is a mapping of +mappings, each key is a legal path, each field name is known, each field's value +has the right shape (scalar vs list). Reported by the command, for its own files +only: a non-numeric `page_id`, an invalid `page_width`, an invalid label, an +empty `title`. + +An **unknown field name inside an entry** is load-time, not per-file, and that +is a judgment call worth stating: it is the same typo class as an unknown +top-level key (`titel:` silently ignored is wrong forever, for that page), and it +means the manifest was written against a different markfluence — which #100 +settles as fatal. A bad *value* is per-file because one bad entry must not block +every invocation in the repo. + +**D5 — Resolution lives in a new package, `internal/pagemeta`.** +`Resolve(key string, mf *frontmatter.MarkdownFile, root *project.Root) +→ (Resolved, error)`, where `Resolved` carries the merged `Fields`/`Lists`, a +`Source` (`frontmatter` / `manifest` / `none`), and the disagreements. + +A package rather than a helper because `update`, `create`, `fix`, `check` **and +`linkindex`** all need the identical merge, and a per-command copy is exactly how +two commands come to publish one file to two different pages. It imports +`frontmatter` and `project` and nothing else, so it stays out of every cycle. + +**D6 — Both locations are legal and agreement is silent; disagreement is graded +by what it can destroy.** #139's table, unchanged: + +| field | on disagreement | | +|---|---|---| +| `page_id`, `space`, `parent` | **error, that file fails** | the clobber case: a `page_id` pasted from an old file publishes over a live page | +| `title`, `page_width`, `labels` | **warning**, frontmatter wins | visible and recoverable, and frontmatter-wins is #100's chain | + +`create` preflights every file, so a coordinate disagreement in **any** file +aborts the whole batch; `update` fails only that file. + +Silent agreement is what makes migration incremental: copy values into the +manifest, verify, delete them from the files later, with everything working +throughout. There is no project "mode" and no all-or-nothing switch. + +**D7 — A file with no metadata anywhere is `skipped`, `ok: true`, exit 0.** +Behavior change: `update` errors on that today. Repositories legitimately hold +markdown that is not published, drafts are a normal state, and a glob-driven CI +run must not go red because someone added a file. A file that *is* registered but +whose entry lacks `page_id` still **errors** — someone claimed it and `create` +has not run. + +**D8 — `update` loses `--title`, `--page-id`, `--page-width`.** The rule is +*flags describe the run; files describe the page*. `--message`/`--force`/ +`--dry-run` stay, being invocation metadata and behavior. `--page-width` is the +real loss, being the only batch-ok one, and #100's project-wide `page_width:` +covers it better and permanently. markfluence is unreleased, so there is no +migration to design. + +`create` **keeps** its flags, for a principled reason rather than squeamishness: +it is the verb that *establishes* metadata and then persists it, so +`--space`/`--parent`/`--title` are how an entry that does not exist yet gets +bootstrapped. `update` only ever consumes metadata, so it should have no way to +invent any. This also settles permanently what #138 kept bumping into: there is +no `--labels`, and no flag for whatever field comes after it. + +**D9 — New metadata is written wherever that file's metadata already is,** and +with none, to the manifest if the project has a `pages:` block, otherwise into +the file's frontmatter. Inferred, no flag: a project that has chosen the manifest +never accidentally grows frontmatter, and a project without one behaves exactly +as today. + +**D10 — `create` writes each entry as that page is published,** read-modify-write +per page, not one deferred write at the end. Fact 4 is the reason: a crash must +leave every already-created page recorded, which is the property the per-file +frontmatter write already has. N reads and writes of one small file is the price, +and N is the number of pages a person creates by hand. + +**D11 — The manifest writer is `project.SetPageEntry`,** a surgical nested edit +holding the same contract `frontmatter`'s writer holds: an existing key keeps its +own key node (so a preceding blank line and comments survive), only values are +replaced, and the result is **re-read and verified** before it is written. It is +built on an exported node-level helper from `internal/frontmatter` rather than a +second serializer, so quoting, the `page_id`/`parent` typing, and the +write-then-re-read fallback exist in one copy. + +**D12 — An entry naming no file on disk is not an error and is not reported.** +A branch that deleted a file, or a sparse checkout, is legitimate, and every +diagnostic here is scoped to the files an invocation names — which never includes +a file that is not there. Auditing a manifest against a tree is a real want and a +separate verb; see Follow-ups. + +**D13 — Land this as two PRs on one issue: read, then write** (settled +2026-09-12). + +- **PR 1 — read and consume.** The dialect's depth, `Config.Pages`, path keys, + `internal/pagemeta`, `linkindex`, `update` (flag removal, manifest lookup, + `skipped`), `check`'s lint, `metadata_source`. This is the entire CI use case + and it is independently useful, testable and reviewable: a person hand-writes + entries, CI updates from them. +- **PR 2 — write.** `frontmatter`'s exported node helper, `project.SetPageEntry`, + `create` persisting entries, `fix` migrating inline keys. + +The reason to split is not size alone: PR 2's nested surgical writer is the part +most likely to need a second pass, and holding the read half hostage to it would +mean re-reviewing all of it. The accepted cost is that between the two, +#139's bootstrap flow needs a `page_id` copied by hand out of `create`'s output. + +## Considered and rejected: `pages:` as a hierarchy + +`pages:` is a **map** of maps, keyed by path — not a list, because the key *is* +the lookup and a list would permit duplicates with no way to ask "what is the +metadata for `docs/foo.md`?". The natural follow-on question is whether it +should instead nest, mirroring the parent/child relationships it currently +states with a `parent:` field: + +```yaml +pages: + docs/engineering-docs.md: + title: Engineering Docs + page_id: 12345 + children: + docs/deploy-runbook.md: + title: Deploy Runbook + page_id: 12346 +``` + +**No.** A Confluence space really is a tree and a flat map really does not show +it, so the legibility argument is genuine — but four things weigh against, and +the first two are decisive. + +**It cannot replace `parent:`, only duplicate it for a subset of cases.** +`parent` has three value domains: `null` (a space-root page), an opaque page +**or folder** id, and a relative `.md` path. Nesting expresses only the third. A +page parented to a Cloud folder, or to a page outside the project, must sit at +the top level *and* carry an explicit `parent: ` regardless — so both +mechanisms ship, which is the "one form, not two" #139 already settled when it +refused the `path: ` shorthand. + +**Every consumer looks up by path, so a hierarchy is flattened on load anyway.** +`update`, `linkindex` and `pagemeta` all ask for the metadata at a path, and a +nested document answers that only after being flattened into the map above. That +makes nesting a pure *serialization* preference, bought with an unbounded-depth +reader in place of D2's `MaxDepth 2`, diagnostics that need a path-within-the- +document rather than a line, and a materially harder writer in PR 2 — `create` +would have to locate a parent's `children:` block, create one when absent, and +insert at the right indent, all under D11's write-then-re-read contract, on the +part of the change already most likely to need a second pass. + +**A page tree and a directory tree are not the same tree.** Nothing stops +`docs/a.md` being the child of `docs/sub/b.md`, so YAML nesting would diverge +from directory nesting and be confusing in a new way rather than a familiar one. + +**Worse diffs, in a file both `create` and humans edit.** Flat, adding a page is +a self-contained hunk at one indent level; nested, it edits its parent's block, +and moving a subtree reindents everything beneath it. The same reason `go.mod` +carries a flat `require` block rather than a dependency tree. + +The one correctness argument in nesting's favour does not survive contact with +the code: the failure modes it would make impossible by construction are already +caught. A cycle among in-set parents is rejected by `create`'s topological sort +(`create.go:948`), and a `parent:` naming a file with no metadata fails that +file. Both would only ever have been covered for in-project parents anyway, per +the first reason. + +What the legibility point deserves is a **read-only view, not a storage +format** — see Follow-ups. Storing the tree in order to display the tree is the +expensive way round. + +## Implementation + +### `internal/frontmatter` + +- `Dialect.MaxDepth int`; `Item.Map []Item`. `reader.items` recurses when the + current depth is under the limit and the value is a mapping, and otherwise + behaves exactly as now. `blockReader` sets nothing, so frontmatter is untouched. +- `TestParseErrorWordingIsExact` guards that: its `nested value` case must still + produce the byte-identical message. +- PR 2: an exported way to build a value node for a field (today's unexported + `valueNodeFor` + `readsBackAs` verification) so `project.SetPageEntry` reuses + the typing and the fallback rather than re-deriving them. + +### `internal/project` + +- `Entry{Fields, Lists}`; `Config.Pages map[string]Entry`. +- `settings` gains `"pages": kindMapping` — the kind table earns its shape here, + exactly as #100 predicted. +- `pagekey.go`: normalization (`path.Clean`, slash-form, root-relative), the + escaping-key and duplicate-after-normalization load-time errors, and the + argument-side normalizer commands call before lookup. One copy, because a + mismatch is a silent skip. +- `entryFields` — the known field names and their shapes, which is the only place + the manifest's schema is written down. +- PR 2: `SetPageEntry(path string, e Entry) error` (D11). + +### `internal/pagemeta` (new) + +`Resolve`, `Resolved{Fields, Lists, Source, Warnings}`, and the graded +disagreement (D6). `Source` is what `--json`'s `metadata_source` reports. + +### `internal/linkindex` + +`Build` resolves each walked file's `page_id`/`title` through `pagemeta` rather +than from `mf` alone. A file with an entry and no frontmatter gets a `PageEntry`; +a mixed tree resolves correctly per file throughout a migration. + +### Commands + +- **`update`** — flags removed (fact 6, `overrideNeedsSingleFile` deleted), + metadata from `pagemeta`, `skipped` for an unmanaged file (D7), coordinate + disagreement fails the file, soft disagreement warns. +- **`check`** — the half-and-half lint: a file carrying inline markfluence keys + in a project that also uses `pages:` is a **warning**, not an error, so "no + half-and-half" is enforceable without becoming a wall someone hits + mid-migration. Plus the per-file entry-value validation of D4. +- **`create`** (PR 2) — writes a complete entry when the project has a `pages:` + block (D9/D10); flags unchanged. +- **`fix`** (PR 2) — moves a file's inline keys into its entry, as an ordinary + `change`. A convenience rather than a prerequisite, since agreement is legal — + but it is what gives the `check` warning an obvious remedy, and it is read-only + against Confluence, so migrating touches no live page. + +### `--json` and the schema + +`metadata_source` (`"frontmatter"` / `"manifest"` / `null`) on `updateResult` and +`createResult`, required, no `omitempty`. Debugging "why did it publish to *that* +page" in CI otherwise means reproducing the resolution by hand. D7's `skipped` +and D6's warnings surface through the existing `status`/`warnings` fields. + +## Tests + +- `internal/frontmatter`: nesting at, below and past `MaxDepth`; that + `blockReader` still refuses a nested value with the byte-identical message. +- `internal/project`: a `pages:` block read into entries; an unknown entry field; + a wrong-shape field; an escaping key; two keys normalizing to one; path + normalization of every spelling; a scalar `pages:`; `pages: {}`. +- `internal/pagemeta`: all nine field/location combinations — absent, one side, + both agreeing, both disagreeing — for a coordinate and for a soft field; that + `Source` is right in each; that a *blank* value on one side is not a + disagreement. +- `internal/linkindex`: a link to a manifest-only sibling resolves (the + regression #139 names); a mixed tree; frontmatter and entry agreeing. +- `cmd/update`: an unmanaged file is `skipped`/`ok`/exit 0 in a batch whose other + file publishes; a registered file with no `page_id` fails; a coordinate + disagreement fails only that file; a soft one warns and frontmatter wins; + `metadata_source` for each source; the three flags are gone (a test that + `Cmd.Flags().Lookup` returns nil, so their removal is pinned rather than + incidental). +- `cmd/check`: the half-and-half warning; an entry's bad value reported only for + a named file; nothing reported about an unnamed file's entry. +- Schema conformance for both new fields, via each command's own builder. +- PR 2: a `SetPageEntry` round-trip preserving comments and a preceding blank + line; an entry whose value needs quoting; `create` recording page N before + publishing page N+1; `fix` moving keys with the file and manifest agreeing + afterwards. + +## Docs + +`docs/markdown_file.md` (an entry is the same block, elsewhere — the field table +already exists and should not be duplicated), `docs/root-model.md` (`pages:` +beside the settings, the path-key rules), README (the CI flow, the removed +flags), `docs/guarantees.md` (**L2** is why keys are lexical and root-relative), +`docs/github-actions.md` (the one-step workflow this exists for), `CLAUDE.md` +(`internal/pagemeta`, and `internal/project`'s bullet gains `pages:`), and +`markfluence --help` for `update`'s flag removal, which regenerates +`docs/commands/`. + +## Out of scope + +- **Creating pages in CI.** A workflow committing a new `page_id` back to the + repo means `contents: write`, a bot commit per new doc, and a race if two runs + create at once. Page creation stays a human act (#139). +- **Sidecar files (#38)** — superseded for the pristine-markdown use case. +- **`export` writing manifest entries** — `export` creates a fresh tree and + writes frontmatter; whether an exported tree can be manifest-shaped is later. +- **A `--persist-to file|manifest` flag** — D9's inference covers every case + anyone has named. +- **`update` enforcing `space`/`parent`** (#10). Until then a manifest `space` is + read for disagreement detection but changes nothing about where `update` + publishes, since the space comes from the live page. + +## Follow-ups + +- **`markfluence status`: the local tree, plus what `update` would do** (filed + as #148). Three + wants meet here, and one command answers all three. Seeing the page tree a + project declares (the legibility the rejected hierarchy was reaching for). + Asking the questions D12 deliberately makes invisible — manifest entries + naming no file, files under root in neither location, entries whose `page_id` + resolves to nothing. And per-page drift. + + **Offline by default, network by opt-in.** The tree, published-vs-not, + unmanaged files and dangling entries all come from disk, and requiring a token + to look at a tree would be wrong; the precedent is `check`, which is its own + command precisely because it is the offline verb (the first whose `run()` + never constructs a client). A flag adds one GET per page for the drift + columns. + + **What drift can honestly mean is the part to get right.** The answerable + comparison is mtime against the page's last-version timestamp — which is what + `update` already acts on, so `status` would report exactly what `update` would + do. Comparing *content* is the trap `docs/confluence/` warns about: the + converter targets semantic, not byte-for-byte, equivalence, and any save + through the Confluence editor re-serializes through ADF (the + `coalesceSplitMarks` case), so a byte comparison would report changes on pages + nobody touched — and "`body.storage` proves only what was stored, never what + takes effect" is one of the two recorded traps that have each already produced + a confident wrong conclusion. A semantic comparison needs a normalizer nobody + has written and would be a second source of truth about equivalence. + + **Much of the drift half already exists**, which is the honest reason this is + a follow-up and not a gap: `update --dry-run docs/**/*.md` reports + skipped-vs-would-publish per file today, honouring the mtime skip (the skip + returns at `update.go:250-258`, before the dry-run branch, so the forecast is + real). The genuine delta is the hierarchy, the audit facts, and **page newer + than file** — the direction nothing reports at all, since `update` simply + skips it, so a stale local copy is currently silent. + + Not a flag on `children`. `children` asks Confluence what is under a node: it + takes a page or a space, needs credentials, reports folders, and reports live + ids. A local view takes the *root*, needs none of that, has no folders to + report, and has no id for an unpublished page — it would share only the output + shape, and `children`'s argument rule is already "exactly one of PAGE or + --space". + +- **Recording the version markfluence last published, which is what makes "the + page is ahead of your copy" answerable at all** (#148 for the display, **#149** + for the defect it fixes). Timestamps cannot answer it: + `version.createdAt` against mtime says which side was written most recently, + not which side has content the other lacks, and it reports every page as + "ahead" immediately after a successful `update` — publishing sets `createdAt` + to now while the file's mtime is from when it was saved. A `status` column + built on timestamps alone would therefore light up for the wrong reason most + of the time. + + One content property holding the version number markfluence last wrote makes + it exact: a version number increments only when somebody saves, so "live + version > recorded version" means precisely *someone other than markfluence + has written to this page since markfluence last did*. The pattern is already + in the codebase — an attachment records a SHA-256 in its comment and that is + how `SyncAttachments` decides skip-vs-update — and so is the machinery + (`SetContentProperty`/`ListContentProperties`, `page_width`'s two properties + per page). A recorded *hash* of the body would be more granular and is worse + here: an ADF round-trip changes the stored bytes when someone opens the editor + and saves without editing, so a hash reports "changed" where a version number + reports, accurately, "somebody saved it". + + **The display is not the valuable part.** `update` currently avoids clobbering + a UI edit only by accident — page newer than file means skip — but a file + edited *after* the UI edit wins on mtime and overwrites it silently, and + `--force` bypasses the check regardless (`update.go:250-258`). A recorded + version turns that into a real warning: "this page was changed in Confluence + since markfluence last published it; publishing will overwrite that." That is + a safety property rather than a convenience, and it is worth more than the + column that prompted it — filed as **#149**, a bug, separately from #148's + feature. Nothing here is needed for #139. +- `layout:` (#21) and any later field: they should need nothing here, and a test + that an unknown-to-the-test field survives a manifest round-trip is what would + prove it. From a0985fbfeca262cf157faed7d21a9c670e6e94b4 Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 18:46:48 -0400 Subject: [PATCH 02/14] feat(frontmatter): let a dialect allow nested mappings markfluence.yaml's pages: key needs two levels -- pages, then a path, then that page's fields -- where the fenced frontmatter block must keep refusing nesting entirely. So the allowance is a depth on Dialect rather than a list of keys permitted to nest, which keeps the reader learning a kind and not a name: the project file asks for two levels and says so, and a mapping under a key whose own table says scalar is refused by that table with its own message. MaxDepth defaults to zero, which is what blockReader keeps, so every message and refusal the fenced block makes is unchanged -- two tests pin that from both sides. Past the allowance a mapping falls through to the scalar path and is refused there, so a flat dialect behaves exactly as the reader did before nesting existed rather than by a separate branch that could drift from it. Item gains Map, and the nil-ness of List and Map is what says which of the three shapes a value has. Both goccy spellings of a one-key mapping are accepted (MappingValueNode as well as MappingNode, the same split soleMappingPair exists for on the write side), or a single-field entry would read as a broken scalar. --- internal/frontmatter/dialect.go | 80 ++++++++++++++-- internal/frontmatter/dialect_test.go | 137 +++++++++++++++++++++++++++ 2 files changed, 208 insertions(+), 9 deletions(-) diff --git a/internal/frontmatter/dialect.go b/internal/frontmatter/dialect.go index 488de32..4f712a5 100644 --- a/internal/frontmatter/dialect.go +++ b/internal/frontmatter/dialect.go @@ -9,7 +9,8 @@ package frontmatter // about, which is why a second copy of them for a second file would be a // second set of the same bugs: the scalar node-kind whitelist (plainScalar), // the single-line rule (spansLines), every null spelling reading as "", and -// the refusal of anything that is not a flat mapping. +// the refusal of anything that is not a flat mapping -- or, where a dialect +// asks for depth (Dialect.MaxDepth, #139), of anything past it. import ( "errors" @@ -38,20 +39,37 @@ import ( type Dialect struct { Doc string Item string + // MaxDepth is how many levels of nested mapping a value may hold. Zero -- + // the default, and what the fenced block keeps -- means flat: a mapping + // value is refused exactly as any other non-scalar is. + // + // A depth rather than a list of keys allowed to nest, so the reader still + // learns a kind and not a name: markfluence.yaml's `pages:` needs two + // levels (pages -> path -> fields) and says so by asking for two, and a + // mapping under a key whose own table says scalar is refused by that table + // with its own message rather than by the reader (#139). + MaxDepth int } // Item is one key/value pair of a mapping, in source order. // -// List is nil for a scalar value and non-nil (possibly empty) for a sequence, -// which is what distinguishes `labels: []` from an absent key. Line is -// 1-based within the text passed to ReadMapping, and is carried because a -// caller rejecting a key -- an unknown setting in a project file -- has to be -// able to say where it is. +// Exactly one of Value, List and Map describes the value, and the nil-ness of +// the latter two is what says which: List is nil for a scalar and non-nil +// (possibly empty) for a sequence, which is what distinguishes `labels: []` +// from an absent key, and Map is non-nil (possibly empty) for a nested +// mapping, which only arrives within Dialect.MaxDepth. A caller reading +// len(List) or len(Map) instead of testing for nil cannot tell an empty +// collection from a scalar. +// +// Line is 1-based within the text passed to ReadMapping, and is carried +// because a caller rejecting a key -- an unknown setting in a project file -- +// has to be able to say where it is. type Item struct { Key string Line int Value string List []string + Map []Item } // ReadMapping reads text as a single flat YAML mapping, applying every rule @@ -67,7 +85,7 @@ func (d Dialect) ReadMapping(text string) ([]Item, error) { if err != nil { return nil, err } - return r.items(b.mapping) + return r.items(b.mapping, 0) } // reader is the dialect plus the two things that differ per document and are @@ -259,7 +277,13 @@ func spansLines(origin string) bool { } // items reads a mapping into its key/value pairs, in source order. -func (r reader) items(m *ast.MappingNode) ([]Item, error) { +// +// depth is how far down this mapping already is, so a nested mapping is read +// only while there is allowance left. Past the allowance a mapping value falls +// through to r.scalar and is refused there, which is what keeps a flat dialect +// (MaxDepth 0) behaving exactly as it did before nesting existed -- including +// the message. +func (r reader) items(m *ast.MappingNode, depth int) ([]Item, error) { out := make([]Item, 0, len(m.Values)) for _, v := range m.Values { key := v.Key.GetToken().Value @@ -277,6 +301,17 @@ func (r reader) items(m *ast.MappingNode) ([]Item, error) { out = append(out, it) continue } + if depth < r.MaxDepth { + nested, ok, err := r.nested(v.Value, depth) + if err != nil { + return nil, err + } + if ok { + it.Map = nested + out = append(out, it) + continue + } + } s, err := r.scalar(key, v.Value) if err != nil { return nil, err @@ -287,6 +322,33 @@ func (r reader) items(m *ast.MappingNode) ([]Item, error) { return out, nil } +// nested reads a mapping value one level down, reporting whether the value was +// a mapping at all. It is not an error for it not to be: a nesting-capable +// dialect still holds plain scalars at every level, so a non-mapping falls back +// to the scalar path rather than being refused for the shape it does have. +// +// goccy renders a one-key mapping as a MappingValueNode and a multi-key one as +// a MappingNode, the same split soleMappingPair exists for on the write side, +// so both have to be accepted -- treating only the plural form as nesting would +// read a single-field entry as a broken scalar. +func (r reader) nested(n ast.Node, depth int) ([]Item, bool, error) { + var m *ast.MappingNode + switch v := n.(type) { + case *ast.MappingNode: + m = v + case *ast.MappingValueNode: + m = emptyMapping() + m.Values = append(m.Values, v) + default: + return nil, false, nil + } + items, err := r.items(m, depth+1) + if err != nil { + return nil, false, err + } + return items, true, nil +} + // maps reads a mapping into the two maps frontmatter's own callers use: scalars // by key, and sequences by key. // @@ -296,7 +358,7 @@ func (r reader) items(m *ast.MappingNode) ([]Item, error) { // scalar map to hold both would touch every caller for no gain. The parser // learns a kind, not a name: nothing here knows which keys are lists. func (r reader) maps(m *ast.MappingNode) (map[string]string, map[string][]string, error) { - items, err := r.items(m) + items, err := r.items(m, 0) if err != nil { return nil, nil, err } diff --git a/internal/frontmatter/dialect_test.go b/internal/frontmatter/dialect_test.go index a848f95..40e2db8 100644 --- a/internal/frontmatter/dialect_test.go +++ b/internal/frontmatter/dialect_test.go @@ -176,3 +176,140 @@ func TestReadMappingDoesNotApplyFrontmattersScalarOnlyKeys(t *testing.T) { t.Error("frontmatter accepted a list-valued parent; scalarFields must still apply there") } } + +// nestedDialect is what internal/project uses for markfluence.yaml: two levels, +// for pages -> path -> fields. +var nestedDialect = Dialect{Doc: "a project file", Item: "setting", MaxDepth: 2} + +func TestReadMappingReadsNestedMappings(t *testing.T) { + items, err := nestedDialect.ReadMapping( + "space: ENG\npages:\n docs/a.md:\n title: A\n page_id: 1\n labels: [x]\n") + if err != nil { + t.Fatalf("ReadMapping: %v", err) + } + if len(items) != 2 { + t.Fatalf("items = %#v, want two", items) + } + if items[0].Key != "space" || items[0].Value != "ENG" || items[0].Map != nil { + t.Errorf("items[0] = %#v, want the scalar space", items[0]) + } + pages := items[1] + if pages.Key != "pages" || pages.Map == nil { + t.Fatalf("items[1] = %#v, want pages as a mapping", pages) + } + if len(pages.Map) != 1 || pages.Map[0].Key != "docs/a.md" { + t.Fatalf("pages.Map = %#v, want one entry keyed by path", pages.Map) + } + entry := pages.Map[0] + if entry.Map == nil { + t.Fatalf("entry = %#v, want its fields as a mapping", entry) + } + got := map[string]string{} + lists := map[string][]string{} + for _, f := range entry.Map { + if f.List != nil { + lists[f.Key] = f.List + continue + } + got[f.Key] = f.Value + } + if got["title"] != "A" || got["page_id"] != "1" { + t.Errorf("entry fields = %#v, want title=A page_id=1", got) + } + if len(lists["labels"]) != 1 || lists["labels"][0] != "x" { + t.Errorf("entry lists = %#v, want labels=[x]", lists) + } +} + +// goccy renders a one-key mapping as a MappingValueNode and a multi-key one as +// a MappingNode. Treating only the plural form as nesting would read a +// single-field entry as a broken scalar. +func TestReadMappingReadsASingleFieldNestedMapping(t *testing.T) { + items, err := nestedDialect.ReadMapping("pages:\n docs/a.md:\n page_id: 1\n") + if err != nil { + t.Fatalf("ReadMapping: %v", err) + } + if len(items) != 1 || items[0].Map == nil || len(items[0].Map) != 1 { + t.Fatalf("items = %#v, want pages with one entry", items) + } + entry := items[0].Map[0] + if entry.Map == nil || len(entry.Map) != 1 || entry.Map[0].Key != "page_id" { + t.Fatalf("entry = %#v, want one page_id field", entry) + } +} + +// An empty nested mapping is non-nil, the same way an empty list is: `pages: {}` +// is a project that has chosen the manifest and registered nothing yet, which +// is not the same as having no pages: key at all. +func TestReadMappingDistinguishesEmptyMapFromAbsent(t *testing.T) { + items, err := nestedDialect.ReadMapping("pages: {}\n") + if err != nil { + t.Fatalf("ReadMapping: %v", err) + } + if len(items) != 1 { + t.Fatalf("items = %#v, want one", items) + } + if items[0].Map == nil { + t.Error("pages: {} read with a nil Map; an empty mapping must stay a mapping") + } + if len(items[0].Map) != 0 { + t.Errorf("Map = %#v, want empty", items[0].Map) + } +} + +// Depth is an allowance, not a requirement: a nesting-capable dialect still +// holds plain scalars at every level. +func TestReadMappingNestingIsOptionalAtEveryLevel(t *testing.T) { + items, err := nestedDialect.ReadMapping("space: ENG\npage_width: wide\n") + if err != nil { + t.Fatalf("ReadMapping: %v", err) + } + for _, it := range items { + if it.Map != nil { + t.Errorf("%s read as a mapping, want a scalar", it.Key) + } + } +} + +// Past the allowance a mapping is refused by the scalar path, with the scalar +// path's own message -- which is what keeps MaxDepth 0 behaving exactly as the +// reader did before nesting existed. +func TestReadMappingRefusesNestingPastMaxDepth(t *testing.T) { + _, err := nestedDialect.ReadMapping( + "pages:\n docs/a.md:\n title:\n deeper: nope\n") + if err == nil { + t.Fatal("ReadMapping accepted three levels under MaxDepth 2, want an error") + } + if want := `setting "title" must be a single scalar value, found Mapping`; err.Error() != want { + t.Errorf("error = %q, want %q", err, want) + } +} + +// The flat dialect is the fenced block's, and its refusal must be untouched by +// nesting support existing at all. +func TestReadMappingFlatDialectStillRefusesAMapping(t *testing.T) { + _, err := testDialect.ReadMapping("space:\n key: ENG\n") + if err == nil { + t.Fatal("the flat dialect accepted a nested mapping, want an error") + } + if want := `setting "space" must be a single scalar value, found Mapping`; err.Error() != want { + t.Errorf("error = %q, want %q", err, want) + } +} + +// A nested key's line is its own, so a caller rejecting a field inside an entry +// can point at the field rather than at the pages: key. +func TestReadMappingNestedLinesAreTheirOwn(t *testing.T) { + items, err := nestedDialect.ReadMapping( + "pages:\n docs/a.md:\n title: A\n page_id: 1\n") + if err != nil { + t.Fatalf("ReadMapping: %v", err) + } + entry := items[0].Map[0] + if entry.Line != 2 { + t.Errorf("entry line = %d, want 2", entry.Line) + } + if got := entry.Map[1]; got.Key != "page_id" || got.Line != 4 { + t.Errorf("page_id at line %d, want 4", got.Line) + } +} From 300252649dc6e7dcde568a05b118bb1d02617b0e Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 18:49:28 -0400 Subject: [PATCH 03/14] feat(project): read the pages: key into per-file entries An Entry is shaped exactly like a parsed frontmatter block -- the same two maps MarkdownFile carries, scalars by key and sequences by key. That is the design and not a convenience: internal/pagewidth and internal/labels both reach internal/client, which holds a *project.Cache, so this package can never import them, and a typed validated Entry would need a broken cycle or a second copy of every field's rules. With two maps it needs neither -- labels.Declared(e.Lists, e.Fields) and pagewidth.Declared(e.Fields) work on an entry unchanged, in the commands that already call them. entryFields is the manifest's schema and the only place it is written down. It mirrors frontmatter's fields deliberately: adding one there without adding it here would make a field expressible in a file and not in an entry, which is the "same keys" half of the equivalence. What is checked at load is structure -- a mapping of mappings, legal paths, known field names, right shapes. What is not checked is any field's value, because #139 requires a semantically bad entry to be reported only when its file is one of the arguments, and this function has no idea which files the command was given. An unknown field *name* is the exception and is load-time: it is the same typo class as an unknown top-level setting, and it means the manifest was written against a different markfluence. NormalizePageKey is used on both sides -- the manifest's keys and a file being published -- because a mismatch means a silent skip rather than an error, so the two have to agree exactly. Lexical only, no symlink resolution (L2 forbids a key whose meaning depends on the checkout's layout); an escaping or absolute key and two keys normalizing to one are load-time errors naming both spellings, since YAML only catches literal duplicates. Config.Pages is nil when there is no pages: key and empty-non-nil for `pages: {}`, which is how a command tells "has not chosen the manifest" from "has, and has registered nothing" -- D9's write-destination inference turns on exactly that. One existing message changed: `space:` holding a mapping is now refused by the settings table ("must be a single value, not a mapping") rather than by the reader, since the project dialect allows depth for pages: and the value reaches the table. That is the intended split and the table's wording is the better one. --- internal/project/config.go | 34 ++++- internal/project/config_test.go | 21 ++- internal/project/pages.go | 176 ++++++++++++++++++++++++ internal/project/pages_test.go | 237 ++++++++++++++++++++++++++++++++ 4 files changed, 460 insertions(+), 8 deletions(-) create mode 100644 internal/project/pages.go create mode 100644 internal/project/pages_test.go diff --git a/internal/project/config.go b/internal/project/config.go index 4a66511..ae7da6d 100644 --- a/internal/project/config.go +++ b/internal/project/config.go @@ -31,12 +31,22 @@ type Config struct { // PageWidth is the default page_width. Declaring it means update asserts // a width on a file that declares none, the same way frontmatter does. PageWidth string + // Pages is per-file page metadata, keyed by NormalizePageKey'd path (#139). + // Nil when the file has no pages: key at all, which is what distinguishes a + // project that has not chosen the manifest from one that has and has + // registered nothing -- `pages: {}` is an empty non-nil map. + Pages map[string]Entry } -// kind is the shape a setting's value must have. Only scalars exist today. +// kind is the shape a value must have -- a top-level setting's, or a field's +// inside a pages: entry (pages.go). type kind int -const kindScalar kind = iota + 1 +const ( + kindScalar kind = iota + 1 + kindList + kindMapping +) // settings is the whitelist of recognized top-level keys, mapped to the shape // each one's value must have. @@ -47,6 +57,7 @@ const kindScalar kind = iota + 1 // should be able to join this table instead of reworking the reader. var settings = map[string]kind{ "page_width": kindScalar, + "pages": kindMapping, "space": kindScalar, } @@ -56,7 +67,7 @@ var settings = map[string]kind{ // probing goccy, and a project file read by a second copy of them would be a // second set of the same bugs (#100 called this "a third minimal parser" and // declined it). -var dialect = frontmatter.Dialect{Doc: "a project file", Item: "setting"} +var dialect = frontmatter.Dialect{Doc: "a project file", Item: "setting", MaxDepth: 2} // ConfigError is a markfluence.yaml that could not be understood. Typed so a // caller can report it as itself rather than under the "resolving the @@ -122,6 +133,14 @@ func loadConfig(path string) (Config, error) { return Config{}, &ConfigError{File: path, Line: it.Line, Err: fmt.Errorf( "setting %q must be a single value, not a list", it.Key)} } + if want == kindScalar && it.Map != nil { + return Config{}, &ConfigError{File: path, Line: it.Line, Err: fmt.Errorf( + "setting %q must be a single value, not a mapping", it.Key)} + } + if want == kindMapping && it.Map == nil { + return Config{}, &ConfigError{File: path, Line: it.Line, Err: fmt.Errorf( + "setting %q must be a mapping", it.Key)} + } // A declared-but-empty setting is unset rather than an error, matching // how frontmatter reads its own fields: `space:` with nothing after it // says no more than an absent key does. @@ -131,6 +150,12 @@ func loadConfig(path string) (Config, error) { cfg.Space = value case "page_width": cfg.PageWidth = value + case "pages": + pages, err := readPages(it.Map) + if err != nil { + return Config{}, &ConfigError{File: path, Line: it.Line, Err: err} + } + cfg.Pages = pages } } return cfg, nil @@ -145,6 +170,9 @@ func (c Config) declared() []string { if c.PageWidth != "" { out = append(out, "page_width="+c.PageWidth) } + if c.Pages != nil { + out = append(out, fmt.Sprintf("pages=%d", len(c.Pages))) + } return out } diff --git a/internal/project/config_test.go b/internal/project/config_test.go index abc9e6f..505f710 100644 --- a/internal/project/config_test.go +++ b/internal/project/config_test.go @@ -52,7 +52,7 @@ func TestDiscoverAcceptsAMarkerWithNoSettings(t *testing.T) { t.Fatalf("Discover: %v", err) } defer func() { _ = root.FS.Close() }() - if root.Config != (Config{}) { + if !configIsZero(root.Config) { t.Errorf("Config = %#v, want zero", root.Config) } if root.Dir != dir { @@ -72,7 +72,7 @@ func TestDiscoverRefusesAnUnknownSetting(t *testing.T) { for _, want := range []string{ filepath.Join(dir, Filename) + ":2", `unknown setting "spce"`, - "known: page_width, space", + "known: page_width, pages, space", "newer markfluence", } { if !strings.Contains(msg, want) { @@ -91,7 +91,12 @@ func TestDiscoverRefusesAMalformedFile(t *testing.T) { "parse error": {"space: [ENG\n", "sequence end token"}, "top-level list": {"- ENG\n", "must be a flat mapping of key: value pairs"}, "list where scalar": {"space: [ENG, OPS]\n", `setting "space" must be a single value, not a list`}, - "nested mapping": {"space:\n key: ENG\n", `setting "space" must be a single scalar value`}, + // Refused by the settings table, not by the reader: the project dialect + // allows depth for pages:, so a mapping under a scalar setting reaches + // the table and earns the table's own wording. + "nested mapping": {"space:\n key: ENG\n", `setting "space" must be a single value, not a mapping`}, + "pages as a scalar": {"pages: nope\n", `setting "pages" must be a mapping`}, + "pages as a list": {"pages: [a]\n", `setting "pages" must be a mapping`}, "duplicate key": {"space: ENG\nspace: OPS\n", "already defined"}, "second document": {"space: ENG\n...\nspace: OPS\n", "must be a single document"}, } @@ -212,7 +217,7 @@ func TestFromPathWithNoProjectFile(t *testing.T) { t.Fatalf("FromPath: %v", err) } defer func() { _ = root.FS.Close() }() - if root.File != "" || root.Config != (Config{}) { + if root.File != "" || !configIsZero(root.Config) { t.Errorf("File = %q, Config = %#v, want empty", root.File, root.Config) } } @@ -252,7 +257,7 @@ func TestLoadConfigTreatsAnEmptySettingAsUnset(t *testing.T) { t.Fatalf("Discover: %v", err) } defer func() { _ = root.FS.Close() }() - if root.Config != (Config{}) { + if !configIsZero(root.Config) { t.Errorf("Config = %#v, want zero", root.Config) } } @@ -423,3 +428,9 @@ func TestLoadConfigDoesNotStripABOMElsewhere(t *testing.T) { t.Fatal("Discover accepted a mid-file BOM, want an error") } } + +// configIsZero reports whether a Config declares nothing. A plain == is not +// available: Config holds a map now. +func configIsZero(c Config) bool { + return c.Space == "" && c.PageWidth == "" && c.Pages == nil +} diff --git a/internal/project/pages.go b/internal/project/pages.go new file mode 100644 index 0000000..4080355 --- /dev/null +++ b/internal/project/pages.go @@ -0,0 +1,176 @@ +package project + +// The pages: key -- per-file page metadata living in the project file instead +// of in the markdown, so a .md can be published while staying pristine (#139). + +import ( + "fmt" + "path" + "sort" + "strings" + + "github.com/mozilla/markfluence/internal/frontmatter" +) + +// Entry is one file's page metadata, shaped exactly like a parsed frontmatter +// block: the same two maps frontmatter.MarkdownFile carries, scalars by key and +// sequences by key. +// +// That shape is the whole design and not a convenience. It makes #139's "an +// entry is a whole frontmatter block that lives elsewhere" literal -- the same +// keys, the same value domains, the same validation -- and it is the only shape +// that avoids a cycle: internal/pagewidth and internal/labels both reach +// internal/client, which holds a *Cache, so this package can never import +// them. With two maps it does not need to. labels.Declared(e.Lists, e.Fields), +// pagewidth.Declared(e.Fields) and pageref.IsDigits(e.Fields["page_id"]) all +// work on an entry unchanged, in the commands that already call them. +// +// Which is also why nothing here validates a *value*. See entryFields. +type Entry struct { + Fields map[string]string + Lists map[string][]string +} + +// entryFields is the manifest's schema: the field names an entry may hold and +// whether each is a scalar or a list. The only place it is written down. +// +// It mirrors frontmatter's own fields deliberately -- adding one there without +// adding it here would make a field expressible in a file and not in an entry, +// which is the "same keys" half of the equivalence above. +var entryFields = map[string]kind{ + "title": kindScalar, + "space": kindScalar, + "parent": kindScalar, + "page_id": kindScalar, + "page_width": kindScalar, + "labels": kindList, +} + +// readPages reads the pages: mapping into entries keyed by normalized path. +// +// What is checked here is *structure*: the value is a mapping of mappings, each +// key is a legal path, each field name is known, and each field's value has the +// right shape. What is not checked is any field's *value* -- a non-numeric +// page_id, an invalid page_width, an invalid label -- because #139 requires a +// semantically bad entry to be reported only when that entry's file is one of +// the arguments. One bad entry must not block every invocation in the repo, +// and this function has no idea which files the command was given. +// +// An unknown field *name* is checked here rather than per-file, and the line is +// worth stating: it is the same typo class as an unknown top-level setting -- +// `titel:` silently ignored is wrong forever, for that page -- and it means the +// manifest was written against a different markfluence, which #100 settles as +// fatal. A bad value is one entry's problem; an unrecognized schema is the +// file's. +func readPages(items []frontmatter.Item) (map[string]Entry, error) { + pages := map[string]Entry{} + // Which spelling each normalized key came from, so a collision can name + // both rather than only the survivor. + origin := map[string]string{} + + for _, it := range items { + if it.Map == nil { + return nil, fmt.Errorf("page %q must be a mapping of fields", it.Key) + } + key, err := NormalizePageKey(it.Key) + if err != nil { + return nil, err + } + if first, dup := origin[key]; dup { + return nil, fmt.Errorf( + "pages %q and %q both name %q; give it one spelling", first, it.Key, key) + } + entry, err := readEntry(it.Key, it.Map) + if err != nil { + return nil, err + } + origin[key] = it.Key + pages[key] = entry + } + return pages, nil +} + +// readEntry reads one entry's fields. named is the key as written, so a message +// points at the spelling the author would search for. +func readEntry(named string, fields []frontmatter.Item) (Entry, error) { + e := Entry{Fields: map[string]string{}, Lists: map[string][]string{}} + for _, f := range fields { + want, ok := entryFields[f.Key] + if !ok { + return Entry{}, fmt.Errorf( + "page %q has an unknown field %q (known: %s) -- an unrecognized field may "+ + "mean this project needs a newer markfluence", named, f.Key, + strings.Join(knownEntryFields(), ", ")) + } + if f.Map != nil { + return Entry{}, fmt.Errorf("page %q field %q must be a value, not a mapping", + named, f.Key) + } + switch want { + case kindList: + if f.List == nil { + // Refused rather than read as a one-element list, matching + // internal/labels' reasoning: labels is destructive, and a + // `labels:` with nothing after it would otherwise mean "strip + // this page" on an entry where somebody typed a key and stopped. + return Entry{}, fmt.Errorf("page %q field %q must be a list", named, f.Key) + } + e.Lists[f.Key] = f.List + default: + if f.List != nil { + return Entry{}, fmt.Errorf("page %q field %q must be a single value, not a list", + named, f.Key) + } + e.Fields[f.Key] = f.Value + } + } + return e, nil +} + +// knownEntryFields lists the recognized field names, sorted so a message is +// stable. +func knownEntryFields() []string { + out := make([]string, 0, len(entryFields)) + for key := range entryFields { + out = append(out, key) + } + sort.Strings(out) + return out +} + +// NormalizePageKey turns a path as written into the form pages: is keyed by and +// arguments are looked up as: lexically cleaned, root-relative, slash form. +// +// Callers use it on both sides -- the manifest's keys and the path of a file +// being published -- because a mismatch now means a silent *skip* rather than +// an error, so the two must agree exactly. +// +// Lexical only, no symlink resolution, matching withinRoot, attachfile.Resolve +// and attachment-download's destPath. A symlinked docs/ is legitimate, and +// resolving it would make a key depend on the checkout's layout, which L2 +// (invocation-independent) forbids. +// +// A key escaping the root is an error rather than a miss: the project file +// declares the project's boundary, so a key outside it is the manifest being +// wrong, not a file being absent. So is an absolute path, which names a +// location no root can contain. +// +// No case folding. Known limit, beside pageslug's NFD/NFC note: on a +// case-insensitive filesystem Docs/a.md opens the file but matches no docs/a.md +// key, so it reads as unmanaged and is skipped. +func NormalizePageKey(p string) (string, error) { + if p == "" { + return "", fmt.Errorf("a page key cannot be empty") + } + if path.IsAbs(p) || strings.HasPrefix(p, "/") { + return "", fmt.Errorf("page %q must be relative to the project root, not absolute", p) + } + clean := path.Clean(strings.ReplaceAll(p, "\\", "/")) + if clean == ".." || strings.HasPrefix(clean, "../") { + return "", fmt.Errorf("page %q is outside the project root", p) + } + if clean == "." { + return "", fmt.Errorf("page %q does not name a file", p) + } + return clean, nil +} diff --git a/internal/project/pages_test.go b/internal/project/pages_test.go new file mode 100644 index 0000000..d65cbbc --- /dev/null +++ b/internal/project/pages_test.go @@ -0,0 +1,237 @@ +package project + +import ( + "errors" + "reflect" + "strings" + "testing" +) + +// discoverPages loads a project file and returns its entries. +func discoverPages(t *testing.T, body string) map[string]Entry { + t.Helper() + root, err := Discover(write(t, body)) + if err != nil { + t.Fatalf("Discover: %v", err) + } + t.Cleanup(func() { _ = root.FS.Close() }) + return root.Config.Pages +} + +func TestReadPagesReadsEntries(t *testing.T) { + pages := discoverPages(t, `space: ENG +pages: + docs/engineering-docs.md: + title: Engineering Docs + page_id: 12345 + labels: [howto] + + docs/deploy-runbook.md: + title: Deploy Runbook + parent: docs/engineering-docs.md + page_id: 12346 + page_width: wide +`) + if len(pages) != 2 { + t.Fatalf("pages = %#v, want two", pages) + } + got := pages["docs/deploy-runbook.md"] + want := map[string]string{ + "title": "Deploy Runbook", + "parent": "docs/engineering-docs.md", + "page_id": "12346", + "page_width": "wide", + } + if !reflect.DeepEqual(got.Fields, want) { + t.Errorf("fields = %#v, want %#v", got.Fields, want) + } + if len(got.Lists) != 0 { + t.Errorf("lists = %#v, want none", got.Lists) + } + if l := pages["docs/engineering-docs.md"].Lists["labels"]; len(l) != 1 || l[0] != "howto" { + t.Errorf("labels = %#v, want [howto]", l) + } +} + +// An entry is shaped exactly like a parsed frontmatter block, which is what +// lets internal/labels and internal/pagewidth validate one unchanged -- this +// package cannot import either of them (both reach internal/client, which +// holds a *Cache), so the two maps are the whole mechanism. +func TestEntryIsShapedLikeAFrontmatterBlock(t *testing.T) { + pages := discoverPages(t, "pages:\n a.md:\n title: A\n labels: [x, y]\n") + e := pages["a.md"] + if e.Fields == nil || e.Lists == nil { + t.Fatalf("entry = %#v, want both maps non-nil", e) + } + // The shapes MarkdownFile.Frontmatter and .Lists have. + var _ map[string]string = e.Fields + var _ map[string][]string = e.Lists +} + +// A project with no pages: key has not chosen the manifest; one with an empty +// pages: has, and has registered nothing. Nil vs empty is how a command tells +// them apart, and D9 turns on exactly that. +func TestPagesNilWhenAbsentAndEmptyWhenDeclared(t *testing.T) { + if pages := discoverPages(t, "space: ENG\n"); pages != nil { + t.Errorf("pages = %#v, want nil when there is no pages: key", pages) + } + pages := discoverPages(t, "pages: {}\n") + if pages == nil { + t.Fatal("pages = nil for `pages: {}`, want an empty non-nil map") + } + if len(pages) != 0 { + t.Errorf("pages = %#v, want empty", pages) + } +} + +func TestReadPagesRefusals(t *testing.T) { + tests := map[string]struct{ body, want string }{ + "entry is a scalar": { + "pages:\n a.md: 123\n", `page "a.md" must be a mapping of fields`}, + "unknown field": { + "pages:\n a.md:\n titel: A\n", + `page "a.md" has an unknown field "titel"`}, + "unknown field names the known ones": { + "pages:\n a.md:\n titel: A\n", + "known: labels, page_id, page_width, parent, space, title"}, + "unknown field suggests a newer markfluence": { + "pages:\n a.md:\n titel: A\n", "needs a newer markfluence"}, + "scalar field given a list": { + "pages:\n a.md:\n title: [A, B]\n", + `page "a.md" field "title" must be a single value, not a list`}, + "list field given a scalar": { + "pages:\n a.md:\n labels: one\n", + `page "a.md" field "labels" must be a list`}, + "field given a mapping": { + "pages:\n a.md:\n title:\n deeper: x\n", + `setting "title" must be a single scalar value, found Mapping`}, + "key escaping the root": { + "pages:\n ../elsewhere.md:\n title: A\n", + `page "../elsewhere.md" is outside the project root`}, + "absolute key": { + "pages:\n /etc/passwd.md:\n title: A\n", + `must be relative to the project root, not absolute`}, + "two keys normalizing to one": { + "pages:\n docs/a.md:\n title: A\n ./docs/a.md:\n title: B\n", + `both name "docs/a.md"`}, + } + for name, tc := range tests { + t.Run(name, func(t *testing.T) { + _, err := Discover(write(t, tc.body)) + if err == nil { + t.Fatalf("Discover accepted %q, want an error", tc.body) + } + if !strings.Contains(err.Error(), tc.want) { + t.Errorf("error = %q, want it to contain %q", err, tc.want) + } + var cfgErr *ConfigError + if !errors.As(err, &cfgErr) { + t.Errorf("error is %T, want it to unwrap to *ConfigError", err) + } + }) + } +} + +// A bad *value* is deliberately not checked here: #139 requires it to be +// reported only when that entry's file is one of the arguments, so one bad +// entry cannot block every invocation in the repo. Structure is this package's +// job; vocabulary is the command's. +func TestReadPagesDoesNotValidateValues(t *testing.T) { + pages := discoverPages(t, `pages: + a.md: + page_id: not-a-number + page_width: huge + labels: [Has Space] +`) + e := pages["a.md"] + if e.Fields["page_id"] != "not-a-number" || e.Fields["page_width"] != "huge" { + t.Errorf("fields = %#v, want the raw values preserved for the caller to judge", e.Fields) + } + if l := e.Lists["labels"]; len(l) != 1 || l[0] != "Has Space" { + t.Errorf("labels = %#v, want the raw value", l) + } +} + +func TestNormalizePageKey(t *testing.T) { + ok := map[string]string{ + "docs/a.md": "docs/a.md", + "./docs/a.md": "docs/a.md", + "docs//a.md": "docs/a.md", + "docs/./a.md": "docs/a.md", + "docs/sub/../a.md": "docs/a.md", + "a.md": "a.md", + // Backslashes are normalized so a key written on Windows matches the + // slash form every other path in markfluence uses. + `docs\a.md`: "docs/a.md", + } + for in, want := range ok { + t.Run(in, func(t *testing.T) { + got, err := NormalizePageKey(in) + if err != nil { + t.Fatalf("NormalizePageKey(%q): %v", in, err) + } + if got != want { + t.Errorf("= %q, want %q", got, want) + } + }) + } + + bad := []string{"", "..", "../x.md", "docs/../../x.md", "/abs.md", "."} + for _, in := range bad { + t.Run("refuses "+in, func(t *testing.T) { + if got, err := NormalizePageKey(in); err == nil { + t.Errorf("NormalizePageKey(%q) = %q, want an error", in, got) + } + }) + } +} + +// Lexical only, no symlink resolution: L2 forbids a key whose meaning depends +// on the checkout's layout, and a symlinked docs/ is legitimate. +func TestNormalizePageKeyIsLexical(t *testing.T) { + // A path that would escape only after resolving a symlink is still just a + // path here, and one that escapes lexically is refused without touching + // the filesystem -- NormalizePageKey never stats anything. + if _, err := NormalizePageKey("docs/../../etc/passwd.md"); err == nil { + t.Error("want a refusal for a lexically escaping key") + } + got, err := NormalizePageKey("docs/link/a.md") + if err != nil || got != "docs/link/a.md" { + t.Errorf("= %q/%v, want the path unchanged and no stat", got, err) + } +} + +// A page key is compared byte-for-byte after normalization, with no case +// folding. Documented limit: on a case-insensitive filesystem the file opens +// and the key does not match, so the file reads as unmanaged. +func TestNormalizePageKeyDoesNotFoldCase(t *testing.T) { + upper, _ := NormalizePageKey("Docs/A.md") + lower, _ := NormalizePageKey("docs/a.md") + if upper == lower { + t.Error("keys were case-folded; the documented behavior is that they are not") + } +} + +// The duplicate-key error names both spellings, since naming only the survivor +// would leave the author hunting for the other one. +func TestDuplicatePageKeyNamesBothSpellings(t *testing.T) { + _, err := Discover(write(t, + "pages:\n docs/a.md:\n title: A\n docs/./a.md:\n title: B\n")) + if err == nil { + t.Fatal("want an error") + } + for _, want := range []string{"docs/a.md", "docs/./a.md"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error = %q, want it to name %q", err, want) + } + } +} + +// Both load-time errors are structural, so they are not scoped per file: an +// escaping or duplicate key means the manifest is wrong, not that one entry is. +func TestStructuralPageErrorsAreNotPerFile(t *testing.T) { + dir := write(t, "pages:\n ../out.md:\n title: A\n fine.md:\n title: B\n") + if _, err := Discover(dir); err == nil { + t.Fatal("Discover succeeded; one escaping key must fail the load outright") + } +} From aded94d3a4067dd7698a80221884fa89f7b11232 Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 18:52:32 -0400 Subject: [PATCH 04/14] feat(pagemeta): resolve a file's metadata from frontmatter and the manifest The merge update, create, fix, check and linkindex all need. A package rather than a helper because a per-command copy is exactly how two commands come to publish one file to two different pages. It imports only frontmatter and project, so it stays out of every cycle -- which is also why it validates no value: pagewidth and labels are unreachable from here, and the commands that need them already call them on the maps this returns. Frontmatter and an entry are two spellings of *one* level, not two levels of a precedence chain. When both speak the rule is not "the higher wins" but grading: page_id/space/parent are coordinates and a disagreement fails the file, while title/page_width/labels are visible and recoverable, so they warn and frontmatter wins. Agreement is silent, which is what makes migration incremental. Three things that took a second pass to get right. Source and Managed are computed from different predicates, deliberately. Source answers "who contributed metadata" for --json's metadata_source; Managed answers "should update act on this file". A file is claimed when it has an entry or names a page_id -- a page_id is the only field that identifies a page, and #139 is precise that a *registered* file is one with an entry. So `a.md: {}` is a claim that must error for want of a page_id rather than be skipped. Contributed metadata is counted over the fields markfluence knows, via a new project.IsPageField, not over any key at all. A docs tree carrying Jekyll or Hugo frontmatter (layout:, date:, draft:) has said nothing about Confluence, and counting every key would have read a whole such tree as claimed. labels is compared as a set. Confluence has no label order, so [a, b] and [b, a] say the same thing and warning that one "overrides" the other would be noise about a difference that cannot reach the page. A blank value is not a disagreement: every null spelling already reads as "", so a key with nothing after it says no more than no key. --- internal/pagemeta/pagemeta.go | 308 +++++++++++++++++++++++++++++ internal/pagemeta/pagemeta_test.go | 282 ++++++++++++++++++++++++++ internal/project/pages.go | 9 + 3 files changed, 599 insertions(+) create mode 100644 internal/pagemeta/pagemeta.go create mode 100644 internal/pagemeta/pagemeta_test.go diff --git a/internal/pagemeta/pagemeta.go b/internal/pagemeta/pagemeta.go new file mode 100644 index 0000000..e4cf362 --- /dev/null +++ b/internal/pagemeta/pagemeta.go @@ -0,0 +1,308 @@ +// Package pagemeta resolves a markdown file's page metadata from the two places +// it may live: the file's own frontmatter, and a pages: entry in the project's +// markfluence.yaml (#139). +// +// A package rather than a helper because update, create, fix, check *and* +// internal/linkindex all need the identical merge, and a per-command copy is +// exactly how two commands come to publish one file to two different pages. +// It imports internal/frontmatter and internal/project and nothing else, so it +// stays out of every import cycle -- which is also why it validates no value: +// internal/pagewidth and internal/labels are unreachable from here, and the +// commands that need them already call them on the maps this returns. +// +// Both locations are legal and agreement is silent. That is what makes +// migration incremental: copy values into the manifest, verify, delete them +// from the files later, with everything working throughout. There is no project +// "mode" and no all-or-nothing switch. +// +// Frontmatter and an entry are two spellings of *one* level, not two levels of +// a precedence chain. When both speak, the rule is not "the higher wins" but +// the grading in Resolve: a coordinate disagreement can publish over a live +// page, so it fails the file, while a visible and recoverable one warns. +package pagemeta + +import ( + "fmt" + "sort" + "strings" + + "github.com/mozilla/markfluence/internal/frontmatter" + "github.com/mozilla/markfluence/internal/project" +) + +// Source says where a file's metadata came from, and is what --json reports as +// metadata_source. Debugging "why did it publish to *that* page" in CI +// otherwise means reproducing the resolution by hand. +type Source string + +const ( + // FromFrontmatter: the file carries markfluence keys. + FromFrontmatter Source = "frontmatter" + // FromManifest: a pages: entry supplied them and the file carries none. + FromManifest Source = "manifest" + // FromBoth: both locations speak. Reported as "frontmatter" to a caller + // asking which one won, since frontmatter does for every soft field -- see + // Resolved.MetadataSource. + FromBoth Source = "both" + // Unmanaged: neither location says anything about this file. + Unmanaged Source = "" +) + +// coordinates are the fields whose disagreement is destructive rather than +// merely visible. A page_id copy-pasted from an old file publishes over a live +// page; space and parent decide where a create lands, and #10 wants update to +// enforce and move, so they are coordinates now rather than after a rule +// change. +var coordinates = map[string]bool{"page_id": true, "space": true, "parent": true} + +// Resolved is a file's effective metadata. +// +// Fields and Lists are the merged maps, in exactly the shape +// frontmatter.MarkdownFile carries them, so every existing consumer -- +// pagewidth.Declared, labels.Declared, pageref.IsDigits -- works on them +// unchanged. +type Resolved struct { + Fields map[string]string + Lists map[string][]string + // Source is where the metadata came from. + Source Source + // Warnings are the soft disagreements, in field order. + Warnings []string + + managed bool +} + +// MetadataSource is the value --json reports. FromBoth collapses to +// "frontmatter", because that is the location that won every field it could +// win: the schema's question is "which location decided this?", and a third +// enum value would make every consumer handle a case that never changes what +// was published. +func (r Resolved) MetadataSource() Source { + if r.Source == FromBoth { + return FromFrontmatter + } + return r.Source +} + +// Managed reports whether this file is claimed: it has a manifest entry, or its +// frontmatter names a page_id. A file nothing claims is skipped rather than +// failed -- repositories legitimately hold markdown that is not published, +// drafts are a normal state, and a glob-driven CI run must not go red because +// somebody added a file. +// +// Deliberately narrower than "declares any markfluence field". A page_id is the +// only field that identifies a page, and an entry is an explicit claim; a file +// carrying `title:` and nothing else has not told anyone which page it is, and +// #139 is precise that a *registered* file lacking a page_id errors -- where +// registered means an entry exists. It is also why Source and this are computed +// from different predicates rather than one: Source answers "who contributed +// metadata", for reporting, and this answers "should update act on it". +func (r Resolved) Managed() bool { return r.managed } + +// Resolve merges a file's frontmatter with its pages: entry, if it has one. +// +// key is the file's path normalized by project.NormalizePageKey -- the caller +// normalizes, because it is the caller that knows the file's path relative to +// the root, and a mismatch here means a silent skip. +// +// A coordinate disagreement is an error, so the caller fails that file (or, for +// create, aborts the batch: it preflights everything, and a coordinate wrong in +// any file means the batch's shape is not what the author thinks). A soft +// disagreement is a warning and frontmatter wins. +// +// A *blank* value on either side is not a disagreement. Every null spelling +// reads as "" already, so `parent:` with nothing after it says no more than an +// absent key does -- treating it as a conflicting answer would fail files that +// say nothing at all. +func Resolve(key string, mf *frontmatter.MarkdownFile, root *project.Root) (Resolved, error) { + entry, hasEntry := entryFor(key, root) + + r := Resolved{Fields: map[string]string{}, Lists: map[string][]string{}} + if !hasEntry { + // The common case, and the one that must stay byte-for-byte what it was + // before this package existed. + for k, v := range mf.Frontmatter { + r.Fields[k] = v + } + for k, v := range mf.Lists { + r.Lists[k] = v + } + r.Source = sourceOf(declaresPageField(mf.Frontmatter, mf.Lists), false) + r.managed = hasPageID(mf.Frontmatter) + return r, nil + } + + // Start from the entry, then let frontmatter override -- after checking + // that where both speak they agree about anything destructive. + for k, v := range entry.Fields { + r.Fields[k] = v + } + for k, v := range entry.Lists { + r.Lists[k] = v + } + + var conflicts []string + for _, k := range sortedKeys(mf.Frontmatter, entry.Fields) { + file, inFile := nonBlank(mf.Frontmatter, k) + manifest, inEntry := nonBlank(entry.Fields, k) + switch { + case inFile && inEntry && file != manifest: + if coordinates[k] { + conflicts = append(conflicts, fmt.Sprintf( + "%s: frontmatter says %q, %s says %q", k, file, project.Filename, manifest)) + continue + } + r.Warnings = append(r.Warnings, fmt.Sprintf( + "%s: frontmatter %q overrides %s's %q", k, file, project.Filename, manifest)) + r.Fields[k] = file + case inFile: + r.Fields[k] = file + } + } + for _, k := range sortedListKeys(mf.Lists, entry.Lists) { + file, inFile := mf.Lists[k] + manifest, inEntry := entry.Lists[k] + if inFile && inEntry && !sameList(file, manifest) { + // labels is the only list field, and it is soft: declaring it + // asserts a set, which is visible on the page and reversible. + r.Warnings = append(r.Warnings, fmt.Sprintf( + "%s: frontmatter [%s] overrides %s's [%s]", + k, strings.Join(file, ", "), project.Filename, strings.Join(manifest, ", "))) + } + if inFile { + r.Lists[k] = file + } + } + if len(conflicts) > 0 { + return Resolved{}, fmt.Errorf( + "%s and this file disagree about where this page is: %s. "+ + "Correct one of them; markfluence will not guess", + project.Filename, strings.Join(conflicts, "; ")) + } + + // inEntry is "an entry exists", not "an entry declares something": `a.md: {}` + // is somebody claiming the path, which #139 says must error for want of a + // page_id rather than be skipped as unmanaged. + r.Source = sourceOf(declaresPageField(mf.Frontmatter, mf.Lists), true) + r.managed = true + return r, nil +} + +// entryFor looks up a file's manifest entry. +func entryFor(key string, root *project.Root) (project.Entry, bool) { + if root == nil || root.Config.Pages == nil { + return project.Entry{}, false + } + e, ok := root.Config.Pages[key] + return e, ok +} + +// HasManifest reports whether the project has chosen the manifest -- a pages: +// key, even an empty one. It is what decides where *new* metadata is written +// (D9): a project that has chosen the manifest never accidentally grows +// frontmatter, and a project without one behaves exactly as it did before. +func HasManifest(root *project.Root) bool { + return root != nil && root.Config.Pages != nil +} + +// declaresPageField reports whether either map holds a non-blank value under a +// field markfluence understands. +// +// Restricted to known fields on purpose: a docs tree carrying Jekyll or Hugo +// frontmatter (layout:, date:, draft:) has said nothing about Confluence, and +// counting any key at all would report every such file as having contributed +// metadata it never had. +func declaresPageField(fields map[string]string, lists map[string][]string) bool { + for k, v := range fields { + if project.IsPageField(k) && strings.TrimSpace(v) != "" { + return true + } + } + for k := range lists { + if project.IsPageField(k) { + return true + } + } + return false +} + +// hasPageID reports whether a page_id is named and non-blank. +func hasPageID(fields map[string]string) bool { + _, ok := nonBlank(fields, "page_id") + return ok +} + +func sourceOf(inFile, inEntry bool) Source { + switch { + case inFile && inEntry: + return FromBoth + case inFile: + return FromFrontmatter + case inEntry: + return FromManifest + default: + return Unmanaged + } +} + +// nonBlank reads a key, reporting absent for a blank value: every null spelling +// reads as "" already, so a key with nothing after it says no more than no key. +func nonBlank(m map[string]string, k string) (string, bool) { + v, ok := m[k] + if !ok { + return "", false + } + v = strings.TrimSpace(v) + return v, v != "" +} + +func sortedKeys(a map[string]string, b map[string]string) []string { + seen := map[string]bool{} + out := []string{} + for _, m := range []map[string]string{a, b} { + for k := range m { + if !seen[k] { + seen[k] = true + out = append(out, k) + } + } + } + sort.Strings(out) + return out +} + +func sortedListKeys(a map[string][]string, b map[string][]string) []string { + seen := map[string]bool{} + out := []string{} + for _, m := range []map[string][]string{a, b} { + for k := range m { + if !seen[k] { + seen[k] = true + out = append(out, k) + } + } + } + sort.Strings(out) + return out +} + +// sameList compares two declared lists as *sets*, not in order. +// +// labels is the only list field and Confluence has no label order, so +// `[a, b]` and `[b, a]` say the same thing -- warning that one "overrides" the +// other would be noise about a difference that cannot reach the page. +func sameList(a, b []string) bool { + if len(a) != len(b) { + return false + } + x := append([]string(nil), a...) + y := append([]string(nil), b...) + sort.Strings(x) + sort.Strings(y) + for i := range x { + if x[i] != y[i] { + return false + } + } + return true +} diff --git a/internal/pagemeta/pagemeta_test.go b/internal/pagemeta/pagemeta_test.go new file mode 100644 index 0000000..54427cf --- /dev/null +++ b/internal/pagemeta/pagemeta_test.go @@ -0,0 +1,282 @@ +package pagemeta + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/mozilla/markfluence/internal/frontmatter" + "github.com/mozilla/markfluence/internal/project" +) + +// rootWith builds a project root whose markfluence.yaml holds body. +func rootWith(t *testing.T, body string) *project.Root { + t.Helper() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), []byte(body), 0o644); err != nil { + t.Fatal(err) + } + root, err := project.Discover(dir) + if err != nil { + t.Fatalf("Discover: %v", err) + } + t.Cleanup(func() { _ = root.FS.Close() }) + return root +} + +// parse builds a MarkdownFile from a frontmatter block (or "" for none). +func parse(t *testing.T, block string) *frontmatter.MarkdownFile { + t.Helper() + content := "body\n" + if block != "" { + content = "---\n" + block + "---\nbody\n" + } + mf, err := frontmatter.Parse("a.md", content) + if err != nil { + t.Fatalf("Parse: %v", err) + } + return mf +} + +func TestResolveFrontmatterOnly(t *testing.T) { + root := rootWith(t, "space: ENG\n") + r, err := Resolve("a.md", parse(t, "title: A\npage_id: 1\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Source != FromFrontmatter || r.MetadataSource() != FromFrontmatter { + t.Errorf("source = %q, want frontmatter", r.Source) + } + if !r.Managed() { + t.Error("not managed; a page_id in frontmatter is a claim") + } + if r.Fields["title"] != "A" || r.Fields["page_id"] != "1" { + t.Errorf("fields = %#v", r.Fields) + } +} + +func TestResolveManifestOnly(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n title: A\n page_id: 1\n labels: [x]\n") + r, err := Resolve("a.md", parse(t, ""), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Source != FromManifest || r.MetadataSource() != FromManifest { + t.Errorf("source = %q, want manifest", r.Source) + } + if !r.Managed() { + t.Error("not managed; an entry is a claim") + } + if r.Fields["title"] != "A" || r.Fields["page_id"] != "1" { + t.Errorf("fields = %#v", r.Fields) + } + if l := r.Lists["labels"]; len(l) != 1 || l[0] != "x" { + t.Errorf("labels = %#v", l) + } +} + +// Agreement is silent, which is what makes migration incremental: copy values +// into the manifest, verify, delete them from the files later. +func TestResolveAgreementIsSilent(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n title: A\n page_id: 1\n") + r, err := Resolve("a.md", parse(t, "title: A\npage_id: 1\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if len(r.Warnings) != 0 { + t.Errorf("warnings = %#v, want none", r.Warnings) + } + if r.Source != FromBoth { + t.Errorf("source = %q, want both", r.Source) + } + // FromBoth collapses for reporting: frontmatter is the location that won + // every field it could win. + if r.MetadataSource() != FromFrontmatter { + t.Errorf("metadata_source = %q, want frontmatter", r.MetadataSource()) + } +} + +// A coordinate disagreement is the clobber case: a page_id pasted from an old +// file publishes over a live page. +func TestResolveCoordinateDisagreementIsAnError(t *testing.T) { + for _, field := range []string{"page_id", "space", "parent"} { + t.Run(field, func(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n "+field+": manifestvalue\n") + _, err := Resolve("a.md", parse(t, field+": filevalue\n"), root) + if err == nil { + t.Fatalf("Resolve accepted a %s disagreement, want an error", field) + } + for _, want := range []string{field, "filevalue", "manifestvalue", "will not guess"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error = %q, want it to contain %q", err, want) + } + } + }) + } +} + +// Soft fields are visible and recoverable, so they warn and frontmatter wins. +func TestResolveSoftDisagreementWarnsAndFrontmatterWins(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n title: Manifest\n page_width: narrow\n") + r, err := Resolve("a.md", parse(t, "title: File\npage_width: wide\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Fields["title"] != "File" || r.Fields["page_width"] != "wide" { + t.Errorf("fields = %#v, want frontmatter's values", r.Fields) + } + if len(r.Warnings) != 2 { + t.Fatalf("warnings = %#v, want two", r.Warnings) + } + // In field order, so output is stable across runs. + if !strings.HasPrefix(r.Warnings[0], "page_width:") || !strings.HasPrefix(r.Warnings[1], "title:") { + t.Errorf("warnings = %#v, want them sorted by field", r.Warnings) + } +} + +// A blank value says no more than an absent key -- every null spelling already +// reads as "" -- so it must not be a conflicting answer. +func TestResolveBlankIsNotADisagreement(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n page_id: 1\n") + for _, block := range []string{"page_id:\n", "page_id: ~\n", "page_id: null\n", "page_id: \" \"\n"} { + t.Run(block, func(t *testing.T) { + r, err := Resolve("a.md", parse(t, block), root) + if err != nil { + t.Fatalf("Resolve(%q): %v", block, err) + } + if r.Fields["page_id"] != "1" { + t.Errorf("page_id = %q, want the manifest's 1", r.Fields["page_id"]) + } + }) + } +} + +// labels is compared as a set: Confluence has no label order, so a reordering +// cannot reach the page and warning about it would be noise. +func TestResolveListOrderIsNotADisagreement(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n labels: [a, b]\n") + r, err := Resolve("a.md", parse(t, "labels: [b, a]\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if len(r.Warnings) != 0 { + t.Errorf("warnings = %#v, want none for a reordering", r.Warnings) + } +} + +func TestResolveListDisagreementWarns(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n labels: [a]\n") + r, err := Resolve("a.md", parse(t, "labels: [b]\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if len(r.Warnings) != 1 || !strings.Contains(r.Warnings[0], "labels") { + t.Errorf("warnings = %#v, want one about labels", r.Warnings) + } + if l := r.Lists["labels"]; len(l) != 1 || l[0] != "b" { + t.Errorf("labels = %#v, want frontmatter's [b]", l) + } +} + +// The point of D7: a pristine file nothing claims is unmanaged, so a +// glob-driven CI run skips it instead of going red. +func TestResolveUnmanaged(t *testing.T) { + root := rootWith(t, "pages:\n other.md:\n page_id: 1\n") + r, err := Resolve("a.md", parse(t, ""), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Source != Unmanaged || r.Managed() { + t.Errorf("source = %q managed = %v, want unmanaged", r.Source, r.Managed()) + } + if r.MetadataSource() != Unmanaged { + t.Errorf("metadata_source = %q, want empty", r.MetadataSource()) + } +} + +// A docs tree carrying Jekyll or Hugo frontmatter has said nothing about +// Confluence. Counting any key at all would report every such file as claimed +// and fail a whole tree. +func TestResolveForeignFrontmatterIsUnmanaged(t *testing.T) { + root := rootWith(t, "space: ENG\n") + r, err := Resolve("a.md", parse(t, "layout: post\ndate: 2026-01-01\ndraft: false\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Source != Unmanaged || r.Managed() { + t.Errorf("source = %q managed = %v, want unmanaged", r.Source, r.Managed()) + } +} + +// A file carrying title: and nothing else has not said which page it is, so it +// is not claimed -- #139 is precise that a *registered* file is one with an +// entry. +func TestResolveTitleAloneDoesNotClaimAFile(t *testing.T) { + root := rootWith(t, "space: ENG\n") + r, err := Resolve("a.md", parse(t, "title: A\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Managed() { + t.Error("managed; a title alone does not identify a page") + } + // It did contribute metadata, though, which is a different question. + if r.Source != FromFrontmatter { + t.Errorf("source = %q, want frontmatter", r.Source) + } +} + +// An empty entry is somebody claiming the path, which must error for want of a +// page_id rather than be skipped as unmanaged. +func TestResolveEmptyEntryStillClaimsTheFile(t *testing.T) { + root := rootWith(t, "pages:\n a.md: {}\n") + r, err := Resolve("a.md", parse(t, ""), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if !r.Managed() { + t.Error("not managed; an entry is an explicit claim even when it declares nothing") + } + if r.Fields["page_id"] != "" { + t.Errorf("page_id = %q, want empty so the caller reports it", r.Fields["page_id"]) + } +} + +// No pages: key at all must behave exactly as markfluence did before #139. +func TestResolveWithNoManifest(t *testing.T) { + root := rootWith(t, "space: ENG\n") + r, err := Resolve("a.md", parse(t, "title: A\npage_id: 1\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Source != FromFrontmatter || len(r.Warnings) != 0 { + t.Errorf("source = %q warnings = %#v", r.Source, r.Warnings) + } + if HasManifest(root) { + t.Error("HasManifest true with no pages: key") + } +} + +// D9 turns on nil-vs-empty: an empty pages: block is a project that has chosen +// the manifest, so new metadata belongs there rather than in frontmatter. +func TestHasManifest(t *testing.T) { + if !HasManifest(rootWith(t, "pages: {}\n")) { + t.Error("HasManifest false for an empty pages: block") + } + if HasManifest(nil) { + t.Error("HasManifest true for a nil root") + } +} + +// A nil root is what a caller has when discovery found nothing; it must not +// panic and must read as frontmatter-only. +func TestResolveNilRoot(t *testing.T) { + r, err := Resolve("a.md", parse(t, "page_id: 1\n"), nil) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Source != FromFrontmatter || !r.Managed() { + t.Errorf("source = %q managed = %v", r.Source, r.Managed()) + } +} diff --git a/internal/project/pages.go b/internal/project/pages.go index 4080355..f1f8786 100644 --- a/internal/project/pages.go +++ b/internal/project/pages.go @@ -127,6 +127,15 @@ func readEntry(named string, fields []frontmatter.Item) (Entry, error) { return e, nil } +// IsPageField reports whether name is a field markfluence understands on a +// page, in frontmatter or in an entry. +// +// Exported because telling a markfluence key from a foreign one is not a +// judgment a caller should make for itself: a docs tree carrying Jekyll's +// layout:/date: frontmatter has said nothing about Confluence, and a caller +// that counted any key at all would read every such file as claimed. +func IsPageField(name string) bool { return entryFields[name] != 0 } + // knownEntryFields lists the recognized field names, sorted so a message is // stable. func knownEntryFields() []string { From e9849bd9b6825c0bda57ba7b589ed307ce11c8ec Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 18:53:47 -0400 Subject: [PATCH 05/14] feat(linkindex): resolve page ids through the manifest too Build read each sibling's own frontmatter for page_id/title, so a file whose coordinates live in markfluence.yaml's pages: block had no entry in the index -- and every link to it would degrade to the "exists on disk, not published yet" warning and republish as plain text. #139 calls this out as the sharp touchpoint, and it is sharp in consequence rather than in plumbing: Build already takes the root and already keys idx.pages by the exact root-relative slash path pages: uses, so the change is to resolve through pagemeta instead of from mf alone. Resolving per path is also what makes a half-migrated tree correct throughout, with some files declaring their own coordinates and some leaving them to the manifest. A disagreement costs the file its page *entry* and nothing else. The first attempt skipped the whole file, matching what an unreadable or malformed one gets, and a test caught why that is worse: the anchors go unrecorded too, so FileExists goes false and every link *to* that file becomes LINK BROKEN across the tree, where the right answer is the ordinary unpublished warning. --- internal/linkindex/linkindex.go | 28 ++++++- internal/linkindex/linkindex_test.go | 108 +++++++++++++++++++++++++++ 2 files changed, 133 insertions(+), 3 deletions(-) diff --git a/internal/linkindex/linkindex.go b/internal/linkindex/linkindex.go index c48acba..a66c34f 100644 --- a/internal/linkindex/linkindex.go +++ b/internal/linkindex/linkindex.go @@ -24,6 +24,7 @@ import ( "strings" "github.com/mozilla/markfluence/internal/frontmatter" + "github.com/mozilla/markfluence/internal/pagemeta" "github.com/mozilla/markfluence/internal/project" ) @@ -43,7 +44,8 @@ type Index struct { } // Build walks root's tree once, via root.FS, collecting every *.md file's -// page_id/title (when it has one) and heading anchors. Walking through +// page_id/title (from either location it may live in -- see pagemeta) and +// heading anchors. Walking through // root.FS is what keeps the walk from ever descending a symlinked directory: // a symlink's directory entry reports its own type (a link, not a // directory), so fs.WalkDir calls the visit function for it once and does @@ -72,8 +74,28 @@ func Build(root *project.Root) (*Index, error) { // converting an unrelated one. return nil } - if id := mf.PageID(); id != "" { - idx.pages[path] = PageEntry{PageID: id, Title: mf.Title()} + // Through pagemeta, not from mf alone: a file whose coordinates live in + // markfluence.yaml's pages: block (#139) has no page_id in its own + // frontmatter, and reading only the file would leave it out of this + // index -- so every link to it would degrade to the "exists on disk, + // not published yet" warning and republish as plain text. Resolving per + // path is also what makes a half-migrated tree correct throughout, with + // some files declaring their own coordinates and some not. + // + // A disagreement is not this walk's business: it is reported, per file, + // by the command processing that file. Here it costs the file its + // *page entry* and nothing else -- one broken entry elsewhere in the + // tree must not block converting an unrelated file. + // + // Deliberately not a skip of the whole file, which is what an + // unreadable or malformed one gets. The anchors below must still be + // recorded, or FileExists goes false for it and every link *to* it + // becomes LINK BROKEN across the whole tree, where the right answer is + // the ordinary "exists on disk, not published yet" warning. + if meta, err := pagemeta.Resolve(path, mf, root); err == nil { + if id := strings.TrimSpace(meta.Fields["page_id"]); id != "" { + idx.pages[path] = PageEntry{PageID: id, Title: meta.Fields["title"]} + } } anchors := map[string]string{} for _, h := range extractHeadings(mf.Body) { diff --git a/internal/linkindex/linkindex_test.go b/internal/linkindex/linkindex_test.go index a486675..73000c2 100644 --- a/internal/linkindex/linkindex_test.go +++ b/internal/linkindex/linkindex_test.go @@ -156,3 +156,111 @@ func TestSetPageOverridesAndInjects(t *testing.T) { t.Errorf("Page(b.md) = %+v, %v; want the injected entry", got, ok) } } + +// writeProjectFile puts a markfluence.yaml at dir. +func writeProjectFile(t *testing.T, dir, body string) { + t.Helper() + if err := os.WriteFile(filepath.Join(dir, project.Filename), []byte(body), 0o644); err != nil { + t.Fatal(err) + } +} + +// The regression #139 names: a file whose coordinates live only in +// markfluence.yaml must be in the index, or every link to it degrades to the +// "exists on disk, not published yet" warning and republishes as plain text. +func TestBuildFindsAManifestOnlyPage(t *testing.T) { + dir := t.TempDir() + writeProjectFile(t, dir, "pages:\n target.md:\n title: Target\n page_id: 4242\n") + // Pristine: no frontmatter at all. + if err := os.WriteFile(filepath.Join(dir, "target.md"), []byte("# Target\n"), 0o644); err != nil { + t.Fatal(err) + } + root, err := project.Discover(dir) + if err != nil { + t.Fatal(err) + } + defer func() { _ = root.FS.Close() }() + + idx, err := Build(root) + if err != nil { + t.Fatalf("Build: %v", err) + } + e, ok := idx.Page("target.md") + if !ok { + t.Fatal("target.md is not in the index; a manifest-only page must be") + } + if e.PageID != "4242" || e.Title != "Target" { + t.Errorf("entry = %#v, want 4242/Target", e) + } +} + +// A half-migrated tree resolves correctly per file: one page declaring its own +// coordinates, one leaving them to the manifest. +func TestBuildHandlesAMixedTree(t *testing.T) { + dir := t.TempDir() + writeProjectFile(t, dir, "pages:\n manifest.md:\n title: M\n page_id: 2\n") + if err := os.WriteFile(filepath.Join(dir, "inline.md"), + []byte("---\ntitle: I\npage_id: 1\n---\n# I\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "manifest.md"), []byte("# M\n"), 0o644); err != nil { + t.Fatal(err) + } + root, err := project.Discover(dir) + if err != nil { + t.Fatal(err) + } + defer func() { _ = root.FS.Close() }() + + idx, err := Build(root) + if err != nil { + t.Fatalf("Build: %v", err) + } + for path, wantID := range map[string]string{"inline.md": "1", "manifest.md": "2"} { + e, ok := idx.Page(path) + if !ok { + t.Errorf("%s missing from the index", path) + continue + } + if e.PageID != wantID { + t.Errorf("%s page_id = %q, want %q", path, e.PageID, wantID) + } + } +} + +// A disagreement is the processing command's business, per file. One bad entry +// elsewhere in the tree must not stop an unrelated file from converting, so +// Build skips it exactly as it skips a malformed sibling. +func TestBuildSkipsAFileWhoseMetadataDisagrees(t *testing.T) { + dir := t.TempDir() + writeProjectFile(t, dir, "pages:\n bad.md:\n page_id: 2\n") + if err := os.WriteFile(filepath.Join(dir, "bad.md"), + []byte("---\npage_id: 999\n---\n# Bad\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "good.md"), + []byte("---\npage_id: 1\n---\n# Good\n"), 0o644); err != nil { + t.Fatal(err) + } + root, err := project.Discover(dir) + if err != nil { + t.Fatal(err) + } + defer func() { _ = root.FS.Close() }() + + idx, err := Build(root) + if err != nil { + t.Fatalf("Build: %v", err) + } + if _, ok := idx.Page("bad.md"); ok { + t.Error("bad.md is in the index; a file whose two locations disagree has no resolved id") + } + if _, ok := idx.Page("good.md"); !ok { + t.Error("good.md is missing; one bad entry must not affect an unrelated file") + } + // Its anchors are still walked -- a link *to* it resolves as an + // unpublished file rather than as a missing one. + if !idx.FileExists("bad.md") { + t.Error("bad.md is not even known to exist; the walk must still record it") + } +} From 6b659a8ed25123c2a02e1a3863b40fcaef3efd74 Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:01:34 -0400 Subject: [PATCH 06/14] feat(update): read page metadata from markfluence.yaml, and drop the page flags update now takes each file's title and page id from its own frontmatter or from a pages: entry for it, indistinguishably. Both places are legal and agreement is silent, which is what makes migration incremental; a coordinate disagreement (page_id/space/parent) fails that file, and a soft one (title/page_width/labels) warns with frontmatter winning. --title, --page-id and --page-width are gone. The rule is that flags describe the run and files describe the page: the first two were single-FILE-only, so docs/**/*.md could not be expressed at all, which is the whole reason #139 exists. --page-width was the only batch-ok one and so the real loss, but a project-wide page_width: says the same thing permanently instead of once per invocation. overrideNeedsSingleFile goes with them, and a test pins their absence so re-adding one is a deliberate act. A file neither place mentions is skipped, ok, exit 0, with no request made. Repositories legitimately hold markdown that is not published and drafts are a normal state, so a glob-driven CI run must not go red because somebody added a file. A file that *is* registered but has no page id still fails -- somebody claimed it and create has not run -- and human output distinguishes the two skips, since "no changes" and "not published by markfluence" are different facts. create reads entries too, which is read-side work and belongs here rather than in the write PR: ignoring an entry's page_id would publish a second page and leave the entry pointing at the first, which is #18's bug reached from the other direction. Its flags are unchanged -- create establishes metadata, so --space/--parent/--title are how an entry that does not exist yet gets bootstrapped. --json gains metadata_source on both commands ("frontmatter", "manifest", or null), because debugging "why did it publish to *that* page" in a CI log otherwise means reproducing the resolution by hand. Required and nullable, no omitempty. --- cmd/create/create.go | 42 +- cmd/create/create_test.go | 22 +- cmd/create/json.go | 19 +- cmd/create/json_test.go | 1 + cmd/update/json.go | 22 +- cmd/update/json_test.go | 6 +- cmd/update/update.go | 198 +-- cmd/update/update_test.go | 355 ++++-- docs/commands/markfluence_update.md | 71 +- internal/project/pages_test.go | 11 +- schema/json-output/v1.json | 1728 ++++++++++++++++++++++----- 11 files changed, 1918 insertions(+), 557 deletions(-) diff --git a/cmd/create/create.go b/cmd/create/create.go index 8c97c86..070c9fb 100644 --- a/cmd/create/create.go +++ b/cmd/create/create.go @@ -34,6 +34,7 @@ import ( "github.com/mozilla/markfluence/internal/labels" "github.com/mozilla/markfluence/internal/linkindex" "github.com/mozilla/markfluence/internal/pagedoc" + "github.com/mozilla/markfluence/internal/pagemeta" "github.com/mozilla/markfluence/internal/pageref" "github.com/mozilla/markfluence/internal/pagewidth" "github.com/mozilla/markfluence/internal/project" @@ -159,6 +160,9 @@ type record struct { // and carried rather than re-read so publish cannot disagree with what was // checked. labels labels.Set + // metadataSource is which location supplied this file's metadata, carried + // from preflight so the published result reports what was actually read. + metadataSource string // root bounds this file's image/parent reads and is what its attachments' // names and recorded Source are relative to. Discovered from the file's own // directory, cached across the batch by internal/project.Cache. @@ -183,6 +187,10 @@ type failure struct { filename, message string pageID, url string code jsonout.Code + // metadataSource is set when the failure happened after metadata + // resolution, so a --json consumer can still see which location supplied + // the coordinates that turned out to be wrong. + metadataSource string } // pageIDFailure is a phase-1 failure about a file's frontmatter page_id. create @@ -696,11 +704,28 @@ func resolveFile( return record{}, fmt.Errorf("building the link index: %w", err) } - title := resolveTitle(titleOpt, mf) + // A file's metadata may live in markfluence.yaml's pages: block rather + // than in the file (#139). create reads it for the same reason update + // does, and one reason more: a file already registered there carries a + // page_id, and ignoring it would publish a second page and leave the + // entry pointing at the first -- #18's bug, arrived at from the other + // direction. + // + // A coordinate disagreement fails the whole batch rather than this file, + // which falls out of preflight: create aborts if any file fails, and a + // page_id or parent wrong in one file means the batch's shape is not what + // the author thinks. + key, _ := pagemeta.KeyFor(root, abs) + meta, err := pagemeta.Resolve(key, mf, root) + if err != nil { + return record{}, err + } + + title := resolveTitle(titleOpt, meta.Fields) if title == "" { return record{}, errors.New("no title given (pass --title or add a 'title:' frontmatter field)") } - width, err := resolveWidth(pageWidthOpt, mf.Frontmatter, root) + width, err := resolveWidth(pageWidthOpt, meta.Fields, root) if err != nil { return record{}, err } @@ -714,7 +739,7 @@ func resolveFile( // Fatal here, and it has to be: a label Confluence splits on a space // publishes *successfully*, so unlike almost anything else phase 1 rejects, // there is no later run that can repair it. - labelSet, err := labels.Declared(mf.Lists, mf.Frontmatter) + labelSet, err := labels.Declared(meta.Lists, meta.Fields) if err != nil { return record{}, err } @@ -722,11 +747,11 @@ func resolveFile( // Before the space, parent, and duplicate-title lookups: a page_id that is // already taken or already broken is the most specific thing wrong with the // file, and reporting it first also spares three API calls the file cannot use. - if err := checkPageID(c, mf.PageID()); err != nil { + if err := checkPageID(c, strings.TrimSpace(meta.Fields["page_id"])); err != nil { return record{}, err } - spaceKey, err := resolveSpace(spaceOpt, mf.Frontmatter, root) + spaceKey, err := resolveSpace(spaceOpt, meta.Fields, root) if err != nil { return record{}, err } @@ -742,7 +767,7 @@ func resolveFile( return record{}, fmt.Errorf("space %q not found", spaceKey) } - parent, err := resolveParent(filename, mf.Frontmatter, inSetAbs, c, spaceID, root) + parent, err := resolveParent(filename, meta.Fields, inSetAbs, c, spaceID, root) if err != nil { return record{}, err } @@ -784,6 +809,7 @@ func resolveFile( return record{ filename: filename, absPath: abs, mdfile: mf, title: title, spaceKey: spaceKey, spaceID: spaceID, parent: parent, width: width, labels: labelSet, root: root, index: index, + metadataSource: string(meta.MetadataSource()), }, nil } @@ -994,11 +1020,11 @@ func writeBackFrontmatter(content string, r record, pageID, parentValue, parentC } // resolveTitle returns the effective title: --title overrides the frontmatter. -func resolveTitle(cliTitle string, mf *frontmatter.MarkdownFile) string { +func resolveTitle(cliTitle string, fields map[string]string) string { if cliTitle != "" { return cliTitle } - return mf.Title() + return strings.TrimSpace(fields["title"]) } // resolveSpace returns the space key to publish into: --space, then the diff --git a/cmd/create/create_test.go b/cmd/create/create_test.go index cb0f61a..c36e4f5 100644 --- a/cmd/create/create_test.go +++ b/cmd/create/create_test.go @@ -94,23 +94,21 @@ func TestCheckPageIDLocalCases(t *testing.T) { } func TestResolveTitle(t *testing.T) { - mf, err := frontmatter.Parse("f.md", "---\ntitle: FM Title\n---\nb\n") - if err != nil { - t.Fatal(err) - } - if got := resolveTitle("CLI Title", mf); got != "CLI Title" { + // Reads resolved metadata now, which may have come from the file's + // frontmatter or from its pages: entry -- create cannot tell, by design. + declared := map[string]string{"title": "FM Title"} + if got := resolveTitle("CLI Title", declared); got != "CLI Title" { t.Errorf("flag override = %q, want CLI Title", got) } - if got := resolveTitle("", mf); got != "FM Title" { - t.Errorf("frontmatter = %q, want FM Title", got) + if got := resolveTitle("", declared); got != "FM Title" { + t.Errorf("declared = %q, want FM Title", got) } - empty, err := frontmatter.Parse("f.md", "body, no frontmatter\n") - if err != nil { - t.Fatal(err) - } - if got := resolveTitle("", empty); got != "" { + if got := resolveTitle("", map[string]string{}); got != "" { t.Errorf("absent = %q, want empty", got) } + if got := resolveTitle("", map[string]string{"title": " "}); got != "" { + t.Errorf("whitespace = %q, want empty", got) + } } func TestResolveSpace(t *testing.T) { diff --git a/cmd/create/json.go b/cmd/create/json.go index c25d6d6..744d2ee 100644 --- a/cmd/create/json.go +++ b/cmd/create/json.go @@ -43,6 +43,11 @@ type createResult struct { warnings []string errMsg string code jsonout.Code + // metadataSource is which location supplied this file's page metadata + // ("frontmatter", "manifest", or empty when nothing claimed it, which for + // create is the ordinary case -- a new page's metadata often comes from + // flags alone). + metadataSource string } // newResult seeds a result with the fields known before creation is attempted. @@ -50,6 +55,7 @@ func newResult(r record) *createResult { return &createResult{ file: r.filename, title: r.title, space: r.spaceKey, dryRun: dryRunOpt, parentFile: nullableStr(r.parent.display), + metadataSource: r.metadataSource, } } @@ -128,8 +134,12 @@ type jsonCreateResult struct { Attachments []jsonout.Attachment `json:"attachments"` Warnings []string `json:"warnings"` Broken []string `json:"broken"` - Error *string `json:"error"` - Code *jsonout.Code `json:"code"` + // MetadataSource is null when nothing in the file or the manifest claimed + // this page -- a pointer rather than "" so null and a source named "" are + // not the same value. + MetadataSource *string `json:"metadata_source"` + Error *string `json:"error"` + Code *jsonout.Code `json:"code"` } func (r *createResult) jsonResult() jsonCreateResult { @@ -151,6 +161,8 @@ func (r *createResult) jsonResult() jsonCreateResult { Attachments: nonNilAttachments(r.attachments), Warnings: nonNilStrings(r.warnings), Broken: nonNilStrings(r.broken), + + MetadataSource: nullableStr(r.metadataSource), } if !r.ok { res.Error = &r.errMsg @@ -252,6 +264,9 @@ func abortedResult(file, status string, f failure) jsonCreateResult { Attachments: []jsonout.Attachment{}, Warnings: []string{}, Broken: []string{}, + // A file rejected in preflight may not have reached metadata + // resolution at all, so this is null rather than guessed at. + MetadataSource: nullableStr(f.metadataSource), } if f.message != "" { msg := f.message diff --git a/cmd/create/json_test.go b/cmd/create/json_test.go index 228c2cd..934e0bf 100644 --- a/cmd/create/json_test.go +++ b/cmd/create/json_test.go @@ -135,6 +135,7 @@ func TestJSONResultCreated(t *testing.T) { "attachments": [], "warnings": [], "broken": [], + "metadata_source": null, "error": null, "code": null }` diff --git a/cmd/update/json.go b/cmd/update/json.go index 9d4787a..b4f1b37 100644 --- a/cmd/update/json.go +++ b/cmd/update/json.go @@ -41,6 +41,15 @@ type updateResult struct { warnings []string errMsg string code jsonout.Code + // metadataSource is which location supplied this file's page metadata + // ("frontmatter", "manifest", or empty when nothing claims the file). + // Debugging "why did it publish to *that* page" in a CI log otherwise means + // reproducing the resolution by hand. + metadataSource string + // unmanaged distinguishes the two reasons a file is skipped: nothing + // claims it, or it is unchanged since the page's last version. Human + // output says which; --json has status plus metadata_source. + unmanaged bool } // fail marks the result failed with an error and code, and returns it for a @@ -63,6 +72,10 @@ func (r *updateResult) renderHuman() { return } if r.status == statusSkipped { + if r.unmanaged { + ui.Info(prefix + " Skipping -- not published by markfluence") + return + } ui.Info(prefix + " Skipping -- no changes") return } @@ -103,8 +116,11 @@ type jsonUpdateResult struct { Attachments []jsonout.Attachment `json:"attachments"` Warnings []string `json:"warnings"` Broken []string `json:"broken"` - Error *string `json:"error"` - Code *jsonout.Code `json:"code"` + // MetadataSource is null for a file nothing claims, which is why it is a + // pointer rather than an empty string: "" would read as a source named "". + MetadataSource *string `json:"metadata_source"` + Error *string `json:"error"` + Code *jsonout.Code `json:"code"` } type jsonUpdateVersion struct { @@ -127,6 +143,8 @@ func (r *updateResult) jsonResult() jsonUpdateResult { Attachments: nonNilAttachments(r.attachments), Warnings: nonNilStrings(r.warnings), Broken: nonNilStrings(r.broken), + + MetadataSource: strOrNil(r.metadataSource), } // version is present once we know the live version (all non-early failures). if r.versionPrev != 0 || r.versionNew != 0 { diff --git a/cmd/update/json_test.go b/cmd/update/json_test.go index 2907472..400dce3 100644 --- a/cmd/update/json_test.go +++ b/cmd/update/json_test.go @@ -43,8 +43,9 @@ func TestJSONResultPublished(t *testing.T) { pageID: "123", title: "Foo", space: "ENG", url: "https://wiki.example.net/wiki/spaces/ENG/pages/123/Foo", versionPrev: 3, versionNew: 4, - width: &jsonout.PageWidth{Value: "max", Default: false}, - attachments: []jsonout.Attachment{{Action: "updated", Filename: "d.png"}}, + width: &jsonout.PageWidth{Value: "max", Default: false}, + attachments: []jsonout.Attachment{{Action: "updated", Filename: "d.png"}}, + metadataSource: "frontmatter", } got, err := json.MarshalIndent(r.jsonResult(), "", " ") if err != nil { @@ -76,6 +77,7 @@ func TestJSONResultPublished(t *testing.T) { ], "warnings": [], "broken": [], + "metadata_source": "frontmatter", "error": null, "code": null }` diff --git a/cmd/update/update.go b/cmd/update/update.go index c5f8fd0..ce4d17b 100644 --- a/cmd/update/update.go +++ b/cmd/update/update.go @@ -19,6 +19,7 @@ import ( "github.com/mozilla/markfluence/internal/labels" "github.com/mozilla/markfluence/internal/linkindex" "github.com/mozilla/markfluence/internal/pagedoc" + "github.com/mozilla/markfluence/internal/pagemeta" "github.com/mozilla/markfluence/internal/pageref" "github.com/mozilla/markfluence/internal/pagewidth" "github.com/mozilla/markfluence/internal/project" @@ -27,12 +28,9 @@ import ( ) var ( - message string - force bool - dryRun bool - titleFlag string - pageIDFlag string - pageWidthFlag string + message string + force bool + dryRun bool ) // Cmd is the update command. @@ -40,38 +38,50 @@ var Cmd = &cobra.Command{ Use: "update FILE...", Short: "Publish one or more markdown files to Confluence pages", Long: "Publish one or more markdown FILEs to Confluence pages.\n\n" + - "Title and page id are read from each file's YAML frontmatter; --title and\n" + - "--page-id override the frontmatter (and require a single FILE). A page id is\n" + - "required (from --page-id or frontmatter); update errors if none is set.\n\n" + - "Page width is asserted only when set via --page-width, a page_width\n" + - "frontmatter line, or a page_width: in markfluence.yaml -- otherwise the\n" + - "live page's width is left untouched.\n" + - "Labels work the same way: a labels: line is asserted exactly (anything on\n" + - "the page the file does not list is removed), and no labels: line means the\n" + - "page's labels are left alone, not even read.\n\n" + - "update never writes back to the file, so fixing a wrong page_id is always\n" + - "safe: the file is exactly as you left it. A page_id that no longer resolves\n" + - "fails that file and says what to do about it; one that is not a numeric id\n" + - "at all is reported without asking Confluence.\n\n" + + "Each file's title and page id come from its own YAML frontmatter, or from\n" + + "a 'pages:' entry for it in markfluence.yaml -- a file can stay pristine and\n" + + "keep its metadata there instead. Both places are legal and agreement is\n" + + "silent; where they disagree about page_id, space or parent the file fails,\n" + + "and where they disagree about title, page_width or labels the frontmatter\n" + + "wins with a warning.\n\n" + + "A file that neither place mentions is skipped, not failed: a repository\n" + + "legitimately holds markdown that is not published, so a glob over a docs\n" + + "tree does not go red because somebody added a draft. A file that IS\n" + + "registered but has no page id fails -- something claimed it and the page\n" + + "has not been created yet.\n\n" + + "There are no per-page flags. Page metadata lives in the file or its entry,\n" + + "which is what lets one invocation publish 'docs/**/*.md'; a flag would have\n" + + "to name a single file. A project-wide 'page_width:' in markfluence.yaml is\n" + + "how a whole tree gets one width.\n\n" + + "Page width is asserted only when something declares it -- the file, its\n" + + "entry, or the project-wide default -- otherwise the live page's width is\n" + + "left untouched. Labels work the same way: a labels: line is asserted\n" + + "exactly (anything on the page the file does not list is removed), and no\n" + + "labels: line means the page's labels are left alone, not even read.\n\n" + + "update never writes back to the file or to markfluence.yaml, so fixing a\n" + + "wrong page_id is always safe: nothing is as you left it by accident. A\n" + + "page_id that no longer resolves fails that file and says what to do about\n" + + "it; one that is not a numeric id at all is reported without asking\n" + + "Confluence.\n\n" + "A file that has not changed since the page's last version is skipped,\n" + "compared by mtime, unless --force is given. Each file is processed\n" + "independently; the command exits non-zero if any file failed.\n\n" + "--dry-run previews the version bump, attachment uploads and any width or\n" + "label change without writing to Confluence. It honours the mtime skip and\n" + "--force exactly as a real run does, so its forecast matches.", - Example: " # Publish a file, taking the page id from its frontmatter\n" + + Example: " # Publish a file, taking the page id from its frontmatter or its entry\n" + " markfluence update docs/managing_an_incident.md\n\n" + + " # Publish a whole tree -- the CI shape: metadata comes from the files\n" + + " # and from markfluence.yaml, so nothing has to be passed per file\n" + + " markfluence update docs/**/*.md\n\n" + " # Publish a batch with a version message\n" + " markfluence update docs/*.md --message \"Bulk update\"\n\n" + " # Republish even though the file has not changed\n" + " markfluence update docs/foo.md --force\n\n" + - " # Override the target page, or rename it\n" + - " markfluence update page.md --page-id 123456\n" + - " markfluence update page.md --title \"New Title\"\n\n" + - " # Set the width across a batch\n" + - " markfluence update docs/*.md --page-width wide\n\n" + " # Preview, write nothing\n" + - " markfluence update docs/*.md --dry-run", + " markfluence update docs/*.md --dry-run\n\n" + + " # See which location supplied each file's metadata\n" + + " markfluence update docs/*.md --json | jq -r '.results[] | \"\\(.file) \\(.metadata_source)\"'", Args: cobra.MinimumNArgs(1), ValidArgsFunction: completion.MarkdownFiles, RunE: run, @@ -82,22 +92,9 @@ func init() { Cmd.Flags().BoolVar(&force, "force", false, "Skip the file-mtime check and always update the page.") Cmd.Flags().BoolVar(&dryRun, "dry-run", false, "Preview what would be published without writing to Confluence.") - Cmd.Flags().StringVar(&titleFlag, "title", "", - "Override the page title (requires a single FILE).") - Cmd.Flags().StringVar(&pageIDFlag, "page-id", "", - "Override the target page id (requires a single FILE).") - Cmd.Flags().StringVar(&pageWidthFlag, "page-width", "", - "Override the page width: narrow, wide, or max.") - - completion.RegisterFlag(Cmd, "page-width", completion.Values(pagewidth.Vocabulary()...)) } func run(cmd *cobra.Command, args []string) error { - if overrideNeedsSingleFile(titleFlag, pageIDFlag, len(args)) { - ui.Error("--title/--page-id apply to a single page; pass exactly one FILE") - return ui.ErrSilent - } - url, _ := cmd.Flags().GetString("url") username, _ := cmd.Flags().GetString("username") cloudID, _ := cmd.Flags().GetString("cloud-id") @@ -180,18 +177,57 @@ func processFile( return r.fail(err, jsonout.CodeValidation) } - title, titlePresent, pageID := resolveTitlePageID(titleFlag, pageIDFlag, mf) + // The root comes first now: a file's metadata may live in the project + // file's pages: block rather than in the file, so nothing local can be + // checked until the root is known. The walk is cached, so asking early + // costs nothing; building the link *index* is still below the mtime check, + // so a file that is skipped never pays for one. + abs, err := filepath.Abs(filename) + if err != nil { + return r.fail(err, jsonout.CodeIO) + } + root, err := roots.Resolve(filepath.Dir(abs)) + if err != nil { + return r.fail(project.RootError(err), rootErrorCode(err)) + } + key, _ := pagemeta.KeyFor(root, abs) + meta, err := pagemeta.Resolve(key, mf, root) + if err != nil { + // A coordinate disagreement: the two locations name different pages, and + // publishing to either would be a guess about which one the author + // means. Only this file fails; the rest of the batch proceeds. + return r.fail(err, jsonout.CodeValidation) + } + r.metadataSource = string(meta.MetadataSource()) + r.warnings = append(r.warnings, meta.Warnings...) + + // Nothing anywhere claims this file, so there is nothing to publish and + // nothing wrong (#139). Repositories legitimately hold markdown that is not + // published to Confluence, drafts are a normal state, and a glob-driven CI + // run must not go red because somebody added a file. A file that *is* + // claimed but has no page_id still fails below: somebody registered it and + // create has not run. + if !meta.Managed() { + r.ok = true + r.status = statusSkipped + r.unmanaged = true + return r + } + + title, titlePresent, pageID := resolveTitlePageID(meta.Fields) // Before the request, like the page-id check below: an empty title is a // local defect, and paying for a round trip to discover it is waste. Only a // title that is *present* and empty is wrong -- an absent title means the // file does not manage the page's title, which is honoured further down. if title == "" && titlePresent { return r.fail(errors.New( - "frontmatter has an empty 'title:'; give it a value, remove it to keep the "+ - "live page title, or pass --title"), jsonout.CodeValidation) + "the title is present but empty; give it a value, or remove it to keep the "+ + "live page title"), jsonout.CodeValidation) } if pageID == "" { - return r.fail(errors.New("no page id: set page_id in frontmatter or pass --page-id"), + return r.fail(errors.New( + "no page id: set page_id in this file's frontmatter or in its "+ + project.Filename+" entry, or create the page first"), jsonout.CodeValidation) } r.pageID = pageID @@ -200,20 +236,7 @@ func processFile( if !pageref.IsDigits(pageID) { return r.fail(errors.New(pageref.NotNumericMessage(pageID)), jsonout.CodeValidation) } - // The root is resolved here rather than beside the link index below, - // because the width chain now ends at the project file and every local - // check has to stay ahead of the first request. The walk is cached, so - // asking early costs nothing; building the *index* is not moved, so a file - // the mtime check skips still never pays for one. - abs, err := filepath.Abs(filename) - if err != nil { - return r.fail(err, jsonout.CodeIO) - } - root, err := roots.Resolve(filepath.Dir(abs)) - if err != nil { - return r.fail(project.RootError(err), rootErrorCode(err)) - } - width, applyWidth, err := resolveWidth(pageWidthFlag, mf, root) + width, applyWidth, err := resolveWidth(meta.Fields, root) if err != nil { return r.fail(err, jsonout.CodeValidation) } @@ -222,7 +245,7 @@ func processFile( // a space publishes *successfully* as several labels that read back as none // of what the file says, so there is no later run that can clean it up -- // see docs/confluence/labels.md. - labelSet, err := labels.Declared(mf.Lists, mf.Frontmatter) + labelSet, err := labels.Declared(meta.Lists, meta.Fields) if err != nil { return r.fail(err, jsonout.CodeValidation) } @@ -366,46 +389,33 @@ func (r *updateResult) previewWidth( r.widthSet = true } -// overrideNeedsSingleFile reports whether a per-page override (--title/--page-id) -// was given with anything other than exactly one FILE. --page-width is exempt (a -// uniform width change across a batch is sensible). -func overrideNeedsSingleFile(cliTitle, cliPageID string, nFiles int) bool { - return (cliTitle != "" || cliPageID != "") && nFiles != 1 -} - -// resolveTitlePageID resolves the effective title and page id, letting the CLI -// flags override the file's frontmatter. An empty page id is an error; an empty -// title is an error only when the frontmatter key is present, which is what -// titlePresent reports. An absent title falls back to the live page title later. +// resolveTitlePageID reads the effective title and page id out of a file's +// resolved metadata, which may have come from its frontmatter or from a pages: +// entry (#139). // -// --title wins over both, as every other override does, so it satisfies a -// present-but-empty frontmatter title rather than tripping over it. -func resolveTitlePageID(cliTitle, cliPageID string, mf *frontmatter.MarkdownFile) ( +// An empty page id is an error; an empty title is an error only when the key is +// *present*, which is what titlePresent reports -- an absent title means the +// file does not manage the page's title and falls back to the live one further +// down. There is no flag to consider any more: update consumes metadata and has +// no way to invent any (#139's "flags describe the run; files describe the +// page"). +func resolveTitlePageID(fields map[string]string) ( title string, titlePresent bool, pageID string) { - title = cliTitle - if title == "" { - title, titlePresent = mf.TitleField() - } - pageID = cliPageID - if pageID == "" { - pageID = mf.PageID() - } - return title, titlePresent, pageID + title, titlePresent = fields["title"] + return strings.TrimSpace(title), titlePresent, strings.TrimSpace(fields["page_id"]) } -// resolveWidth resolves the page width to assert: --page-width, then the -// frontmatter page_width, then the project file's default (#100's chain, flag > -// frontmatter > project file). It returns apply=false when -// neither --page-width nor a frontmatter page_width is set, meaning the live -// page's width should be left untouched. -func resolveWidth(cliPageWidth string, mf *frontmatter.MarkdownFile, - root *project.Root) (pagewidth.Width, bool, error) { - if cliPageWidth != "" { - w, err := pagewidth.Declared(map[string]string{"page_width": cliPageWidth}) - return w, err == nil, err - } - if raw, ok := mf.Frontmatter["page_width"]; ok && strings.TrimSpace(raw) != "" { - w, err := pagewidth.Declared(mf.Frontmatter) +// resolveWidth resolves the page width to assert: the file's own page_width -- +// from its frontmatter or its pages: entry, whichever supplied it -- then the +// project file's project-wide default. It returns apply=false when neither is +// set, meaning the live page's width is left untouched. +// +// --page-width is gone with the other page-metadata flags (#139): a uniform +// width across a batch is what the project-wide default is for, and it says so +// permanently rather than per invocation. +func resolveWidth(fields map[string]string, root *project.Root) (pagewidth.Width, bool, error) { + if raw, ok := fields["page_width"]; ok && strings.TrimSpace(raw) != "" { + w, err := pagewidth.Declared(fields) return w, err == nil, err } // The project file's default, and the one level of the chain that changes diff --git a/cmd/update/update_test.go b/cmd/update/update_test.go index 1945ffd..66a6d50 100644 --- a/cmd/update/update_test.go +++ b/cmd/update/update_test.go @@ -22,121 +22,108 @@ import ( ) func TestResolveTitlePageID(t *testing.T) { - mf, err := frontmatter.Parse("f.md", "---\ntitle: FM Title\npage_id: 111\n---\nbody\n") - if err != nil { - t.Fatal(err) - } - - tests := []struct { - name string - cliTitle, cliPageID string + // The flags are gone (#139): update consumes metadata and has no way to + // invent any, so this reads whatever pagemeta resolved -- from the file's + // frontmatter or from its pages: entry, indistinguishably by design. + tests := map[string]struct { + fields map[string]string wantTitle, wantPageID string + wantPresent bool }{ - {"flags override frontmatter", "CLI Title", "222", "CLI Title", "222"}, - {"frontmatter when no flags", "", "", "FM Title", "111"}, - {"only page-id overridden", "", "222", "FM Title", "222"}, - {"only title overridden", "CLI Title", "", "CLI Title", "111"}, - } - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - title, _, pageID := resolveTitlePageID(tc.cliTitle, tc.cliPageID, mf) - if title != tc.wantTitle || pageID != tc.wantPageID { - t.Errorf("resolveTitlePageID = %q/%q, want %q/%q", - title, pageID, tc.wantTitle, tc.wantPageID) + "both present": { + map[string]string{"title": "T", "page_id": "111"}, "T", "111", true}, + "page id only": { + map[string]string{"page_id": "111"}, "", "111", false}, + "title present but empty": { + map[string]string{"title": "", "page_id": "111"}, "", "111", true}, + "whitespace title is empty but present": { + map[string]string{"title": " ", "page_id": "111"}, "", "111", true}, + "nothing": {map[string]string{}, "", "", false}, + } + for name, tc := range tests { + t.Run(name, func(t *testing.T) { + title, present, pageID := resolveTitlePageID(tc.fields) + if title != tc.wantTitle || pageID != tc.wantPageID || present != tc.wantPresent { + t.Errorf("= %q/%v/%q, want %q/%v/%q", + title, present, pageID, tc.wantTitle, tc.wantPresent, tc.wantPageID) } }) } } -func TestResolveTitlePageIDEmptyWhenAbsent(t *testing.T) { - mf, err := frontmatter.Parse("f.md", "body only, no frontmatter\n") - if err != nil { - t.Fatal(err) - } - title, present, pageID := resolveTitlePageID("", "", mf) - if present { - t.Error("titlePresent = true, want false for a file with no frontmatter") +// The three page-metadata flags are gone, and their absence is pinned rather +// than incidental: re-adding one would put back the single-FILE-only shape that +// made docs/**/*.md inexpressible from CI, which is the whole reason #139 +// exists. +func TestPageMetadataFlagsAreGone(t *testing.T) { + for _, name := range []string{"title", "page-id", "page-width"} { + if f := Cmd.Flags().Lookup(name); f != nil { + t.Errorf("--%s exists; page metadata belongs in the file or its entry, not in a flag", name) + } } - if title != "" || pageID != "" { - t.Errorf("resolveTitlePageID = %q/%q, want empty/empty", title, pageID) + // The invocation flags stay: they describe the run, not the page. + for _, name := range []string{"message", "force", "dry-run"} { + if f := Cmd.Flags().Lookup(name); f == nil { + t.Errorf("--%s is missing; it describes the run and should have stayed", name) + } } } func TestResolveWidth(t *testing.T) { - withFM, err := frontmatter.Parse("f.md", "---\ntitle: T\npage_width: wide\n---\nb\n") - if err != nil { - t.Fatal(err) - } - noWidth, err := frontmatter.Parse("f.md", "---\ntitle: T\n---\nb\n") - if err != nil { - t.Fatal(err) - } - noFM, err := frontmatter.Parse("f.md", "b\n") - if err != nil { - t.Fatal(err) - } + withWidth := map[string]string{"title": "T", "page_width": "wide"} + noWidth := map[string]string{"title": "T"} // A project file declaring a width, and one declaring nothing. declared := &project.Root{File: "/repo/markfluence.yaml", Config: project.Config{PageWidth: "narrow"}} bare := &project.Root{File: "/repo/markfluence.yaml"} - t.Run("flag overrides frontmatter", func(t *testing.T) { - w, apply, err := resolveWidth("narrow", withFM, bare) - if err != nil || !apply || w != pagewidth.Narrow { - t.Fatalf("= %q/%v/%v, want narrow/true/nil", w, apply, err) - } - }) - t.Run("frontmatter when no flag", func(t *testing.T) { - w, apply, err := resolveWidth("", withFM, bare) + t.Run("the file's own width", func(t *testing.T) { + w, apply, err := resolveWidth(withWidth, bare) if err != nil || !apply || w != pagewidth.Wide { t.Fatalf("= %q/%v/%v, want wide/true/nil", w, apply, err) } }) - t.Run("no flag and no frontmatter width -> skip", func(t *testing.T) { - if _, apply, err := resolveWidth("", noWidth, bare); err != nil || apply { + t.Run("no width anywhere means no width request", func(t *testing.T) { + if _, apply, err := resolveWidth(noWidth, bare); err != nil || apply { t.Fatalf("= apply %v err %v, want false/nil", apply, err) } - if _, apply, err := resolveWidth("", noFM, bare); err != nil || apply { - t.Fatalf("(no frontmatter) = apply %v err %v, want false/nil", apply, err) + if _, apply, err := resolveWidth(map[string]string{}, bare); err != nil || apply { + t.Fatalf("(no fields) = apply %v err %v, want false/nil", apply, err) } }) - t.Run("invalid flag errors", func(t *testing.T) { - if _, apply, err := resolveWidth("huge", noFM, bare); err == nil || apply { + t.Run("an invalid width errors", func(t *testing.T) { + bad := map[string]string{"page_width": "huge"} + if _, apply, err := resolveWidth(bad, bare); err == nil || apply { t.Fatalf("= apply %v err %v, want false/error", apply, err) } }) - // The behavior change: a project-wide page_width makes update assert a - // width on a file that declares none. Before, that file's live width was - // left alone. It is what "declared means asserted" (L9) means one level up. + // The behavior change #100 made: a project-wide page_width makes update + // assert a width on a file that declares none, where before that file's + // live width was left alone. It is what "declared means asserted" (L9) + // means one level up. t.Run("project file makes update assert a width", func(t *testing.T) { - w, apply, err := resolveWidth("", noWidth, declared) + w, apply, err := resolveWidth(noWidth, declared) if err != nil || !apply || w != pagewidth.Narrow { t.Fatalf("= %q/%v/%v, want narrow/true/nil", w, apply, err) } }) - t.Run("flag beats the project file", func(t *testing.T) { - w, apply, err := resolveWidth("wide", noWidth, declared) - if err != nil || !apply || w != pagewidth.Wide { - t.Fatalf("= %q/%v/%v, want wide/true/nil", w, apply, err) - } - }) - t.Run("frontmatter beats the project file", func(t *testing.T) { - w, apply, err := resolveWidth("", withFM, declared) + t.Run("the file beats the project file", func(t *testing.T) { + w, apply, err := resolveWidth(withWidth, declared) if err != nil || !apply || w != pagewidth.Wide { t.Fatalf("= %q/%v/%v, want wide/true/nil", w, apply, err) } }) // The escape hatch has to keep working: a project that omits the key gets - // no width request at all, which is the pre-#100 behavior. + // no width request at all. t.Run("no project width means no width request", func(t *testing.T) { - if _, apply, err := resolveWidth("", noWidth, declaredNothing()); err != nil || apply { + if _, apply, err := resolveWidth(noWidth, declaredNothing()); err != nil || apply { t.Fatalf("= apply %v err %v, want false/nil", apply, err) } }) t.Run("invalid project width names the project file", func(t *testing.T) { bad := &project.Root{File: "/repo/markfluence.yaml", Config: project.Config{PageWidth: "huge"}} - _, apply, err := resolveWidth("", noWidth, bad) + _, apply, err := resolveWidth(noWidth, bad) if err == nil || apply { t.Fatalf("= apply %v err %v, want false/error", apply, err) } @@ -145,7 +132,7 @@ func TestResolveWidth(t *testing.T) { } }) t.Run("nil root is not a panic", func(t *testing.T) { - if _, apply, err := resolveWidth("", noWidth, nil); err != nil || apply { + if _, apply, err := resolveWidth(noWidth, nil); err != nil || apply { t.Fatalf("= apply %v err %v, want false/nil", apply, err) } }) @@ -180,27 +167,6 @@ func TestRootErrorCode(t *testing.T) { } } -func TestOverrideNeedsSingleFile(t *testing.T) { - tests := []struct { - name string - cliTitle, cliPageID string - nFiles int - want bool - }{ - {"title with two files", "T", "", 2, true}, - {"page-id with two files", "", "9", 2, true}, - {"page-id with one file", "", "9", 1, false}, - {"no overrides, many files", "", "", 3, false}, - } - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - if got := overrideNeedsSingleFile(tc.cliTitle, tc.cliPageID, tc.nFiles); got != tc.want { - t.Errorf("overrideNeedsSingleFile = %v, want %v", got, tc.want) - } - }) - } -} - // TestProcessFileRejectsNonNumericPageID covers the local half of the fix: a // page_id that is not an id never reaches the API, so the reader gets a sentence // instead of a 400 body. The client points at a host that does not resolve, so a @@ -396,9 +362,6 @@ func TestResolveTitlePageIDSeparatesAbsentFromEmpty(t *testing.T) { {"present and empty", "---\ntitle:\npage_id: 1\n---\nb\n", "", "", true}, {"present and null", "---\ntitle: null\npage_id: 1\n---\nb\n", "", "", true}, {"present with value", "---\ntitle: T\npage_id: 1\n---\nb\n", "", "T", true}, - // --title wins, as every other override does, so it satisfies a - // present-but-empty frontmatter title rather than tripping over it. - {"flag over empty", "---\ntitle:\npage_id: 1\n---\nb\n", "CLI", "CLI", false}, } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { @@ -406,7 +369,7 @@ func TestResolveTitlePageIDSeparatesAbsentFromEmpty(t *testing.T) { if err != nil { t.Fatal(err) } - title, present, _ := resolveTitlePageID(tc.cliTitle, "", mf) + title, present, _ := resolveTitlePageID(mf.Frontmatter) if title != tc.wantTitle || present != tc.wantPresent { t.Errorf("resolveTitlePageID = %q/%v, want %q/%v", title, present, tc.wantTitle, tc.wantPresent) @@ -431,7 +394,7 @@ func TestProcessFileRejectsEmptyTitle(t *testing.T) { if r.ok { t.Fatal("a present-but-empty title must fail the file") } - if !strings.Contains(r.errMsg, "empty 'title:'") { + if !strings.Contains(r.errMsg, "present but empty") { t.Errorf("errMsg = %q, want the empty-title sentence", r.errMsg) } if r.code != jsonout.CodeValidation { @@ -892,3 +855,201 @@ func TestProcessFileFrontmatterWidthBeatsProjectWidth(t *testing.T) { } } } + +// --- manifest metadata -------------------------------------------------------- + +// writeManifestProject writes a markfluence.yaml and a markdown file under one +// root, returning the file's path. +func writeManifestProject(t *testing.T, projectFile, md string) string { + t.Helper() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), []byte(projectFile), 0o644); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, "f.md") + if err := os.WriteFile(path, []byte(md), 0o644); err != nil { + t.Fatal(err) + } + return path +} + +// The point of #139: a pristine file, published from its manifest entry. +func TestProcessFilePublishesFromAManifestEntry(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + switch r.Method { + case http.MethodGet: + _, _ = w.Write([]byte(pageWithVersion("1", 3, "2020-01-01T00:00:00Z"))) + default: + _, _ = w.Write([]byte(pageWithVersion("1", 4, "2026-01-01T00:00:00Z"))) + } + }) + // No frontmatter at all. + path := writeManifestProject(t, + "pages:\n f.md:\n title: From The Manifest\n page_id: 1\n", "# Hello\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if !r.ok || r.status != statusPublished { + t.Fatalf("result = %+v, want published", r) + } + if r.title != "From The Manifest" { + t.Errorf("title = %q, want the manifest's", r.title) + } + if r.metadataSource != "manifest" { + t.Errorf("metadata_source = %q, want manifest", r.metadataSource) + } +} + +// A file nothing claims is skipped, ok, with no request made -- which is what +// keeps a glob over a docs tree from going red when somebody adds a draft. The +// server fails any request so the skip has to be local. +func TestProcessFileSkipsAnUnmanagedFile(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + path := writeManifestProject(t, "pages:\n other.md:\n page_id: 9\n", "# Draft\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if !r.ok || r.status != statusSkipped { + t.Fatalf("result = %+v, want a successful skip", r) + } + if !r.unmanaged { + t.Error("unmanaged = false; the skip reason must be distinguishable from the mtime skip") + } + if r.metadataSource != "" { + t.Errorf("metadata_source = %q, want empty for an unclaimed file", r.metadataSource) + } +} + +// A docs tree carrying Jekyll frontmatter has said nothing about Confluence. +func TestProcessFileSkipsForeignFrontmatter(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + path := writeManifestProject(t, "space: ENG\n", + "---\nlayout: post\ndate: 2026-01-01\n---\n# Post\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if !r.ok || r.status != statusSkipped || !r.unmanaged { + t.Fatalf("result = %+v, want a successful unmanaged skip", r) + } +} + +// A file that IS registered but has no page id fails: something claimed it and +// create has not run. This is the case that must not be swept into the skip. +func TestProcessFileRegisteredWithNoPageIDFails(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + path := writeManifestProject(t, "pages:\n f.md:\n title: Claimed\n", "# Hello\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if r.ok { + t.Fatalf("result = %+v, want a failure", r) + } + if !strings.Contains(r.errMsg, "no page id") { + t.Errorf("errMsg = %q, want the no-page-id message", r.errMsg) + } + if !strings.Contains(r.errMsg, project.Filename) { + t.Errorf("errMsg = %q, want it to mention the project file as a place to set one", r.errMsg) + } +} + +// The clobber case: a page_id in two places naming two pages. Publishing to +// either would be a guess, so the file fails and nothing is written. +func TestProcessFileCoordinateDisagreementFails(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + path := writeManifestProject(t, "pages:\n f.md:\n page_id: 1\n", + "---\npage_id: 999\n---\n# Hello\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if r.ok { + t.Fatalf("result = %+v, want a failure", r) + } + if !strings.Contains(r.errMsg, "disagree about where this page is") { + t.Errorf("errMsg = %q, want the disagreement message", r.errMsg) + } + if r.code != jsonout.CodeValidation { + t.Errorf("code = %q, want VALIDATION", r.code) + } +} + +// A soft disagreement is visible and recoverable, so it warns and the file wins. +func TestProcessFileSoftDisagreementWarns(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + switch r.Method { + case http.MethodGet: + _, _ = w.Write([]byte(pageWithVersion("1", 3, "2020-01-01T00:00:00Z"))) + default: + _, _ = w.Write([]byte(pageWithVersion("1", 4, "2026-01-01T00:00:00Z"))) + } + }) + path := writeManifestProject(t, + "pages:\n f.md:\n title: Manifest Title\n page_id: 1\n", + "---\ntitle: File Title\n---\n# Hello\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if !r.ok { + t.Fatalf("result = %+v, want success", r) + } + if r.title != "File Title" { + t.Errorf("title = %q, want the file's", r.title) + } + if len(r.warnings) == 0 { + t.Fatal("no warnings; a soft disagreement must be reported") + } + if !strings.Contains(r.warnings[0], "title") { + t.Errorf("warnings = %#v, want one about title", r.warnings) + } + // Both locations spoke, and frontmatter is the one that won. + if r.metadataSource != "frontmatter" { + t.Errorf("metadata_source = %q, want frontmatter", r.metadataSource) + } +} + +// A batch mixes the two: one file unmanaged, one publishing. The unmanaged one +// must not fail the batch. +func TestRunBatchSkipsUnmanagedAndPublishesTheRest(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + switch r.Method { + case http.MethodGet: + _, _ = w.Write([]byte(pageWithVersion("1", 3, "2020-01-01T00:00:00Z"))) + default: + _, _ = w.Write([]byte(pageWithVersion("1", 4, "2026-01-01T00:00:00Z"))) + } + }) + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), + []byte("pages:\n published.md:\n title: P\n page_id: 1\n"), 0o644); err != nil { + t.Fatal(err) + } + for _, name := range []string{"published.md", "draft.md"} { + if err := os.WriteFile(filepath.Join(dir, name), []byte("# "+name+"\n"), 0o644); err != nil { + t.Fatal(err) + } + } + roots := project.NewCache("") + defer roots.Close() + indexes := linkindex.NewCache() + users := pagedoc.NewUserCache() + + got := map[string]string{} + for _, name := range []string{"published.md", "draft.md"} { + r := processFile(filepath.Join(dir, name), c, roots, indexes, users) + if !r.ok { + t.Fatalf("%s failed: %s", name, r.errMsg) + } + got[name] = r.status + } + if got["published.md"] != statusPublished { + t.Errorf("published.md status = %q, want published", got["published.md"]) + } + if got["draft.md"] != statusSkipped { + t.Errorf("draft.md status = %q, want skipped", got["draft.md"]) + } +} diff --git a/docs/commands/markfluence_update.md b/docs/commands/markfluence_update.md index 3765666..a914839 100644 --- a/docs/commands/markfluence_update.md +++ b/docs/commands/markfluence_update.md @@ -6,21 +6,35 @@ Publish one or more markdown files to Confluence pages Publish one or more markdown FILEs to Confluence pages. -Title and page id are read from each file's YAML frontmatter; --title and ---page-id override the frontmatter (and require a single FILE). A page id is -required (from --page-id or frontmatter); update errors if none is set. - -Page width is asserted only when set via --page-width, a page_width -frontmatter line, or a page_width: in markfluence.yaml -- otherwise the -live page's width is left untouched. -Labels work the same way: a labels: line is asserted exactly (anything on -the page the file does not list is removed), and no labels: line means the -page's labels are left alone, not even read. - -update never writes back to the file, so fixing a wrong page_id is always -safe: the file is exactly as you left it. A page_id that no longer resolves -fails that file and says what to do about it; one that is not a numeric id -at all is reported without asking Confluence. +Each file's title and page id come from its own YAML frontmatter, or from +a 'pages:' entry for it in markfluence.yaml -- a file can stay pristine and +keep its metadata there instead. Both places are legal and agreement is +silent; where they disagree about page_id, space or parent the file fails, +and where they disagree about title, page_width or labels the frontmatter +wins with a warning. + +A file that neither place mentions is skipped, not failed: a repository +legitimately holds markdown that is not published, so a glob over a docs +tree does not go red because somebody added a draft. A file that IS +registered but has no page id fails -- something claimed it and the page +has not been created yet. + +There are no per-page flags. Page metadata lives in the file or its entry, +which is what lets one invocation publish 'docs/**/*.md'; a flag would have +to name a single file. A project-wide 'page_width:' in markfluence.yaml is +how a whole tree gets one width. + +Page width is asserted only when something declares it -- the file, its +entry, or the project-wide default -- otherwise the live page's width is +left untouched. Labels work the same way: a labels: line is asserted +exactly (anything on the page the file does not list is removed), and no +labels: line means the page's labels are left alone, not even read. + +update never writes back to the file or to markfluence.yaml, so fixing a +wrong page_id is always safe: nothing is as you left it by accident. A +page_id that no longer resolves fails that file and says what to do about +it; one that is not a numeric id at all is reported without asking +Confluence. A file that has not changed since the page's last version is skipped, compared by mtime, unless --force is given. Each file is processed @@ -37,36 +51,33 @@ markfluence update FILE... [flags] ### Examples ``` - # Publish a file, taking the page id from its frontmatter + # Publish a file, taking the page id from its frontmatter or its entry markfluence update docs/managing_an_incident.md + # Publish a whole tree -- the CI shape: metadata comes from the files + # and from markfluence.yaml, so nothing has to be passed per file + markfluence update docs/**/*.md + # Publish a batch with a version message markfluence update docs/*.md --message "Bulk update" # Republish even though the file has not changed markfluence update docs/foo.md --force - # Override the target page, or rename it - markfluence update page.md --page-id 123456 - markfluence update page.md --title "New Title" - - # Set the width across a batch - markfluence update docs/*.md --page-width wide - # Preview, write nothing markfluence update docs/*.md --dry-run + + # See which location supplied each file's metadata + markfluence update docs/*.md --json | jq -r '.results[] | "\(.file) \(.metadata_source)"' ``` ### Options ``` - --dry-run Preview what would be published without writing to Confluence. - --force Skip the file-mtime check and always update the page. - -h, --help help for update - --message string Version message. (default "Updated via markfluence") - --page-id string Override the target page id (requires a single FILE). - --page-width string Override the page width: narrow, wide, or max. - --title string Override the page title (requires a single FILE). + --dry-run Preview what would be published without writing to Confluence. + --force Skip the file-mtime check and always update the page. + -h, --help help for update + --message string Version message. (default "Updated via markfluence") ``` ### Options inherited from parent commands diff --git a/internal/project/pages_test.go b/internal/project/pages_test.go index d65cbbc..00f5ac0 100644 --- a/internal/project/pages_test.go +++ b/internal/project/pages_test.go @@ -63,9 +63,11 @@ func TestEntryIsShapedLikeAFrontmatterBlock(t *testing.T) { if e.Fields == nil || e.Lists == nil { t.Fatalf("entry = %#v, want both maps non-nil", e) } - // The shapes MarkdownFile.Frontmatter and .Lists have. - var _ map[string]string = e.Fields - var _ map[string][]string = e.Lists + // The shapes MarkdownFile.Frontmatter and .Lists have -- asserted by + // handing them to functions typed for those, which is what every consumer + // does and what would break if Entry were ever re-typed. + takesFrontmatter(e.Fields) + takesLists(e.Lists) } // A project with no pages: key has not chosen the manifest; one with an empty @@ -235,3 +237,6 @@ func TestStructuralPageErrorsAreNotPerFile(t *testing.T) { t.Fatal("Discover succeeded; one escaping key must fail the load outright") } } + +func takesFrontmatter(map[string]string) {} +func takesLists(map[string][]string) {} diff --git a/schema/json-output/v1.json b/schema/json-output/v1.json index cd60461..1f25a6a 100644 --- a/schema/json-output/v1.json +++ b/schema/json-output/v1.json @@ -5,359 +5,896 @@ "description": "The single JSON document markfluence writes to stdout under --json. results holds one object per target (a single element for info/read; one per attachment for attachment-list); its item shape and the summary shape depend on command. The typed error object written to stderr on a fatal/pre-flight failure conforms to #/$defs/errorObject.", "type": "object", "additionalProperties": false, - "required": ["schema_version", "markfluence_version", "command", "roots", "warnings", "results", "summary"], + "required": [ + "schema_version", + "markfluence_version", + "command", + "roots", + "warnings", + "results", + "summary" + ], "properties": { - "schema_version": { "const": 1 }, - "markfluence_version": { "type": "string" }, - "command": { "enum": ["info", "read", "update", "create", "fix", "check", "children", "find", "search", "attachment-list", "attachment-upload", "attachment-download", "export"] }, - "roots": { "type": "array", "items": { "type": "string" }, "description": "Every distinct documentation root the command resolved, sorted. Empty for a command with no per-file root concept, and for a pre-flight failure that never reached root resolution." }, - "warnings": { "type": "array", "items": { "type": "string" }, "description": "Warnings about the invocation itself rather than about any page or file -- currently only the .env permission warning, raised when the .env that supplied the API token is reachable by anyone but its owner. Always present, [] when there is nothing to report, the same convention roots and results follow. Not per-result warnings: those live on the result." }, - "results": { "type": "array" }, - "summary": { "type": "object" } + "schema_version": { + "const": 1 + }, + "markfluence_version": { + "type": "string" + }, + "command": { + "enum": [ + "info", + "read", + "update", + "create", + "fix", + "check", + "children", + "find", + "search", + "attachment-list", + "attachment-upload", + "attachment-download", + "export" + ] + }, + "roots": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Every distinct documentation root the command resolved, sorted. Empty for a command with no per-file root concept, and for a pre-flight failure that never reached root resolution." + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Warnings about the invocation itself rather than about any page or file -- currently only the .env permission warning, raised when the .env that supplied the API token is reachable by anyone but its owner. Always present, [] when there is nothing to report, the same convention roots and results follow. Not per-result warnings: those live on the result." + }, + "results": { + "type": "array" + }, + "summary": { + "type": "object" + } }, "allOf": [ { - "if": { "properties": { "command": { "const": "info" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "info" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "oneOf": [{ "$ref": "#/$defs/infoResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, - "summary": { "$ref": "#/$defs/basicSummary" } + "results": { + "items": { + "oneOf": [ + { + "$ref": "#/$defs/infoResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } + ] + } + }, + "summary": { + "$ref": "#/$defs/basicSummary" + } } } }, { - "if": { "properties": { "command": { "const": "read" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "read" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "oneOf": [{ "$ref": "#/$defs/readResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, - "summary": { "$ref": "#/$defs/basicSummary" } + "results": { + "items": { + "oneOf": [ + { + "$ref": "#/$defs/readResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } + ] + } + }, + "summary": { + "$ref": "#/$defs/basicSummary" + } } } }, { - "if": { "properties": { "command": { "const": "update" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "update" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "$ref": "#/$defs/updateResult" } }, - "summary": { "$ref": "#/$defs/updateSummary" } + "results": { + "items": { + "$ref": "#/$defs/updateResult" + } + }, + "summary": { + "$ref": "#/$defs/updateSummary" + } } } }, { - "if": { "properties": { "command": { "const": "create" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "create" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "$ref": "#/$defs/createResult" } }, - "summary": { "$ref": "#/$defs/createSummary" } + "results": { + "items": { + "$ref": "#/$defs/createResult" + } + }, + "summary": { + "$ref": "#/$defs/createSummary" + } } } }, { - "if": { "properties": { "command": { "const": "fix" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "fix" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "$ref": "#/$defs/fixResult" } }, - "summary": { "$ref": "#/$defs/fixSummary" } + "results": { + "items": { + "$ref": "#/$defs/fixResult" + } + }, + "summary": { + "$ref": "#/$defs/fixSummary" + } } } }, { - "if": { "properties": { "command": { "const": "check" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "check" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "$ref": "#/$defs/checkResult" } }, - "summary": { "$ref": "#/$defs/checkSummary" } + "results": { + "items": { + "$ref": "#/$defs/checkResult" + } + }, + "summary": { + "$ref": "#/$defs/checkSummary" + } } } }, { - "if": { "properties": { "command": { "const": "attachment-upload" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "attachment-upload" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { "results": { "items": { - "oneOf": [{ "$ref": "#/$defs/attachmentUploadResult" }, { "$ref": "#/$defs/singleOpFailure" }] + "oneOf": [ + { + "$ref": "#/$defs/attachmentUploadResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } + ] } }, - "summary": { "$ref": "#/$defs/attachmentSummary" } + "summary": { + "$ref": "#/$defs/attachmentSummary" + } } } }, { - "if": { "properties": { "command": { "const": "export" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "export" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { "results": { - "items": { "oneOf": [{ "$ref": "#/$defs/exportResult" }, { "$ref": "#/$defs/singleOpFailure" }] } + "items": { + "oneOf": [ + { + "$ref": "#/$defs/exportResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } + ] + } }, - "summary": { "$ref": "#/$defs/exportSummary" } + "summary": { + "$ref": "#/$defs/exportSummary" + } } } }, { - "if": { "properties": { "command": { "const": "attachment-download" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "attachment-download" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { "results": { "items": { - "oneOf": [{ "$ref": "#/$defs/attachmentDownloadResult" }, { "$ref": "#/$defs/singleOpFailure" }] + "oneOf": [ + { + "$ref": "#/$defs/attachmentDownloadResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } + ] } }, - "summary": { "$ref": "#/$defs/attachmentSummary" } + "summary": { + "$ref": "#/$defs/attachmentSummary" + } } } }, { - "if": { "properties": { "command": { "const": "children" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "children" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { "results": { "items": { "oneOf": [ - { "$ref": "#/$defs/childrenResult" }, - { "$ref": "#/$defs/singleOpFailure" } + { + "$ref": "#/$defs/childrenResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } ] } }, - "summary": { "$ref": "#/$defs/basicSummary" } + "summary": { + "$ref": "#/$defs/basicSummary" + } } } }, { - "if": { "properties": { "command": { "const": "find" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "find" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "$ref": "#/$defs/findResult" } }, - "summary": { "$ref": "#/$defs/basicSummary" } + "results": { + "items": { + "$ref": "#/$defs/findResult" + } + }, + "summary": { + "$ref": "#/$defs/basicSummary" + } } } }, { - "if": { "properties": { "command": { "const": "search" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "search" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { - "results": { "items": { "$ref": "#/$defs/searchResult" } }, - "summary": { "$ref": "#/$defs/searchSummary" } + "results": { + "items": { + "$ref": "#/$defs/searchResult" + } + }, + "summary": { + "$ref": "#/$defs/searchSummary" + } } } }, { - "if": { "properties": { "command": { "const": "attachment-list" } }, "required": ["command"] }, + "if": { + "properties": { + "command": { + "const": "attachment-list" + } + }, + "required": [ + "command" + ] + }, "then": { "properties": { "results": { "items": { - "oneOf": [{ "$ref": "#/$defs/attachmentListResult" }, { "$ref": "#/$defs/singleOpFailure" }] + "oneOf": [ + { + "$ref": "#/$defs/attachmentListResult" + }, + { + "$ref": "#/$defs/singleOpFailure" + } + ] } }, - "summary": { "$ref": "#/$defs/basicSummary" } + "summary": { + "$ref": "#/$defs/basicSummary" + } } } } ], "$defs": { "code": { - "enum": ["CONFIG", "AUTH", "NOT_FOUND", "VALIDATION", "CONVERT", "IO", "NETWORK", "API"] + "enum": [ + "CONFIG", + "AUTH", + "NOT_FOUND", + "VALIDATION", + "CONVERT", + "IO", + "NETWORK", + "API" + ] }, "codeOrNull": { - "oneOf": [{ "$ref": "#/$defs/code" }, { "type": "null" }] + "oneOf": [ + { + "$ref": "#/$defs/code" + }, + { + "type": "null" + } + ] }, "childrenResult": { "description": "One page or folder under the requested node, or under the requested space's root. The results array is flat and in walk order (depth-first, siblings in the order Confluence displays them); parent_id and depth carry the hierarchy. parent_id is null for a page at the root of a space, which hangs off no node -- under --space, that is every depth-1 row. space and url are both derived from the row's webui link, so a row missing one is missing both.", "type": "object", "additionalProperties": false, - "required": ["ok", "id", "type", "title", "status", "parent_id", "depth", "space", "url"], + "required": [ + "ok", + "id", + "type", + "title", + "status", + "parent_id", + "depth", + "space", + "url" + ], "properties": { - "ok": { "const": true }, - "id": { "type": "string" }, - "type": { "enum": ["page", "folder"] }, - "title": { "type": "string" }, - "status": { "type": "string" }, - "parent_id": { "$ref": "#/$defs/stringOrNull" }, - "depth": { "type": "integer", "minimum": 1 }, - "space": { "$ref": "#/$defs/stringOrNull" }, - "url": { "$ref": "#/$defs/stringOrNull" } + "ok": { + "const": true + }, + "id": { + "type": "string" + }, + "type": { + "enum": [ + "page", + "folder" + ] + }, + "title": { + "type": "string" + }, + "status": { + "type": "string" + }, + "parent_id": { + "$ref": "#/$defs/stringOrNull" + }, + "depth": { + "type": "integer", + "minimum": 1 + }, + "space": { + "$ref": "#/$defs/stringOrNull" + }, + "url": { + "$ref": "#/$defs/stringOrNull" + } } }, "findResult": { "description": "One page or folder whose title matches exactly. There is no failure variant: find names no page to fail about, so a failed search is an error object on stderr with no envelope. status may be \"archived\" -- such a page is absent from the page tree but still reserves its title, so it blocks creating a page with that title in the same space. A folder reserves nothing, so a folder match is discovery only and never explains a creation conflict. space and url are both derived from the match's link, so a row missing one is missing both.", "type": "object", "additionalProperties": false, - "required": ["ok", "id", "type", "title", "space", "status", "url"], + "required": [ + "ok", + "id", + "type", + "title", + "space", + "status", + "url" + ], "properties": { - "ok": { "const": true }, - "id": { "type": "string" }, - "type": { "enum": ["page", "folder"] }, - "title": { "type": "string" }, - "space": { "$ref": "#/$defs/stringOrNull" }, - "status": { "type": "string" }, - "url": { "$ref": "#/$defs/stringOrNull" } + "ok": { + "const": true + }, + "id": { + "type": "string" + }, + "type": { + "enum": [ + "page", + "folder" + ] + }, + "title": { + "type": "string" + }, + "space": { + "$ref": "#/$defs/stringOrNull" + }, + "status": { + "type": "string" + }, + "url": { + "$ref": "#/$defs/stringOrNull" + } } }, "searchResult": { "description": "One full-text match. The array is in Confluence's relevance order, best first; that order is the only ranking available (the API reports score as 0.0 on every row), so re-sorting results discards it. The ordering is stable in membership but not in position: two identical searches return the same set of hits, but equally-ranked neighbours can trade places, so do not diff two runs expecting the same sequence or treat a position as an identifier. There is no failure variant and no status field: search names no page to fail about, so a failed search is an error object on stderr with no envelope, and the search index cannot see archived content, so every hit is current -- use find to discover an archived page. type is an open string rather than an enum because --type all and --cql can return whiteboard, database, or any content type Atlassian adds. space and url are both derived from the hit's link, so a result missing one is missing both; excerpt is null when the match was in the title alone, and is always a single line with Confluence's highlight markers removed and entities unescaped.", "type": "object", "additionalProperties": false, - "required": ["ok", "id", "type", "title", "space", "url", "excerpt"], + "required": [ + "ok", + "id", + "type", + "title", + "space", + "url", + "excerpt" + ], "properties": { - "ok": { "const": true }, - "id": { "type": "string" }, - "type": { "type": "string" }, - "title": { "type": "string" }, - "space": { "$ref": "#/$defs/stringOrNull" }, - "url": { "$ref": "#/$defs/stringOrNull" }, - "excerpt": { "$ref": "#/$defs/stringOrNull" } + "ok": { + "const": true + }, + "id": { + "type": "string" + }, + "type": { + "type": "string" + }, + "title": { + "type": "string" + }, + "space": { + "$ref": "#/$defs/stringOrNull" + }, + "url": { + "$ref": "#/$defs/stringOrNull" + }, + "excerpt": { + "$ref": "#/$defs/stringOrNull" + } } }, "searchSummary": { "description": "search's summary. failed is always 0, since a failed search is an error object with no envelope rather than a failed result. truncated says --limit was reached with matches left over; it is a flag and not a count because the API's totalSize is an estimate that has been observed both to drift between pages and to be nonzero against an empty results array. skipped counts index rows that carried no addressable content object -- a `type = space` hit, for example -- which is reachable only via --cql or --type all; it is reported rather than dropped quietly, because otherwise such a query yields total 0 and reads as a genuine miss.", "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "truncated", "skipped"], + "required": [ + "total", + "succeeded", + "failed", + "truncated", + "skipped" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "truncated": { "type": "boolean" }, - "skipped": { "type": "integer", "minimum": 0 } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "truncated": { + "type": "boolean" + }, + "skipped": { + "type": "integer", + "minimum": 0 + } } }, "stringOrNull": { - "type": ["string", "null"] + "type": [ + "string", + "null" + ] }, "parentTypeOrNull": { "description": "What kind of object the parent id names. A folder is a Cloud-only content type that can parent a page; null when the page is top-level.", - "enum": ["page", "folder", null] + "enum": [ + "page", + "folder", + null + ] }, "pageWidth": { "type": "object", "additionalProperties": false, - "required": ["value", "default"], + "required": [ + "value", + "default" + ], "properties": { - "value": { "type": "string" }, - "default": { "type": "boolean" } + "value": { + "type": "string" + }, + "default": { + "type": "boolean" + } } }, "pageWidthOrNull": { - "oneOf": [{ "$ref": "#/$defs/pageWidth" }, { "type": "null" }] + "oneOf": [ + { + "$ref": "#/$defs/pageWidth" + }, + { + "type": "null" + } + ] }, "author": { "type": "object", "additionalProperties": false, - "required": ["account_id", "name"], + "required": [ + "account_id", + "name" + ], "properties": { - "account_id": { "type": "string" }, - "name": { "type": "string" } + "account_id": { + "type": "string" + }, + "name": { + "type": "string" + } } }, "stamp": { "type": "object", "additionalProperties": false, - "required": ["at", "by"], + "required": [ + "at", + "by" + ], "properties": { - "at": { "type": "string" }, - "by": { "oneOf": [{ "$ref": "#/$defs/author" }, { "type": "null" }] } + "at": { + "type": "string" + }, + "by": { + "oneOf": [ + { + "$ref": "#/$defs/author" + }, + { + "type": "null" + } + ] + } } }, "stampOrNull": { - "oneOf": [{ "$ref": "#/$defs/stamp" }, { "type": "null" }] + "oneOf": [ + { + "$ref": "#/$defs/stamp" + }, + { + "type": "null" + } + ] }, "attachment": { "type": "object", "additionalProperties": false, - "required": ["action", "filename"], + "required": [ + "action", + "filename" + ], "properties": { - "action": { "type": "string" }, - "filename": { "type": "string" } + "action": { + "type": "string" + }, + "filename": { + "type": "string" + } } }, "labelAction": { "description": "What happened to one label in an asserted set.", "type": "object", "additionalProperties": false, - "required": ["action", "name"], + "required": [ + "action", + "name" + ], "properties": { "action": { "description": "kept means a surplus label was left on the page because an unmanaged label shares its name: Confluence's removal takes a name with no prefix and deletes the personal label first, so there is no request that removes the intended one.", - "enum": ["added", "removed", "unchanged", "kept"] + "enum": [ + "added", + "removed", + "unchanged", + "kept" + ] }, - "name": { "type": "string" } + "name": { + "type": "string" + } } }, "labelActionsOrNull": { "description": "The full label set asserted this run, or null when the file declares no labels key and the page's labels were left alone.", "oneOf": [ - { "type": "array", "items": { "$ref": "#/$defs/labelAction" } }, - { "type": "null" } + { + "type": "array", + "items": { + "$ref": "#/$defs/labelAction" + } + }, + { + "type": "null" + } ] }, "labelInfo": { "description": "One label a page carries. managed is true for the global prefix, the only one markfluence writes or removes.", "type": "object", "additionalProperties": false, - "required": ["name", "prefix", "managed"], + "required": [ + "name", + "prefix", + "managed" + ], "properties": { - "name": { "type": "string" }, - "prefix": { "type": "string" }, - "managed": { "type": "boolean" } + "name": { + "type": "string" + }, + "prefix": { + "type": "string" + }, + "managed": { + "type": "boolean" + } } }, "labelInfoListOrNull": { "description": "Every label on a page, or null when the fetch failed. An empty array means the page genuinely has none.", "oneOf": [ - { "type": "array", "items": { "$ref": "#/$defs/labelInfo" } }, - { "type": "null" } + { + "type": "array", + "items": { + "$ref": "#/$defs/labelInfo" + } + }, + { + "type": "null" + } ] }, "singleOpFailure": { "description": "An operational failure for a single-target command (info/read): page not found, fetch error, etc.", "type": "object", "additionalProperties": false, - "required": ["ok", "page_id", "error", "code"], + "required": [ + "ok", + "page_id", + "error", + "code" + ], "properties": { - "ok": { "const": false }, - "page_id": { "type": "string" }, - "error": { "type": "string" }, - "code": { "$ref": "#/$defs/code" } + "ok": { + "const": false + }, + "page_id": { + "type": "string" + }, + "error": { + "type": "string" + }, + "code": { + "$ref": "#/$defs/code" + } } }, "infoResult": { "type": "object", "additionalProperties": false, "required": [ - "ok", "page_id", "title", "page_status", "space", "parent", "parent_type", - "version", "page_width", "labels", "created", "updated", "message", "url", "properties" + "ok", + "page_id", + "title", + "page_status", + "space", + "parent", + "parent_type", + "version", + "page_width", + "labels", + "created", + "updated", + "message", + "url", + "properties" ], "properties": { - "ok": { "const": true }, - "page_id": { "type": "string" }, - "title": { "type": "string" }, - "page_status": { "type": "string" }, - "space": { "type": "string" }, - "parent": { "$ref": "#/$defs/stringOrNull" }, - "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, + "ok": { + "const": true + }, + "page_id": { + "type": "string" + }, + "title": { + "type": "string" + }, + "page_status": { + "type": "string" + }, + "space": { + "type": "string" + }, + "parent": { + "$ref": "#/$defs/stringOrNull" + }, + "parent_type": { + "$ref": "#/$defs/parentTypeOrNull" + }, "version": { "type": "object", "additionalProperties": false, - "required": ["number"], - "properties": { "number": { "type": "integer" } } - }, - "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, - "labels": { "$ref": "#/$defs/labelInfoListOrNull" }, - "created": { "$ref": "#/$defs/stampOrNull" }, - "updated": { "$ref": "#/$defs/stampOrNull" }, - "message": { "type": "string" }, - "url": { "type": "string" }, + "required": [ + "number" + ], + "properties": { + "number": { + "type": "integer" + } + } + }, + "page_width": { + "$ref": "#/$defs/pageWidthOrNull" + }, + "labels": { + "$ref": "#/$defs/labelInfoListOrNull" + }, + "created": { + "$ref": "#/$defs/stampOrNull" + }, + "updated": { + "$ref": "#/$defs/stampOrNull" + }, + "message": { + "type": "string" + }, + "url": { + "type": "string" + }, "properties": { "oneOf": [ - { "type": "null" }, + { + "type": "null" + }, { "type": "array", "items": { "type": "object", "additionalProperties": false, - "required": ["key", "value"], - "properties": { "key": { "type": "string" }, "value": {} } + "required": [ + "key", + "value" + ], + "properties": { + "key": { + "type": "string" + }, + "value": {} + } } } ] @@ -368,59 +905,183 @@ "type": "object", "additionalProperties": false, "required": [ - "ok", "page_id", "title", "space", "parent", "parent_type", "page_width", "labels", - "format", "body" + "ok", + "page_id", + "title", + "space", + "parent", + "parent_type", + "page_width", + "labels", + "format", + "body" ], "properties": { - "ok": { "const": true }, - "page_id": { "type": "string" }, - "title": { "type": "string" }, - "space": { "type": "string" }, - "parent": { "$ref": "#/$defs/stringOrNull" }, - "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, - "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, + "ok": { + "const": true + }, + "page_id": { + "type": "string" + }, + "title": { + "type": "string" + }, + "space": { + "type": "string" + }, + "parent": { + "$ref": "#/$defs/stringOrNull" + }, + "parent_type": { + "$ref": "#/$defs/parentTypeOrNull" + }, + "page_width": { + "$ref": "#/$defs/pageWidthOrNull" + }, "labels": { "description": "The managed labels that went into the rendered frontmatter, sorted; null when the fetch failed. A different shape from info's labels on purpose: info describes the page, read describes the document it produced.", - "oneOf": [{ "type": "array", "items": { "type": "string" } }, { "type": "null" }] + "oneOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ] }, - "format": { "enum": ["markdown", "storage"] }, - "body": { "type": "string" } + "format": { + "enum": [ + "markdown", + "storage" + ] + }, + "body": { + "type": "string" + } } }, "updateResult": { "type": "object", "additionalProperties": false, "required": [ - "ok", "status", "dry_run", "file", "page_id", "title", "space", "url", - "version", "page_width", "labels", "attachments", "warnings", "broken", "error", "code" + "ok", + "status", + "dry_run", + "file", + "page_id", + "title", + "space", + "url", + "version", + "page_width", + "labels", + "attachments", + "warnings", + "broken", + "metadata_source", + "error", + "code" ], "properties": { - "ok": { "type": "boolean" }, - "status": { "enum": ["published", "skipped", "failed"] }, - "dry_run": { "type": "boolean" }, - "file": { "type": "string" }, - "page_id": { "$ref": "#/$defs/stringOrNull" }, - "title": { "$ref": "#/$defs/stringOrNull" }, - "space": { "$ref": "#/$defs/stringOrNull" }, - "url": { "$ref": "#/$defs/stringOrNull" }, + "ok": { + "type": "boolean" + }, + "status": { + "enum": [ + "published", + "skipped", + "failed" + ] + }, + "dry_run": { + "type": "boolean" + }, + "file": { + "type": "string" + }, + "page_id": { + "$ref": "#/$defs/stringOrNull" + }, + "title": { + "$ref": "#/$defs/stringOrNull" + }, + "space": { + "$ref": "#/$defs/stringOrNull" + }, + "url": { + "$ref": "#/$defs/stringOrNull" + }, "version": { "oneOf": [ - { "type": "null" }, + { + "type": "null" + }, { "type": "object", "additionalProperties": false, - "required": ["previous", "new"], - "properties": { "previous": { "type": "integer" }, "new": { "type": "integer" } } + "required": [ + "previous", + "new" + ], + "properties": { + "previous": { + "type": "integer" + }, + "new": { + "type": "integer" + } + } } ] }, - "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, - "labels": { "$ref": "#/$defs/labelActionsOrNull" }, - "attachments": { "type": "array", "items": { "$ref": "#/$defs/attachment" } }, - "warnings": { "type": "array", "items": { "type": "string" } }, - "broken": { "type": "array", "items": { "type": "string" } }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "page_width": { + "$ref": "#/$defs/pageWidthOrNull" + }, + "labels": { + "$ref": "#/$defs/labelActionsOrNull" + }, + "attachments": { + "type": "array", + "items": { + "$ref": "#/$defs/attachment" + } + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + } + }, + "broken": { + "type": "array", + "items": { + "type": "string" + } + }, + "metadata_source": { + "oneOf": [ + { + "type": "string", + "enum": [ + "frontmatter", + "manifest" + ] + }, + { + "type": "null" + } + ], + "description": "Which location supplied this file's page metadata: its own frontmatter, or a pages: entry in markfluence.yaml. Null when nothing claims the file. Where both locations speak, frontmatter is reported, since it wins every field it can win." + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "createResult": { @@ -428,51 +1089,170 @@ "type": "object", "additionalProperties": false, "required": [ - "ok", "status", "dry_run", "file", "page_id", "title", "space", "parent", "parent_type", "parent_file", "url", - "page_width", "labels", "persisted", "attachments", "warnings", "broken", "error", "code" + "ok", + "status", + "dry_run", + "file", + "page_id", + "title", + "space", + "parent", + "parent_type", + "parent_file", + "url", + "page_width", + "labels", + "persisted", + "attachments", + "warnings", + "broken", + "metadata_source", + "error", + "code" ], "properties": { - "ok": { "type": "boolean" }, - "status": { "enum": ["created", "not_created", "failed"] }, - "dry_run": { "type": "boolean" }, - "file": { "type": "string" }, - "page_id": { "$ref": "#/$defs/stringOrNull" }, - "title": { "$ref": "#/$defs/stringOrNull" }, - "space": { "$ref": "#/$defs/stringOrNull" }, - "parent": { "$ref": "#/$defs/stringOrNull" }, - "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, - "parent_file": { "$ref": "#/$defs/stringOrNull" }, - "url": { "$ref": "#/$defs/stringOrNull" }, - "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, - "labels": { "$ref": "#/$defs/labelActionsOrNull" }, - "persisted": { "type": "boolean" }, - "attachments": { "type": "array", "items": { "$ref": "#/$defs/attachment" } }, - "warnings": { "type": "array", "items": { "type": "string" } }, - "broken": { "type": "array", "items": { "type": "string" } }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "ok": { + "type": "boolean" + }, + "status": { + "enum": [ + "created", + "not_created", + "failed" + ] + }, + "dry_run": { + "type": "boolean" + }, + "file": { + "type": "string" + }, + "page_id": { + "$ref": "#/$defs/stringOrNull" + }, + "title": { + "$ref": "#/$defs/stringOrNull" + }, + "space": { + "$ref": "#/$defs/stringOrNull" + }, + "parent": { + "$ref": "#/$defs/stringOrNull" + }, + "parent_type": { + "$ref": "#/$defs/parentTypeOrNull" + }, + "parent_file": { + "$ref": "#/$defs/stringOrNull" + }, + "url": { + "$ref": "#/$defs/stringOrNull" + }, + "page_width": { + "$ref": "#/$defs/pageWidthOrNull" + }, + "labels": { + "$ref": "#/$defs/labelActionsOrNull" + }, + "persisted": { + "type": "boolean" + }, + "attachments": { + "type": "array", + "items": { + "$ref": "#/$defs/attachment" + } + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + } + }, + "broken": { + "type": "array", + "items": { + "type": "string" + } + }, + "metadata_source": { + "oneOf": [ + { + "type": "string", + "enum": [ + "frontmatter", + "manifest" + ] + }, + { + "type": "null" + } + ], + "description": "Which location supplied this file's page metadata: its own frontmatter, or a pages: entry in markfluence.yaml. Null when nothing claims the file. Where both locations speak, frontmatter is reported, since it wins every field it can win." + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "fixResult": { "type": "object", "additionalProperties": false, - "required": ["ok", "status", "file", "page_id", "dry_run", "changes", "reordered", "warnings", "error", "code"], + "required": [ + "ok", + "status", + "file", + "page_id", + "dry_run", + "changes", + "reordered", + "warnings", + "error", + "code" + ], "properties": { - "ok": { "type": "boolean" }, - "status": { "enum": ["changed", "consistent", "failed"] }, - "file": { "type": "string" }, - "page_id": { "$ref": "#/$defs/stringOrNull" }, - "dry_run": { "type": "boolean" }, + "ok": { + "type": "boolean" + }, + "status": { + "enum": [ + "changed", + "consistent", + "failed" + ] + }, + "file": { + "type": "string" + }, + "page_id": { + "$ref": "#/$defs/stringOrNull" + }, + "dry_run": { + "type": "boolean" + }, "changes": { "type": "array", "items": { "type": "object", "additionalProperties": false, - "required": ["field", "old", "new"], + "required": [ + "field", + "old", + "new" + ], "properties": { - "field": { "type": "string" }, - "old": { "$ref": "#/$defs/stringOrNull" }, - "new": { "type": "string" } + "field": { + "type": "string" + }, + "old": { + "$ref": "#/$defs/stringOrNull" + }, + "new": { + "type": "string" + } } } }, @@ -480,77 +1260,189 @@ "description": "Whether fix rewrote the frontmatter into canonical field order. Independent of changes: a file whose values all match its live page can still be reordered, and that counts as changed.", "type": "boolean" }, - "warnings": { "type": "array", "items": { "type": "string" } }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "warnings": { + "type": "array", + "items": { + "type": "string" + } + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "checkResult": { "description": "One checked file. broken/warnings are always [] (never null), matching ConfluencePage's own convention. status=broken means broken is non-empty (frontmatter or converter); status=failed means the file never reached a clean answer at all (unreadable, unterminated frontmatter, frontmatter that is not valid YAML or not a flat mapping of single-line scalars, bad page_width, non-numeric page_id) -- code is VALIDATION in that case. debug is non-null only when --show-html was passed and the file reached the converter (never on a failed file).", "type": "object", "additionalProperties": false, - "required": ["ok", "status", "file", "broken", "warnings", "debug", "error", "code"], + "required": [ + "ok", + "status", + "file", + "broken", + "warnings", + "debug", + "error", + "code" + ], "properties": { - "ok": { "type": "boolean" }, - "status": { "enum": ["clean", "warnings", "broken", "failed"] }, - "file": { "type": "string" }, - "broken": { "type": "array", "items": { "type": "string" } }, - "warnings": { "type": "array", "items": { "type": "string" } }, + "ok": { + "type": "boolean" + }, + "status": { + "enum": [ + "clean", + "warnings", + "broken", + "failed" + ] + }, + "file": { + "type": "string" + }, + "broken": { + "type": "array", + "items": { + "type": "string" + } + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + } + }, "debug": { "oneOf": [ - { "type": "null" }, + { + "type": "null" + }, { "type": "object", "additionalProperties": false, - "required": ["html", "attachments"], + "required": [ + "html", + "attachments" + ], "properties": { - "html": { "type": "string" }, - "attachments": { "type": "array", "items": { "$ref": "#/$defs/checkAttachment" } } + "html": { + "type": "string" + }, + "attachments": { + "type": "array", + "items": { + "$ref": "#/$defs/checkAttachment" + } + } } } ] }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "checkAttachment": { "description": "ConfluencePage.Attachments verbatim: a local image check's --show-html surfaces, not an upload outcome (contrast update/create's attachments array, which reports an action).", "type": "object", "additionalProperties": false, - "required": ["filename", "path", "source"], + "required": [ + "filename", + "path", + "source" + ], "properties": { - "filename": { "type": "string" }, - "path": { "type": "string" }, - "source": { "type": "string" } + "filename": { + "type": "string" + }, + "path": { + "type": "string" + }, + "source": { + "type": "string" + } } }, "attachmentUploadResult": { "description": "One uploaded file. status uses the same verbs as the attachments array on update/create. dest_path is always null: upload has no local destination to report, and only exists here so upload and download share one result shape.", "type": "object", "additionalProperties": false, - "required": ["ok", "status", "dry_run", "filename", "dest_path", "error", "code"], + "required": [ + "ok", + "status", + "dry_run", + "filename", + "dest_path", + "error", + "code" + ], "properties": { - "ok": { "type": "boolean" }, - "status": { "enum": ["created", "updated", "skipped", "failed"] }, - "dry_run": { "type": "boolean" }, - "filename": { "type": "string" }, - "dest_path": { "const": null }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "ok": { + "type": "boolean" + }, + "status": { + "enum": [ + "created", + "updated", + "skipped", + "failed" + ] + }, + "dry_run": { + "type": "boolean" + }, + "filename": { + "type": "string" + }, + "dest_path": { + "const": null + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "exportSummary": { "description": "export batch summary. skipped counts pages whose file was already on disk; a run that exports nothing new is all skipped and still succeeded. project_file says what happened to the markfluence.yaml a multi-page export needs to be republishable: null for a single-page export, which needs none.", "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "skipped", "project_file"], + "required": [ + "total", + "succeeded", + "failed", + "skipped", + "project_file" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "skipped": { "type": "integer" }, - "project_file": { "enum": ["wrote", "exists", null] } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "skipped": { + "type": "integer" + }, + "project_file": { + "enum": [ + "wrote", + "exists", + null + ] + } } }, "exportResult": { @@ -558,157 +1450,379 @@ "type": "object", "additionalProperties": false, "required": [ - "ok", "page_id", "title", "space", "parent", "parent_type", "parent_file", "dry_run", - "status", "dest_path", "attachments", "warnings", "error", "code" + "ok", + "page_id", + "title", + "space", + "parent", + "parent_type", + "parent_file", + "dry_run", + "status", + "dest_path", + "attachments", + "warnings", + "error", + "code" ], "properties": { - "ok": { "type": "boolean" }, - "page_id": { "type": "string" }, - "title": { "type": "string" }, - "space": { "type": "string" }, - "parent": { "$ref": "#/$defs/stringOrNull" }, - "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, + "ok": { + "type": "boolean" + }, + "page_id": { + "type": "string" + }, + "title": { + "type": "string" + }, + "space": { + "type": "string" + }, + "parent": { + "$ref": "#/$defs/stringOrNull" + }, + "parent_type": { + "$ref": "#/$defs/parentTypeOrNull" + }, "parent_file": { "$ref": "#/$defs/stringOrNull", "description": "The parent: value written into the exported file, when it is a path to the parent's own .md. Null when the parent stayed an id -- the export root, or a page whose parent is a folder." }, - "dry_run": { "type": "boolean" }, - "status": { "enum": ["wrote", "skipped", ""] }, - "dest_path": { "$ref": "#/$defs/stringOrNull" }, + "dry_run": { + "type": "boolean" + }, + "status": { + "enum": [ + "wrote", + "skipped", + "" + ] + }, + "dest_path": { + "$ref": "#/$defs/stringOrNull" + }, "attachments": { "type": "array", "items": { "type": "object", "additionalProperties": false, - "required": ["status", "filename", "dest_path", "error", "code"], + "required": [ + "status", + "filename", + "dest_path", + "error", + "code" + ], "properties": { - "status": { "enum": ["downloaded", "skipped", "skipped_unreferenced", "failed"] }, - "filename": { "type": "string" }, - "dest_path": { "$ref": "#/$defs/stringOrNull" }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "status": { + "enum": [ + "downloaded", + "skipped", + "skipped_unreferenced", + "failed" + ] + }, + "filename": { + "type": "string" + }, + "dest_path": { + "$ref": "#/$defs/stringOrNull" + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } } }, - "warnings": { "type": "array", "items": { "type": "string" } }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "warnings": { + "type": "array", + "items": { + "type": "string" + } + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "attachmentDownloadResult": { "description": "One attachment written to disk. filename is the stored attachment name; dest_path is the local path written, which depends on the recorded source path, --flat, and --dest. dest_path is null only when resolving it is what failed.", "type": "object", "additionalProperties": false, - "required": ["ok", "status", "dry_run", "filename", "dest_path", "error", "code"], + "required": [ + "ok", + "status", + "dry_run", + "filename", + "dest_path", + "error", + "code" + ], "properties": { - "ok": { "type": "boolean" }, - "status": { "enum": ["downloaded", "skipped", "failed"] }, - "dry_run": { "type": "boolean" }, - "filename": { "type": "string" }, - "dest_path": { "$ref": "#/$defs/stringOrNull" }, - "error": { "$ref": "#/$defs/stringOrNull" }, - "code": { "$ref": "#/$defs/codeOrNull" } + "ok": { + "type": "boolean" + }, + "status": { + "enum": [ + "downloaded", + "skipped", + "failed" + ] + }, + "dry_run": { + "type": "boolean" + }, + "filename": { + "type": "string" + }, + "dest_path": { + "$ref": "#/$defs/stringOrNull" + }, + "error": { + "$ref": "#/$defs/stringOrNull" + }, + "code": { + "$ref": "#/$defs/codeOrNull" + } } }, "attachmentListResult": { "description": "One attachment on the page. filename is the name Confluence stores; for an attachment markfluence published that is the encoded source path, and source is the markdown image path it came from. managed is false for a hand-uploaded attachment (sha256 and source both null). source may also be null on a managed attachment published before markfluence recorded source paths, in which case sha256 is still set.", "type": "object", "additionalProperties": false, - "required": ["ok", "id", "filename", "size", "media_type", "version", "comment", "managed", "sha256", "source"], + "required": [ + "ok", + "id", + "filename", + "size", + "media_type", + "version", + "comment", + "managed", + "sha256", + "source" + ], "properties": { - "ok": { "const": true }, - "id": { "type": "string" }, - "filename": { "type": "string" }, - "size": { "type": "integer" }, - "media_type": { "type": "string" }, - "version": { "type": "integer" }, - "comment": { "type": "string" }, - "managed": { "type": "boolean" }, - "sha256": { "$ref": "#/$defs/stringOrNull" }, - "source": { "$ref": "#/$defs/stringOrNull" } + "ok": { + "const": true + }, + "id": { + "type": "string" + }, + "filename": { + "type": "string" + }, + "size": { + "type": "integer" + }, + "media_type": { + "type": "string" + }, + "version": { + "type": "integer" + }, + "comment": { + "type": "string" + }, + "managed": { + "type": "boolean" + }, + "sha256": { + "$ref": "#/$defs/stringOrNull" + }, + "source": { + "$ref": "#/$defs/stringOrNull" + } } }, "attachmentSummary": { "description": "attachment-upload/attachment-download batch summary.", "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "skipped"], + "required": [ + "total", + "succeeded", + "failed", + "skipped" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "skipped": { "type": "integer" } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "skipped": { + "type": "integer" + } } }, "basicSummary": { "description": "info/read batch summary (total:1), and attachment-list (total: the attachment count).", "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed"], + "required": [ + "total", + "succeeded", + "failed" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + } } }, "updateSummary": { "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "skipped"], + "required": [ + "total", + "succeeded", + "failed", + "skipped" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "skipped": { "type": "integer" } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "skipped": { + "type": "integer" + } } }, "createSummary": { "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "aborted"], + "required": [ + "total", + "succeeded", + "failed", + "aborted" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "aborted": { "type": "boolean" } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "aborted": { + "type": "boolean" + } } }, "fixSummary": { "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "changed", "consistent"], + "required": [ + "total", + "succeeded", + "failed", + "changed", + "consistent" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "changed": { "type": "integer" }, - "consistent": { "type": "integer" } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "changed": { + "type": "integer" + }, + "consistent": { + "type": "integer" + } } }, "checkSummary": { "description": "clean/warnings count files on the ok:true side (clean has neither broken nor warnings; warnings has only warnings); failed already covers both the broken and failed statuses on the ok:false side, the same granularity fixSummary uses.", "type": "object", "additionalProperties": false, - "required": ["total", "succeeded", "failed", "clean", "warnings"], + "required": [ + "total", + "succeeded", + "failed", + "clean", + "warnings" + ], "properties": { - "total": { "type": "integer" }, - "succeeded": { "type": "integer" }, - "failed": { "type": "integer" }, - "clean": { "type": "integer" }, - "warnings": { "type": "integer" } + "total": { + "type": "integer" + }, + "succeeded": { + "type": "integer" + }, + "failed": { + "type": "integer" + }, + "clean": { + "type": "integer" + }, + "warnings": { + "type": "integer" + } } }, "errorObject": { "description": "The typed error object written to stderr on a fatal/pre-flight failure. command may be empty for a pre-parse (bad-flag) error.", "type": "object", "additionalProperties": false, - "required": ["schema_version", "command", "error", "code", "warnings"], + "required": [ + "schema_version", + "command", + "error", + "code", + "warnings" + ], "properties": { - "schema_version": { "const": 1 }, - "command": { "type": "string" }, - "error": { "type": "string" }, - "code": { "$ref": "#/$defs/code" }, - "warnings": { "type": "array", "items": { "type": "string" }, "description": "As the envelope's warnings, carried here too because a fatal failure emits no envelope -- and a credential-resolution failure is exactly when a warning about the .env matters most." } + "schema_version": { + "const": 1 + }, + "command": { + "type": "string" + }, + "error": { + "type": "string" + }, + "code": { + "$ref": "#/$defs/code" + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + }, + "description": "As the envelope's warnings, carried here too because a fatal failure emits no envelope -- and a credential-resolution failure is exactly when a warning about the .env matters most." + } } } } From ceb3b2eee7efe3b5d4177a120201ecb3b64a2005 Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:03:50 -0400 Subject: [PATCH 07/14] feat(check): validate a file's metadata wherever it lives check now resolves each file's metadata through pagemeta before validating it, so an entry's page_width, page_id, labels and title are checked exactly as a file's own are -- and only for a file the invocation names, which is #139's per-file scoping made real: one bad entry never blocks checking the rest of a repository. Two locations naming different pages is an error, and offline-visible, which is the kind of defect check exists to catch before a publish does. The half-and-half lint is a *warning*, never an error, and that is the whole point: agreement between the two locations is legal, so "keep it in one place" has to be sayable without becoming a wall somebody hits halfway through a migration. fix moving the keys is the remedy (PR 2). One bug found by its own test: r.warnings was *assigned* from the label and converter warnings, so anything set earlier -- the new lint -- was silently discarded. It accumulates now. --- cmd/check/check.go | 80 ++++++++++++----- cmd/check/check_test.go | 138 ++++++++++++++++++++++++++++- docs/commands/markfluence_check.md | 9 +- internal/pagemeta/pagemeta.go | 35 ++++++++ internal/pagemeta/pagemeta_test.go | 23 +++++ 5 files changed, 259 insertions(+), 26 deletions(-) diff --git a/cmd/check/check.go b/cmd/check/check.go index 1bd5ab9..444b5c7 100644 --- a/cmd/check/check.go +++ b/cmd/check/check.go @@ -18,6 +18,7 @@ import ( "github.com/mozilla/markfluence/internal/jsonout" "github.com/mozilla/markfluence/internal/labels" "github.com/mozilla/markfluence/internal/linkindex" + "github.com/mozilla/markfluence/internal/pagemeta" "github.com/mozilla/markfluence/internal/pageref" "github.com/mozilla/markfluence/internal/pagewidth" "github.com/mozilla/markfluence/internal/project" @@ -45,10 +46,16 @@ var Cmd = &cobra.Command{ Long: "Validate one or more markdown FILEs against the converter and frontmatter\n" + "rules, with no network access and no credentials -- fast, safe, and\n" + "CI/agent-friendly. Reports conversion warnings and broken image/link\n" + - "references, and frontmatter sanity (parseable, page_width valid, page_id\n" + + "references, and metadata sanity (parseable, page_width valid, page_id\n" + "numeric when present). Each file is processed independently; the command\n" + "exits non-zero if any file is broken or failed outright. Warnings alone do\n" + "not fail.\n\n" + + "A file's metadata is checked wherever it lives -- its own frontmatter or a\n" + + "'pages:' entry for it in markfluence.yaml -- and an entry is reported only\n" + + "when its file is one of the FILEs given, so one bad entry never blocks\n" + + "checking the rest of a repository. Two locations naming different pages is\n" + + "an error; a file keeping its own keys in a project that uses 'pages:' is a\n" + + "warning, since both work.\n\n" + "\"link not resolved: TARGET\" means TARGET is a sibling .md file that exists\n" + "under the documentation root but has no page_id yet -- the normal state of\n" + "a tree that hasn't been published, not a defect. \"same-page anchor not\n" + @@ -130,23 +137,11 @@ func processFile(filename string, roots *project.Cache, indexes *linkindex.Cache return r.fail(err, jsonout.CodeValidation) } - if _, err := pagewidth.Declared(mf.Frontmatter); err != nil { - return r.fail(err, jsonout.CodeValidation) - } - // An invalid label is a guaranteed publish defect that needs no network to - // see, the same class as an invalid page_width -- and worse in one way: a - // name Confluence splits on a space publishes successfully, as the wrong - // labels, and then cannot be removed by any spelling of the file (see - // docs/confluence/labels.md). Catching it offline is the cheapest place it - // can be caught. - labelSet, err := labels.Declared(mf.Lists, mf.Frontmatter) - if err != nil { - return r.fail(err, jsonout.CodeValidation) - } - if pageID := mf.PageID(); pageID != "" && !pageref.IsDigits(pageID) { - return r.fail(errors.New(pageref.NotNumericMessage(pageID)), jsonout.CodeValidation) - } - + // The root first, because a file's metadata may live in the project file's + // pages: block rather than in the file -- and then everything below + // validates the *resolved* metadata, so an entry's values are checked + // exactly as a file's own are. That is #139's per-file scoping made real: + // an entry is diagnosed when its file is named, and never otherwise. abs, err := filepath.Abs(filename) if err != nil { return r.fail(err, jsonout.CodeIO) @@ -163,6 +158,30 @@ func processFile(filename string, roots *project.Cache, indexes *linkindex.Cache } return r.fail(project.RootError(err), code) } + key, _ := pagemeta.KeyFor(root, abs) + meta, err := pagemeta.Resolve(key, mf, root) + if err != nil { + // The two locations name different pages. Offline-visible, and exactly + // the kind of defect check exists to catch before a publish does. + return r.fail(err, jsonout.CodeValidation) + } + + if _, err := pagewidth.Declared(meta.Fields); err != nil { + return r.fail(err, jsonout.CodeValidation) + } + // An invalid label is a guaranteed publish defect that needs no network to + // see, the same class as an invalid page_width -- and worse in one way: a + // name Confluence splits on a space publishes successfully, as the wrong + // labels, and then cannot be removed by any spelling of the file (see + // docs/confluence/labels.md). Catching it offline is the cheapest place it + // can be caught. + labelSet, err := labels.Declared(meta.Lists, meta.Fields) + if err != nil { + return r.fail(err, jsonout.CodeValidation) + } + if pageID := strings.TrimSpace(meta.Fields["page_id"]); pageID != "" && !pageref.IsDigits(pageID) { + return r.fail(errors.New(pageref.NotNumericMessage(pageID)), jsonout.CodeValidation) + } index, err := indexes.Get(root) if err != nil { return r.fail(fmt.Errorf("building the link index: %w", err), jsonout.CodeIO) @@ -204,9 +223,21 @@ func processFile(filename string, roots *project.Cache, indexes *linkindex.Cache localBroken = append(localBroken, fmt.Sprintf("%s: %s", root.File, err)) } } - if title, present := mf.TitleField(); present && title == "" { + if title, present := meta.Fields["title"]; present && strings.TrimSpace(title) == "" { localBroken = append(localBroken, - "frontmatter has an empty 'title:'; give it a value or remove it") + "the title is present but empty; give it a value or remove it") + } + + // "No half-and-half" (#139): a file carrying its own markfluence keys in a + // project that has chosen the manifest. A *warning*, never an error, and + // that is the whole point -- agreement between the two locations is legal, + // so this has to be sayable without becoming a wall somebody hits halfway + // through a migration. `fix` moving the keys is the remedy. + if pagemeta.HasManifest(root) && meta.InFile() { + r.warnings = append(r.warnings, fmt.Sprintf( + "this file carries markfluence frontmatter in a project that keeps page "+ + "metadata in %s; both work, but keeping it in one place is clearer", + project.Filename)) } page, err := convert.MdToConfluence(mf, root, index, checkBaseURL, checkSpaceKey, buildinfo.Stamp()) @@ -232,9 +263,12 @@ func processFile(filename string, roots *project.Cache, indexes *linkindex.Cache return r.fail(err, jsonout.CodeConvert) } r.broken = append(localBroken, page.Broken...) - // Label warnings lead: they are a property of the frontmatter, so they hold - // whatever the converter went on to find in the body. - r.warnings = append(labelSet.Warnings, page.Warnings...) + // Appended rather than assigned: the half-and-half lint above has already + // put a warning here, and assigning discarded it. Label warnings still + // lead the converter's -- they are a property of the declared metadata, so + // they hold whatever the converter went on to find in the body. + r.warnings = append(r.warnings, labelSet.Warnings...) + r.warnings = append(r.warnings, page.Warnings...) if showHTML { r.debugHTML = page.HTML r.debugAttachments = page.Attachments diff --git a/cmd/check/check_test.go b/cmd/check/check_test.go index 5211d8e..9b31b99 100644 --- a/cmd/check/check_test.go +++ b/cmd/check/check_test.go @@ -354,7 +354,7 @@ func TestRunEmptyTitleIsBroken(t *testing.T) { if !ui.IsSilent(err) || ui.ExitCode(err) != 1 { t.Fatalf("run = %v, want a silent exit-1 error", err) } - if !strings.Contains(out, "empty 'title:'") { + if !strings.Contains(out, "title is present but empty") { t.Errorf("output = %q, want the empty-title message", out) } } @@ -388,7 +388,7 @@ func TestRunEmptyTitleReportedEvenWhenConversionFails(t *testing.T) { if !ui.IsSilent(err) || ui.ExitCode(err) != 1 { t.Fatalf("run = %v, want a silent exit-1 error", err) } - if !strings.Contains(out, "empty 'title:'") { + if !strings.Contains(out, "title is present but empty") { t.Errorf("output = %q, want the empty-title message alongside the collision", out) } } @@ -629,3 +629,137 @@ func TestRunInvalidProjectPageWidthIsNotReportedForAFileThatOverridesIt(t *testi t.Errorf("output = %q, want no complaint: the file's own width wins", out) } } + +// --- pages: entries ----------------------------------------------------------- + +// An entry's values are validated exactly as a file's own are -- and only for +// a file the invocation names, which is #139's scoping made real. +func TestRunValidatesAnEntrysValues(t *testing.T) { + tests := map[string]struct{ project, want string }{ + "invalid page_width": { + "pages:\n main.md:\n page_id: 1\n page_width: huge\n", + "invalid page_width", + }, + "non-numeric page_id": { + "pages:\n main.md:\n page_id: TODO\n", + "not a numeric page id", + }, + "invalid label": { + "pages:\n main.md:\n page_id: 1\n labels: [\"has space\"]\n", + "label", + }, + "empty title": { + "pages:\n main.md:\n page_id: 1\n title: \"\"\n", + "title is present but empty", + }, + } + for name, tc := range tests { + t.Run(name, func(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), tc.project) + write(t, filepath.Join(dir, "main.md"), "# Main\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if !ui.IsSilent(err) || ui.ExitCode(err) != 1 { + t.Fatalf("run = %v, want a silent exit-1 error\n%s", err, out) + } + if !strings.Contains(out, tc.want) { + t.Errorf("output = %q, want it to contain %q", out, tc.want) + } + }) + } +} + +// One bad entry must not block checking an unrelated file: the diagnostic is +// scoped to the file it belongs to, not to the run. +func TestRunEntryDefectIsScopedToItsOwnFile(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), + "pages:\n bad.md:\n page_id: TODO\n good.md:\n page_id: 2\n") + write(t, filepath.Join(dir, "bad.md"), "# Bad\n") + write(t, filepath.Join(dir, "good.md"), "# Good\n") + + if _, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "good.md")}) + }); err != nil { + t.Fatalf("checking good.md = %v, want success: bad.md's entry is not its business", err) + } +} + +// The two locations naming different pages is offline-visible and exactly what +// check exists to catch before a publish does. +func TestRunReportsACoordinateDisagreement(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), "pages:\n main.md:\n page_id: 1\n") + write(t, filepath.Join(dir, "main.md"), "---\npage_id: 999\n---\n# Main\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if !ui.IsSilent(err) || ui.ExitCode(err) != 1 { + t.Fatalf("run = %v, want a silent exit-1 error\n%s", err, out) + } + if !strings.Contains(out, "disagree about where this page is") { + t.Errorf("output = %q, want the disagreement message", out) + } +} + +// "No half-and-half" is a warning, never an error -- agreement between the two +// locations is legal, so this has to be sayable without becoming a wall +// somebody hits halfway through a migration. +func TestRunWarnsAboutHalfAndHalf(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), + "pages:\n other.md:\n page_id: 2\n") + write(t, filepath.Join(dir, "main.md"), "---\ntitle: Main\npage_id: 1\n---\n# Main\n") + write(t, filepath.Join(dir, "other.md"), "# Other\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if err != nil { + t.Fatalf("run = %v, want success: half-and-half is a warning\n%s", err, out) + } + if !strings.Contains(out, "keeps page metadata in markfluence.yaml") { + t.Errorf("output = %q, want the half-and-half warning", out) + } +} + +// A project with no pages: block has not chosen the manifest, so a file's own +// frontmatter is simply how it works -- no warning. +func TestRunNoHalfAndHalfWarningWithoutAManifest(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), "space: ENG\n") + write(t, filepath.Join(dir, "main.md"), "---\ntitle: Main\npage_id: 1\n---\n# Main\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if err != nil { + t.Fatalf("run = %v, want success\n%s", err, out) + } + if strings.Contains(out, "keeps page metadata") { + t.Errorf("output = %q, want no half-and-half warning", out) + } +} + +// A pristine file in a manifest project is the shape #139 exists for, and +// check must have nothing to say about it. +func TestRunPristineManifestFileIsClean(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), + "pages:\n main.md:\n title: Main\n page_id: 1\n") + write(t, filepath.Join(dir, "main.md"), "# Main\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if err != nil { + t.Fatalf("run = %v, want success\n%s", err, out) + } + if !strings.Contains(out, "clean") { + t.Errorf("output = %q, want it reported clean", out) + } +} diff --git a/docs/commands/markfluence_check.md b/docs/commands/markfluence_check.md index 85b711c..9a61354 100644 --- a/docs/commands/markfluence_check.md +++ b/docs/commands/markfluence_check.md @@ -7,11 +7,18 @@ Validate markdown files against the converter and frontmatter rules, offline Validate one or more markdown FILEs against the converter and frontmatter rules, with no network access and no credentials -- fast, safe, and CI/agent-friendly. Reports conversion warnings and broken image/link -references, and frontmatter sanity (parseable, page_width valid, page_id +references, and metadata sanity (parseable, page_width valid, page_id numeric when present). Each file is processed independently; the command exits non-zero if any file is broken or failed outright. Warnings alone do not fail. +A file's metadata is checked wherever it lives -- its own frontmatter or a +'pages:' entry for it in markfluence.yaml -- and an entry is reported only +when its file is one of the FILEs given, so one bad entry never blocks +checking the rest of a repository. Two locations naming different pages is +an error; a file keeping its own keys in a project that uses 'pages:' is a +warning, since both work. + "link not resolved: TARGET" means TARGET is a sibling .md file that exists under the documentation root but has no page_id yet -- the normal state of a tree that hasn't been published, not a defect. "same-page anchor not diff --git a/internal/pagemeta/pagemeta.go b/internal/pagemeta/pagemeta.go index e4cf362..45f4679 100644 --- a/internal/pagemeta/pagemeta.go +++ b/internal/pagemeta/pagemeta.go @@ -23,6 +23,7 @@ package pagemeta import ( "fmt" + "path/filepath" "sort" "strings" @@ -84,6 +85,15 @@ func (r Resolved) MetadataSource() Source { return r.Source } +// InFile reports whether the file's own frontmatter contributed any field +// markfluence understands. It is what check's half-and-half lint asks: a file +// carrying inline keys in a project that has chosen the manifest is the shape +// "no half-and-half" is about. +func (r Resolved) InFile() bool { return r.Source == FromFrontmatter || r.Source == FromBoth } + +// InManifest reports whether a pages: entry claimed this file. +func (r Resolved) InManifest() bool { return r.Source == FromManifest || r.Source == FromBoth } + // Managed reports whether this file is claimed: it has a manifest entry, or its // frontmatter names a page_id. A file nothing claims is skipped rather than // failed -- repositories legitimately hold markdown that is not published, @@ -306,3 +316,28 @@ func sameList(a, b []string) bool { } return true } + +// KeyFor returns the manifest key for a file: its path relative to the root, in +// the normalized slash form pages: is keyed by. +// +// One copy, used by every command that looks a file up, because the manifest +// side and the argument side must agree exactly -- a mismatch is a silent skip +// (Managed), not an error, so there is nothing to notice if they drift. +// +// A file outside the root has no key and is not an error: a batch may span more +// than one project (docs/root-model.md), and a file belonging to a different +// root than the one being consulted simply has no entry there. +func KeyFor(root *project.Root, absPath string) (string, bool) { + if root == nil { + return "", false + } + rel, err := filepath.Rel(root.Dir, absPath) + if err != nil { + return "", false + } + key, err := project.NormalizePageKey(filepath.ToSlash(rel)) + if err != nil { + return "", false + } + return key, true +} diff --git a/internal/pagemeta/pagemeta_test.go b/internal/pagemeta/pagemeta_test.go index 54427cf..4102dcf 100644 --- a/internal/pagemeta/pagemeta_test.go +++ b/internal/pagemeta/pagemeta_test.go @@ -280,3 +280,26 @@ func TestResolveNilRoot(t *testing.T) { t.Errorf("source = %q managed = %v", r.Source, r.Managed()) } } + +func TestKeyFor(t *testing.T) { + root := rootWith(t, "pages: {}\n") + got, ok := KeyFor(root, filepath.Join(root.Dir, "docs", "a.md")) + if !ok || got != "docs/a.md" { + t.Errorf("= %q/%v, want docs/a.md", got, ok) + } + if got, ok := KeyFor(root, filepath.Join(root.Dir, "a.md")); !ok || got != "a.md" { + t.Errorf("= %q/%v, want a.md", got, ok) + } +} + +// A batch may span more than one project, so a file outside the root being +// consulted has no key there -- and that is not an error, just no entry. +func TestKeyForOutsideTheRoot(t *testing.T) { + root := rootWith(t, "pages: {}\n") + if got, ok := KeyFor(root, filepath.Join(filepath.Dir(root.Dir), "elsewhere.md")); ok { + t.Errorf("= %q/%v, want no key for a file outside the root", got, ok) + } + if _, ok := KeyFor(nil, "/tmp/a.md"); ok { + t.Error("want no key for a nil root") + } +} From f9c87df8f3649ac9b0e2c698fb15f6fb643d774e Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:07:10 -0400 Subject: [PATCH 08/14] docs: record pages:, and what update losing its flags means docs/root-model.md gains a pages: section beside the settings: that an entry is a frontmatter block living elsewhere, the disagreement grading, that an unclaimed file is skipped rather than failed, and the path-key rules -- lexical, root-relative, no case folding, with an escaping or duplicate key failing the load. docs/github-actions.md is the one that changes most, because it was the document #139 exists to make true. Its example was `update --page-id=12345 --force docs/some_doc.md`; it is now `update docs/**/*.md` with no per-file inputs at all, plus the pristine-markdown shape, and notes on the skip, on metadata_source, and on why creating pages stays a human act. docs/guarantees.md extends the L2 note: a pages: entry is #100's argument taken further, and it is *why* the keys are lexical -- resolving a symlink would make a key mean different things on two checkouts. update losing its three page flags narrows L2's carve-out for flags, since which page a file publishes to no longer depends on how the command was invoked. CLAUDE.md gets an internal/pagemeta bullet, and internal/project's gains pages: -- including the three things that took a second pass: Source and Managed computed from different predicates, contributed metadata counted only over known fields (or a Jekyll tree reads as claimed), and labels compared as a set. docs/json-output.md documents metadata_source. The README block and docs/markdown_file.md both point at the new section rather than restating the field table. --- CLAUDE.md | 3 ++- README.md | 17 ++++++++++++++++ docs/github-actions.md | 31 ++++++++++++++++++++++++++-- docs/guarantees.md | 14 +++++++++++-- docs/json-output.md | 9 +++++++++ docs/markdown_file.md | 26 +++++++++++++++++++++--- docs/root-model.md | 46 ++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 138 insertions(+), 8 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index b625fbb..af6b5bb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -70,7 +70,8 @@ Module `github.com/mozilla/markfluence` (`go 1.25`). `main.go` is a shim to `cmd - `cmd/attachment{list,upload,download}/` — the flat `attachment-list`/`attachment-upload`/`attachment-download` commands (noun-first so cobra's alphabetized help keeps them together and `attachment-` completes as a group). `upload` reuses the checksum skip/update logic, with `--force` (`client.ForceUploadAttachments`) and `--dry-run` (`PlanAttachments`); its `--name` takes a *path* whose base name becomes the stored name, and the recorded `path=` is the path as given, so a later publish can't create a duplicate under a different name; a batch whose base names collide is refused, since `planAttachments` reads the page's attachments once before its loop and would otherwise plan two creates for one name. `download` restores an attachment to its recorded `path=`, using the stored name verbatim when there is none (never interpreting it — a file really called `a%2Fb.png` must not be scattered into `a/b.png`, and `convert.sourceFor` answers the same way on the markdown side, which is what keeps a downloaded file where the markdown says it is), with `--flat` to opt out; `destPath` is the only place server data becomes a filesystem path and clamps to `--dest`, refusing rather than clipping an escape, since `..` is legitimate in a source path. - `cmd/schema/` — `schema`: print the embedded `--json` schema to stdout verbatim (no args, no credentials, no Confluence call). `--json` is deliberately a no-op — the output is already the schema document, not an envelope — which is also why `schema` is absent from the schema's own `command` enum. - `schema/` — the published JSON Schema (`json-output/v1.json`) *and* the `schema` Go package that embeds it (`V1`). The Go file lives beside the schema because `go:embed` cannot reach outside its own directory, and the schema stays at a top-level path a non-Go consumer can browse, mirroring its own `$id`. `internal/schematest` validates against the embed rather than reading the file, which is what makes "what ships" and "what the tests checked" the same bytes — do not reintroduce a disk read or a second copy. The version number is **not** restated here: `jsonout.SchemaVersion` and the document's own `schema_version` const are the two copies, tied together by a test in `cmd/schema`. The envelope's and the error object's top-level **`warnings`** are the one field no command fills: `jsonout.NewEnvelope`/`EmitError` drain a package-level collector (`AddWarning`), because the only thing in it is raised during credential resolution — below any command, before either document exists. -- `internal/project` — the documentation root: `Discover` (walk up from a directory looking for `markfluence.yaml`), `FromPath` (`--root`), `Resolve`, and `Cache`, which consults itself at every level of the walk so a batch spanning a subtree pays for the walk — and `os.OpenRoot` — once rather than per directory (the quadratic cost `_plans/025` measured). A `Root` carries `Dir`, `File`, `Config` and an `os.Root` that refuses an escape even through a symlink partway down. `Discover` is called from **two starting points for two reasons** — once per invocation from the working directory to locate `.env`, and once per markdown file from its own directory to bound its reads and name its attachments — which is why it returns a type rather than a string; the two diverge legitimately, so a multi-root batch is allowed and nothing refuses it ([docs/root-model.md](docs/root-model.md)). `config.go` reads the project file's **settings** (#100): `space` and `page_width`, resolving **flag > frontmatter > project file**, which is *not* the credentials chain and must never be conflated with it. Three things about it are load-bearing. It is read through `frontmatter.Dialect.ReadMapping` rather than a second parser, since every rule there was found by probing goccy and a second copy would be a second set of the same bugs. An **unknown top-level key is fatal**, and that is the point rather than a cost — a silently ignored `spce: ENG` is wrong for every file at once, and a file written for a newer markfluence holds keys this binary would ignore, so there is no schema version and this must not be loosened; an empty or comment-only file stays valid, being what ships and what `export` plants. And loading happens in `open()`, the single place a `Root` is built from a marker hit, so `Discover`/`Cache`/`FromPath` cannot disagree that a file which cannot be understood **is not a valid marker**: the walk does not continue upward and does not fall back to the starting directory, because the root decides every attachment name and guessing at it is worse than stopping. `ConfigError`/`IsConfigError`/`RootError` exist so a caller reports that as a local defect (`VALIDATION`) rather than under `resolving the documentation root` as I/O. What it validates is **structure only**: `internal/pagewidth` cannot be imported here (`pagewidth` → `client` → `project`), so a width's vocabulary is checked by `pagewidth.Declared` where it already runs and by `check`'s offline lint. `Config` deliberately holds **no `url` or token**, and the reason is sharper than "those are credentials": basic auth goes to whatever host the resolved URL names, so a committed, walked-up file naming one would decide where `CONFLUENCE_TOKEN` is sent — a worse version of the `.env` hole #136 records. +- `internal/project` — the documentation root: `Discover` (walk up from a directory looking for `markfluence.yaml`), `FromPath` (`--root`), `Resolve`, and `Cache`, which consults itself at every level of the walk so a batch spanning a subtree pays for the walk — and `os.OpenRoot` — once rather than per directory (the quadratic cost `_plans/025` measured). A `Root` carries `Dir`, `File`, `Config` and an `os.Root` that refuses an escape even through a symlink partway down. `Discover` is called from **two starting points for two reasons** — once per invocation from the working directory to locate `.env`, and once per markdown file from its own directory to bound its reads and name its attachments — which is why it returns a type rather than a string; the two diverge legitimately, so a multi-root batch is allowed and nothing refuses it ([docs/root-model.md](docs/root-model.md)). `config.go` reads the project file's **settings** (#100): `space` and `page_width`, resolving **flag > frontmatter > project file**, which is *not* the credentials chain and must never be conflated with it. Three things about it are load-bearing. It is read through `frontmatter.Dialect.ReadMapping` rather than a second parser, since every rule there was found by probing goccy and a second copy would be a second set of the same bugs. An **unknown top-level key is fatal**, and that is the point rather than a cost — a silently ignored `spce: ENG` is wrong for every file at once, and a file written for a newer markfluence holds keys this binary would ignore, so there is no schema version and this must not be loosened; an empty or comment-only file stays valid, being what ships and what `export` plants. And loading happens in `open()`, the single place a `Root` is built from a marker hit, so `Discover`/`Cache`/`FromPath` cannot disagree that a file which cannot be understood **is not a valid marker**: the walk does not continue upward and does not fall back to the starting directory, because the root decides every attachment name and guessing at it is worse than stopping. `ConfigError`/`IsConfigError`/`RootError` exist so a caller reports that as a local defect (`VALIDATION`) rather than under `resolving the documentation root` as I/O. `pages.go` holds the **`pages:`** key (#139): an `Entry` is `{Fields, Lists}`, the same two maps `frontmatter.MarkdownFile` carries, which is the design rather than a convenience — `pagewidth` and `labels` both reach `client`, which holds a `*Cache`, so a typed validated entry would need a broken cycle or a second copy of every field's rules, and with two maps `labels.Declared(e.Lists, e.Fields)` works unchanged. `entryFields` is the manifest's schema and the only place it is written down; it mirrors frontmatter's fields deliberately, since adding one there and not here would make a field expressible in a file and not in an entry. Load checks **structure** — a mapping of mappings, legal paths, known field *names*, right shapes — and never a field's **value**, because #139 requires a semantically bad entry to be reported only when its file is one of the arguments, and this package has no idea which files the command was given. An unknown field *name* is the exception and is fatal at load, being the same typo class as an unknown setting. `NormalizePageKey` is lexical (L2 forbids a key whose meaning depends on the checkout's layout), and an escaping key or two keys normalizing to one are load-time errors naming both spellings. `Config.Pages` is **nil when there is no `pages:` key and empty-non-nil for `pages: {}`**, which is how a command tells "has not chosen the manifest" from "has, and has registered nothing". What it validates is **structure only**: `internal/pagewidth` cannot be imported here (`pagewidth` → `client` → `project`), so a width's vocabulary is checked by `pagewidth.Declared` where it already runs and by `check`'s offline lint. `Config` deliberately holds **no `url` or token**, and the reason is sharper than "those are credentials": basic auth goes to whatever host the resolved URL names, so a committed, walked-up file naming one would decide where `CONFLUENCE_TOKEN` is sent — a worse version of the `.env` hole #136 records. +- `internal/pagemeta` — `Resolve`, the one merge of a file's page metadata from the two places it may live: its own frontmatter and a `pages:` entry in the project file (#139). A package because `update`, `create`, `check` **and `internal/linkindex`** all need it, and a per-command copy is how two commands come to publish one file to two different pages. It imports `frontmatter` and `project` and nothing else, which is also why it validates no *value*: `pagewidth` and `labels` are unreachable from here, and the commands that need them already call them on the maps it returns. **Frontmatter and an entry are two spellings of one level, not two levels of a precedence chain** — when both speak the rule is not "the higher wins" but a grading: `page_id`/`space`/`parent` are coordinates and a disagreement fails the file (for `create`, the batch, since it preflights everything), while `title`/`page_width`/`labels` are visible and recoverable, so they warn and frontmatter wins. Agreement is silent, which is what makes migration incremental. Three things took a second pass and should not be flattened: `Source` (what `--json` reports as `metadata_source`) and `Managed` are computed from **different predicates** — the first answers "who contributed metadata", the second "should `update` act on this file", which is true when an entry exists or a `page_id` is named, so `a.md: {}` is a claim that must fail for want of an id rather than be skipped; contributed metadata is counted only over fields `project.IsPageField` knows, or a docs tree carrying Jekyll's `layout:`/`date:` reads as claimed and a whole tree fails; and `labels` is compared as a **set**, since Confluence has no label order and a reordering cannot reach the page. A blank value is not a disagreement — every null spelling already reads as `""`. `KeyFor` is the one place a file's path becomes a manifest key, used on both sides because a mismatch is a silent skip rather than an error. - `internal/pageslug` — `Slug`/`For`/`Filename`: a title to a filename-safe slug. A package rather than a helper because `export`, `read` and `attachment-download` all place attachments under a page's own directory and must agree. It lowercases (so case-variant titles collide and can be caught) and drops `/` (so no title can inject a path separator); it is lossy, and no readable slug can avoid being, so the caller decides what a collision means. Known limit: NFD and NFC spellings of one title are different Go strings but one filename on APFS, so that pair is not disambiguated. - `internal/pagedoc` — a fetched page as a markdown document: `Render` (frontmatter + converted body), `Frontmatter`, and the lookups the converter can't do for itself — `Sources`/`SourcesFrom` (attachment name → recorded source path) and `PageLinks` (the page an `` points at → its URL). **One conversion, parameterized by a `Placement`**: where the page's file sits, where its unrecorded attachments go, what `parent:` says, and the attachment listing the caller already has. `read`, `export` and `attachment-download` all go through `Options`/`AttachmentDirFor` rather than assembling their own, so they cannot drift by accident — only by argument. For a page at the top level of what is being written, which is what `read` prints and what a single-page export writes, `read` and `export` are byte-identical; deeper in a tree they differ in exactly the position-dependent parts (a sourced attachment's `../` prefix, a `-` suffix a sibling forced, and `parent:`), because `read` has no tree to be positioned in. It needs a client (page width, attachment list, title lookups), which is why it isn't in `internal/convert` — that package is deliberately client-free, and it's why `StorageToMarkdown` takes those maps rather than fetching them. Every one of them is best-effort in the same shape: no references in the body means no request at all, and a lookup that fails is omitted rather than fatal (an omitted page link renders as raw storage, not as a link with no destination). It also owns **`UserCache`**, the user-name lookup both mention directions share (#91): a per-run, **cross-page** cache, threaded in from the caller the way `project.Cache`/`linkindex.Cache` are, because the obvious structure is wrong — `PageLinks` builds its space-id map per page and `Options` is built per page, so a user map written that way would re-resolve the same twelve people on every page of a 200-page export. It **remembers misses**, or a page mentioning deactivated people costs a request each, every page, to learn the same failures. Not persisted to disk, and the reason is **L2**: output must depend only on the files on disk, not on what a cache happens to hold. `MentionWarnings` is the forward direction's use of it, and sharing the cache is what makes that warning affordable — publishing needs no display names at all, so that lookup exists purely to report an id that names nobody. `PageLinks` resolves a space id **once per space key**, not once per link, and refuses to search site-wide when it can't scope a title to a space — a same-titled page in the wrong space is a wrong answer, which is worse than the passthrough a miss produces. - `internal/attachfile` — `Resolve` (where an attachment goes under a destination root, **including the traversal clamp**) and `Write` (download it there, honoring force/dry-run). Shared by `attachment-download` and `export`; the clamp must never exist in two copies. diff --git a/README.md b/README.md index 1b2f57c..f686c91 100644 --- a/README.md +++ b/README.md @@ -498,6 +498,23 @@ project-wide default is wrong for every file at once. Credentials are deliberately not settings here; see [docs/root-model.md](docs/root-model.md#what-it-deliberately-does-not-hold). +It can also hold a **`pages:` block**, giving each file its page metadata so the +markdown itself stays pristine — no frontmatter at all: + +```yaml +pages: + docs/deploy-runbook.md: + title: Deploy Runbook + page_id: 12346 +``` + +That is what makes `markfluence update docs/**/*.md` work from CI with no +per-file inputs, and it is why `update` has no `--page-id`/`--title` flags: +those would each have to name one file. Both locations are legal and agreement +is silent, so you can move metadata in a file at a time; a file neither place +mentions is skipped rather than failed. Details: +[docs/root-model.md](docs/root-model.md#pages--page-metadata-for-a-pristine-file). + The rest of this section is the precise version of the same idea. Every markdown file has a **documentation root**: the directory holding `markfluence.yaml`, found by walking up from the file's own directory, or — diff --git a/docs/github-actions.md b/docs/github-actions.md index 5f4e1eb..66ce03d 100644 --- a/docs/github-actions.md +++ b/docs/github-actions.md @@ -65,15 +65,42 @@ jobs: # A variable, not a secret: the cloud ID is public. Omit it if you're # using an unscoped personal token. CONFLUENCE_CLOUD_ID: ${{ vars.CONFLUENCE_CLOUD_ID }} - run: - markfluence update --page-id=12345 --force docs/some_doc.md + run: markfluence update docs/**/*.md ``` +That step takes no per-file inputs, and that is the point: each file's page id +and title come from its own frontmatter or from a `pages:` entry in +`markfluence.yaml`, so adding a page is a repository change rather than a +workflow change. There are deliberately no `--page-id`/`--title`/`--page-width` +flags — they would each have to name a single file, which is what made +`docs/**/*.md` inexpressible before. + +If your markdown must stay pristine — a README, or a docs tree with other +readers — put every page's metadata in `markfluence.yaml`: + +```yaml +space: ENG + +pages: + docs/deploy-runbook.md: + title: Deploy Runbook + page_id: 12346 +``` + +See [the project file](root-model.md#pages--page-metadata-for-a-pristine-file). + Notes: - **Exit codes.** `update` exits non-zero if any file fails, so the job fails loudly. Add `--json` to get machine-readable per-file results on stdout (see [`--json` output](../README.md#--json-output)) if a later step needs to parse them. +- **A file nothing claims is skipped, not failed**, so a glob over a docs tree + does not turn the job red when somebody adds a draft. `metadata_source` in + `--json` says which location supplied each published page's metadata, which is + what to look at when a page lands somewhere unexpected. +- **Creating pages stays a human act.** A workflow creating one would have to + commit the new `page_id` back to the repository. Create locally, commit the + entry, and let CI update from then on. A reusable composite/Docker action wrapping this is tracked in [#29](https://github.com/mozilla/markfluence/issues/29). diff --git a/docs/guarantees.md b/docs/guarantees.md index f9cb1aa..dfbd81b 100644 --- a/docs/guarantees.md +++ b/docs/guarantees.md @@ -185,8 +185,9 @@ how a link to `sub/dup.md` reached `./dup.md`. `internal/linkindex` resolves by path instead, so a basename can no longer match the wrong file (`_plans/026` commit 5). -**L2** is deliberately narrow. `--title` and `--page-width` change what gets -published and are meant to, so the law constrains resolution and naming only. +**L2** is deliberately narrow. A flag like `create`'s `--title` changes what +gets published and is meant to, so the law constrains resolution and naming +only. Within that scope it rules out a root derived from the working directory, and equally one derived from the *set* of arguments — the same file would otherwise be named differently depending on what else was in the batch. `internal/project` @@ -201,6 +202,15 @@ working-directory-independent walk, so two people in different directories resolve it identically. It is strictly better for L2 than the `--space` flag it replaces, which is invocation state by definition. Status unchanged. +A `pages:` entry (#139) is the same argument taken further, and it is why the +path keys are **lexical and root-relative** rather than resolved. Resolving a +symlink would make a key's meaning depend on how a checkout was laid out, so +the same repository could publish to different pages on two machines — exactly +what L2 forbids. It is also why `update` lost `--title`, `--page-id` and +`--page-width`: page metadata now lives entirely in files on disk, so which +page a file publishes to no longer depends on how the command was invoked at +all. L2's carve-out for flags is narrower than it was. + **L3** is what makes moving a page free. `images.go` records an attachment's `Source` relative to the root rather than to the referencing page, so identity follows the asset alone (`_plans/026` commit 4). diff --git a/docs/json-output.md b/docs/json-output.md index d359907..97eee9c 100644 --- a/docs/json-output.md +++ b/docs/json-output.md @@ -38,6 +38,15 @@ per-command details a script author hits once and then needs to look up. `.results[] | .filename` works and `summary.total` is the attachment count. `export` nests the files it wrote in an `attachments` array on its page result, the way `update`/`create` do. +- **`metadata_source`** on `update`/`create` says which location supplied that + file's page metadata: `"frontmatter"`, `"manifest"` (a `pages:` entry in + `markfluence.yaml`), or `null` when nothing claimed the file. Where both + locations speak it reports `"frontmatter"`, since that is the one that wins + every field it can win. It exists because "why did this publish to *that* + page?" is otherwise only answerable by reproducing the resolution by hand, + which is hard from a CI log. A `null` source on `update` goes with + `status: skipped`: nothing claims the file, which is not a failure — + repositories legitimately hold markdown that is not published. - **`check`'s `broken` status is `ok: false` with no `error`/`code`** — unlike every other failure, its `broken`/`warnings` arrays already say everything there is to say, so there's no separate operational error to attach. Only diff --git a/docs/markdown_file.md b/docs/markdown_file.md index 737b998..dee697c 100644 --- a/docs/markdown_file.md +++ b/docs/markdown_file.md @@ -64,9 +64,29 @@ page_width: max The chain is **flag > frontmatter > project file** — the answer closest to the content wins — and the project file is only consulted when both levels above it -are silent, so it never conflicts with either. No other frontmatter field has a -project-wide form: `title` and `page_id` are per page by definition, and -`parent` varies per file. +are silent, so it never conflicts with either. + +### The same block, somewhere else + +Every field above can live in a `pages:` entry in `markfluence.yaml` instead of +in the file, which is how a markdown file stays pristine and still gets +published: + +```yaml +pages: + docs/deploy-runbook.md: + title: Deploy Runbook + page_id: 12346 + labels: [runbook] +``` + +An entry is the same block, moved — the same field names, the same value +domains, the same canonical order. Both locations are legal, agreement is +silent, and an entry is *not* a fourth precedence level: frontmatter and an +entry are two spellings of one level, so when both speak the rule is a +disagreement rule rather than a precedence one. The details, and the path-key +rules, are in +[root-model.md](root-model.md#pages--page-metadata-for-a-pristine-file). ## Body diff --git a/docs/root-model.md b/docs/root-model.md index 73e64c2..b07e8db 100644 --- a/docs/root-model.md +++ b/docs/root-model.md @@ -90,6 +90,52 @@ key out. Settings are per-root, so an invocation spanning two projects gets each project's own defaults — see [Multi-root batches](#multi-root-batches-are-allowed). +### `pages:` — page metadata for a pristine file + +A `pages:` block maps a path to that file's page metadata, so a markdown file +can be published while carrying no markfluence keys at all: + +```yaml +space: ENG + +pages: + docs/deploy-runbook.md: + title: Deploy Runbook + page_id: 12346 + labels: [runbook] +``` + +An entry **is a frontmatter block that lives elsewhere** — the same field names, +the same value domains, the same canonical order. Both locations are legal and +agreement is silent, which is what makes moving metadata into the manifest +something you can do a file at a time. + +Where the two disagree, what happens depends on what the disagreement can +destroy: + +| field | on disagreement | +|---|---| +| `page_id`, `space`, `parent` | **error** — the file fails. A `page_id` pasted from an old file would publish over a live page | +| `title`, `page_width`, `labels` | **warning**, and the frontmatter wins. Visible and recoverable | + +A file **neither location mentions is skipped**, not failed: a repository +legitimately holds markdown that is not published, so `markfluence update +docs/**/*.md` does not go red because somebody added a draft. A file that *is* +registered but has no `page_id` fails — something claimed it and the page has +not been created yet. + +**Path keys** are relative to the root, in slash form, and lexically cleaned +(`./docs/a.md` and `docs/a.md` are the same key). Two rules are load-time +errors, because either means the manifest's structure is wrong rather than one +entry being bad: a key that escapes the root, and two keys that normalize to one +path. Keys are compared exactly, with no case folding — on a case-insensitive +filesystem `Docs/a.md` opens the file but matches no `docs/a.md` key, so the +file reads as unmanaged and is skipped. + +Resolution is lexical and never follows symlinks, for the same reason **L2** +requires of everything else here: a key whose meaning depended on how the +checkout was laid out would resolve differently on two machines. + ### A file that cannot be understood stops the command An unparseable file, or one holding a key markfluence does not recognise, is an From 8d86b9a39f74ab71e7806b99f5ea08ea69533793 Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:08:22 -0400 Subject: [PATCH 09/14] fix(schema): keep the hand-authored formatting The metadata_source addition went in via a JSON round-trip, which reformatted the whole document -- every compact inline object exploded across lines, for a 1728-line diff in a file that is hand-formatted and read by people. Reverted and applied as text: 10 lines, and only the two definitions that change. The enum also becomes a shared metadataSourceOrNull $def rather than being written twice, matching stringOrNull and the other *OrNull refs beside it. --- schema/json-output/v1.json | 1736 +++++++----------------------------- 1 file changed, 315 insertions(+), 1421 deletions(-) diff --git a/schema/json-output/v1.json b/schema/json-output/v1.json index 1f25a6a..6222146 100644 --- a/schema/json-output/v1.json +++ b/schema/json-output/v1.json @@ -5,896 +5,359 @@ "description": "The single JSON document markfluence writes to stdout under --json. results holds one object per target (a single element for info/read; one per attachment for attachment-list); its item shape and the summary shape depend on command. The typed error object written to stderr on a fatal/pre-flight failure conforms to #/$defs/errorObject.", "type": "object", "additionalProperties": false, - "required": [ - "schema_version", - "markfluence_version", - "command", - "roots", - "warnings", - "results", - "summary" - ], + "required": ["schema_version", "markfluence_version", "command", "roots", "warnings", "results", "summary"], "properties": { - "schema_version": { - "const": 1 - }, - "markfluence_version": { - "type": "string" - }, - "command": { - "enum": [ - "info", - "read", - "update", - "create", - "fix", - "check", - "children", - "find", - "search", - "attachment-list", - "attachment-upload", - "attachment-download", - "export" - ] - }, - "roots": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Every distinct documentation root the command resolved, sorted. Empty for a command with no per-file root concept, and for a pre-flight failure that never reached root resolution." - }, - "warnings": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Warnings about the invocation itself rather than about any page or file -- currently only the .env permission warning, raised when the .env that supplied the API token is reachable by anyone but its owner. Always present, [] when there is nothing to report, the same convention roots and results follow. Not per-result warnings: those live on the result." - }, - "results": { - "type": "array" - }, - "summary": { - "type": "object" - } + "schema_version": { "const": 1 }, + "markfluence_version": { "type": "string" }, + "command": { "enum": ["info", "read", "update", "create", "fix", "check", "children", "find", "search", "attachment-list", "attachment-upload", "attachment-download", "export"] }, + "roots": { "type": "array", "items": { "type": "string" }, "description": "Every distinct documentation root the command resolved, sorted. Empty for a command with no per-file root concept, and for a pre-flight failure that never reached root resolution." }, + "warnings": { "type": "array", "items": { "type": "string" }, "description": "Warnings about the invocation itself rather than about any page or file -- currently only the .env permission warning, raised when the .env that supplied the API token is reachable by anyone but its owner. Always present, [] when there is nothing to report, the same convention roots and results follow. Not per-result warnings: those live on the result." }, + "results": { "type": "array" }, + "summary": { "type": "object" } }, "allOf": [ { - "if": { - "properties": { - "command": { - "const": "info" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "info" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "oneOf": [ - { - "$ref": "#/$defs/infoResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } - ] - } - }, - "summary": { - "$ref": "#/$defs/basicSummary" - } + "results": { "items": { "oneOf": [{ "$ref": "#/$defs/infoResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, + "summary": { "$ref": "#/$defs/basicSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "read" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "read" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "oneOf": [ - { - "$ref": "#/$defs/readResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } - ] - } - }, - "summary": { - "$ref": "#/$defs/basicSummary" - } + "results": { "items": { "oneOf": [{ "$ref": "#/$defs/readResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, + "summary": { "$ref": "#/$defs/basicSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "update" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "update" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "$ref": "#/$defs/updateResult" - } - }, - "summary": { - "$ref": "#/$defs/updateSummary" - } + "results": { "items": { "$ref": "#/$defs/updateResult" } }, + "summary": { "$ref": "#/$defs/updateSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "create" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "create" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "$ref": "#/$defs/createResult" - } - }, - "summary": { - "$ref": "#/$defs/createSummary" - } + "results": { "items": { "$ref": "#/$defs/createResult" } }, + "summary": { "$ref": "#/$defs/createSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "fix" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "fix" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "$ref": "#/$defs/fixResult" - } - }, - "summary": { - "$ref": "#/$defs/fixSummary" - } + "results": { "items": { "$ref": "#/$defs/fixResult" } }, + "summary": { "$ref": "#/$defs/fixSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "check" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "check" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "$ref": "#/$defs/checkResult" - } - }, - "summary": { - "$ref": "#/$defs/checkSummary" - } + "results": { "items": { "$ref": "#/$defs/checkResult" } }, + "summary": { "$ref": "#/$defs/checkSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "attachment-upload" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "attachment-upload" } }, "required": ["command"] }, "then": { "properties": { "results": { "items": { - "oneOf": [ - { - "$ref": "#/$defs/attachmentUploadResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } - ] + "oneOf": [{ "$ref": "#/$defs/attachmentUploadResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, - "summary": { - "$ref": "#/$defs/attachmentSummary" - } + "summary": { "$ref": "#/$defs/attachmentSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "export" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "export" } }, "required": ["command"] }, "then": { "properties": { "results": { - "items": { - "oneOf": [ - { - "$ref": "#/$defs/exportResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } - ] - } + "items": { "oneOf": [{ "$ref": "#/$defs/exportResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, - "summary": { - "$ref": "#/$defs/exportSummary" - } + "summary": { "$ref": "#/$defs/exportSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "attachment-download" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "attachment-download" } }, "required": ["command"] }, "then": { "properties": { "results": { "items": { - "oneOf": [ - { - "$ref": "#/$defs/attachmentDownloadResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } - ] + "oneOf": [{ "$ref": "#/$defs/attachmentDownloadResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, - "summary": { - "$ref": "#/$defs/attachmentSummary" - } + "summary": { "$ref": "#/$defs/attachmentSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "children" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "children" } }, "required": ["command"] }, "then": { "properties": { "results": { "items": { "oneOf": [ - { - "$ref": "#/$defs/childrenResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } + { "$ref": "#/$defs/childrenResult" }, + { "$ref": "#/$defs/singleOpFailure" } ] } }, - "summary": { - "$ref": "#/$defs/basicSummary" - } + "summary": { "$ref": "#/$defs/basicSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "find" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "find" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "$ref": "#/$defs/findResult" - } - }, - "summary": { - "$ref": "#/$defs/basicSummary" - } + "results": { "items": { "$ref": "#/$defs/findResult" } }, + "summary": { "$ref": "#/$defs/basicSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "search" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "search" } }, "required": ["command"] }, "then": { "properties": { - "results": { - "items": { - "$ref": "#/$defs/searchResult" - } - }, - "summary": { - "$ref": "#/$defs/searchSummary" - } + "results": { "items": { "$ref": "#/$defs/searchResult" } }, + "summary": { "$ref": "#/$defs/searchSummary" } } } }, { - "if": { - "properties": { - "command": { - "const": "attachment-list" - } - }, - "required": [ - "command" - ] - }, + "if": { "properties": { "command": { "const": "attachment-list" } }, "required": ["command"] }, "then": { "properties": { "results": { "items": { - "oneOf": [ - { - "$ref": "#/$defs/attachmentListResult" - }, - { - "$ref": "#/$defs/singleOpFailure" - } - ] + "oneOf": [{ "$ref": "#/$defs/attachmentListResult" }, { "$ref": "#/$defs/singleOpFailure" }] } }, - "summary": { - "$ref": "#/$defs/basicSummary" - } + "summary": { "$ref": "#/$defs/basicSummary" } } } } ], "$defs": { "code": { - "enum": [ - "CONFIG", - "AUTH", - "NOT_FOUND", - "VALIDATION", - "CONVERT", - "IO", - "NETWORK", - "API" - ] + "enum": ["CONFIG", "AUTH", "NOT_FOUND", "VALIDATION", "CONVERT", "IO", "NETWORK", "API"] }, "codeOrNull": { - "oneOf": [ - { - "$ref": "#/$defs/code" - }, - { - "type": "null" - } - ] + "oneOf": [{ "$ref": "#/$defs/code" }, { "type": "null" }] }, "childrenResult": { "description": "One page or folder under the requested node, or under the requested space's root. The results array is flat and in walk order (depth-first, siblings in the order Confluence displays them); parent_id and depth carry the hierarchy. parent_id is null for a page at the root of a space, which hangs off no node -- under --space, that is every depth-1 row. space and url are both derived from the row's webui link, so a row missing one is missing both.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "id", - "type", - "title", - "status", - "parent_id", - "depth", - "space", - "url" - ], + "required": ["ok", "id", "type", "title", "status", "parent_id", "depth", "space", "url"], "properties": { - "ok": { - "const": true - }, - "id": { - "type": "string" - }, - "type": { - "enum": [ - "page", - "folder" - ] - }, - "title": { - "type": "string" - }, - "status": { - "type": "string" - }, - "parent_id": { - "$ref": "#/$defs/stringOrNull" - }, - "depth": { - "type": "integer", - "minimum": 1 - }, - "space": { - "$ref": "#/$defs/stringOrNull" - }, - "url": { - "$ref": "#/$defs/stringOrNull" - } + "ok": { "const": true }, + "id": { "type": "string" }, + "type": { "enum": ["page", "folder"] }, + "title": { "type": "string" }, + "status": { "type": "string" }, + "parent_id": { "$ref": "#/$defs/stringOrNull" }, + "depth": { "type": "integer", "minimum": 1 }, + "space": { "$ref": "#/$defs/stringOrNull" }, + "url": { "$ref": "#/$defs/stringOrNull" } } }, "findResult": { "description": "One page or folder whose title matches exactly. There is no failure variant: find names no page to fail about, so a failed search is an error object on stderr with no envelope. status may be \"archived\" -- such a page is absent from the page tree but still reserves its title, so it blocks creating a page with that title in the same space. A folder reserves nothing, so a folder match is discovery only and never explains a creation conflict. space and url are both derived from the match's link, so a row missing one is missing both.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "id", - "type", - "title", - "space", - "status", - "url" - ], + "required": ["ok", "id", "type", "title", "space", "status", "url"], "properties": { - "ok": { - "const": true - }, - "id": { - "type": "string" - }, - "type": { - "enum": [ - "page", - "folder" - ] - }, - "title": { - "type": "string" - }, - "space": { - "$ref": "#/$defs/stringOrNull" - }, - "status": { - "type": "string" - }, - "url": { - "$ref": "#/$defs/stringOrNull" - } + "ok": { "const": true }, + "id": { "type": "string" }, + "type": { "enum": ["page", "folder"] }, + "title": { "type": "string" }, + "space": { "$ref": "#/$defs/stringOrNull" }, + "status": { "type": "string" }, + "url": { "$ref": "#/$defs/stringOrNull" } } }, "searchResult": { "description": "One full-text match. The array is in Confluence's relevance order, best first; that order is the only ranking available (the API reports score as 0.0 on every row), so re-sorting results discards it. The ordering is stable in membership but not in position: two identical searches return the same set of hits, but equally-ranked neighbours can trade places, so do not diff two runs expecting the same sequence or treat a position as an identifier. There is no failure variant and no status field: search names no page to fail about, so a failed search is an error object on stderr with no envelope, and the search index cannot see archived content, so every hit is current -- use find to discover an archived page. type is an open string rather than an enum because --type all and --cql can return whiteboard, database, or any content type Atlassian adds. space and url are both derived from the hit's link, so a result missing one is missing both; excerpt is null when the match was in the title alone, and is always a single line with Confluence's highlight markers removed and entities unescaped.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "id", - "type", - "title", - "space", - "url", - "excerpt" - ], + "required": ["ok", "id", "type", "title", "space", "url", "excerpt"], "properties": { - "ok": { - "const": true - }, - "id": { - "type": "string" - }, - "type": { - "type": "string" - }, - "title": { - "type": "string" - }, - "space": { - "$ref": "#/$defs/stringOrNull" - }, - "url": { - "$ref": "#/$defs/stringOrNull" - }, - "excerpt": { - "$ref": "#/$defs/stringOrNull" - } + "ok": { "const": true }, + "id": { "type": "string" }, + "type": { "type": "string" }, + "title": { "type": "string" }, + "space": { "$ref": "#/$defs/stringOrNull" }, + "url": { "$ref": "#/$defs/stringOrNull" }, + "excerpt": { "$ref": "#/$defs/stringOrNull" } } }, "searchSummary": { "description": "search's summary. failed is always 0, since a failed search is an error object with no envelope rather than a failed result. truncated says --limit was reached with matches left over; it is a flag and not a count because the API's totalSize is an estimate that has been observed both to drift between pages and to be nonzero against an empty results array. skipped counts index rows that carried no addressable content object -- a `type = space` hit, for example -- which is reachable only via --cql or --type all; it is reported rather than dropped quietly, because otherwise such a query yields total 0 and reads as a genuine miss.", "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "truncated", - "skipped" - ], + "required": ["total", "succeeded", "failed", "truncated", "skipped"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "truncated": { - "type": "boolean" - }, - "skipped": { - "type": "integer", - "minimum": 0 - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "truncated": { "type": "boolean" }, + "skipped": { "type": "integer", "minimum": 0 } } }, "stringOrNull": { - "type": [ - "string", - "null" - ] + "type": ["string", "null"] }, "parentTypeOrNull": { "description": "What kind of object the parent id names. A folder is a Cloud-only content type that can parent a page; null when the page is top-level.", - "enum": [ - "page", - "folder", - null - ] + "enum": ["page", "folder", null] }, "pageWidth": { "type": "object", "additionalProperties": false, - "required": [ - "value", - "default" - ], + "required": ["value", "default"], "properties": { - "value": { - "type": "string" - }, - "default": { - "type": "boolean" - } + "value": { "type": "string" }, + "default": { "type": "boolean" } } }, "pageWidthOrNull": { - "oneOf": [ - { - "$ref": "#/$defs/pageWidth" - }, - { - "type": "null" - } - ] + "oneOf": [{ "$ref": "#/$defs/pageWidth" }, { "type": "null" }] }, "author": { "type": "object", "additionalProperties": false, - "required": [ - "account_id", - "name" - ], + "required": ["account_id", "name"], "properties": { - "account_id": { - "type": "string" - }, - "name": { - "type": "string" - } + "account_id": { "type": "string" }, + "name": { "type": "string" } } }, "stamp": { "type": "object", "additionalProperties": false, - "required": [ - "at", - "by" - ], + "required": ["at", "by"], "properties": { - "at": { - "type": "string" - }, - "by": { - "oneOf": [ - { - "$ref": "#/$defs/author" - }, - { - "type": "null" - } - ] - } + "at": { "type": "string" }, + "by": { "oneOf": [{ "$ref": "#/$defs/author" }, { "type": "null" }] } } }, "stampOrNull": { - "oneOf": [ - { - "$ref": "#/$defs/stamp" - }, - { - "type": "null" - } - ] + "oneOf": [{ "$ref": "#/$defs/stamp" }, { "type": "null" }] }, "attachment": { "type": "object", "additionalProperties": false, - "required": [ - "action", - "filename" - ], + "required": ["action", "filename"], "properties": { - "action": { - "type": "string" - }, - "filename": { - "type": "string" - } + "action": { "type": "string" }, + "filename": { "type": "string" } } }, "labelAction": { "description": "What happened to one label in an asserted set.", "type": "object", "additionalProperties": false, - "required": [ - "action", - "name" - ], + "required": ["action", "name"], "properties": { "action": { "description": "kept means a surplus label was left on the page because an unmanaged label shares its name: Confluence's removal takes a name with no prefix and deletes the personal label first, so there is no request that removes the intended one.", - "enum": [ - "added", - "removed", - "unchanged", - "kept" - ] + "enum": ["added", "removed", "unchanged", "kept"] }, - "name": { - "type": "string" - } + "name": { "type": "string" } } }, "labelActionsOrNull": { "description": "The full label set asserted this run, or null when the file declares no labels key and the page's labels were left alone.", "oneOf": [ - { - "type": "array", - "items": { - "$ref": "#/$defs/labelAction" - } - }, - { - "type": "null" - } + { "type": "array", "items": { "$ref": "#/$defs/labelAction" } }, + { "type": "null" } ] }, "labelInfo": { "description": "One label a page carries. managed is true for the global prefix, the only one markfluence writes or removes.", "type": "object", "additionalProperties": false, - "required": [ - "name", - "prefix", - "managed" - ], + "required": ["name", "prefix", "managed"], "properties": { - "name": { - "type": "string" - }, - "prefix": { - "type": "string" - }, - "managed": { - "type": "boolean" - } + "name": { "type": "string" }, + "prefix": { "type": "string" }, + "managed": { "type": "boolean" } } }, "labelInfoListOrNull": { "description": "Every label on a page, or null when the fetch failed. An empty array means the page genuinely has none.", "oneOf": [ - { - "type": "array", - "items": { - "$ref": "#/$defs/labelInfo" - } - }, - { - "type": "null" - } + { "type": "array", "items": { "$ref": "#/$defs/labelInfo" } }, + { "type": "null" } ] }, "singleOpFailure": { "description": "An operational failure for a single-target command (info/read): page not found, fetch error, etc.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "page_id", - "error", - "code" - ], + "required": ["ok", "page_id", "error", "code"], "properties": { - "ok": { - "const": false - }, - "page_id": { - "type": "string" - }, - "error": { - "type": "string" - }, - "code": { - "$ref": "#/$defs/code" - } + "ok": { "const": false }, + "page_id": { "type": "string" }, + "error": { "type": "string" }, + "code": { "$ref": "#/$defs/code" } } }, "infoResult": { "type": "object", "additionalProperties": false, "required": [ - "ok", - "page_id", - "title", - "page_status", - "space", - "parent", - "parent_type", - "version", - "page_width", - "labels", - "created", - "updated", - "message", - "url", - "properties" + "ok", "page_id", "title", "page_status", "space", "parent", "parent_type", + "version", "page_width", "labels", "created", "updated", "message", "url", "properties" ], "properties": { - "ok": { - "const": true - }, - "page_id": { - "type": "string" - }, - "title": { - "type": "string" - }, - "page_status": { - "type": "string" - }, - "space": { - "type": "string" - }, - "parent": { - "$ref": "#/$defs/stringOrNull" - }, - "parent_type": { - "$ref": "#/$defs/parentTypeOrNull" - }, + "ok": { "const": true }, + "page_id": { "type": "string" }, + "title": { "type": "string" }, + "page_status": { "type": "string" }, + "space": { "type": "string" }, + "parent": { "$ref": "#/$defs/stringOrNull" }, + "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, "version": { "type": "object", "additionalProperties": false, - "required": [ - "number" - ], - "properties": { - "number": { - "type": "integer" - } - } - }, - "page_width": { - "$ref": "#/$defs/pageWidthOrNull" - }, - "labels": { - "$ref": "#/$defs/labelInfoListOrNull" - }, - "created": { - "$ref": "#/$defs/stampOrNull" - }, - "updated": { - "$ref": "#/$defs/stampOrNull" - }, - "message": { - "type": "string" - }, - "url": { - "type": "string" - }, + "required": ["number"], + "properties": { "number": { "type": "integer" } } + }, + "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, + "labels": { "$ref": "#/$defs/labelInfoListOrNull" }, + "created": { "$ref": "#/$defs/stampOrNull" }, + "updated": { "$ref": "#/$defs/stampOrNull" }, + "message": { "type": "string" }, + "url": { "type": "string" }, "properties": { "oneOf": [ - { - "type": "null" - }, + { "type": "null" }, { "type": "array", "items": { "type": "object", "additionalProperties": false, - "required": [ - "key", - "value" - ], - "properties": { - "key": { - "type": "string" - }, - "value": {} - } + "required": ["key", "value"], + "properties": { "key": { "type": "string" }, "value": {} } } } ] @@ -905,183 +368,65 @@ "type": "object", "additionalProperties": false, "required": [ - "ok", - "page_id", - "title", - "space", - "parent", - "parent_type", - "page_width", - "labels", - "format", - "body" + "ok", "page_id", "title", "space", "parent", "parent_type", "page_width", "labels", + "format", "body" ], "properties": { - "ok": { - "const": true - }, - "page_id": { - "type": "string" - }, - "title": { - "type": "string" - }, - "space": { - "type": "string" - }, - "parent": { - "$ref": "#/$defs/stringOrNull" - }, - "parent_type": { - "$ref": "#/$defs/parentTypeOrNull" - }, - "page_width": { - "$ref": "#/$defs/pageWidthOrNull" - }, + "ok": { "const": true }, + "page_id": { "type": "string" }, + "title": { "type": "string" }, + "space": { "type": "string" }, + "parent": { "$ref": "#/$defs/stringOrNull" }, + "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, + "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, "labels": { "description": "The managed labels that went into the rendered frontmatter, sorted; null when the fetch failed. A different shape from info's labels on purpose: info describes the page, read describes the document it produced.", - "oneOf": [ - { - "type": "array", - "items": { - "type": "string" - } - }, - { - "type": "null" - } - ] - }, - "format": { - "enum": [ - "markdown", - "storage" - ] + "oneOf": [{ "type": "array", "items": { "type": "string" } }, { "type": "null" }] }, - "body": { - "type": "string" - } + "format": { "enum": ["markdown", "storage"] }, + "body": { "type": "string" } } }, + "metadataSourceOrNull": { + "description": "Which location supplied a file's page metadata: its own frontmatter, or a pages: entry in markfluence.yaml. Null when nothing claims the file, which for update goes with status: skipped. Where both locations speak this reports frontmatter, since that is the one that wins every field it can win.", + "oneOf": [{ "enum": ["frontmatter", "manifest"] }, { "type": "null" }] + }, "updateResult": { "type": "object", "additionalProperties": false, "required": [ - "ok", - "status", - "dry_run", - "file", - "page_id", - "title", - "space", - "url", - "version", - "page_width", - "labels", - "attachments", - "warnings", - "broken", - "metadata_source", - "error", - "code" + "ok", "status", "dry_run", "file", "page_id", "title", "space", "url", + "version", "page_width", "labels", "attachments", "warnings", "broken", + "metadata_source", "error", "code" ], "properties": { - "ok": { - "type": "boolean" - }, - "status": { - "enum": [ - "published", - "skipped", - "failed" - ] - }, - "dry_run": { - "type": "boolean" - }, - "file": { - "type": "string" - }, - "page_id": { - "$ref": "#/$defs/stringOrNull" - }, - "title": { - "$ref": "#/$defs/stringOrNull" - }, - "space": { - "$ref": "#/$defs/stringOrNull" - }, - "url": { - "$ref": "#/$defs/stringOrNull" - }, + "ok": { "type": "boolean" }, + "status": { "enum": ["published", "skipped", "failed"] }, + "dry_run": { "type": "boolean" }, + "file": { "type": "string" }, + "page_id": { "$ref": "#/$defs/stringOrNull" }, + "title": { "$ref": "#/$defs/stringOrNull" }, + "space": { "$ref": "#/$defs/stringOrNull" }, + "url": { "$ref": "#/$defs/stringOrNull" }, "version": { "oneOf": [ - { - "type": "null" - }, + { "type": "null" }, { "type": "object", "additionalProperties": false, - "required": [ - "previous", - "new" - ], - "properties": { - "previous": { - "type": "integer" - }, - "new": { - "type": "integer" - } - } + "required": ["previous", "new"], + "properties": { "previous": { "type": "integer" }, "new": { "type": "integer" } } } ] }, - "page_width": { - "$ref": "#/$defs/pageWidthOrNull" - }, - "labels": { - "$ref": "#/$defs/labelActionsOrNull" - }, - "attachments": { - "type": "array", - "items": { - "$ref": "#/$defs/attachment" - } - }, - "warnings": { - "type": "array", - "items": { - "type": "string" - } - }, - "broken": { - "type": "array", - "items": { - "type": "string" - } - }, - "metadata_source": { - "oneOf": [ - { - "type": "string", - "enum": [ - "frontmatter", - "manifest" - ] - }, - { - "type": "null" - } - ], - "description": "Which location supplied this file's page metadata: its own frontmatter, or a pages: entry in markfluence.yaml. Null when nothing claims the file. Where both locations speak, frontmatter is reported, since it wins every field it can win." - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, + "labels": { "$ref": "#/$defs/labelActionsOrNull" }, + "attachments": { "type": "array", "items": { "$ref": "#/$defs/attachment" } }, + "warnings": { "type": "array", "items": { "type": "string" } }, + "broken": { "type": "array", "items": { "type": "string" } }, + "metadata_source": { "$ref": "#/$defs/metadataSourceOrNull" }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "createResult": { @@ -1089,170 +434,53 @@ "type": "object", "additionalProperties": false, "required": [ - "ok", - "status", - "dry_run", - "file", - "page_id", - "title", - "space", - "parent", - "parent_type", - "parent_file", - "url", - "page_width", - "labels", - "persisted", - "attachments", - "warnings", - "broken", - "metadata_source", - "error", - "code" + "ok", "status", "dry_run", "file", "page_id", "title", "space", "parent", "parent_type", "parent_file", "url", + "page_width", "labels", "persisted", "attachments", "warnings", "broken", + "metadata_source", "error", "code" ], "properties": { - "ok": { - "type": "boolean" - }, - "status": { - "enum": [ - "created", - "not_created", - "failed" - ] - }, - "dry_run": { - "type": "boolean" - }, - "file": { - "type": "string" - }, - "page_id": { - "$ref": "#/$defs/stringOrNull" - }, - "title": { - "$ref": "#/$defs/stringOrNull" - }, - "space": { - "$ref": "#/$defs/stringOrNull" - }, - "parent": { - "$ref": "#/$defs/stringOrNull" - }, - "parent_type": { - "$ref": "#/$defs/parentTypeOrNull" - }, - "parent_file": { - "$ref": "#/$defs/stringOrNull" - }, - "url": { - "$ref": "#/$defs/stringOrNull" - }, - "page_width": { - "$ref": "#/$defs/pageWidthOrNull" - }, - "labels": { - "$ref": "#/$defs/labelActionsOrNull" - }, - "persisted": { - "type": "boolean" - }, - "attachments": { - "type": "array", - "items": { - "$ref": "#/$defs/attachment" - } - }, - "warnings": { - "type": "array", - "items": { - "type": "string" - } - }, - "broken": { - "type": "array", - "items": { - "type": "string" - } - }, - "metadata_source": { - "oneOf": [ - { - "type": "string", - "enum": [ - "frontmatter", - "manifest" - ] - }, - { - "type": "null" - } - ], - "description": "Which location supplied this file's page metadata: its own frontmatter, or a pages: entry in markfluence.yaml. Null when nothing claims the file. Where both locations speak, frontmatter is reported, since it wins every field it can win." - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "ok": { "type": "boolean" }, + "status": { "enum": ["created", "not_created", "failed"] }, + "dry_run": { "type": "boolean" }, + "file": { "type": "string" }, + "page_id": { "$ref": "#/$defs/stringOrNull" }, + "title": { "$ref": "#/$defs/stringOrNull" }, + "space": { "$ref": "#/$defs/stringOrNull" }, + "parent": { "$ref": "#/$defs/stringOrNull" }, + "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, + "parent_file": { "$ref": "#/$defs/stringOrNull" }, + "url": { "$ref": "#/$defs/stringOrNull" }, + "page_width": { "$ref": "#/$defs/pageWidthOrNull" }, + "labels": { "$ref": "#/$defs/labelActionsOrNull" }, + "persisted": { "type": "boolean" }, + "attachments": { "type": "array", "items": { "$ref": "#/$defs/attachment" } }, + "warnings": { "type": "array", "items": { "type": "string" } }, + "broken": { "type": "array", "items": { "type": "string" } }, + "metadata_source": { "$ref": "#/$defs/metadataSourceOrNull" }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "fixResult": { "type": "object", "additionalProperties": false, - "required": [ - "ok", - "status", - "file", - "page_id", - "dry_run", - "changes", - "reordered", - "warnings", - "error", - "code" - ], + "required": ["ok", "status", "file", "page_id", "dry_run", "changes", "reordered", "warnings", "error", "code"], "properties": { - "ok": { - "type": "boolean" - }, - "status": { - "enum": [ - "changed", - "consistent", - "failed" - ] - }, - "file": { - "type": "string" - }, - "page_id": { - "$ref": "#/$defs/stringOrNull" - }, - "dry_run": { - "type": "boolean" - }, + "ok": { "type": "boolean" }, + "status": { "enum": ["changed", "consistent", "failed"] }, + "file": { "type": "string" }, + "page_id": { "$ref": "#/$defs/stringOrNull" }, + "dry_run": { "type": "boolean" }, "changes": { "type": "array", "items": { "type": "object", "additionalProperties": false, - "required": [ - "field", - "old", - "new" - ], + "required": ["field", "old", "new"], "properties": { - "field": { - "type": "string" - }, - "old": { - "$ref": "#/$defs/stringOrNull" - }, - "new": { - "type": "string" - } + "field": { "type": "string" }, + "old": { "$ref": "#/$defs/stringOrNull" }, + "new": { "type": "string" } } } }, @@ -1260,189 +488,77 @@ "description": "Whether fix rewrote the frontmatter into canonical field order. Independent of changes: a file whose values all match its live page can still be reordered, and that counts as changed.", "type": "boolean" }, - "warnings": { - "type": "array", - "items": { - "type": "string" - } - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "warnings": { "type": "array", "items": { "type": "string" } }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "checkResult": { "description": "One checked file. broken/warnings are always [] (never null), matching ConfluencePage's own convention. status=broken means broken is non-empty (frontmatter or converter); status=failed means the file never reached a clean answer at all (unreadable, unterminated frontmatter, frontmatter that is not valid YAML or not a flat mapping of single-line scalars, bad page_width, non-numeric page_id) -- code is VALIDATION in that case. debug is non-null only when --show-html was passed and the file reached the converter (never on a failed file).", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "status", - "file", - "broken", - "warnings", - "debug", - "error", - "code" - ], + "required": ["ok", "status", "file", "broken", "warnings", "debug", "error", "code"], "properties": { - "ok": { - "type": "boolean" - }, - "status": { - "enum": [ - "clean", - "warnings", - "broken", - "failed" - ] - }, - "file": { - "type": "string" - }, - "broken": { - "type": "array", - "items": { - "type": "string" - } - }, - "warnings": { - "type": "array", - "items": { - "type": "string" - } - }, + "ok": { "type": "boolean" }, + "status": { "enum": ["clean", "warnings", "broken", "failed"] }, + "file": { "type": "string" }, + "broken": { "type": "array", "items": { "type": "string" } }, + "warnings": { "type": "array", "items": { "type": "string" } }, "debug": { "oneOf": [ - { - "type": "null" - }, + { "type": "null" }, { "type": "object", "additionalProperties": false, - "required": [ - "html", - "attachments" - ], + "required": ["html", "attachments"], "properties": { - "html": { - "type": "string" - }, - "attachments": { - "type": "array", - "items": { - "$ref": "#/$defs/checkAttachment" - } - } + "html": { "type": "string" }, + "attachments": { "type": "array", "items": { "$ref": "#/$defs/checkAttachment" } } } } ] }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "checkAttachment": { "description": "ConfluencePage.Attachments verbatim: a local image check's --show-html surfaces, not an upload outcome (contrast update/create's attachments array, which reports an action).", "type": "object", "additionalProperties": false, - "required": [ - "filename", - "path", - "source" - ], + "required": ["filename", "path", "source"], "properties": { - "filename": { - "type": "string" - }, - "path": { - "type": "string" - }, - "source": { - "type": "string" - } + "filename": { "type": "string" }, + "path": { "type": "string" }, + "source": { "type": "string" } } }, "attachmentUploadResult": { "description": "One uploaded file. status uses the same verbs as the attachments array on update/create. dest_path is always null: upload has no local destination to report, and only exists here so upload and download share one result shape.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "status", - "dry_run", - "filename", - "dest_path", - "error", - "code" - ], + "required": ["ok", "status", "dry_run", "filename", "dest_path", "error", "code"], "properties": { - "ok": { - "type": "boolean" - }, - "status": { - "enum": [ - "created", - "updated", - "skipped", - "failed" - ] - }, - "dry_run": { - "type": "boolean" - }, - "filename": { - "type": "string" - }, - "dest_path": { - "const": null - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "ok": { "type": "boolean" }, + "status": { "enum": ["created", "updated", "skipped", "failed"] }, + "dry_run": { "type": "boolean" }, + "filename": { "type": "string" }, + "dest_path": { "const": null }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "exportSummary": { "description": "export batch summary. skipped counts pages whose file was already on disk; a run that exports nothing new is all skipped and still succeeded. project_file says what happened to the markfluence.yaml a multi-page export needs to be republishable: null for a single-page export, which needs none.", "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "skipped", - "project_file" - ], + "required": ["total", "succeeded", "failed", "skipped", "project_file"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "skipped": { - "type": "integer" - }, - "project_file": { - "enum": [ - "wrote", - "exists", - null - ] - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "skipped": { "type": "integer" }, + "project_file": { "enum": ["wrote", "exists", null] } } }, "exportResult": { @@ -1450,379 +566,157 @@ "type": "object", "additionalProperties": false, "required": [ - "ok", - "page_id", - "title", - "space", - "parent", - "parent_type", - "parent_file", - "dry_run", - "status", - "dest_path", - "attachments", - "warnings", - "error", - "code" + "ok", "page_id", "title", "space", "parent", "parent_type", "parent_file", "dry_run", + "status", "dest_path", "attachments", "warnings", "error", "code" ], "properties": { - "ok": { - "type": "boolean" - }, - "page_id": { - "type": "string" - }, - "title": { - "type": "string" - }, - "space": { - "type": "string" - }, - "parent": { - "$ref": "#/$defs/stringOrNull" - }, - "parent_type": { - "$ref": "#/$defs/parentTypeOrNull" - }, + "ok": { "type": "boolean" }, + "page_id": { "type": "string" }, + "title": { "type": "string" }, + "space": { "type": "string" }, + "parent": { "$ref": "#/$defs/stringOrNull" }, + "parent_type": { "$ref": "#/$defs/parentTypeOrNull" }, "parent_file": { "$ref": "#/$defs/stringOrNull", "description": "The parent: value written into the exported file, when it is a path to the parent's own .md. Null when the parent stayed an id -- the export root, or a page whose parent is a folder." }, - "dry_run": { - "type": "boolean" - }, - "status": { - "enum": [ - "wrote", - "skipped", - "" - ] - }, - "dest_path": { - "$ref": "#/$defs/stringOrNull" - }, + "dry_run": { "type": "boolean" }, + "status": { "enum": ["wrote", "skipped", ""] }, + "dest_path": { "$ref": "#/$defs/stringOrNull" }, "attachments": { "type": "array", "items": { "type": "object", "additionalProperties": false, - "required": [ - "status", - "filename", - "dest_path", - "error", - "code" - ], + "required": ["status", "filename", "dest_path", "error", "code"], "properties": { - "status": { - "enum": [ - "downloaded", - "skipped", - "skipped_unreferenced", - "failed" - ] - }, - "filename": { - "type": "string" - }, - "dest_path": { - "$ref": "#/$defs/stringOrNull" - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "status": { "enum": ["downloaded", "skipped", "skipped_unreferenced", "failed"] }, + "filename": { "type": "string" }, + "dest_path": { "$ref": "#/$defs/stringOrNull" }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } } }, - "warnings": { - "type": "array", - "items": { - "type": "string" - } - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "warnings": { "type": "array", "items": { "type": "string" } }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "attachmentDownloadResult": { "description": "One attachment written to disk. filename is the stored attachment name; dest_path is the local path written, which depends on the recorded source path, --flat, and --dest. dest_path is null only when resolving it is what failed.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "status", - "dry_run", - "filename", - "dest_path", - "error", - "code" - ], + "required": ["ok", "status", "dry_run", "filename", "dest_path", "error", "code"], "properties": { - "ok": { - "type": "boolean" - }, - "status": { - "enum": [ - "downloaded", - "skipped", - "failed" - ] - }, - "dry_run": { - "type": "boolean" - }, - "filename": { - "type": "string" - }, - "dest_path": { - "$ref": "#/$defs/stringOrNull" - }, - "error": { - "$ref": "#/$defs/stringOrNull" - }, - "code": { - "$ref": "#/$defs/codeOrNull" - } + "ok": { "type": "boolean" }, + "status": { "enum": ["downloaded", "skipped", "failed"] }, + "dry_run": { "type": "boolean" }, + "filename": { "type": "string" }, + "dest_path": { "$ref": "#/$defs/stringOrNull" }, + "error": { "$ref": "#/$defs/stringOrNull" }, + "code": { "$ref": "#/$defs/codeOrNull" } } }, "attachmentListResult": { "description": "One attachment on the page. filename is the name Confluence stores; for an attachment markfluence published that is the encoded source path, and source is the markdown image path it came from. managed is false for a hand-uploaded attachment (sha256 and source both null). source may also be null on a managed attachment published before markfluence recorded source paths, in which case sha256 is still set.", "type": "object", "additionalProperties": false, - "required": [ - "ok", - "id", - "filename", - "size", - "media_type", - "version", - "comment", - "managed", - "sha256", - "source" - ], + "required": ["ok", "id", "filename", "size", "media_type", "version", "comment", "managed", "sha256", "source"], "properties": { - "ok": { - "const": true - }, - "id": { - "type": "string" - }, - "filename": { - "type": "string" - }, - "size": { - "type": "integer" - }, - "media_type": { - "type": "string" - }, - "version": { - "type": "integer" - }, - "comment": { - "type": "string" - }, - "managed": { - "type": "boolean" - }, - "sha256": { - "$ref": "#/$defs/stringOrNull" - }, - "source": { - "$ref": "#/$defs/stringOrNull" - } + "ok": { "const": true }, + "id": { "type": "string" }, + "filename": { "type": "string" }, + "size": { "type": "integer" }, + "media_type": { "type": "string" }, + "version": { "type": "integer" }, + "comment": { "type": "string" }, + "managed": { "type": "boolean" }, + "sha256": { "$ref": "#/$defs/stringOrNull" }, + "source": { "$ref": "#/$defs/stringOrNull" } } }, "attachmentSummary": { "description": "attachment-upload/attachment-download batch summary.", "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "skipped" - ], + "required": ["total", "succeeded", "failed", "skipped"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "skipped": { - "type": "integer" - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "skipped": { "type": "integer" } } }, "basicSummary": { "description": "info/read batch summary (total:1), and attachment-list (total: the attachment count).", "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed" - ], + "required": ["total", "succeeded", "failed"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" } } }, "updateSummary": { "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "skipped" - ], + "required": ["total", "succeeded", "failed", "skipped"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "skipped": { - "type": "integer" - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "skipped": { "type": "integer" } } }, "createSummary": { "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "aborted" - ], + "required": ["total", "succeeded", "failed", "aborted"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "aborted": { - "type": "boolean" - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "aborted": { "type": "boolean" } } }, "fixSummary": { "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "changed", - "consistent" - ], + "required": ["total", "succeeded", "failed", "changed", "consistent"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "changed": { - "type": "integer" - }, - "consistent": { - "type": "integer" - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "changed": { "type": "integer" }, + "consistent": { "type": "integer" } } }, "checkSummary": { "description": "clean/warnings count files on the ok:true side (clean has neither broken nor warnings; warnings has only warnings); failed already covers both the broken and failed statuses on the ok:false side, the same granularity fixSummary uses.", "type": "object", "additionalProperties": false, - "required": [ - "total", - "succeeded", - "failed", - "clean", - "warnings" - ], + "required": ["total", "succeeded", "failed", "clean", "warnings"], "properties": { - "total": { - "type": "integer" - }, - "succeeded": { - "type": "integer" - }, - "failed": { - "type": "integer" - }, - "clean": { - "type": "integer" - }, - "warnings": { - "type": "integer" - } + "total": { "type": "integer" }, + "succeeded": { "type": "integer" }, + "failed": { "type": "integer" }, + "clean": { "type": "integer" }, + "warnings": { "type": "integer" } } }, "errorObject": { "description": "The typed error object written to stderr on a fatal/pre-flight failure. command may be empty for a pre-parse (bad-flag) error.", "type": "object", "additionalProperties": false, - "required": [ - "schema_version", - "command", - "error", - "code", - "warnings" - ], + "required": ["schema_version", "command", "error", "code", "warnings"], "properties": { - "schema_version": { - "const": 1 - }, - "command": { - "type": "string" - }, - "error": { - "type": "string" - }, - "code": { - "$ref": "#/$defs/code" - }, - "warnings": { - "type": "array", - "items": { - "type": "string" - }, - "description": "As the envelope's warnings, carried here too because a fatal failure emits no envelope -- and a credential-resolution failure is exactly when a warning about the .env matters most." - } + "schema_version": { "const": 1 }, + "command": { "type": "string" }, + "error": { "type": "string" }, + "code": { "$ref": "#/$defs/code" }, + "warnings": { "type": "array", "items": { "type": "string" }, "description": "As the envelope's warnings, carried here too because a fatal failure emits no envelope -- and a credential-resolution failure is exactly when a warning about the .env matters most." } } } } From 52553539e745c92f31d6523e4d7bf5da10a393bd Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:29:24 -0400 Subject: [PATCH 10/14] fix(create): resolve a manifest parent, and stop persisting over an entry Two merge blockers, both in create, which "read entries" for title/space/page_id/page_width/labels while two other paths still read frontmatter alone. **A .md parent could not resolve in a manifest project, and this plan's own example was one of the failing cases.** Two independent halves. resolveParent joined the parent path onto the *file's* directory, so a root-relative spelling looked for docs/docs/engineering-docs.md; and it read the parent's id with pmf.PageID(), frontmatter only, so a parent whose page_id lived in its own entry reported "not yet published". The second is #139's linkindex trap in a second place, where it fails a create rather than degrading a link. A path in the manifest is root-relative, like every pages: key, and #139's own example spells parent: that way -- but a parent: path everywhere else in markfluence is relative to the file that names it. pagemeta now translates at the boundary, so the manifest stays internally consistent and nothing downstream has to learn where a value came from. The plan claimed the two resolved "exactly the same way", which was self-contradictory and was implemented as neither; it is corrected. **create no longer persists frontmatter into a file a pages: entry claims.** It was writing page_id and a resolved `parent: ` into a file whose entry said something else, so the next update or check failed as a coordinate disagreement. D9's rule is that new metadata goes where that file's metadata already is, and PR 1 has no manifest writer -- so the right behaviour is not to write, and to say so: the result carries a warning naming the new page id and the entry to put it in. D13 called the cost "a page_id copied by hand"; this was silently creating the thing that needed repairing. Also from review: a present-but-blank frontmatter key no longer vanishes when an entry exists (the same file was broken without one and clean with one); metadata_source is set after the Managed() check, so an unmanaged skip reports null as the schema says; and create reports meta.Warnings, which D6 promises wherever metadata is resolved and only update delivered. Nothing in cmd/create exercised pagemeta at all, which is how both blockers hid. Both halves of the parent fix are sabotage-checked. --- cmd/create/create.go | 83 +++++++++++++++-- cmd/create/parent_test.go | 113 ++++++++++++++++++++++- internal/pagemeta/pagemeta.go | 50 +++++++++- internal/pagemeta/pagemeta_test.go | 141 ++++++++++++++++++++++++++++- 4 files changed, 368 insertions(+), 19 deletions(-) diff --git a/cmd/create/create.go b/cmd/create/create.go index 070c9fb..131256b 100644 --- a/cmd/create/create.go +++ b/cmd/create/create.go @@ -163,6 +163,12 @@ type record struct { // metadataSource is which location supplied this file's metadata, carried // from preflight so the published result reports what was actually read. metadataSource string + // inManifest reports whether a pages: entry claims this file, which is + // what suppresses the frontmatter write-back (see createAll). + inManifest bool + // warnings are the metadata-resolution warnings, carried from preflight so + // the published result reports them (D6). + warnings []string // root bounds this file's image/parent reads and is what its attachments' // names and recorded Source are relative to. Discovered from the file's own // directory, cached across the batch by internal/project.Cache. @@ -187,10 +193,6 @@ type failure struct { filename, message string pageID, url string code jsonout.Code - // metadataSource is set when the failure happened after metadata - // resolution, so a --json consumer can still see which location supplied - // the coordinates that turned out to be wrong. - metadataSource string } // pageIDFailure is a phase-1 failure about a file's frontmatter page_id. create @@ -482,7 +484,16 @@ func createAll(ordered []record, c *client.ConfluenceClient, doPersist bool) []* } } - res, pageID, version, ok := reserveOne(r, parentID, c, doPersist) + // A file whose metadata lives in markfluence.yaml must not have + // frontmatter written into it. D9's rule is that new metadata goes + // wherever that file's metadata already is, and PR 1 of #139 has no + // manifest writer yet -- so persisting would put a page_id (and a + // resolved `parent: `) into a file whose entry says something + // else, and every later update and check of that file would then fail + // as a coordinate disagreement. Skipping the write leaves the author + // one copy-paste, which is the cost the plan accepted; writing it + // would have been self-inflicted corruption. + res, pageID, version, ok := reserveOne(r, parentID, c, doPersist && !r.inManifest) if !ok { final[r.absPath] = res continue @@ -535,6 +546,7 @@ func reserveOne( r record, parentID string, c *client.ConfluenceClient, persist bool, ) (res *createResult, pageID string, version int, ok bool) { res = newResult(r) + res.warnings = append(res.warnings, r.warnings...) res.parent = nullableStr(parentID) // parent_type tracks parent: both null for a top-level page, and both null in // a dry-run whose parent is an in-set page that has no id yet. @@ -544,6 +556,9 @@ func reserveOne( if dryRunOpt { res.persisted = persist + if r.inManifest { + res.warnings = append(res.warnings, manifestPersistNotice("")) + } return res, "", 0, true } @@ -554,6 +569,11 @@ func reserveOne( pageID = result.ID res.pageID = pageID res.url = c.PageURL(result, pageID) + if r.inManifest { + // Said out loud, because a silent non-write is how somebody ends up + // with a created page nothing records. + res.warnings = append(res.warnings, manifestPersistNotice(pageID)) + } if persist { parentValue, parentComment := parentField(r.parent, parentID) @@ -721,9 +741,15 @@ func resolveFile( return record{}, err } + // A soft disagreement is reported here as well as by update: D6 promises + // the warning wherever metadata is resolved, and create is the verb most + // likely to be run right after somebody edited one of the two locations. + warnings := meta.Warnings + title := resolveTitle(titleOpt, meta.Fields) if title == "" { - return record{}, errors.New("no title given (pass --title or add a 'title:' frontmatter field)") + return record{}, fmt.Errorf("no title given (pass --title, add a 'title:' frontmatter "+ + "field, or set 'title:' in this file's %s entry)", project.Filename) } width, err := resolveWidth(pageWidthOpt, meta.Fields, root) if err != nil { @@ -809,10 +835,19 @@ func resolveFile( return record{ filename: filename, absPath: abs, mdfile: mf, title: title, spaceKey: spaceKey, spaceID: spaceID, parent: parent, width: width, labels: labelSet, root: root, index: index, - metadataSource: string(meta.MetadataSource()), + metadataSource: string(meta.MetadataSource()), inManifest: meta.InManifest(), + warnings: warnings, }, nil } +// parentKey is the manifest key for a parent file, or "" when it has none -- +// which pagemeta.Resolve reads as "no entry", the right answer for a parent +// outside this root. +func parentKey(root *project.Root, abs string) string { + key, _ := pagemeta.KeyFor(root, abs) + return key +} + // resolveParent resolves a file's parent: reference. A ".md" reference is read // through root's os.Root -- root.FS -- rather than the bare filesystem: a // parent escaping root is a hard error (S2), not an unresolved-and-reported @@ -827,7 +862,9 @@ func resolveParent( fmParent := fm["parent"] fmParentSet := fmParent != "" && fmParent != "null" if parentOpt != "" && fmParentSet { - return parentInfo{}, errors.New("both --parent and a frontmatter 'parent' are set; use only one") + return parentInfo{}, fmt.Errorf( + "both --parent and a declared 'parent' (in the frontmatter or in %s) are set; use only one", + project.Filename) } parentValue := fmParent if parentOpt != "" { @@ -884,9 +921,20 @@ func resolveParent( if err != nil { return parentInfo{}, fmt.Errorf("parent %s: %w", parentValue, err) } - pID := pmf.PageID() + // Through pagemeta, not pmf.PageID(): the *parent's* coordinates may + // live in its own pages: entry rather than in its frontmatter, and + // reading only the file reported a published parent as "not yet + // published" -- which is #139's linkindex trap in a second place, + // where it fails a create rather than degrading a link. + pMeta, err := pagemeta.Resolve(parentKey(root, parentAbs), pmf, root) + if err != nil { + return parentInfo{}, fmt.Errorf("parent %s: %w", parentValue, err) + } + pID := strings.TrimSpace(pMeta.Fields["page_id"]) if pID == "" { - return parentInfo{}, fmt.Errorf("parent not yet published (no page_id): %s", parentValue) + return parentInfo{}, fmt.Errorf( + "parent not yet published (no page_id in the file or in %s): %s", + project.Filename, parentValue) } parentType, err := checkParentInSpace(c, pID, spaceID) if err != nil { @@ -991,6 +1039,21 @@ func parentField(p parentInfo, parentID string) (value, comment string) { // wantPersist resolves the --persist/--no-persist pair; --no-persist wins. func wantPersist(persist, noPersist bool) bool { return persist && !noPersist } +// manifestPersistNotice says why nothing was written back, and what to do +// instead. Until #139's write half lands, a page created for a file whose +// metadata lives in markfluence.yaml has to have its id put in the entry by +// hand -- and that has to be *said*, or the author is left with a created page +// nothing on disk records. +func manifestPersistNotice(pageID string) string { + if pageID == "" { + return fmt.Sprintf("this file's metadata lives in %s, so no frontmatter will be "+ + "written; the new page id has to be added to its pages: entry by hand", + project.Filename) + } + return fmt.Sprintf("this file's metadata lives in %s, so no frontmatter was written; "+ + "add \"page_id: %s\" to its pages: entry", project.Filename, pageID) +} + // overrideNeedsSingleFile reports whether --title was given with anything other // than exactly one FILE. --page-width and the persist toggle are batch-ok. func overrideNeedsSingleFile(cliTitle string, nFiles int) bool { diff --git a/cmd/create/parent_test.go b/cmd/create/parent_test.go index d0e603c..a16d8fd 100644 --- a/cmd/create/parent_test.go +++ b/cmd/create/parent_test.go @@ -7,6 +7,8 @@ import ( "strings" "testing" + "github.com/mozilla/markfluence/internal/frontmatter" + "github.com/mozilla/markfluence/internal/pagemeta" "github.com/mozilla/markfluence/internal/project" ) @@ -165,7 +167,7 @@ func TestResolveParentBothSetIsAnError(t *testing.T) { _, err := resolveParent(filepath.Join(root.Dir, "a.md"), map[string]string{"parent": "other.md"}, nil, nil, "", root) - if err == nil || !strings.Contains(err.Error(), "both --parent and a frontmatter 'parent' are set") { + if err == nil || !strings.Contains(err.Error(), "both --parent and a declared 'parent'") { t.Errorf("err = %v, want the both-set conflict error", err) } } @@ -248,3 +250,112 @@ func TestResolveParentRefusesSymlinkedTarget(t *testing.T) { t.Errorf("err = %v, want a symlink refusal", err) } } + +// --- a parent whose coordinates live in the manifest -------------------------- + +// rootWithManifest builds a root whose markfluence.yaml holds body. +func rootWithManifest(t *testing.T, body string) *project.Root { + t.Helper() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), []byte(body), 0o644); err != nil { + t.Fatal(err) + } + root, err := project.Discover(dir) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = root.FS.Close() }) + return root +} + +// The parent is a pristine file whose page_id lives only in its pages: entry. +// Reading the parent's frontmatter alone reported it "not yet published", +// which is #139's linkindex trap in a second place -- failing a create rather +// than degrading a link. +func TestResolveParentReadsThePageIDFromTheManifest(t *testing.T) { + root := rootWithManifest(t, "pages:\n parent.md:\n title: Parent\n page_id: 100\n") + if err := os.WriteFile(filepath.Join(root.Dir, "parent.md"), []byte("# Parent\n"), 0o644); err != nil { + t.Fatal(err) + } + c := parentServer(t, map[string]string{"100": `{"id":"100","spaceId":"space1"}`}, nil) + + p, err := resolveParent(filepath.Join(root.Dir, "a.md"), + map[string]string{"parent": "parent.md"}, nil, c, "space1", root) + if err != nil { + t.Fatalf("resolveParent: %v", err) + } + if p.kind != parentPublished || p.id != "100" { + t.Errorf("p = %+v, want kind=published id=100", p) + } +} + +// A parent registered with no page_id yet: still an error, and the message has +// to name both places the id could go. +func TestResolveParentManifestEntryWithNoPageID(t *testing.T) { + root := rootWithManifest(t, "pages:\n parent.md:\n title: Parent\n") + if err := os.WriteFile(filepath.Join(root.Dir, "parent.md"), []byte("# Parent\n"), 0o644); err != nil { + t.Fatal(err) + } + _, err := resolveParent(filepath.Join(root.Dir, "a.md"), + map[string]string{"parent": "parent.md"}, nil, nil, "space1", root) + if err == nil { + t.Fatal("want an error for a parent with no page id anywhere") + } + for _, want := range []string{"not yet published", project.Filename} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error = %q, want it to mention %q", err, want) + } + } +} + +// The end-to-end shape from _plans/039's own example: the child's entry names +// its parent root-relative, and the parent's id is in the parent's entry. The +// plan's example failed both halves of this before the fix. +func TestParentFromAnEntryMatchesThePlansExample(t *testing.T) { + root := rootWithManifest(t, `pages: + docs/engineering-docs.md: + title: Engineering Docs + page_id: 100 + docs/deploy-runbook.md: + title: Deploy Runbook + parent: docs/engineering-docs.md + page_id: 101 +`) + if err := os.MkdirAll(filepath.Join(root.Dir, "docs"), 0o755); err != nil { + t.Fatal(err) + } + for _, name := range []string{"engineering-docs.md", "deploy-runbook.md"} { + if err := os.WriteFile(filepath.Join(root.Dir, "docs", name), []byte("# x\n"), 0o644); err != nil { + t.Fatal(err) + } + } + c := parentServer(t, map[string]string{"100": `{"id":"100","spaceId":"space1"}`}, nil) + + // The value resolveParent receives is what pagemeta hands it: the + // file-relative translation of the entry's root-relative path. + meta, err := pagemeta.Resolve("docs/deploy-runbook.md", + mustParse(t, filepath.Join(root.Dir, "docs", "deploy-runbook.md")), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got := meta.Fields["parent"]; got != "engineering-docs.md" { + t.Fatalf("translated parent = %q, want engineering-docs.md", got) + } + p, err := resolveParent(filepath.Join(root.Dir, "docs", "deploy-runbook.md"), + meta.Fields, nil, c, "space1", root) + if err != nil { + t.Fatalf("resolveParent: %v", err) + } + if p.kind != parentPublished || p.id != "100" { + t.Errorf("p = %+v, want kind=published id=100", p) + } +} + +func mustParse(t *testing.T, path string) *frontmatter.MarkdownFile { + t.Helper() + mf, err := frontmatter.ParseFile(path) + if err != nil { + t.Fatal(err) + } + return mf +} diff --git a/internal/pagemeta/pagemeta.go b/internal/pagemeta/pagemeta.go index 45f4679..95da748 100644 --- a/internal/pagemeta/pagemeta.go +++ b/internal/pagemeta/pagemeta.go @@ -23,6 +23,7 @@ package pagemeta import ( "fmt" + "path" "path/filepath" "sort" "strings" @@ -147,6 +148,15 @@ func Resolve(key string, mf *frontmatter.MarkdownFile, root *project.Root) (Reso for k, v := range entry.Fields { r.Fields[k] = v } + // One translation at the boundary: a path inside the manifest is + // root-relative, like every pages: key, but a `parent:` path everywhere + // else in markfluence is relative to the file that names it (create's + // resolveParent, and what export writes). Converting here keeps both true + // -- the manifest stays internally consistent, and nothing downstream has + // to learn where a value came from. + if rel, ok := fileRelativeParent(key, r.Fields["parent"]); ok { + r.Fields["parent"] = rel + } for k, v := range entry.Lists { r.Lists[k] = v } @@ -167,6 +177,16 @@ func Resolve(key string, mf *frontmatter.MarkdownFile, root *project.Root) (Reso r.Fields[k] = file case inFile: r.Fields[k] = file + case !inEntry: + // Present in the file and blank, with nothing in the entry to fall + // back to. The *blank* has to survive rather than vanish: a + // present-but-empty `title:` is a defect update and check report, + // and dropping the key here made the same file clean with an entry + // and broken without one. A blank value is not a disagreement + // (nonBlank), but it is still something the author wrote. + if v, present := mf.Frontmatter[k]; present { + r.Fields[k] = v + } } } for _, k := range sortedListKeys(mf.Lists, entry.Lists) { @@ -198,6 +218,24 @@ func Resolve(key string, mf *frontmatter.MarkdownFile, root *project.Root) (Reso return r, nil } +// fileRelativeParent converts a manifest entry's root-relative .md parent into +// the file-relative form used everywhere else, reporting whether it did. +// +// Pure path arithmetic -- nothing is stat'd -- so it is safe to do for a file +// that may not exist, and it cannot depend on the checkout's layout (L2). +// Anything that is not a .md path is left alone: an id and a null mean the same +// thing in both locations. +func fileRelativeParent(key, parent string) (string, bool) { + if key == "" || parent == "" || !strings.HasSuffix(parent, ".md") { + return "", false + } + rel, err := filepath.Rel(path.Dir(key), parent) + if err != nil { + return "", false + } + return filepath.ToSlash(rel), true +} + // entryFor looks up a file's manifest entry. func entryFor(key string, root *project.Root) (project.Entry, bool) { if root == nil || root.Config.Pages == nil { @@ -218,10 +256,14 @@ func HasManifest(root *project.Root) bool { // declaresPageField reports whether either map holds a non-blank value under a // field markfluence understands. // -// Restricted to known fields on purpose: a docs tree carrying Jekyll or Hugo -// frontmatter (layout:, date:, draft:) has said nothing about Confluence, and -// counting any key at all would report every such file as having contributed -// metadata it never had. +// Restricted to known fields on purpose, and the reason is inside markfluence +// rather than hypothetical: frontmatter deliberately preserves keys markfluence +// knows nothing about (a test there pins `reviewers: [ana, bo]` surviving a +// write, since a labels special case would break #21/#100). So a file whose +// only frontmatter is unknown keys is a shape markfluence explicitly supports, +// and counting any key at all read it as having contributed metadata it never +// had. The same holds for frontmatter another tool wrote, which is the premise +// of #139 -- "a README or a docs tree that has other readers". func declaresPageField(fields map[string]string, lists map[string][]string) bool { for k, v := range fields { if project.IsPageField(k) && strings.TrimSpace(v) != "" { diff --git a/internal/pagemeta/pagemeta_test.go b/internal/pagemeta/pagemeta_test.go index 4102dcf..c81e598 100644 --- a/internal/pagemeta/pagemeta_test.go +++ b/internal/pagemeta/pagemeta_test.go @@ -195,12 +195,12 @@ func TestResolveUnmanaged(t *testing.T) { } } -// A docs tree carrying Jekyll or Hugo frontmatter has said nothing about -// Confluence. Counting any key at all would report every such file as claimed -// and fail a whole tree. +// A file whose frontmatter markfluence knows nothing about has said nothing +// about Confluence. markfluence deliberately preserves such keys, so counting +// any key at all would read every such file as claimed and fail a whole tree. func TestResolveForeignFrontmatterIsUnmanaged(t *testing.T) { root := rootWith(t, "space: ENG\n") - r, err := Resolve("a.md", parse(t, "layout: post\ndate: 2026-01-01\ndraft: false\n"), root) + r, err := Resolve("a.md", parse(t, "reviewers: [ana, bo]\nowner: sre\n"), root) if err != nil { t.Fatalf("Resolve: %v", err) } @@ -303,3 +303,136 @@ func TestKeyForOutsideTheRoot(t *testing.T) { t.Error("want no key for a nil root") } } + +// A path in the manifest is root-relative like every pages: key, but a parent: +// path everywhere else in markfluence is relative to the file that names it. +// Resolve translates at the boundary so both stay true. +func TestResolveTranslatesAnEntrysParentToFileRelative(t *testing.T) { + root := rootWith(t, `pages: + docs/deploy-runbook.md: + page_id: 2 + parent: docs/engineering-docs.md +`) + mf := parse(t, "") + r, err := Resolve("docs/deploy-runbook.md", mf, root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got := r.Fields["parent"]; got != "engineering-docs.md" { + t.Errorf("parent = %q, want engineering-docs.md (file-relative)", got) + } +} + +func TestResolveParentTranslationAcrossDirectories(t *testing.T) { + tests := map[string]struct{ key, entry, want string }{ + "same directory": {"docs/a.md", "docs/b.md", "b.md"}, + "one level up": {"docs/team/a.md", "docs/b.md", "../b.md"}, + "one level down": {"a.md", "docs/b.md", "docs/b.md"}, + "root to root": {"a.md", "b.md", "b.md"}, + "two levels up": {"a/b/c.md", "d.md", "../../d.md"}, + } + for name, tc := range tests { + t.Run(name, func(t *testing.T) { + root := rootWith(t, "pages:\n "+tc.key+":\n page_id: 1\n parent: "+tc.entry+"\n") + r, err := Resolve(tc.key, parse(t, ""), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got := r.Fields["parent"]; got != tc.want { + t.Errorf("parent = %q, want %q", got, tc.want) + } + }) + } +} + +// A parent that is not a path means the same thing in both locations, so it is +// left exactly as written. +func TestResolveLeavesANonPathParentAlone(t *testing.T) { + for _, value := range []string{"12345", "null"} { + root := rootWith(t, "pages:\n docs/a.md:\n page_id: 1\n parent: "+value+"\n") + r, err := Resolve("docs/a.md", parse(t, ""), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := value + if value == "null" { + want = "" // every null spelling reads as empty + } + if got := r.Fields["parent"]; got != want { + t.Errorf("parent = %q, want %q", got, want) + } + } +} + +// A frontmatter parent is already file-relative and must not be translated. +func TestResolveDoesNotTranslateAFrontmatterParent(t *testing.T) { + root := rootWith(t, "space: ENG\n") + r, err := Resolve("docs/team/a.md", parse(t, "parent: ../index.md\npage_id: 1\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got := r.Fields["parent"]; got != "../index.md" { + t.Errorf("parent = %q, want it unchanged", got) + } +} + +// A present-but-blank frontmatter key has to survive the merge. Dropping it +// made the same file clean with an entry and broken without one, since a +// present-but-empty title: is a defect update and check report. +func TestResolveKeepsAPresentButBlankFrontmatterKey(t *testing.T) { + root := rootWith(t, "pages:\n a.md:\n page_id: 1\n") + r, err := Resolve("a.md", parse(t, "title:\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + v, present := r.Fields["title"] + if !present { + t.Fatal("title vanished; a blank the author wrote is still something they wrote") + } + if v != "" { + t.Errorf("title = %q, want empty", v) + } +} + +// The same input with no entry at all must behave identically, which is the +// asymmetry the bug created. +func TestResolveBlankKeySurvivesWithAndWithoutAnEntry(t *testing.T) { + withEntry := rootWith(t, "pages:\n a.md:\n page_id: 1\n") + without := rootWith(t, "space: ENG\n") + for name, root := range map[string]*project.Root{"with entry": withEntry, "no entry": without} { + t.Run(name, func(t *testing.T) { + r, err := Resolve("a.md", parse(t, "title:\npage_id: 1\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if _, present := r.Fields["title"]; !present { + t.Error("title not present; the two paths must agree") + } + }) + } +} + +// Foreign frontmatter plus a key markfluence shares with other tools: the +// realistic shape, and the one where the two fixes have to work together. +// markfluence deliberately preserves keys it knows nothing about (a test in +// internal/frontmatter pins `reviewers: [ana, bo]` surviving a write), so a +// file whose only frontmatter is unknown keys must not read as claimed -- and +// `title:` alone must not either, since it does not say which page this is. +func TestResolveUnknownKeysWithAKnownOneDoNotClaimAFile(t *testing.T) { + root := rootWith(t, "space: ENG\n") + r, err := Resolve("a.md", parse(t, "reviewers: [ana, bo]\ntitle: Shared Key\n"), root) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if r.Managed() { + t.Error("managed; neither an unknown key nor a bare title identifies a page") + } + // It did contribute a known field, which is a different question. + if r.Source != FromFrontmatter { + t.Errorf("source = %q, want frontmatter", r.Source) + } + // And the unknown key is still carried, since markfluence preserves it. + if l := r.Lists["reviewers"]; len(l) != 2 { + t.Errorf("reviewers = %#v, want it preserved", l) + } +} From d52b37d97d164595f11f6902c402e9598224b94c Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:29:41 -0400 Subject: [PATCH 11/14] fix(check): lint the resolved width, and report soft disagreements check's project-width lint read mf.Frontmatter rather than the resolved metadata, so a file whose *entry* declared a width was failed over the project's invalid one -- the exact false positive the surrounding comment forbids, while update published the same file fine. check also dropped meta.Warnings, so a soft disagreement between the two locations was reported only by update, against D6's promise that it is reported wherever metadata is resolved. Two more from review, both in the same class of "the code and its own comment disagreed": NormalizePageKey judged absoluteness before normalizing separators, so `\foo.md`, `C:\foo.md` and a UNC path were accepted as keys KeyFor can never produce -- a silently unreachable entry, worse than an error because nothing says so. And failure.metadataSource was never assigned while its comment claimed it was populated, so it is gone and abortedResult's null is explained. create's "no title given" and the --parent conflict message now name the entry as a place a value can live. Tests: the width false positive, the check-side warning, the Windows absolute forms, that a kind is never zero (IsPageField reads 0 as absent), that a valueless `pages:` is refused, the enum values of metadata_source against the schema, the unmanaged skip's own document, and that the two skips render differently -- which the PR claimed and nothing checked. --- cmd/check/check.go | 11 +++++- cmd/check/check_test.go | 40 +++++++++++++++++++ cmd/create/json.go | 8 ++-- cmd/update/json_test.go | 34 +++++++++++++++++ cmd/update/update.go | 8 +++- cmd/update/update_test.go | 70 +++++++++++++++++++++++++++++++++- internal/project/pages.go | 28 +++++++++++--- internal/project/pages_test.go | 45 ++++++++++++++++++++++ 8 files changed, 231 insertions(+), 13 deletions(-) diff --git a/cmd/check/check.go b/cmd/check/check.go index 444b5c7..f069d9d 100644 --- a/cmd/check/check.go +++ b/cmd/check/check.go @@ -166,6 +166,10 @@ func processFile(filename string, roots *project.Cache, indexes *linkindex.Cache return r.fail(err, jsonout.CodeValidation) } + // A soft disagreement between the two locations, which D6 promises is + // reported wherever metadata is resolved -- not only by update. + r.warnings = append(r.warnings, meta.Warnings...) + if _, err := pagewidth.Declared(meta.Fields); err != nil { return r.fail(err, jsonout.CodeValidation) } @@ -217,7 +221,12 @@ func processFile(filename string, roots *project.Cache, indexes *linkindex.Cache // reported per file rather than once for the run, which is what keeps every // diagnostic scoped to the files actually named -- a file under a different // project hears nothing about this one. - if root.Config.PageWidth != "" && strings.TrimSpace(mf.Frontmatter["page_width"]) == "" { + // meta.Fields, not mf.Frontmatter: a width declared in this file's pages: + // entry wins over the project default exactly as one in its frontmatter + // does, so reading the file alone reported the project's bad value against + // a file that publishes perfectly well -- the false positive the paragraph + // above says check must not produce. + if root.Config.PageWidth != "" && strings.TrimSpace(meta.Fields["page_width"]) == "" { if _, err := pagewidth.Declared( map[string]string{"page_width": root.Config.PageWidth}); err != nil { localBroken = append(localBroken, fmt.Sprintf("%s: %s", root.File, err)) diff --git a/cmd/check/check_test.go b/cmd/check/check_test.go index 9b31b99..61152ea 100644 --- a/cmd/check/check_test.go +++ b/cmd/check/check_test.go @@ -763,3 +763,43 @@ func TestRunPristineManifestFileIsClean(t *testing.T) { t.Errorf("output = %q, want it reported clean", out) } } + +// A width declared in this file's entry wins over the project default exactly +// as one in its frontmatter does, so the project's bad value must not fail it. +// Reading mf.Frontmatter instead of the resolved metadata produced the false +// positive check's own rule forbids: update publishes this file fine. +func TestRunInvalidProjectWidthNotReportedWhenAnEntryOverridesIt(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), + "page_width: huge\npages:\n main.md:\n page_id: 1\n page_width: wide\n") + write(t, filepath.Join(dir, "main.md"), "# Main\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if err != nil { + t.Fatalf("run = %v, want success: the entry declares its own width\n%s", err, out) + } + if strings.Contains(out, "invalid page_width") { + t.Errorf("output = %q, want no complaint", out) + } +} + +// D6 promises the soft-disagreement warning wherever metadata is resolved, not +// only from update. +func TestRunReportsASoftDisagreement(t *testing.T) { + dir := t.TempDir() + write(t, filepath.Join(dir, "markfluence.yaml"), + "pages:\n main.md:\n title: Manifest\n page_id: 1\n") + write(t, filepath.Join(dir, "main.md"), "---\ntitle: File\n---\n# Main\n") + + out, err := captureOutput(t, func() error { + return run(testCmd(t, ""), []string{filepath.Join(dir, "main.md")}) + }) + if err != nil { + t.Fatalf("run = %v, want success: a soft disagreement is a warning\n%s", err, out) + } + if !strings.Contains(out, "overrides") { + t.Errorf("output = %q, want the title-override warning", out) + } +} diff --git a/cmd/create/json.go b/cmd/create/json.go index 744d2ee..f28065c 100644 --- a/cmd/create/json.go +++ b/cmd/create/json.go @@ -264,9 +264,11 @@ func abortedResult(file, status string, f failure) jsonCreateResult { Attachments: []jsonout.Attachment{}, Warnings: []string{}, Broken: []string{}, - // A file rejected in preflight may not have reached metadata - // resolution at all, so this is null rather than guessed at. - MetadataSource: nullableStr(f.metadataSource), + // Always null: a file rejected in preflight may not have reached + // metadata resolution at all, and one that did has no result of its + // own to carry it -- abortedResult is built from the failure, not from + // the record. + MetadataSource: nil, } if f.message != "" { msg := f.message diff --git a/cmd/update/json_test.go b/cmd/update/json_test.go index 400dce3..d278a60 100644 --- a/cmd/update/json_test.go +++ b/cmd/update/json_test.go @@ -154,3 +154,37 @@ func TestSummarize(t *testing.T) { type errTest string func (e errTest) Error() string { return string(e) } + +// The enum values metadata_source can actually hold were never validated +// against the schema: the conformance fixtures left the field empty, so only +// null was ever checked. +func TestSchemaConformanceMetadataSourceValues(t *testing.T) { + for _, src := range []string{"frontmatter", "manifest"} { + t.Run(src, func(t *testing.T) { + results := []*updateResult{{ + file: "a.md", ok: true, status: statusPublished, + pageID: "1", title: "A", space: "ENG", url: "https://x/1", + versionPrev: 1, versionNew: 2, metadataSource: src, + }} + items := []any{results[0].jsonResult()} + env := jsonout.NewEnvelope("update", items, summarize(results)) + var buf bytes.Buffer + if err := jsonout.Emit(&buf, env); err != nil { + t.Fatalf("Emit: %v", err) + } + schematest.ValidateEnvelope(t, buf.Bytes()) + }) + } +} + +// And the unmanaged skip's own document, which no fixture covered. +func TestSchemaConformanceUnmanagedSkip(t *testing.T) { + results := []*updateResult{{file: "a.md", ok: true, status: statusSkipped, unmanaged: true}} + items := []any{results[0].jsonResult()} + env := jsonout.NewEnvelope("update", items, summarize(results)) + var buf bytes.Buffer + if err := jsonout.Emit(&buf, env); err != nil { + t.Fatalf("Emit: %v", err) + } + schematest.ValidateEnvelope(t, buf.Bytes()) +} diff --git a/cmd/update/update.go b/cmd/update/update.go index ce4d17b..5924950 100644 --- a/cmd/update/update.go +++ b/cmd/update/update.go @@ -198,7 +198,6 @@ func processFile( // means. Only this file fails; the rest of the batch proceeds. return r.fail(err, jsonout.CodeValidation) } - r.metadataSource = string(meta.MetadataSource()) r.warnings = append(r.warnings, meta.Warnings...) // Nothing anywhere claims this file, so there is nothing to publish and @@ -211,8 +210,15 @@ func processFile( r.ok = true r.status = statusSkipped r.unmanaged = true + // metadata_source stays null, deliberately, even for a file whose + // frontmatter holds a known field: the question it answers is "which + // location supplied the metadata this page was published from", and + // nothing was published. Setting it before this check reported + // "frontmatter" on a skip, contradicting the schema and leaving + // --json unable to tell an unmanaged skip from an unchanged one. return r } + r.metadataSource = string(meta.MetadataSource()) title, titlePresent, pageID := resolveTitlePageID(meta.Fields) // Before the request, like the page-id check below: an empty title is a diff --git a/cmd/update/update_test.go b/cmd/update/update_test.go index 66a6d50..fdae0ef 100644 --- a/cmd/update/update_test.go +++ b/cmd/update/update_test.go @@ -921,14 +921,15 @@ func TestProcessFileSkipsAnUnmanagedFile(t *testing.T) { } } -// A docs tree carrying Jekyll frontmatter has said nothing about Confluence. +// Frontmatter markfluence knows nothing about -- a shape it explicitly +// preserves -- says nothing about Confluence. func TestProcessFileSkipsForeignFrontmatter(t *testing.T) { c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) w.WriteHeader(http.StatusInternalServerError) }) path := writeManifestProject(t, "space: ENG\n", - "---\nlayout: post\ndate: 2026-01-01\n---\n# Post\n") + "---\nreviewers: [ana, bo]\nowner: sre\n---\n# Post\n") r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) if !r.ok || r.status != statusSkipped || !r.unmanaged { @@ -1053,3 +1054,68 @@ func TestRunBatchSkipsUnmanagedAndPublishesTheRest(t *testing.T) { t.Errorf("draft.md status = %q, want skipped", got["draft.md"]) } } + +// metadata_source is null on an unmanaged skip even for a file whose +// frontmatter holds a known field: the question it answers is which location +// supplied the metadata a page was published from, and nothing was published. +// Setting it before the Managed() check reported "frontmatter" here, which +// contradicted the schema and left --json unable to tell the two skips apart. +func TestProcessFileUnmanagedSkipReportsNoSource(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + // title and space are fields markfluence knows, so this file *contributes* + // metadata -- but names no page, so nothing claims it. + path := writeManifestProject(t, "space: ENG\n", "---\ntitle: Draft\nspace: ENG\n---\n# Draft\n") + + r := processFile(path, c, project.NewCache(""), linkindex.NewCache(), pagedoc.NewUserCache()) + if !r.ok || r.status != statusSkipped || !r.unmanaged { + t.Fatalf("result = %+v, want an unmanaged skip", r) + } + if r.metadataSource != "" { + t.Errorf("metadata_source = %q, want empty", r.metadataSource) + } + if got := r.jsonResult().MetadataSource; got != nil { + t.Errorf("--json metadata_source = %q, want null", *got) + } +} + +// The two skips have to be distinguishable, which is the whole reason the +// unmanaged flag exists. +func TestRenderHumanDistinguishesTheTwoSkips(t *testing.T) { + unmanaged := &updateResult{file: "a.md", ok: true, status: statusSkipped, unmanaged: true} + unchanged := &updateResult{file: "a.md", ok: true, status: statusSkipped} + + got := captureStdout(t, unmanaged.renderHuman) + if !strings.Contains(got, "not published by markfluence") { + t.Errorf("unmanaged skip rendered %q", got) + } + got = captureStdout(t, unchanged.renderHuman) + if !strings.Contains(got, "no changes") { + t.Errorf("unchanged skip rendered %q", got) + } +} + +// captureStdout runs fn with os.Stdout redirected, returning what it printed. +func captureStdout(t *testing.T, fn func()) string { + t.Helper() + rd, w, err := os.Pipe() + if err != nil { + t.Fatal(err) + } + saved := os.Stdout + os.Stdout = w + done := make(chan string, 1) + go func() { + var b strings.Builder + _, _ = io.Copy(&b, rd) + done <- b.String() + }() + fn() + os.Stdout = saved + _ = w.Close() + out := <-done + _ = rd.Close() + return out +} diff --git a/internal/project/pages.go b/internal/project/pages.go index f1f8786..63b042b 100644 --- a/internal/project/pages.go +++ b/internal/project/pages.go @@ -127,13 +127,24 @@ func readEntry(named string, fields []frontmatter.Item) (Entry, error) { return e, nil } +// volumeRelative reports whether p carries a Windows drive letter ("C:/x"), +// which is absolute in meaning while not starting with a separator. +func volumeRelative(p string) bool { + if len(p) < 2 || p[1] != ':' { + return false + } + c := p[0] + return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') +} + // IsPageField reports whether name is a field markfluence understands on a // page, in frontmatter or in an entry. // -// Exported because telling a markfluence key from a foreign one is not a -// judgment a caller should make for itself: a docs tree carrying Jekyll's -// layout:/date: frontmatter has said nothing about Confluence, and a caller -// that counted any key at all would read every such file as claimed. +// Exported because telling a markfluence key from one it merely preserves is +// not a judgment a caller should make for itself. internal/frontmatter keeps +// keys markfluence knows nothing about on purpose (a test there pins +// `reviewers: [ana, bo]` surviving a write), so a caller that counted any key +// at all would read a file carrying only those as claimed. func IsPageField(name string) bool { return entryFields[name] != 0 } // knownEntryFields lists the recognized field names, sorted so a message is @@ -171,10 +182,15 @@ func NormalizePageKey(p string) (string, error) { if p == "" { return "", fmt.Errorf("a page key cannot be empty") } - if path.IsAbs(p) || strings.HasPrefix(p, "/") { + // Separators first, then absoluteness: judging the raw string let + // "\\foo.md", "C:\\foo.md" and "\\\\server\\share\\a.md" through as keys, which + // normalize to absolute paths KeyFor can never produce -- a silently + // unreachable entry, which is worse than an error because nothing says so. + slashed := strings.ReplaceAll(p, "\\", "/") + if path.IsAbs(slashed) || strings.HasPrefix(slashed, "/") || volumeRelative(slashed) { return "", fmt.Errorf("page %q must be relative to the project root, not absolute", p) } - clean := path.Clean(strings.ReplaceAll(p, "\\", "/")) + clean := path.Clean(slashed) if clean == ".." || strings.HasPrefix(clean, "../") { return "", fmt.Errorf("page %q is outside the project root", p) } diff --git a/internal/project/pages_test.go b/internal/project/pages_test.go index 00f5ac0..62e36a7 100644 --- a/internal/project/pages_test.go +++ b/internal/project/pages_test.go @@ -240,3 +240,48 @@ func TestStructuralPageErrorsAreNotPerFile(t *testing.T) { func takesFrontmatter(map[string]string) {} func takesLists(map[string][]string) {} + +// Separators are normalized before absoluteness is judged. Judging the raw +// string let these through as keys that normalize to absolute paths KeyFor can +// never produce -- a silently unreachable entry, which is worse than an error +// because nothing says so. +func TestNormalizePageKeyRefusesWindowsAbsoluteForms(t *testing.T) { + for _, in := range []string{`\foo.md`, `C:\foo.md`, `c:/foo.md`, `\\server\share\a.md`} { + t.Run(in, func(t *testing.T) { + if got, err := NormalizePageKey(in); err == nil { + t.Errorf("NormalizePageKey(%q) = %q, want an error", in, got) + } + }) + } +} + +// A `pages:` key with nothing after it is a mapping-shaped setting given no +// mapping, which is refused rather than read as an empty block -- unlike a +// scalar setting, where blank means unset. An author who typed the key and +// stopped gets told, instead of silently having a project with no entries. +func TestPagesWithNoValueIsRefused(t *testing.T) { + _, err := Discover(write(t, "pages:\n")) + if err == nil { + t.Fatal("Discover accepted a valueless pages:, want an error") + } + if !strings.Contains(err.Error(), `setting "pages" must be a mapping`) { + t.Errorf("error = %q, want the must-be-a-mapping message", err) + } +} + +// IsPageField is entryFields[name] != 0, which is only sound because kind +// starts at iota + 1. A zero-valued kind would make every unknown name look +// known, so this pins the invariant rather than the lookup. +func TestKindZeroValueIsNotAValidKind(t *testing.T) { + if kindScalar == 0 || kindList == 0 || kindMapping == 0 { + t.Fatal("a kind is zero; IsPageField and the settings lookups both read 0 as absent") + } + if IsPageField("titel") || IsPageField("") { + t.Error("an unknown name reported as a page field") + } + for _, name := range []string{"title", "space", "parent", "page_id", "page_width", "labels"} { + if !IsPageField(name) { + t.Errorf("%s is not reported as a page field", name) + } + } +} From c7034b4a8cecbe9df9bc5e4a4b7ab4398b2dca8c Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:29:41 -0400 Subject: [PATCH 12/14] docs: record the review, and drop a framing nobody here can check _plans/039 gains the nine findings and the corrected parent claim: an entry's .md parent is root-relative, translated at the boundary, which is *not* "the same way as frontmatter" however much the equivalence elsewhere holds. Separately: the comments justifying IsPageField cited Jekyll and Hugo frontmatter as if that were a known scenario here. It is not, and the grounded version is inside markfluence -- internal/frontmatter deliberately preserves keys it does not understand, pinned by a test on `reviewers: [ana, bo]`, so a file carrying only such keys is a shape markfluence explicitly supports and must not read as claimed. No third-party tool required, and verifiable in this repo. --- CLAUDE.md | 2 +- _plans/039_project-file-pages.md | 77 +++++++++++++++++++++++++++++++- 2 files changed, 76 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index af6b5bb..6b89913 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,7 +71,7 @@ Module `github.com/mozilla/markfluence` (`go 1.25`). `main.go` is a shim to `cmd - `cmd/schema/` — `schema`: print the embedded `--json` schema to stdout verbatim (no args, no credentials, no Confluence call). `--json` is deliberately a no-op — the output is already the schema document, not an envelope — which is also why `schema` is absent from the schema's own `command` enum. - `schema/` — the published JSON Schema (`json-output/v1.json`) *and* the `schema` Go package that embeds it (`V1`). The Go file lives beside the schema because `go:embed` cannot reach outside its own directory, and the schema stays at a top-level path a non-Go consumer can browse, mirroring its own `$id`. `internal/schematest` validates against the embed rather than reading the file, which is what makes "what ships" and "what the tests checked" the same bytes — do not reintroduce a disk read or a second copy. The version number is **not** restated here: `jsonout.SchemaVersion` and the document's own `schema_version` const are the two copies, tied together by a test in `cmd/schema`. The envelope's and the error object's top-level **`warnings`** are the one field no command fills: `jsonout.NewEnvelope`/`EmitError` drain a package-level collector (`AddWarning`), because the only thing in it is raised during credential resolution — below any command, before either document exists. - `internal/project` — the documentation root: `Discover` (walk up from a directory looking for `markfluence.yaml`), `FromPath` (`--root`), `Resolve`, and `Cache`, which consults itself at every level of the walk so a batch spanning a subtree pays for the walk — and `os.OpenRoot` — once rather than per directory (the quadratic cost `_plans/025` measured). A `Root` carries `Dir`, `File`, `Config` and an `os.Root` that refuses an escape even through a symlink partway down. `Discover` is called from **two starting points for two reasons** — once per invocation from the working directory to locate `.env`, and once per markdown file from its own directory to bound its reads and name its attachments — which is why it returns a type rather than a string; the two diverge legitimately, so a multi-root batch is allowed and nothing refuses it ([docs/root-model.md](docs/root-model.md)). `config.go` reads the project file's **settings** (#100): `space` and `page_width`, resolving **flag > frontmatter > project file**, which is *not* the credentials chain and must never be conflated with it. Three things about it are load-bearing. It is read through `frontmatter.Dialect.ReadMapping` rather than a second parser, since every rule there was found by probing goccy and a second copy would be a second set of the same bugs. An **unknown top-level key is fatal**, and that is the point rather than a cost — a silently ignored `spce: ENG` is wrong for every file at once, and a file written for a newer markfluence holds keys this binary would ignore, so there is no schema version and this must not be loosened; an empty or comment-only file stays valid, being what ships and what `export` plants. And loading happens in `open()`, the single place a `Root` is built from a marker hit, so `Discover`/`Cache`/`FromPath` cannot disagree that a file which cannot be understood **is not a valid marker**: the walk does not continue upward and does not fall back to the starting directory, because the root decides every attachment name and guessing at it is worse than stopping. `ConfigError`/`IsConfigError`/`RootError` exist so a caller reports that as a local defect (`VALIDATION`) rather than under `resolving the documentation root` as I/O. `pages.go` holds the **`pages:`** key (#139): an `Entry` is `{Fields, Lists}`, the same two maps `frontmatter.MarkdownFile` carries, which is the design rather than a convenience — `pagewidth` and `labels` both reach `client`, which holds a `*Cache`, so a typed validated entry would need a broken cycle or a second copy of every field's rules, and with two maps `labels.Declared(e.Lists, e.Fields)` works unchanged. `entryFields` is the manifest's schema and the only place it is written down; it mirrors frontmatter's fields deliberately, since adding one there and not here would make a field expressible in a file and not in an entry. Load checks **structure** — a mapping of mappings, legal paths, known field *names*, right shapes — and never a field's **value**, because #139 requires a semantically bad entry to be reported only when its file is one of the arguments, and this package has no idea which files the command was given. An unknown field *name* is the exception and is fatal at load, being the same typo class as an unknown setting. `NormalizePageKey` is lexical (L2 forbids a key whose meaning depends on the checkout's layout), and an escaping key or two keys normalizing to one are load-time errors naming both spellings. `Config.Pages` is **nil when there is no `pages:` key and empty-non-nil for `pages: {}`**, which is how a command tells "has not chosen the manifest" from "has, and has registered nothing". What it validates is **structure only**: `internal/pagewidth` cannot be imported here (`pagewidth` → `client` → `project`), so a width's vocabulary is checked by `pagewidth.Declared` where it already runs and by `check`'s offline lint. `Config` deliberately holds **no `url` or token**, and the reason is sharper than "those are credentials": basic auth goes to whatever host the resolved URL names, so a committed, walked-up file naming one would decide where `CONFLUENCE_TOKEN` is sent — a worse version of the `.env` hole #136 records. -- `internal/pagemeta` — `Resolve`, the one merge of a file's page metadata from the two places it may live: its own frontmatter and a `pages:` entry in the project file (#139). A package because `update`, `create`, `check` **and `internal/linkindex`** all need it, and a per-command copy is how two commands come to publish one file to two different pages. It imports `frontmatter` and `project` and nothing else, which is also why it validates no *value*: `pagewidth` and `labels` are unreachable from here, and the commands that need them already call them on the maps it returns. **Frontmatter and an entry are two spellings of one level, not two levels of a precedence chain** — when both speak the rule is not "the higher wins" but a grading: `page_id`/`space`/`parent` are coordinates and a disagreement fails the file (for `create`, the batch, since it preflights everything), while `title`/`page_width`/`labels` are visible and recoverable, so they warn and frontmatter wins. Agreement is silent, which is what makes migration incremental. Three things took a second pass and should not be flattened: `Source` (what `--json` reports as `metadata_source`) and `Managed` are computed from **different predicates** — the first answers "who contributed metadata", the second "should `update` act on this file", which is true when an entry exists or a `page_id` is named, so `a.md: {}` is a claim that must fail for want of an id rather than be skipped; contributed metadata is counted only over fields `project.IsPageField` knows, or a docs tree carrying Jekyll's `layout:`/`date:` reads as claimed and a whole tree fails; and `labels` is compared as a **set**, since Confluence has no label order and a reordering cannot reach the page. A blank value is not a disagreement — every null spelling already reads as `""`. `KeyFor` is the one place a file's path becomes a manifest key, used on both sides because a mismatch is a silent skip rather than an error. +- `internal/pagemeta` — `Resolve`, the one merge of a file's page metadata from the two places it may live: its own frontmatter and a `pages:` entry in the project file (#139). A package because `update`, `create`, `check` **and `internal/linkindex`** all need it, and a per-command copy is how two commands come to publish one file to two different pages. It imports `frontmatter` and `project` and nothing else, which is also why it validates no *value*: `pagewidth` and `labels` are unreachable from here, and the commands that need them already call them on the maps it returns. **Frontmatter and an entry are two spellings of one level, not two levels of a precedence chain** — when both speak the rule is not "the higher wins" but a grading: `page_id`/`space`/`parent` are coordinates and a disagreement fails the file (for `create`, the batch, since it preflights everything), while `title`/`page_width`/`labels` are visible and recoverable, so they warn and frontmatter wins. Agreement is silent, which is what makes migration incremental. Three things took a second pass and should not be flattened: `Source` (what `--json` reports as `metadata_source`) and `Managed` are computed from **different predicates** — the first answers "who contributed metadata", the second "should `update` act on this file", which is true when an entry exists or a `page_id` is named, so `a.md: {}` is a claim that must fail for want of an id rather than be skipped; contributed metadata is counted only over fields `project.IsPageField` knows, or a file carrying only keys markfluence *preserves but does not understand* (`reviewers:`, pinned by a frontmatter test) reads as claimed and a whole tree of them fails; and `labels` is compared as a **set**, since Confluence has no label order and a reordering cannot reach the page. A blank value is not a disagreement — every null spelling already reads as `""`. `KeyFor` is the one place a file's path becomes a manifest key, used on both sides because a mismatch is a silent skip rather than an error. - `internal/pageslug` — `Slug`/`For`/`Filename`: a title to a filename-safe slug. A package rather than a helper because `export`, `read` and `attachment-download` all place attachments under a page's own directory and must agree. It lowercases (so case-variant titles collide and can be caught) and drops `/` (so no title can inject a path separator); it is lossy, and no readable slug can avoid being, so the caller decides what a collision means. Known limit: NFD and NFC spellings of one title are different Go strings but one filename on APFS, so that pair is not disambiguated. - `internal/pagedoc` — a fetched page as a markdown document: `Render` (frontmatter + converted body), `Frontmatter`, and the lookups the converter can't do for itself — `Sources`/`SourcesFrom` (attachment name → recorded source path) and `PageLinks` (the page an `` points at → its URL). **One conversion, parameterized by a `Placement`**: where the page's file sits, where its unrecorded attachments go, what `parent:` says, and the attachment listing the caller already has. `read`, `export` and `attachment-download` all go through `Options`/`AttachmentDirFor` rather than assembling their own, so they cannot drift by accident — only by argument. For a page at the top level of what is being written, which is what `read` prints and what a single-page export writes, `read` and `export` are byte-identical; deeper in a tree they differ in exactly the position-dependent parts (a sourced attachment's `../` prefix, a `-` suffix a sibling forced, and `parent:`), because `read` has no tree to be positioned in. It needs a client (page width, attachment list, title lookups), which is why it isn't in `internal/convert` — that package is deliberately client-free, and it's why `StorageToMarkdown` takes those maps rather than fetching them. Every one of them is best-effort in the same shape: no references in the body means no request at all, and a lookup that fails is omitted rather than fatal (an omitted page link renders as raw storage, not as a link with no destination). It also owns **`UserCache`**, the user-name lookup both mention directions share (#91): a per-run, **cross-page** cache, threaded in from the caller the way `project.Cache`/`linkindex.Cache` are, because the obvious structure is wrong — `PageLinks` builds its space-id map per page and `Options` is built per page, so a user map written that way would re-resolve the same twelve people on every page of a 200-page export. It **remembers misses**, or a page mentioning deactivated people costs a request each, every page, to learn the same failures. Not persisted to disk, and the reason is **L2**: output must depend only on the files on disk, not on what a cache happens to hold. `MentionWarnings` is the forward direction's use of it, and sharing the cache is what makes that warning affordable — publishing needs no display names at all, so that lookup exists purely to report an id that names nobody. `PageLinks` resolves a space id **once per space key**, not once per link, and refuses to search site-wide when it can't scope a title to a space — a same-titled page in the wrong space is a wrong answer, which is worse than the passthrough a miss produces. - `internal/attachfile` — `Resolve` (where an attachment goes under a destination root, **including the traversal clamp**) and `Write` (download it there, honoring force/dry-run). Shared by `attachment-download` and `export`; the clamp must never exist in two copies. diff --git a/_plans/039_project-file-pages.md b/_plans/039_project-file-pages.md index 0393fd2..1e146f2 100644 --- a/_plans/039_project-file-pages.md +++ b/_plans/039_project-file-pages.md @@ -115,8 +115,14 @@ Six things to read off it, each of which a decision below has to hold up: both speak the rule is not "the higher wins" but D6's grading: a coordinate disagreement is an error and a soft one is a warning. Only *below* them is there precedence, and only for a field neither declares. -- **`parent` is a relative `.md` path here**, exactly as in frontmatter, and is - resolved the same way. The `page_id` form works too; nothing new. +- **`parent` is a relative `.md` path here, but relative to the *root*, not to + the file** — like every other path in the manifest, and as #139's own example + spells it. That is *not* how frontmatter's `parent:` works, which is + file-relative; the two are reconciled by `pagemeta` translating an entry's + value at the boundary, so the manifest stays internally consistent and + nothing downstream has to learn where a value came from. An earlier draft of + this plan claimed it resolved "exactly as in frontmatter, the same way", + which was self-contradictory and was implemented as neither. - **Keys are root-relative and lexical** (D3), which is the same key space `linkindex` already uses (fact 1) — so `docs/deploy-runbook.md` linking to `engineering-docs.md` resolves through the manifest with no new path vocabulary. @@ -534,6 +540,73 @@ flags), `docs/guarantees.md` (**L2** is why keys are lexical and root-relative), read for disagreement detection but changes nothing about where `update` publishes, since the space comes from the live page. +## What the review found — 2026-09-12 + +Nine findings. Two were merge blockers and both were in `create`, which is the +seam this PR left weakest: it "read entries" for `title`/`space`/`page_id`/ +`page_width`/`labels` while two other paths still read frontmatter alone. + +1. **A `.md` parent could not resolve in a manifest project — this plan's own + example failed.** Two independent halves. `resolveParent` joined the parent + path onto the *file's* directory, so the root-relative spelling looked for + `docs/docs/engineering-docs.md`; and it read the parent's id with + `pmf.PageID()`, frontmatter only, so a parent whose `page_id` lived in its + own entry reported "not yet published". That is #139's `linkindex` trap in a + second place, where it fails a `create` rather than degrading a link. Fixed + by the boundary translation above and by resolving the *parent's* metadata + through `pagemeta` too. The plan's example is now an end-to-end test. + +2. **`create` persisting frontmatter poisoned every later run of a registered + file.** Publishing wrote `page_id` and a resolved `parent: ` into a file + whose entry said something else, so the next `update` or `check` failed as a + coordinate disagreement. D9's rule is that new metadata goes where that + file's metadata already is, and PR 1 has no manifest writer — so the correct + behaviour is to *not write*, and say so. D13 called the cost "a `page_id` + copied by hand"; it was silently creating the thing that needed repairing. + +3. **`check`'s project-width lint read `mf.Frontmatter`, not the resolved + metadata**, so a file whose *entry* declared a width was failed over the + project's bad one — the exact false positive the surrounding comment forbids, + and `update` published the same file fine. + +4. **Soft-disagreement warnings were delivered only by `update`.** `check` and + `create` dropped `meta.Warnings` on the floor, against D6. + +5. **`metadata_source` was `"frontmatter"` on an unmanaged skip**, set before + the `Managed()` check — contradicting the schema, `docs/json-output.md`, and + the `unmanaged` field's own comment about `--json` telling the two skips + apart. + +6. **A present-but-blank frontmatter key vanished when an entry existed.** So a + file with `title:` (empty) was reported broken with no entry and clean with + one. A blank is not a *disagreement*, but it is still something the author + wrote. + +7. **`NormalizePageKey` judged absoluteness before normalizing separators**, so + `\foo.md`, `C:\foo.md` and `\\server\share\a.md` were accepted as keys + `KeyFor` can never produce — a silently unreachable entry. + +8. Minor: a dead `failure.metadataSource` field whose comment claimed it was + populated; `create`'s "add a 'title:' frontmatter field" and the + `--parent` conflict message not mentioning an entry. + +9. Noted, deliberate: a frontmatter-only file with `page_id:` blank now exits 0 + `skipped` where it used to fail. D7 covers it, and "registered" is + entry-only by design. + +Test quality: nothing in `cmd/create` exercised `pagemeta` at all, which is how +1 and 2 hid; the conformance fixtures left `metadata_source` empty so its enum +values were never validated; and the human "not published by markfluence" line +was untested despite the PR claiming the two skips are distinguishable. All +added, and 1's two halves were sabotage-checked. + +One framing correction that came out of review discussion rather than from +fable: the comments justifying `IsPageField` cited Jekyll and Hugo frontmatter, +which is a scenario nobody here has. The grounded version is inside markfluence +— `internal/frontmatter` deliberately preserves keys it does not understand, +pinned by a test on `reviewers: [ana, bo]` — so a file carrying only such keys +is a shape markfluence explicitly supports, with no third-party tool involved. + ## Follow-ups - **`markfluence status`: the local tree, plus what `update` would do** (filed From bec4e71d83b4914cbac432213d4b14aa34de55e5 Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:58:39 -0400 Subject: [PATCH 13/14] feat(pageref): resolve a page id from the manifest too Found by asking what the review's pattern implied rather than by the review itself, which was scoped to the diff: every one of its nine findings was at a seam -- a call site updated incompletely, an ordering, or old code receiving new inputs -- and not one was in the new core. That shape is enumerable, so one grep over every place page metadata is read from a file turned up a whole class still unfixed. pageref.Resolve read mf.PageID() and nothing else, so seven commands could not name a file update could publish: info, read, children, export and the three attachment verbs. They failed with "no page_id in frontmatter", safely -- but the page argument meant one thing to update and another to everything else, which is the inconsistency this package exists to prevent. Fixed in pageref rather than at seven call sites. Resolve already stats the file, so it discovers the root from that file's own directory -- the same per-file discovery every other read does -- and resolves through pagemeta. No signature change, no call-site churn, no import cycle. One deliberate asymmetry: a malformed markfluence.yaml is *not* fatal here. A disagreement between the two locations is, since that is exactly the question being asked and either id would be a guess; but a project file this function never consults should not make `info` fail, and the commands that bound reads by the root report the problem themselves. fix had no project concept at all, being the one verb that reads the page and writes the file. It gets one, which is what lets it locate a pristine registered page -- and it refuses to write into a file whose metadata lives only in the manifest, because reconciling that means editing a pages: entry and attempting it would reproduce the create bug review caught. A file carrying some inline keys is still fixable: those keys exist already, and the two locations agreed about the coordinates or resolution would have failed first. Its planners take the resolved maps rather than a MarkdownFile, so they reconcile what the file effectively declares. --- _plans/039_project-file-pages.md | 39 +++++++- cmd/fix/fix.go | 72 ++++++++++++--- cmd/fix/fix_test.go | 151 +++++++++++++++++++++++++++---- cmd/fix/json_test.go | 3 +- internal/pageref/pageref.go | 62 ++++++++++++- internal/pageref/pageref_test.go | 117 ++++++++++++++++++++++++ 6 files changed, 407 insertions(+), 37 deletions(-) diff --git a/_plans/039_project-file-pages.md b/_plans/039_project-file-pages.md index 1e146f2..a23d32b 100644 --- a/_plans/039_project-file-pages.md +++ b/_plans/039_project-file-pages.md @@ -607,7 +607,44 @@ which is a scenario nobody here has. The grounded version is inside markfluence pinned by a test on `reviewers: [ana, bo]` — so a file carrying only such keys is a shape markfluence explicitly supports, with no third-party tool involved. -## Follow-ups +## The seam the review's *pattern* revealed — 2026-09-12 + +Every one of the nine findings was at a seam: a call site updated incompletely, +an ordering, or old code receiving new inputs. Not one was in the new core, +which fable checked and cleared. That shape is enumerable, so it was enumerated +— one grep over every place page metadata is read from a file outside +`internal/pagemeta` — and it found a whole class the review had not, because the +review was scoped to the diff. + +**`internal/pageref.Resolve` read `mf.PageID()` and nothing else**, so seven +commands could not name a file `update` could publish: `info`, `read`, +`children`, `export`, `attachment-list`, `attachment-upload`, +`attachment-download`. They failed with `no page_id in frontmatter of docs/a.md` +— safely, but the page argument meant one thing to `update` and another to +everything else, which is exactly what the package exists to prevent. + +Fixed there rather than at seven call sites. `Resolve` already stats the file, +so it discovers the root from the file's own directory — the same per-file +discovery every other read does — and resolves through `pagemeta`. No signature +change and no call-site churn. One deliberate asymmetry: a *malformed* +`markfluence.yaml` is not fatal here, because this function answers "which page +does this argument name" and a project file it never consults should not fail +that; the commands that bound reads by the root report it themselves. + +**`fix` had no project concept at all** — the one verb that reads the page and +writes the file. It now resolves through the root too, which is what lets it +locate a pristine registered page. And it **refuses to write** into a file whose +metadata lives only in the manifest: reconciling that means editing a `pages:` +entry, which needs the write half, and attempting it would be finding 2 all over +again in a second command. A file carrying *some* inline keys is still fixable, +usefully so — those keys exist already, and the two locations agreed about the +coordinates or resolution would have failed first. + +So PR 1's real scope is three seams wide, not two: read, consume, and *name*. +What is left for PR 2 is the writer, plus `fix` reconciling an entry once there +is something to reconcile it with. + + - **`markfluence status`: the local tree, plus what `update` would do** (filed as #148). Three diff --git a/cmd/fix/fix.go b/cmd/fix/fix.go index 5fb836e..e2860bd 100644 --- a/cmd/fix/fix.go +++ b/cmd/fix/fix.go @@ -7,6 +7,7 @@ import ( "errors" "fmt" "os" + "path/filepath" "strings" "github.com/mozilla/markfluence/internal/client" @@ -14,8 +15,10 @@ import ( "github.com/mozilla/markfluence/internal/frontmatter" "github.com/mozilla/markfluence/internal/jsonout" "github.com/mozilla/markfluence/internal/labels" + "github.com/mozilla/markfluence/internal/pagemeta" "github.com/mozilla/markfluence/internal/pageref" "github.com/mozilla/markfluence/internal/pagewidth" + "github.com/mozilla/markfluence/internal/project" "github.com/mozilla/markfluence/internal/ui" "github.com/spf13/cobra" ) @@ -64,8 +67,15 @@ func run(cmd *cobra.Command, args []string) error { username, _ := cmd.Flags().GetString("username") cloudID, _ := cmd.Flags().GetString("cloud-id") envFile, _ := cmd.Flags().GetString("env-file") + rootOverride, _ := cmd.Flags().GetString("root") + // fix had no project concept at all, being the one verb that reads the + // page and writes the file. It needs one now: a file's metadata may live in + // markfluence.yaml's pages: block, so locating the page at all requires + // resolving through the root (#139). + roots := project.NewCache(rootOverride) + defer roots.Close() c, err := client.Resolve(client.ResolveOptions{ - URL: url, Username: username, CloudID: cloudID, EnvFile: envFile, + URL: url, Username: username, CloudID: cloudID, EnvFile: envFile, Roots: roots, }) if err != nil { if ui.IsJSON() { @@ -83,7 +93,7 @@ func run(cmd *cobra.Command, args []string) error { failures := 0 results := make([]*fixResult, 0, len(args)) for _, filename := range args { - r := processFile(filename, c) + r := processFile(filename, c, roots) results = append(results, r) if !ui.IsJSON() { r.renderHuman() @@ -129,13 +139,49 @@ type change struct { // processFile reconciles one file and returns a result. It performs no output; // the caller renders the result. -func processFile(filename string, c *client.ConfluenceClient) *fixResult { +func processFile(filename string, c *client.ConfluenceClient, roots *project.Cache) *fixResult { r := &fixResult{file: filename, dryRun: dryRun} mf, err := frontmatter.ParseFile(filename) if err != nil { return r.fail(err, jsonout.CodeValidation) } - page, err := locatePage(mf.Frontmatter, c) + abs, err := filepath.Abs(filename) + if err != nil { + return r.fail(err, jsonout.CodeIO) + } + root, err := roots.Resolve(filepath.Dir(abs)) + if err != nil { + code := jsonout.CodeIO + if project.IsConfigError(err) { + code = jsonout.CodeValidation + } + return r.fail(project.RootError(err), code) + } + key, _ := pagemeta.KeyFor(root, abs) + meta, err := pagemeta.Resolve(key, mf, root) + if err != nil { + return r.fail(err, jsonout.CodeValidation) + } + r.warnings = append(r.warnings, meta.Warnings...) + + // A file whose metadata lives *only* in markfluence.yaml has nothing fix + // can write: reconciling it means editing its pages: entry, and #139's + // write half is not here yet. Refused rather than attempted, because the + // alternative is what create was doing before review caught it -- writing + // frontmatter into a pristine file, which turns every later update and + // check of it into a coordinate disagreement. + // + // A file that carries *some* inline keys as well is still fixable, and + // usefully so: those keys exist already, and the two locations agreed about + // the coordinates or Resolve would have failed above. + if meta.InManifest() && !meta.InFile() { + return r.fail(fmt.Errorf( + "this file's metadata lives in %s; fix cannot reconcile a pages: entry yet, "+ + "so there is nothing here to correct", project.Filename), + jsonout.CodeValidation) + } + + page, err := locatePage(meta.Fields, c) if err != nil { // locatePage mixes server failures (GetPageOrNil, SearchPagesByTitle) // with local ones (no page_id or title, an ambiguous title), so the code @@ -168,7 +214,7 @@ func processFile(filename string, c *client.ConfluenceClient) *fixResult { } } - r.changes = plannedChanges(mf, page, liveWidth, liveLabels) + r.changes = plannedChanges(meta.Fields, meta.Lists, page, liveWidth, liveLabels) // Field order is reconciled too, and counts as a change: reporting a // jumbled file "consistent" would mean running fix, being told there is // nothing to do, and still having a jumbled file. Computed before any edit, @@ -260,9 +306,9 @@ func locatePage(fm map[string]string, c *client.ConfluenceClient) (*client.Page, // plannedChanges computes the field edits needed to reconcile mf to page. Only // fields that actually differ are returned. func plannedChanges( - mf *frontmatter.MarkdownFile, page *client.Page, liveWidth string, liveLabels []string, + fm map[string]string, lists map[string][]string, + page *client.Page, liveWidth string, liveLabels []string, ) []change { - fm := mf.Frontmatter live := []struct{ field, value string }{ {"page_id", page.ID}, {"space", client.SpaceKeyFromWebUI(page.Links.WebUI)}, @@ -308,7 +354,7 @@ func plannedChanges( } } - if ch, ok := labelChange(mf, liveLabels); ok { + if ch, ok := labelChange(fm, lists, liveLabels); ok { changes = append(changes, ch) } return changes @@ -331,19 +377,21 @@ func plannedChanges( // set is what is about to be written, and it came from the server, so it is // valid by construction. That is the one place fix repairs a file check would // have failed. -func labelChange(mf *frontmatter.MarkdownFile, liveLabels []string) (change, bool) { +func labelChange( + fm map[string]string, lists map[string][]string, liveLabels []string, +) (change, bool) { // nil means the read failed. Planning nothing is right: a change here would // propose the file's own labels be replaced by a set nobody could see. if liveLabels == nil { return change{}, false } - declared, present := mf.Lists[labels.Field] + declared, present := lists[labels.Field] // A scalar labels: value is a file every other verb refuses, so fix has to // offer a way out of it whatever the page's labels are -- including none, // where the repair is "labels: []". Reading it as absent meant fix reported // "already consistent" for a file check, update and create all reject, and // only repaired it when the page happened to carry labels. - scalar := !present && hasKey(mf.Frontmatter, labels.Field) + scalar := !present && hasKey(fm, labels.Field) if present { // Compared raw, not normalized. Normalizing first made a case mismatch @@ -366,7 +414,7 @@ func labelChange(mf *frontmatter.MarkdownFile, liveLabels []string) (change, boo case present: old = renderLabelList(declared) case scalar: - old = strings.TrimSpace(mf.Frontmatter[labels.Field]) + old = strings.TrimSpace(fm[labels.Field]) if old == "" { old = noneDisplay } diff --git a/cmd/fix/fix_test.go b/cmd/fix/fix_test.go index 2a87e56..2bdc0e3 100644 --- a/cmd/fix/fix_test.go +++ b/cmd/fix/fix_test.go @@ -11,6 +11,7 @@ import ( "github.com/mozilla/markfluence/internal/client" "github.com/mozilla/markfluence/internal/clienttest" "github.com/mozilla/markfluence/internal/frontmatter" + "github.com/mozilla/markfluence/internal/project" ) // --- locatePage -------------------------------------------------------------- @@ -283,7 +284,7 @@ func TestProcessFileConsistentDoesNotWrite(t *testing.T) { path := writeFixture(t, content) c := fixServer(t, pageJSON("1", "X", "", "/spaces/ENG/pages/1/X"), `"max"`) - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if !r.ok || r.status != statusConsistent { t.Fatalf("result = %+v, want ok/consistent", r) } @@ -304,7 +305,7 @@ func TestProcessFileDryRunDoesNotWrite(t *testing.T) { dryRun = true t.Cleanup(func() { dryRun = false }) - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if !r.ok || r.status != statusChanged || len(r.changes) == 0 { t.Fatalf("result = %+v, want ok/changed with a nonempty diff", r) } @@ -322,7 +323,7 @@ func TestProcessFileWritesOnRealChange(t *testing.T) { path := writeFixture(t, content) c := fixServer(t, pageJSON("123", "X", "", "/spaces/ENG/pages/123/X"), `"max"`) - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if !r.ok || r.status != statusChanged { t.Fatalf("result = %+v, want ok/changed", r) } @@ -346,7 +347,7 @@ func TestProcessFileFailsWhenPageNotFound(t *testing.T) { _, _ = w.Write([]byte(`{"errors":[{"status":404,"title":"Cannot find a page with id 999"}]}`)) }) - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if r.ok { t.Fatal("want a failure when the page_id resolves to nothing") } @@ -395,7 +396,7 @@ func TestProcessFileNormalizesFieldOrder(t *testing.T) { path := writeFixture(t, content) c := fixServer(t, pageJSON("1", "X", "", "/spaces/ENG/pages/1/X"), `"max"`) - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if !r.ok || r.status != statusChanged { t.Fatalf("result = %+v, want ok/changed", r) } @@ -423,7 +424,7 @@ func TestProcessFileTopLevelPageConverges(t *testing.T) { path := writeFixture(t, content) c := fixServer(t, pageJSON("1", "X", "", "/spaces/ENG/pages/1/X"), `"max"`) - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if r.status != statusConsistent { t.Fatalf("status = %q with changes %+v, want consistent", r.status, r.changes) } @@ -435,7 +436,7 @@ func TestProcessFileTopLevelPageConverges(t *testing.T) { // wants -- one field under test and nothing else moving. func plannedChangesFM(fm map[string]string, page *client.Page, liveWidth string) []change { mf := &frontmatter.MarkdownFile{Frontmatter: fm, Lists: map[string][]string{}} - return plannedChanges(mf, page, liveWidth, nil) + return plannedChanges(mf.Frontmatter, mf.Lists, page, liveWidth, nil) } // --- labels ------------------------------------------------------------------- @@ -466,7 +467,8 @@ func labelChangeIn(changes []change) (change, bool) { // the file to the page and update the page to the file. func TestFixAdoptsHandLabels(t *testing.T) { page := &client.Page{ID: "1", Title: "T"} - got := plannedChanges(mdFile(t, "page_id: 1"), page, "", []string{"ci/cd", "runbook"}) + fm, lists := mapsOf(t, "page_id: 1") + got := plannedChanges(fm, lists, page, "", []string{"ci/cd", "runbook"}) ch, ok := labelChangeIn(got) if !ok { @@ -493,7 +495,8 @@ func TestFixLeavesAMatchingSetAlone(t *testing.T) { "page_id: 1\nlabels: [runbook, ci/cd]", "page_id: 1\nlabels: [runbook, ci/cd, runbook]", } { - got := plannedChanges(mdFile(t, block), page, "", []string{"ci/cd", "runbook"}) + fm, lists := mapsOf(t, block) + got := plannedChanges(fm, lists, page, "", []string{"ci/cd", "runbook"}) if ch, ok := labelChangeIn(got); ok { t.Errorf("%q planned %+v, want no labels change", block, ch) } @@ -504,7 +507,8 @@ func TestFixLeavesAMatchingSetAlone(t *testing.T) { // no labels must not gain a "labels: []" that says nothing. func TestFixPlansNothingWhenNeitherHasLabels(t *testing.T) { page := &client.Page{ID: "1", Title: "T"} - got := plannedChanges(mdFile(t, "page_id: 1"), page, "", []string{}) + fm, lists := mapsOf(t, "page_id: 1") + got := plannedChanges(fm, lists, page, "", []string{}) if ch, ok := labelChangeIn(got); ok { t.Errorf("planned %+v, want no labels change", ch) } @@ -515,7 +519,8 @@ func TestFixPlansNothingWhenNeitherHasLabels(t *testing.T) { // would propose stripping every label from the file. func TestFixPlansNothingWhenTheReadFailed(t *testing.T) { page := &client.Page{ID: "1", Title: "T"} - got := plannedChanges(mdFile(t, "page_id: 1\nlabels: [runbook]"), page, "", nil) + fm, lists := mapsOf(t, "page_id: 1\nlabels: [runbook]") + got := plannedChanges(fm, lists, page, "", nil) if ch, ok := labelChangeIn(got); ok { t.Errorf("planned %+v, want no labels change when the read failed", ch) } @@ -525,7 +530,8 @@ func TestFixPlansNothingWhenTheReadFailed(t *testing.T) { // to the empty set, which is a real state a page can be in. func TestFixRemovesLabelsThePageNoLongerHas(t *testing.T) { page := &client.Page{ID: "1", Title: "T"} - got := plannedChanges(mdFile(t, "page_id: 1\nlabels: [runbook, gone]"), page, "", []string{}) + fm, lists := mapsOf(t, "page_id: 1\nlabels: [runbook, gone]") + got := plannedChanges(fm, lists, page, "", []string{}) ch, ok := labelChangeIn(got) if !ok { @@ -544,8 +550,8 @@ func TestFixRemovesLabelsThePageNoLongerHas(t *testing.T) { // fix repairs a file check would have refused. func TestFixReconcilesAnInvalidLabel(t *testing.T) { page := &client.Page{ID: "1", Title: "T"} - got := plannedChanges(mdFile(t, `page_id: 1 -labels: ["Runbook Two"]`), page, "", []string{"runbook", "two"}) + fm, lists := mapsOf(t, "page_id: 1\nlabels: [\"Runbook Two\"]") + got := plannedChanges(fm, lists, page, "", []string{"runbook", "two"}) ch, ok := labelChangeIn(got) if !ok { @@ -587,7 +593,7 @@ func TestProcessFileKeepsBlockLabelStyle(t *testing.T) { path := writeFixture(t, content) c := labelFixServer(t, pageJSON("1", "X", "", "/spaces/ENG/pages/1/X"), "runbook", "howto") - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if !r.ok || r.status != statusChanged { t.Fatalf("result = %+v, want ok/changed", r) } @@ -617,7 +623,7 @@ func TestProcessFileLabelFixConverges(t *testing.T) { path := writeFixture(t, content) c := labelFixServer(t, pageJSON("1", "X", "", "/spaces/ENG/pages/1/X"), "ci/cd", "runbook") - first := processFile(path, c) + first := processFile(path, c, project.NewCache("")) if !first.ok || first.status != statusChanged { t.Fatalf("first run = %+v, want ok/changed", first) } @@ -629,7 +635,7 @@ func TestProcessFileLabelFixConverges(t *testing.T) { t.Errorf("file = %q, want a sorted flow list for a newly added key", got) } - second := processFile(path, c) + second := processFile(path, c, project.NewCache("")) if !second.ok || second.status != statusConsistent { t.Fatalf("second run = %+v (changes %+v), want ok/consistent", second, second.changes) } @@ -643,7 +649,8 @@ func TestProcessFileLabelFixConverges(t *testing.T) { // on every run and no command that would silence it. func TestFixCorrectsALabelCaseMismatch(t *testing.T) { page := &client.Page{ID: "1", Title: "T"} - got := plannedChanges(mdFile(t, "page_id: 1\nlabels: [Runbook]"), page, "", []string{"runbook"}) + fm, lists := mapsOf(t, "page_id: 1\nlabels: [Runbook]") + got := plannedChanges(fm, lists, page, "", []string{"runbook"}) ch, ok := labelChangeIn(got) if !ok { @@ -676,7 +683,8 @@ func TestFixRepairsAScalarLabelsValue(t *testing.T) { } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - got := plannedChanges(mdFile(t, tt.block), page, "", tt.live) + fm, lists := mapsOf(t, tt.block) + got := plannedChanges(fm, lists, page, "", tt.live) ch, ok := labelChangeIn(got) if !ok { t.Fatalf("changes = %+v, want the scalar value repaired", got) @@ -690,3 +698,108 @@ func TestFixRepairsAScalarLabelsValue(t *testing.T) { }) } } + +// mapsOf gives a test block's two maps, the shapes the planners take now that +// fix reconciles resolved metadata rather than a file's own frontmatter. +func mapsOf(t *testing.T, block string) (map[string]string, map[string][]string) { + t.Helper() + mf := mdFile(t, block) + return mf.Frontmatter, mf.Lists +} + +// --- pristine files ---------------------------------------------------------- + +// projectDir writes a markfluence.yaml plus a markdown file and returns the +// file's path. +func projectDir(t *testing.T, manifest, md string) string { + t.Helper() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, "a.md") + if err := os.WriteFile(path, []byte(md), 0o644); err != nil { + t.Fatal(err) + } + return path +} + +// A file whose metadata lives only in markfluence.yaml has nothing fix can +// write: reconciling it means editing its pages: entry, which needs #139's +// write half. Refused rather than attempted -- the alternative is writing +// frontmatter into a pristine file, which turns every later update and check +// of it into a coordinate disagreement. +func TestProcessFileRefusesAManifestOnlyFile(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + path := projectDir(t, "pages:\n a.md:\n title: A\n page_id: 1\n", "# A\n") + roots := project.NewCache("") + defer roots.Close() + + r := processFile(path, c, roots) + if r.ok { + t.Fatalf("result = %+v, want a refusal", r) + } + if !strings.Contains(r.errMsg, "cannot reconcile a pages: entry yet") { + t.Errorf("errMsg = %q, want the not-yet message", r.errMsg) + } + // And the file is untouched, which is the property that matters. + body, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if string(body) != "# A\n" { + t.Errorf("file was rewritten:\n%s", body) + } +} + +// A file carrying *some* inline keys is still fixable, and usefully so: those +// keys exist already, and the two locations agreed about the coordinates or +// resolution would have failed first. +func TestProcessFileFixesAHalfAndHalfFile(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + if strings.Contains(r.URL.Path, "properties") { + _, _ = w.Write([]byte(`{"results":[]}`)) + return + } + if strings.Contains(r.URL.Path, "label") { + _, _ = w.Write([]byte(`{"results":[]}`)) + return + } + _, _ = w.Write([]byte(pageJSON("1", "Live Title", "", "/spaces/ENG/pages/1/Live"))) + }) + // page_id inline and agreeing with the entry; no title, which fix fills in. + path := projectDir(t, "pages:\n a.md:\n page_id: 1\n", "---\npage_id: 1\n---\n# A\n") + roots := project.NewCache("") + defer roots.Close() + + r := processFile(path, c, roots) + if !r.ok { + t.Fatalf("result = %+v, want success", r) + } + if len(r.changes) == 0 { + t.Error("no changes planned; the live title should have been adopted") + } +} + +// A pristine file with no entry either is the case fix has always reported: +// nothing locates the page. The message should not have regressed. +func TestProcessFileUnmanagedFileStillReportsNoID(t *testing.T) { + c := clienttest.New(t, func(w http.ResponseWriter, r *http.Request) { + t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) + w.WriteHeader(http.StatusInternalServerError) + }) + path := projectDir(t, "space: ENG\n", "# A\n") + roots := project.NewCache("") + defer roots.Close() + + r := processFile(path, c, roots) + if r.ok { + t.Fatalf("result = %+v, want a failure", r) + } + if !strings.Contains(r.errMsg, "no page_id or title") { + t.Errorf("errMsg = %q, want the locate message", r.errMsg) + } +} diff --git a/cmd/fix/json_test.go b/cmd/fix/json_test.go index 7f24e79..cc73824 100644 --- a/cmd/fix/json_test.go +++ b/cmd/fix/json_test.go @@ -10,6 +10,7 @@ import ( "github.com/mozilla/markfluence/internal/clienttest" "github.com/mozilla/markfluence/internal/jsonout" + "github.com/mozilla/markfluence/internal/project" "github.com/mozilla/markfluence/internal/schematest" ) @@ -175,7 +176,7 @@ func TestProcessFileClassifiesALocateFailureByOrigin(t *testing.T) { if err := os.WriteFile(path, []byte(tt.body), 0o644); err != nil { t.Fatal(err) } - r := processFile(path, c) + r := processFile(path, c, project.NewCache("")) if r.ok { t.Fatal("processFile should have failed") } diff --git a/internal/pageref/pageref.go b/internal/pageref/pageref.go index f3fb52f..c540793 100644 --- a/internal/pageref/pageref.go +++ b/internal/pageref/pageref.go @@ -3,21 +3,69 @@ // // Three spellings are accepted, because all three are things a user naturally // has to hand: a bare numeric id, a Confluence page or folder URL (pasted from a -// browser), and a markdown file whose frontmatter carries a page_id. Every command that +// browser), and a markdown file that carries a page_id. Every command that // takes a page argument accepts all three, so the meaning of that argument does // not depend on which command it was given to. +// +// "Carries a page_id" means either location it may live in: the file's own +// frontmatter, or a pages: entry for it in the project's markfluence.yaml +// (#139). Reading only the frontmatter left seven commands -- info, read, +// children, export and the three attachment-* verbs -- unable to name a file +// they could publish, which is a worse inconsistency than the extra lookup +// costs: the argument would have meant different things to update and to info. package pageref import ( "fmt" "net/url" "os" + "path/filepath" "regexp" "strings" "github.com/mozilla/markfluence/internal/frontmatter" + "github.com/mozilla/markfluence/internal/pagemeta" + "github.com/mozilla/markfluence/internal/project" ) +// declaredPageID reads a file's page_id from wherever it lives. +// +// The root is discovered from the file's own directory, which is the same +// per-file discovery every other read does (docs/root-model.md) -- and it has to +// be discovered here rather than passed in: these seven commands take a single +// page and build no project.Cache, so threading a root through would mean +// seven signature changes to reach one lookup. A file with no project file above +// it, or one whose project declares no pages:, resolves exactly as it did +// before. +// +// A malformed markfluence.yaml is *not* fatal here, deliberately. This function +// answers "which page does this argument name", and a project file that cannot +// be understood does not stop the frontmatter from answering it; the commands +// that bound reads by the root resolve it themselves and report the problem +// there. Refusing here would make `info 123` fail for a broken file it never +// consults. +func declaredPageID(arg string, mf *frontmatter.MarkdownFile) (string, error) { + abs, err := filepath.Abs(arg) + if err != nil { + return strings.TrimSpace(mf.PageID()), nil + } + root, err := project.Discover(filepath.Dir(abs)) + if err != nil { + return strings.TrimSpace(mf.PageID()), nil + } + defer func() { _ = root.FS.Close() }() + + key, _ := pagemeta.KeyFor(root, abs) + meta, err := pagemeta.Resolve(key, mf, root) + if err != nil { + // The two locations name different pages. Unlike the malformed-file + // case above, this is exactly the question being asked, and answering + // with either id would be a guess. + return "", err + } + return strings.TrimSpace(meta.Fields["page_id"]), nil +} + // pagePathRE matches the numeric id in a modern Confluence content URL path, // e.g. /wiki/spaces/ENG/pages/123456/Some+Title (the trailing slug is optional). // @@ -41,10 +89,16 @@ func Resolve(arg string) (string, error) { if err != nil { return "", err } - if mf.PageID() == "" { - return "", fmt.Errorf("no page_id in frontmatter of %s", arg) + id, err := declaredPageID(arg, mf) + if err != nil { + return "", err + } + if id == "" { + return "", fmt.Errorf( + "no page_id for %s: set one in its frontmatter or in its %s entry", + arg, project.Filename) } - return mf.PageID(), nil + return id, nil } if IsDigits(arg) { return arg, nil diff --git a/internal/pageref/pageref_test.go b/internal/pageref/pageref_test.go index 35b5eff..a2ea403 100644 --- a/internal/pageref/pageref_test.go +++ b/internal/pageref/pageref_test.go @@ -3,7 +3,10 @@ package pageref import ( "os" "path/filepath" + "strings" "testing" + + "github.com/mozilla/markfluence/internal/project" ) func TestResolveNumericID(t *testing.T) { @@ -97,3 +100,117 @@ func TestIsDigits(t *testing.T) { } } } + +// --- a file whose page_id lives in markfluence.yaml --------------------------- + +// The seam an audit found after review: seven commands take a page argument +// through Resolve, and all of them read mf.PageID() alone -- so a file that +// update could publish from its pages: entry could not be named to info, read, +// children, export or the three attachment verbs. +func TestResolveReadsThePageIDFromTheManifest(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), + []byte("pages:\n docs/a.md:\n title: A\n page_id: 12345\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Join(dir, "docs"), 0o755); err != nil { + t.Fatal(err) + } + // Pristine: no frontmatter at all. + path := filepath.Join(dir, "docs", "a.md") + if err := os.WriteFile(path, []byte("# A\n"), 0o644); err != nil { + t.Fatal(err) + } + + got, err := Resolve(path) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got != "12345" { + t.Errorf("= %q, want 12345", got) + } +} + +// Frontmatter still wins where it speaks, and a file with neither still says +// so -- naming both places the id could go. +func TestResolveFrontmatterStillWinsAndNeitherIsReported(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), + []byte("pages:\n inline.md:\n page_id: 999\n"), 0o644); err != nil { + t.Fatal(err) + } + // Agreement, so no disagreement error: the entry and the file say the same. + inline := filepath.Join(dir, "inline.md") + if err := os.WriteFile(inline, []byte("---\npage_id: 999\n---\n# I\n"), 0o644); err != nil { + t.Fatal(err) + } + if got, err := Resolve(inline); err != nil || got != "999" { + t.Errorf("= %q/%v, want 999", got, err) + } + + bare := filepath.Join(dir, "bare.md") + if err := os.WriteFile(bare, []byte("# B\n"), 0o644); err != nil { + t.Fatal(err) + } + _, err := Resolve(bare) + if err == nil { + t.Fatal("want an error for a file with no page_id anywhere") + } + for _, want := range []string{"no page_id", project.Filename} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error = %q, want it to mention %q", err, want) + } + } +} + +// The two locations naming different pages is the question being asked, so it +// is an error rather than a guess. +func TestResolveRefusesADisagreement(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), + []byte("pages:\n a.md:\n page_id: 111\n"), 0o644); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, "a.md") + if err := os.WriteFile(path, []byte("---\npage_id: 222\n---\n# A\n"), 0o644); err != nil { + t.Fatal(err) + } + if _, err := Resolve(path); err == nil { + t.Fatal("want an error when the two locations name different pages") + } +} + +// A malformed project file must not stop the frontmatter from answering. This +// function answers "which page does this argument name", and a project file it +// never consults should not fail that -- the commands that bound reads by the +// root report the problem themselves. +func TestResolveIgnoresAMalformedProjectFile(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, project.Filename), + []byte("spce: ENG\n"), 0o644); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, "a.md") + if err := os.WriteFile(path, []byte("---\npage_id: 777\n---\n# A\n"), 0o644); err != nil { + t.Fatal(err) + } + got, err := Resolve(path) + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got != "777" { + t.Errorf("= %q, want 777", got) + } +} + +// A file with no project file above it resolves exactly as it did before. +func TestResolveWithNoProjectFile(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "a.md") + if err := os.WriteFile(path, []byte("---\npage_id: 42\n---\n# A\n"), 0o644); err != nil { + t.Fatal(err) + } + if got, err := Resolve(path); err != nil || got != "42" { + t.Errorf("= %q/%v, want 42", got, err) + } +} From 1d2a38b039833595314e5d4ad76430619c95fb1e Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sat, 12 Sep 2026 19:59:30 -0400 Subject: [PATCH 14/14] docs: record that every page-taking command accepts a pristine file CLAUDE.md's internal/pageref bullet said a .md argument resolves by its frontmatter page_id, which is now only half of it -- and the reason the lookup belongs there rather than at the seven call sites is worth writing down, since that is what keeps the page argument meaning the same thing to update and to info. The malformed-vs-disagreement asymmetry goes with it. fix's passage gains what it refuses and why: it can locate a page whose coordinates live in an entry, but writing frontmatter into such a file is the bug review caught in create, so it says so instead. docs/root-model.md notes the reader-facing consequence: `markfluence info docs/deploy-runbook.md` works on a file that says nothing about Confluence. --- CLAUDE.md | 4 ++-- docs/root-model.md | 6 ++++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6b89913..aef3c4a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,7 +76,7 @@ Module `github.com/mozilla/markfluence` (`go 1.25`). `main.go` is a shim to `cmd - `internal/pagedoc` — a fetched page as a markdown document: `Render` (frontmatter + converted body), `Frontmatter`, and the lookups the converter can't do for itself — `Sources`/`SourcesFrom` (attachment name → recorded source path) and `PageLinks` (the page an `` points at → its URL). **One conversion, parameterized by a `Placement`**: where the page's file sits, where its unrecorded attachments go, what `parent:` says, and the attachment listing the caller already has. `read`, `export` and `attachment-download` all go through `Options`/`AttachmentDirFor` rather than assembling their own, so they cannot drift by accident — only by argument. For a page at the top level of what is being written, which is what `read` prints and what a single-page export writes, `read` and `export` are byte-identical; deeper in a tree they differ in exactly the position-dependent parts (a sourced attachment's `../` prefix, a `-` suffix a sibling forced, and `parent:`), because `read` has no tree to be positioned in. It needs a client (page width, attachment list, title lookups), which is why it isn't in `internal/convert` — that package is deliberately client-free, and it's why `StorageToMarkdown` takes those maps rather than fetching them. Every one of them is best-effort in the same shape: no references in the body means no request at all, and a lookup that fails is omitted rather than fatal (an omitted page link renders as raw storage, not as a link with no destination). It also owns **`UserCache`**, the user-name lookup both mention directions share (#91): a per-run, **cross-page** cache, threaded in from the caller the way `project.Cache`/`linkindex.Cache` are, because the obvious structure is wrong — `PageLinks` builds its space-id map per page and `Options` is built per page, so a user map written that way would re-resolve the same twelve people on every page of a 200-page export. It **remembers misses**, or a page mentioning deactivated people costs a request each, every page, to learn the same failures. Not persisted to disk, and the reason is **L2**: output must depend only on the files on disk, not on what a cache happens to hold. `MentionWarnings` is the forward direction's use of it, and sharing the cache is what makes that warning affordable — publishing needs no display names at all, so that lookup exists purely to report an id that names nobody. `PageLinks` resolves a space id **once per space key**, not once per link, and refuses to search site-wide when it can't scope a title to a space — a same-titled page in the wrong space is a wrong answer, which is worse than the passthrough a miss produces. - `internal/attachfile` — `Resolve` (where an attachment goes under a destination root, **including the traversal clamp**) and `Write` (download it there, honoring force/dry-run). Shared by `attachment-download` and `export`; the clamp must never exist in two copies. - `internal/pagetree` — `Walk`, the traversal of pages *and folders* under a node, plus `WalkSpace` (the same traversal seeded from a space's root pages, via `client.ListSpaceRootPages`) and `AllDepths`. Both go through one `walker`, so the depth rule and the visited guard exist in a single copy. It is a package rather than command-local because listing a subtree and exporting one (#59) need the identical walk, and its rules must not exist in two copies: siblings arrive from two requests (`/child/page`, `/child/folder`) and are **merged by `extensions.position`**, or the output loses the order Confluence displays; a folder **counts as a level** like a page, which is only reasonable because folders are reported rather than silently traversed; and the walk descends folders even when only pages matter, since a folder may hold the only pages in a subtree. `nodeURL` uses `SiteURL()` — a v1 child row carries `webui` but no `base`. A visited set guards the unbounded case. -- `internal/pageref` — `Resolve`, the single page-argument resolver: a numeric id, a Confluence page **or folder** URL (`pagePathRE` matches both `/pages/` and `/folder/`, since `children` takes a folder and a folder URL is what a browser hands you — the id is all it returns, so a command that can only use a page reports its own not-found), or a `.md` file whose frontmatter has a `page_id` (stat'd first, so `123.md` is a file). Every command taking a page uses it. `message.go` also owns the wording for the two ways a *frontmatter* `page_id` is wrong — `NotFoundMessage` (caller supplies the remedy, which differs per command) and `NotNumericMessage` — because `create`, `update`, and `fix` all report them and a reader should recognize the same problem across all three. They return strings, not errors: `create` wraps the text in its typed `pageIDFailure` (which also carries the `--json` fields), the others want a plain error. Anything checking a `page_id` before a request uses `IsDigits`, since the API answers a non-numeric id with a 400 whose body says nothing useful. +- `internal/pageref` — `Resolve`, the single page-argument resolver: a numeric id, a Confluence page **or folder** URL (`pagePathRE` matches both `/pages/` and `/folder/`, since `children` takes a folder and a folder URL is what a browser hands you — the id is all it returns, so a command that can only use a page reports its own not-found), or a `.md` file that **declares** a `page_id` — in its own frontmatter or in a `pages:` entry for it (#139), resolved through `pagemeta` after discovering the root from the file's own directory (stat'd first, so `123.md` is a file). Every command taking a page uses it, which is why the manifest lookup lives here rather than at the seven call sites: without it the page argument meant one thing to `update` and another to `info`/`read`/`children`/`export`/`attachment-*`, so a file `update` could publish could not be named to any of them. A **disagreement** between the two locations is fatal here — it is the question being asked — while a **malformed** `markfluence.yaml` is not: a project file this resolver never consults must not make `info 123` fail, and the commands that bound reads by the root report it themselves. `message.go` also owns the wording for the two ways a *frontmatter* `page_id` is wrong — `NotFoundMessage` (caller supplies the remedy, which differs per command) and `NotNumericMessage` — because `create`, `update`, and `fix` all report them and a reader should recognize the same problem across all three. They return strings, not errors: `create` wraps the text in its typed `pageIDFailure` (which also carries the `--json` fields), the others want a plain error. Anything checking a `page_id` before a request uses `IsDigits`, since the API answers a non-numeric id with a 400 whose body says nothing useful. - `internal/client` — `ConfluenceClient` over `net/http` with basic auth. Built from a `Config` (site URL, cloud ID, username, token) via `New`; it carries **two bases**: `BaseURL()` is where requests go (the gateway when a cloud ID is set) and `SiteURL()` is always the site. Anything a reader sees uses `SiteURL()` — printed page URLs and, critically, the `baseURL` handed to `convert.MdToConfluence`, since rewritten links are published *into* the page. Pages are Confluence **v2**; attachment writes and the user lookup are **v1** (`/wiki/rest/api/...`). A **folder** — the Cloud content type that can parent a page — has its own v2 route, `GetFolderOrNil` against `/wiki/api/v2/folders/{id}`, because every v2 *page* route answers a folder id with 404; enumerating children, if it is ever added, must be v1, since v2 cannot list inside a folder at all and its page-children route silently omits folders ([docs/confluence/folders.md](docs/confluence/folders.md)). Typed `HTTPError`, per-attempt context timeouts, centralized retry/backoff in `send`. `HTTPError.Error()` appends a **hint** for the three auth failures whose status misleads, matched on the response *body* rather than deduced from the status and always **appended** to it, never replacing it. The one that matters: **a rejected credential is a 404 on every v2 route**, so a revoked token used to make `read` answer `page ... not found` about a page that exists. `RejectedCredential` tells it apart by the fact that every genuine v2 404 *names* what it could not find and the auth one does not, `notFound` gates the three `…OrNil` helpers on it so they stop reading it as "absent", and `jsonout.CodeFor` checks it before the status switch so `--json` reports `AUTH` rather than `NOT_FOUND`. **Two error types on the request path, and one predicate for them**: an `*HTTPError` once a response has a status, an unexported `requestError` when there is none (a transport failure, a request that would not build, a body that would not decode), and `FromRequest` answers whether an error is either. That is what lets a caller tell a server failure from a local one — `jsonout.CodeOr(err, fallback)` is the whole point of it, since `CodeFor` alone reports every non-`HTTPError` as `NETWORK` and so turns `no title given` into a network problem (#133). The rule is deliberately scoped to the request: `DownloadAttachment` writing to the caller's writer, `uploadAttachment` opening the caller's file, and `Resolve` reading the environment stay untyped, because tagging them would misreport an unreadable file as a network failure. The wrapper carries no message of its own, so `Error()` is the inner text verbatim and nothing a reader sees changed. A 403 that is not one of the two measured credential phrasings gets no hint, because that is what a genuine permission denial looks like ([docs/confluence/api.md](docs/confluence/api.md#scopes)). **Retry rules**: 429 for any method; 502/503/504 for idempotent methods; **any other 5xx only when the response carries `Retry-After`** — that is how a 500 becomes retryable, and it is why `parseRetryAfter` reports the header's *presence* apart from its delay (`Retry-After: 0` means "retry now", not "no header"). The exponential delay is jittered, a server-supplied `Retry-After` never is. Decisions go to a package-level hook (`SetRetryLogger`, set once in `root.go` beside `ui.SetDebug`) and fire whichever way they went, because `internal/client` prints nothing and a silent twelve-minute retry storm is otherwise indistinguishable from a hang. **A versioned PUT is not as idempotent as its method**: `SetContentProperty` retry-once on top (recovers a lost create-POST response) and `UpdatePage`'s `updateLanded` both exist for the same reason — a write whose response was lost gets re-sent, and the re-sent version is refused. `updateLanded` requires version *and* title *and* body to match what was sent, since a concurrent edit could have produced the version alone and claiming success over someone else's content is worse than a false failure ([docs/confluence/api.md](docs/confluence/api.md)). `SyncAttachments` (skip/update by a SHA-256 recorded in the attachment's comment, alongside the source path so `read` recovers image paths exactly; only the current comment form is parsed — an attachment stamped by a markfluence predating a comment-format change reads as unmanaged and is re-uploaded once, the same as any hand-uploaded file — except that a *recorded path disagreeing with the local source* is an update even when the checksum matches, so a mangled path repairs itself instead of surviving every later publish; a comment with no source recorded at all is not a disagreement. Every text part of the upload form must go through `writeTextField`, never `multipart.Writer.WriteField`, which emits no charset and gets decoded as Latin-1), `_links.next` pagination. **Three pagination schemes, and picking the wrong one truncates silently.** v1 *child/attachment* collections page through the generic `listV1` helper by `start`/`limit` offset, never `_links.next` (absent when the results fit one page, so it cannot terminate a loop); `ListAttachments`, `ListChildPages`, and `ListChildFolders` all go through it. v2 collections page through `listV2` by the cursor in `_links.next`, which is a `/wiki`-prefixed absolute path `resolveNext` handles unchanged; `ListContentProperties` and `SearchPagesByTitle` share it. **`/wiki/rest/api/search` is neither**: it ignores `start` outright, its `next` is context-relative so it needs the `/wiki` prefix `resolveNext` does not add, a short page does *not* mean the end, and `totalSize` can be nonzero against an empty `results` — so `searchCQL` terminates only on a missing `next` and nothing may branch on `totalSize` ([docs/confluence/search.md](docs/confluence/search.md)). `searchCQLBounded` adds a row bound under it (`SearchCQL` is that call with no bound, which is why `find` is unaffected): it asks for `max+1` and reports the surplus as `more`, since `totalSize` cannot supply a count. Full text goes through `SearchText`/`SearchRawCQL`, which return the cleaned `SearchMatch` the way `FindByTitle` returns `TitleMatch` — and **every field of a match comes from the row's `content` object**, because the row-level `title` is HTML-escaped *and* wrapped in `@@@hl@@@` markers where `content.title` is neither. The `excerpt` exists only at row level, so `cleanExcerpt` strips those markers, unescapes once, and collapses to one line — in the client, so the human and `--json` paths cannot disagree about it. `excerpt=highlight` is passed explicitly and **re-attached when following the cursor** (the `next` link carries `cql` and `limit` but not `excerpt`, and `doJSON` appends params with a bare `?`); an unrecognized value there yields an empty excerpt with a 200, so a rename by Atlassian degrades to no excerpts rather than an error. A row with no `content` object is skipped and **counted** — `type = space` answers with hundreds of them, and a silent skip would report a successful empty result. A bare v1 child row already carries `webui`, `status`, and `extensions.position`, so child listing needs no `expand`. `DownloadAttachment` goes through `send` (inheriting retry/backoff) against `_links.download`; **never** add a `CheckRedirect` that forwards headers — it would leak site credentials to Atlassian's media host, which neither needs nor wants them. `config.go` holds `Resolve` and the `.env` reader, plus the **`.env` permission warning** (#136): a `.env` reachable by anyone but its owner (`mode.Perm()&0o077`) *and* containing `CONFLUENCE_TOKEN` earns a warning naming the file, its mode, and the `chmod`. Both halves matter — a `.env` holding only the URL and username leaks nothing, and a warning that fires on a file with no secret in it is how one becomes something people scroll past. It stats rather than lstats (a link's own `0777` would cry wolf over a `0600` target), lives in `loadDotenv` because that is the one function both the discovered `.env` and `--env-file` pass through, and reaches the reader through `SetSecurityWarner` for the same reason `SetRetryLogger` exists — wired to `cmd/root.go`'s `reportSecurityWarning`, which prints it (human mode) *and* records it via `jsonout.AddWarning`, since stderr under `--json` is a schema-validated document with no room for a stray line. A group/world-*writable* `.env` with no token in it is knowingly **not** covered, though `CONFLUENCE_URL` resolves from there too and rewriting it would redirect the token: see #136. Why each of these is shaped this way, with the evidence: [docs/confluence/api.md](docs/confluence/api.md) and [attachments.md](docs/confluence/attachments.md). - `internal/convert` — the converter (the crux). `MdToConfluence(md *frontmatter.MarkdownFile, root *project.Root, index *linkindex.Index, baseURL, spaceKey, version string) (*ConfluencePage, error)`. `root` bounds which images and parent references may be read (S1/S2) and is what an image's recorded `Source` is relative to; `index` is the tree-wide link/anchor index for `root` (`internal/linkindex.Build`), built once and shared across every file converted under it rather than rebuilt per conversion — both are discovered/built by the caller (`internal/project`/`internal/linkindex`), which is why this package stays client-free. It parses with goldmark (GFM) and renders through a custom `storageRenderer` registered at priority 100 (below the default HTML=1000 and table=500 renderers) that emits Confluence storage format. `shield.go` renames raw `ac:`/`ri:` tags to colon-free sentinels around the goldmark step so pasted storage passes through; `callouts.go` is an AST transformer + blockquote renderer for GitHub alerts; `mention.go` owns the user-mention mapping in both directions (#91): a mention is 80% of all `` usage, and it converts to `[@Display Name](https://home.atlassian.com/people/{accountId})`. Three things decide its shape, each measured rather than reasoned. **The URL is Atlassian Home, not the site** — Confluence's own renderer still emits `{site}/wiki/people/{id}`, which no longer resolves usefully in a browser, so a mention in markdown names *no site*, needs nothing from configuration, and is therefore recognisable by `check` with no client at all. **Matching is on the path, ignoring host and query**, because several spellings of one target circulate (the Home URL, the modal's `?cloudId=` copy, the `/o/{orgId}` redirect, both Confluence forms, root-relative) and none of `cloudId`/`ref`/the org segment identifies the person — only the id does, and `ri:user` stores nothing else. **The `@` on the link text is the marker**, load-bearing rather than decoration: the URL cannot tell "mention this person" from "link to their profile", so without it anyone writing the second would silently get the first. The account id is **not** pattern-validated (two shapes are live on one instance, so a pattern tight enough for one rejects the other) and `ri:local-id` is never emitted (a mention carrying only the id resolves to the same person, verified via ADF). `ConfluencePage.Mentions` reports the ids the *forward* direction emitted so the caller can warn about one that names nobody — the `Attachments` arrangement, and necessary because Confluence accepts any id and renders `@Unlicensed user` rather than failing, and the profile URL 200s either way. An unresolvable mention still renders as a link, `[@Unlicensed user](…)` — that wording mirrors Confluence because the only ids reaching it are the ones the page labels that way: a **deactivated account resolves normally** and keeps its name (measured across every mention on a real page — 18 of them, six departed, all 200, returning e.g. `Mark Reid (Deactivated)`), so a departed colleague never takes that branch. Name resolution is `pagedoc.UserCache`, a per-run cross-page cache, and the tri-state is the part to preserve: `client.LookupUser` separates a name from `ErrNoSuchUser` from an unaskable question, `StorageOptions.UserNames` carries that as name / `""` / absent, and only a **confirmed** absence renders the placeholder. Flattening those would write a fabricated name over a real one the moment a VPN dropped mid-export, across a whole tree, into a file that then looks authoritative — which is also why the cache remembers a 404 but not a timeout (one is an answer, the other is not) and why `MentionWarnings` warns only about a confirmed absence; `aclink.go` is the *inverse* direction's one element with enough shape to need its own file — ``, which the editor writes for every internal link and `MdToConfluence` never emits, so nothing in the regression suite covers it. One rule decides its whole mapping: **convert when the markdown republishes to a link resolving to the same target, pass the storage through when it would not** — so a page link and a space link convert, while a mention and a space link convert, and an attachment link (only images are uploaded, so a relative href would be dead) and an unresolvable target stay raw, which the shield republishes byte-identical. A page target is a **title, never an id**, so `PageLinkTargets` reports what needs resolving and `StorageOptions.PageLinks` carries the answers back. An `ac:anchor` is **percent-encoded** where `confluenceSlug` output is not: decode it before matching a heading, leave it encoded inside a URL. A same-page anchor recovers its heading from the document rather than inverting the slug, which is impossible — `confluenceSlug` turns both a space and a hyphen into `-`. The survey the mapping rests on, and the `xml.HTMLAutoClose` trap that made `` crash the parser outright (#88), are in [docs/confluence/links-and-anchors.md](docs/confluence/links-and-anchors.md); Plain text used as a markdown link's text goes through `escapeLinkText` (`\`, `[`, `]`), applied to the *raw* sources only — a page title, a space key, an anchor, a display name — and via `inlineTextForLink` to a body whose every descendant is a text node. Never to already-rendered output: an `ac:link-body` holding markup has been converted to markdown already, and escaping it yields a literal `\*\*bold\*\*`. Both directions are tested, because a fix at either extreme passes one and fails the other. `attachname.go` owns the source-path→attachment-name mapping, which is now the path's **base name** and nothing else (#59/`_plans/029`): the name is the attachment's identity, so an encoded path moved the name every time the file moved and orphaned the old attachment, and the path is recorded in the comment anyway. The mapping is therefore lossy, and what the bijection used to buy is an explicit refusal — two assets in one document whose base names agree return a typed `NameCollisionError` from `MdToConfluence`, which is a *failure* and not a `Broken` entry, since nothing blocks a publish on `Broken`. `check` catches that error and reports it as `Broken` anyway, because there it is a document defect like a dead link rather than a converter failure. A stored name is never interpreted in the other direction either: `sourceFor` reads the recorded path or uses the name verbatim. What names Confluence accepts is in [docs/confluence/attachments.md](docs/confluence/attachments.md); `destination.go` owns the **other** codec, destination↔path (`decodeDestination`/`encodeDestination`), shared by images *and* doc links — a markdown destination is a URL, so decode inbound (**before** `withinRoot`, or an encoded `..%2F` slips the clamp) and encode outbound in `storage_to_md.go` (or `export` emits markdown that no longer parses, and `sourceFor`'s absolute-path refusal is undone by the next read); an undecodable destination is a literal `%` in a filename, not an error; the reasoning is in [docs/confluence/links-and-anchors.md](docs/confluence/links-and-anchors.md); `images.go` (resolution stays page-relative like GitHub; the documentation root — cwd — bounds what may be published, and an image above it is `IMAGE BROKEN`), `links.go` (GitHub/Confluence slugs, doc-link + anchor rewriting against `internal/linkindex`'s tree-wide index; `resolveDocKey` resolves a destination to the index's root-relative key and reports `escapes` — a purely lexical check on the *query* side, since the index itself needs no clamp: an escaping key can never be in it, built by walking downward from root). A doc-link target is one of four severities, #42: missing entirely or escaping root is **Broken** (`LINK BROKEN: … (not found|outside the documentation root)`) and replaces the whole `` element — tags and visible text alike — with that literal message, matching `images.go`'s precedent for a missing image (`renderLink` needs a small per-node flag, `linkBrokenText`, since goldmark still invokes a container node's renderer on the matching leaving call regardless of `WalkSkipChildren` on entering, and there is no `` to write in the broken case); existing on disk with no `page_id` yet is unchanged — a **warning**, the normal state of an unpublished tree; a `#fragment` matching no heading on an otherwise-resolving target also **warns**, gated on `linkindex.Index.FileExists` so a missing/escaping target isn't double-reported. `tables.go` (the `` tag, stamped with `data-layout="align-start"` so tables auto-size and left-align — this must stay if column widths are ever emitted, or a `` silently induces a layout; plus cells: an AST transformer consumes a leading `` comment in a cell and `renderTableCell` emits it as `data-highlight-colour`; `storage_to_md.go`'s `cellTexts` reverses this, reading `data-highlight-colour` back into a `bg:` marker (`cellBGNames`, the reverse of `tables.go`'s swatch map — a hex outside the 21 swatches round-trips as the literal hex, and where two names share a hex the British spelling wins, matching Confluence's own `-colour`), and a column's GFM alignment becomes a `

` wrapper **inside** the cell — never the `align` attribute the GFM renderer would emit, which is the one form Confluence discards. Only center and right are emitted: Confluence has no explicit left, so `:---` publishes bare and `read` recovers it as `---`. Since alignment is per-paragraph there and per-column in GFM, `columnSeparators` in `storage_to_md.go` takes each column's most common declared alignment (ties to the first seen) and drops the rest. Rows still fall through to the GFM renderer. A multi-line cell is one `

` per line — Enter in the editor starts a new `

`, it does not insert a `
` — so `renderCellLines` in `storage_to_md.go` joins sibling `

` children with a literal `
` rather than nothing: a GFM table row is exactly one physical line, so a real newline isn't an option, and the same substitution catches a bare mid-line `
` (Shift+Enter) that would otherwise render as the two-space hard break valid in ordinary block content but not inside a table row. A `