TINYDOC-3592 - Generate the LLM files during the main build

[kemister85]

Ticket: TINYDOC-3592

Site: Not applicable — this changes the build pipeline, not documentation content. The PR preview workflow builds from the content branches and does not run this step, so the change is only observable on a main deploy.

Changes:

• Add -scripts/generate-llm-files.js to main. The build runs here, but the script previously only existed on tinymce/8, so nothing could invoke it during a deploy.
• Replace the hardcoded OUTPUT_DIR with a resolver that accepts a build directory, a sitemap file, or a remote sitemap URL. Writing into the source tree is what forced the manual commit-and-PR loop.
• Add one workflow step, Generate LLM files, before the existing Copy llms.txt files to root. The generator writes into the build's _attachments directory and the copy step propagates to the site root exactly as it does today, so both published locations keep being fed by the mechanism already in place.
• Add the generate-llm-files and generate-llm-files-from-url package scripts, and sanitize-html to devDependencies — the generator requires it and main has never needed it before.

Why:

llms.txt and llms-full.txt were only refreshed when someone ran the generator locally and merged the result. They were last regenerated on 7 May and not refreshed again until a manual hotfix on 1 September (#4331), so 8.5 through 8.9 shipped without appearing in either file. After this, every production deploy regenerates them.

Backwards compatibility:

The manual form is unchanged. A remote sitemap URL still writes to modules/ROOT/attachments for review and commit, so yarn generate-llm-files-from-url behaves exactly as it does now.

Both published locations are preserved:

• /docs/llms.txt and /docs/llms-full.txt
• /docs/tinymce/latest/_attachments/llms.txt and /docs/tinymce/latest/_attachments/llms-full.txt

The _attachments directory is discovered rather than hardcoded, so a change to the version segment will not silently break it.

Validation:

Verified against a full local Antora build — 452 pages, 431 sitemap URLs, matching the production /latest/ count.

• Generator writes into _attachments; copy step propagates to the site root.
• Root and _attachments copies are byte-identical.
• Both are byte-identical to the files committed by TINYDOC: Regenerate llms.txt files to match latest build. #4331, so the automated output reproduces the manual output exactly.
• Sibling attachments in the same directory are untouched.
• The manual URL form writes only the source tree and reproduces the committed files with no diff.
• Step ordering confirmed empirically: generating after the copy step leaves the stale file published, so the new step must precede it.

Known limitation, accepted:

Page titles are still read from production during the build, so a page new in a release falls back to a title derived from its path and corrects itself on the next deploy. main is built close to daily, so this self-heals quickly.

Follow-ups, not in scope:

• Read titles from the generated Markdown instead of production, removing the network dependency during builds.
• Emit page content into llms-full.txt rather than a list of links.
• Retire the committed copies in modules/ROOT/attachments once the build is authoritative.

---

Pre-checks:

• Build passes without console errors, warnings, or issues.
• modules/ROOT/nav.adoc has been updated — n/a, no content change.
• Files added for New product features include a release note entry — n/a.
• Major or minor version changes have updated the supported-versions.adoc table — n/a.