Skip to content

feat: incremental building - #1211

Open
avivkeller wants to merge 1 commit into
mainfrom
feat/experimental-incremental
Open

avivkeller wants to merge 1 commit into
mainfrom
feat/experimental-incremental

Conversation

@avivkeller

Copy link
Copy Markdown
Member

This adds --experimental-incremental to doc-kit generate. With it on, a build looks at what the previous build produced and only redoes what changed: html renders only the pages whose inputs changed, json builds only the modules whose entries changed, unchanged files keep their modification times, and files whose inputs are gone get deleted.

@avivkeller
avivkeller requested a review from a team as a code owner October 31, 2026 16:15
@vercel

vercel Bot commented Oct 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
api-docs-tooling Ready Ready Preview Oct 31, 2026 4:15pm UTC

Request Review

@cloudflare-workers-and-pages

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ 48fb2ea 2026-10-31T16:14:49.171Z View logs ↗

@avivkeller avivkeller changed the title åfeat: incremental building feat: incremental building Oct 31, 2026
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.29428% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.16%. Comparing base (b8d7b77) to head (48fb2ea).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
packages/cli/bin/commands/generate.mjs 0.00% 5 Missing ⚠️
packages/core/src/incremental/manifest.mjs 95.38% 3 Missing ⚠️
packages/react/src/html/bundlers/vite.mjs 88.88% 1 Missing ⚠️
packages/react/src/html/generate.mjs 99.06% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1211      +/-   ##
==========================================
+ Coverage   93.77%   94.16%   +0.39%     
==========================================
  Files         289      296       +7     
  Lines       28813    30196    +1383     
  Branches     2913     3055     +142     
==========================================
+ Hits        27018    28433    +1415     
+ Misses       1773     1740      -33     
- Partials       22       23       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

api-links Generator

Performance estimate (single CI run)

  • Generation time: 35.3% faster (1.02 s → 660.00 ms)
  • Peak memory: 6.3% higher (282.89 MB → 300.79 MB)

json Generator

Performance estimate (single CI run)

  • Generation time: 25.2% faster (9.67 s → 7.23 s)
  • Peak memory: 3.4% higher (952.80 MB → 985.29 MB)

legacy-html Generator

Performance estimate (single CI run)

  • Generation time: 0.6% slower (16.34 s → 16.43 s)
  • Peak memory: 1.3% higher (1.50 GB → 1.52 GB)

legacy-json Generator

Performance estimate (single CI run)

  • Generation time: 7.0% faster (9.43 s → 8.77 s)
  • Peak memory: 1.8% higher (907.74 MB → 924.20 MB)

llms-txt Generator

Performance estimate (single CI run)

  • Generation time: 14.1% faster (6.18 s → 5.31 s)
  • Peak memory: 1.3% lower (934.21 MB → 921.80 MB)

orama-db Generator

Performance estimate (single CI run)

  • Generation time: 83.5% slower (5.20 s → 9.54 s)
  • Peak memory: 9.6% lower (1.11 GB → 1.01 GB)

web Generator

Performance estimate (single CI run)

  • Generation time: 24.5% faster (25.60 s → 19.33 s)
  • Peak memory: 1.6% lower (1.55 GB → 1.53 GB)

@ovflowd ovflowd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a round of review comments. But cool stuff so far ;)

Also please add benchmarks and e2e tests!

for await (const chunk of worker.stream(modules, dependencies)) {
const { fresh, stale } = incremental.partition(modules, {
/**
*

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: add an actual comment

* @param {(item: T) => string | Array<string>} [options.files]
* @returns {{ fresh: Array<T>, stale: Array<T> }}
*/
partition(items, { key, files } = {}) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of me feels this needs polishing/using fs mid this unsure how I feel about it. Should have also failsafe mechanisms.

@@ -0,0 +1,308 @@
'use strict';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parts of this file could benefit from some refactoring/cleanup.

// declared one
partition<T>(
items: Array<T>,
options?: {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dedicated type for options pretty please

*/
const holds = async (file, data) => {
try {
return (await fs.readFile(file)).equals(Buffer.from(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: await in its own line

Comment thread packages/react/src/html/index.mjs Resolved

This branch was successfully deployed

1 active deployment
Preview – api-docs-tooling — 48fb2eae Deployed Oct 31, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants