Skip to content

Support fully offline builds from a pre-populated deno cache#14708

Open
eejd wants to merge 21 commits into
quarto-dev:mainfrom
eejd:macports/offline-vendor-build
Open

Support fully offline builds from a pre-populated deno cache#14708
eejd wants to merge 21 commits into
quarto-dev:mainfrom
eejd:macports/offline-vendor-build

Conversation

@eejd

@eejd eejd commented Jul 19, 2026

Copy link
Copy Markdown

Summary

Two small changes needed to build quarto.js from a source tarball (no .git, no network) against a pre-populated, checksummed DENO_DIR, as required by distro packaging (we're authoring a MacPorts port — see macports-ports-local#212).

  • src/webui/quarto-preview/build.ts: buildFromGit() unconditionally triggers a network-dependent npm install && npm run build whenever git ls-files fails — which it always does when building from a release tarball rather than a git checkout, even though quarto-preview.js is already committed to the tree. This adds an explicit QUARTO_SKIP_PREVIEW_BUILD opt-out so packagers can use the shipped build instead of triggering a spurious (and network-dependent) rebuild.
  • package/scripts/vendoring/vendor.sh: the deno install calls that resolve quarto's TypeScript module graph have no way to enforce that resolution stays local to an already-populated cache. This adds QUARTO_VENDOR_LOCK (passes --lock=<path> --frozen) and QUARTO_VENDOR_OFFLINE=1 (passes --cached-only) so a cache miss fails loudly instead of silently reaching out to the network.

Both are additive, env-var-gated, and no-ops unless explicitly opted into — no change to existing default behavior.

Test plan

Verified end-to-end on macOS/arm64: a full configure.sh + quarto-bld prepare-dist run against system-provided deno/pandoc/typst/dart-sass plus from-source esbuild/deno_dom, with QUARTO_SKIP_PREVIEW_BUILD=1 QUARTO_VENDOR_OFFLINE=1 and a pre-populated DENO_DIR, completes with zero network fetches and produces a working quarto.js that renders documents correctly (quarto check all green, quarto render produces valid self-contained HTML).

  • configure.sh completes with 0 downloads
  • quarto-bld prepare-dist completes with 0 downloads
  • quarto check passes
  • quarto render hello.qmd --to html produces correct output

cderv and others added 21 commits March 27, 2026 20:25
@cderv
[backport] Fix Windows paths with accented characters breaking dart-s…
8dd9762 
…ass (quarto-dev#14274)

* fix: set UTF-8 code page in safeWindowsExec batch files (quarto-dev#14267)

safeWindowsExec writes temp .bat files as UTF-8 (Deno default), but
cmd.exe reads them using the OEM code page (e.g., CP850). Multi-byte
UTF-8 characters like é (0xC3 0xA9) get misinterpreted, breaking
paths with accented characters (e.g., C:\Users\Sébastien\).

Add `chcp 65001 >nul` to switch cmd.exe to UTF-8 before the command
line is parsed. Use CRLF line endings for correct .bat parsing.

* docs: add changelog entry for quarto-dev#14267 backport
@cderv
[backport] Fix transient .quarto_ipynb files accumulating during prev…
c829202 
…iew (quarto-dev#14288)

* Fix transient .quarto_ipynb files accumulating during preview (quarto-dev#14281)

Preview re-renders delete the fileInformationCache entry to pick up
file changes, but this loses track of the transient .quarto_ipynb path.
The collision-avoidance loop in jupyter.ts target() then sees the old
file on disk and creates numbered variants (_1, _2, etc.) that are
never cleaned up until preview exits.

Add invalidateForFile() to FileInformationCacheMap that cleans up any
transient notebook file from disk before removing the cache entry.
Replace the bare delete() call in renderForPreview() with this method.

Fixes quarto-dev#14281
@cderv
Add missing lua-types declarations for public Lua API (quarto-dev#14295)
f2b075f 
Add type declarations for functions exported in init.lua that were
missing from the lua-types annotations. This feeds into the autogen
docs effort (quarto-dev/quarto-web#1649).

quarto.doc: add_resource, add_supporting, is_filter_active
quarto.utils: type, table.isarray, table.contains, table.sortedPairs,
  resolve_path_relative_to_document, as_inlines, as_blocks,
  is_empty_node, render, match, add_to_blocks

(cherry picked from commit 6ce64dc)
@cderv
Update remaining GitHub Actions for Node.js 24 compatibility (quarto-…
e8126a5 
…dev#14294)

- EndBug/add-and-commit: v9.1.4 (node20) → v10.0.0 (node24)
- softprops/action-gh-release: v1-era (node20) → v2.6.1, remove deprecated env GITHUB_TOKEN
- julia-actions/setup-julia: @v2 (node20) → master SHA with node24 (no release yet)

Follow-up to quarto-dev#14199 which updated official actions/* and other third-party actions.

(cherry picked from commit 0ae540e)
@cderv
[backport] Fix preview browse URL for single-file documents (quarto-d…
448eea6 
…ev#14301)

[backport] Fix preview browse URL for single-file documents

Backport of quarto-dev#14300.

After quarto-dev#13804 made the project variable always non-null via
singleFileProjectContext(), single-file preview broke: the browse
URL included the output filename and GET / returned 404. Guard
both the URL path computation and handler selection with
!project.isSingleFile.

- Fix initialPath using relative path instead of empty string
- Fix handler selection using wrong default file for single files
- Add unit tests for previewInitialPath()
- Add manual preview test entries and plain.qmd fixture

Fixes quarto-dev#14298
@cderv
Fix skill about preview testing
5f5993c
@gordonwoodhull
Add changelog entry for Typst margin layout (quarto-dev#13879)
da25f51 
Also sort quarto-dev#13878 into numerical order.
@cderv
[backport] Fix quarto inspect standalone file project emission for RS…
cff3319 
…tudio

Backport from quarto-dev#14307.

Gate `project` emission in `inspectDocumentConfig` on
`!(isSingleFile && isRStudio())` so RStudio's publishing wizard
doesn't crash on standalone .qmd files with Quarto >= 1.9.35.

Ref: rstudio/rstudio#17333
@cderv
[backport] Add arm64 Linux support for chrome-headless-shell and depr…
9db6863 
…ecate chromium installer (quarto-dev#14335)

Add arm64 Linux support for `quarto install chrome-headless-shell`
using Microsoft's Playwright CDN as the download source, since Chrome
for Testing has no arm64 Linux builds. Version metadata comes from
Playwright's browsers.json, and the arm64 binary name difference
(`headless_shell` vs `chrome-headless-shell`) is abstracted by a
platform-aware helper.

Add deprecation warnings guiding users from `chromium` to
`chrome-headless-shell`:

- `quarto install chromium` and `quarto update chromium` show
  deprecation before proceeding (including on WSL)
- `quarto check install` shows migration guidance when legacy
  Chromium is detected in the installed tools list

Add CI workflow (`test-install.yml`) covering arm64 Linux and macOS
tool installation, and chromium deprecation warnings across all
platforms.

Partial backport from quarto-dev#14334.

Fixes quarto-dev#1187
@github-actions
Update version.txt
35a1f14
@cderv
[release checklist] Move release items
01754ec
@cderv
[backport] Fix tinytex install silently ignoring extraction failures (q…
e541ca7 
…uarto-dev#14360)

Backport from quarto-dev#14357.

`quarto install tinytex` silently succeeded when archive extraction
failed, leaving users with a broken TinyTeX installation. Check the
`unzip()` return value and surface a clear error message, including
an xz-utils install hint on Linux for `.tar.xz` archives. Add
`xz-utils`/`xz` to `.deb`/`.rpm` Recommends so the dependency is
available on fresh installs.

Fixes quarto-dev#14304
@cderv
[backport] Fix Dart Sass failing in enterprise Windows with Group Pol…
8b184cb 
…icy restrictions (quarto-dev#14369)

* Fall back to direct sass.bat execution when safeWindowsExec fails

Enterprise Windows environments with Group Policy / AppLocker rules
that block .bat execution from %TEMP% cause the safeWindowsExec temp
wrapper to fail, breaking dart-sass invocation (regression from quarto-dev#14002).

Try safeWindowsExec first (handles spaces in paths), then fall back to
direct execProcess call (v1.8 behavior) when it fails.

Fixes quarto-dev#14367

* Add changelog entry for quarto-dev#14367
@cderv
[backport] Bump julia-actions/cache to v3 (quarto-dev#14412)
f26f7f6 
v2 is a composite action whose post-save and delete-old-caches steps
don't run reliably, letting per-run julia caches (~250 MB each)
accumulate on refs/heads/main until the repository hits its 10 GB
cache cap. v3 is a JavaScript rewrite with a proper post hook. It also
uses Node.js 24 directly, dropping the latent Node 20 exposure from
v2's transitive actions/cache dependency.
@cderv
[backport] Cache Deno development cache in CI (quarto-dev#14452)
a8748c3 
* Cache Deno development cache in CI (quarto-dev#14408)

* Gate vendor.sh cache wipe behind QUARTO_SKIP_DENO_CACHE_WIPE

Default behavior unchanged: local ./configure.sh still wipes deno_cache
before re-vendoring. CI opts into cache preservation by setting
QUARTO_SKIP_DENO_CACHE_WIPE=1, enabling actions/cache to restore the
cache and have it survive vendor.sh.

* Gate vendor.cmd cache wipe behind QUARTO_SKIP_DENO_CACHE_WIPE

Windows counterpart to the vendor.sh change. Behavior matches:
default wipes, QUARTO_SKIP_DENO_CACHE_WIPE=1 preserves.

* Cache Deno development cache in CI

Adds actions/cache restore for ./package/dist/bin/deno_cache, keyed on
the five files that influence Deno dep resolution: configuration,
src/import_map.json, src/vendor_deps.ts, tests/test-deps.ts, and
package/scripts/deno_std/deno_std.ts.

Both configure steps set QUARTO_SKIP_DENO_CACHE_WIPE=1 so the restored
cache survives the configure.sh / configure.cmd run.

Adds dev-docs/ci-deno-caching.md explaining the cache layout, key
composition, env var contract, and how to force invalidation. Adds a
cross-link from dev-docs/upgrade-dependencies.md so a contributor
bumping deno_std version sees the cache consequences.

Expected per-OS cache size ~150-200 MB; total footprint ~600 MB across
Linux/macOS/Windows, well under GitHub's 10 GB repo quota.

(cherry picked from commit e4b75c1)

* Fix configure scripts destroying restored Deno CI cache (quarto-dev#14450)

The CI Deno dev cache (Cache Deno development cache step in
.github/workflows/actions/quarto-dev/action.yml) restores
package/dist/bin/deno_cache before configure runs. configure.sh
and configure.cmd then unconditionally wipe package/dist when
bootstrapping Deno, destroying the just-restored cache before
vendor.sh reaches it. The vendor scripts already gate their own
cache wipe with QUARTO_SKIP_DENO_CACHE_WIPE=1; the configure-time
wipe was missed.

Apply the same gate to configure.sh and configure.cmd so the
restored cache survives. Local devs (env var unset) keep the
original wipe behavior.

(cherry picked from commit 3e11b83)
@cderv
[backport] fix(preview): restore --output-dir for single-file preview…
89ff73c 
… without _quarto.yml (quarto-dev#14492)

* fix(preview): restore --output-dir for single-file preview without _quarto.yml

When `quarto preview file.qmd --output-dir <dir>` runs in a directory
without _quarto.yml, preview pre-creates a single-file ProjectContext
and passes it to render() as pContext. The synthetic-project trigger
in render-shared only fired when context was null, bypassing it for
the preview path and surfacing the project-only-flag check.

Extend the trigger to also fire when pContext is a single-file context.
Discarded single-file context cleans up via globalTempContext.

Adds a unit test that exercises the exact preview pattern.

Fixes quarto-dev#14489. Preview side of posit-dev/positron#13370.

* docs(changelog): add 1.9 entry for quarto-dev#14489 preview --output-dir fix

Pairs with the backported render-shared.ts fix in the previous commit.

* docs(changelog): move 1.9 entry under v1.10 backports section

Backport entries belong under the "v1.10 backports > In this release"
heading on the v1.9 release branch, not under "v1.9 changes > Regression
fixes" (which tracks original v1.9 fixes).
@cderv
[backport] Fix dart-sass failure on Windows usernames containing ampe…
b4c00ec 
…rsand (quarto-dev#14535)

* [backport] Bypass sass.bat on Windows to fix ampersand-in-username regression (quarto-dev#14534)

The v1.9 introduction of safeWindowsExec broke dart-sass for users whose
Windows profile path contains `&` (e.g., C:\Users\Tom & Jerry\). The
ampersand is a cmd.exe command separator, so the temp .bat wrapper that
safeWindowsExec writes is split mid-path and fails. The existing quarto-dev#14367
fallback to direct sass.bat execution does not help: Windows still spawns
cmd.exe internally to interpret any .bat file, and the ampersand splits
again.

Backport of the dart.exe + sass.snapshot direct-invocation rewrite from
PR quarto-dev#14273 (commit 4a7b6ce). Calling dart.exe through Deno.Command
bypasses cmd.exe entirely and removes the whole class of .bat issues on
Windows. The bundled dart-sass version is identical between v1.9 and main
(DARTSASS=1.87.0), so the src/dart.exe + src/sass.snapshot layout is the
same on disk.

This also subsumes the quarto-dev#14367 enterprise group-policy fallback (no .bat =
no policy block) and reinforces the quarto-dev#14267 accented-path fix already
backported via quarto-dev#14274.

Out of scope: other safeWindowsExec callers (texlive, zip, shell) still
go through cmd.exe and remain vulnerable to ampersand in paths. Those
have separate fixes upstream and are not covered by this backport.

* Add regression test for ampersand in dart-sass paths (quarto-dev#14534)

Cover the exact path pattern that broke v1.9 (e.g. C:\Users\Tom & Jerry\)
to prevent re-regression. Mirrors the existing spaced/accented tests by
junctioning the real dart-sass install dir into an ampersand-containing
parent directory and asserting compilation succeeds.

The junction itself is created via cmd /c mklink — Deno.Command passes
each arg as a separate CreateProcess argument, so the ampersand in the
target path is not interpreted by cmd.exe as a command separator.
@github-actions
Update version.txt
4df84fd
@cderv
Revert version.txt update (4df84fd)
86d05a5 
Publishing release has failed.
@github-actions
Update version.txt
6ebb5db
@eejd
Support fully offline builds from a pre-populated deno cache
778425b 
Two changes needed to build quarto.js from a source tarball (no .git,
no network) against a pre-populated, checksummed DENO_DIR, as required
by distro packaging (MacPorts, etc.):

- src/webui/quarto-preview/build.ts: buildFromGit() unconditionally
  triggers a network-dependent `npm install && npm run build` whenever
  `git ls-files` fails, which it always does when building from a
  release tarball rather than a git checkout -- even though
  quarto-preview.js is already committed to the tree. Add an explicit
  QUARTO_SKIP_PREVIEW_BUILD opt-out so packagers can use the shipped
  build instead of triggering a spurious rebuild.

- package/scripts/vendoring/vendor.sh: the `deno install` calls that
  resolve quarto's TypeScript module graph have no way to enforce that
  resolution stays local to an already-populated cache. Add
  QUARTO_VENDOR_LOCK (passes --lock=<path> --frozen) and
  QUARTO_VENDOR_OFFLINE=1 (passes --cached-only) so a cache miss fails
  loudly instead of silently reaching out to the network.

Verified end-to-end: a full `configure.sh` + `quarto-bld prepare-dist`
run against MacPorts-provided deno/pandoc/typst/dart-sass plus
from-source esbuild/deno_dom dependencies, with these two env vars set
and a pre-populated DENO_DIR, completes with zero network fetches and
produces a working quarto.js that renders documents correctly.
@posit-snyk-bot

posit-snyk-bot commented Jul 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

cderv
cderv requested changes Jul 20, 2026
View reviewed changes

@cderv cderv left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR - before we can review can you rebase you commits unto quarto-cli main branch from upstream. Looking at the number of commits in this PR, and the conflict, I believe your feature branch is not based on the correct HEAD.

Thank you

@cderv cderv added the invalid This doesn't seem right label Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cderv cderv cderv requested changes

Assignees

No one assigned

Labels

invalid This doesn't seem right

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@eejd @posit-snyk-bot @cderv @gordonwoodhull