Skip to content

perf: Reuse one shared context parser + document loader across all files#170

Draft
jeswr wants to merge 1 commit into
LinkedSoftwareDependencies:masterfrom
jeswr:perf/shared-context-parser
Draft

perf: Reuse one shared context parser + document loader across all files#170
jeswr wants to merge 1 commit into
LinkedSoftwareDependencies:masterfrom
jeswr:perf/shared-context-parser

Conversation

@jeswr

@jeswr jeswr commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

This is a DRAFT PR opened by @jeswr's AI performance agent. @jeswr will review, rebase, mark it ready for review, and tag the maintainers when it is ready. Please do not merge yet.

Supersedes: n/a (no prior PR for this change).

Motivation

CPU-profiling a Community Solid Server boot (bin/server.js -c config/default.json) showed that JSON-LD context parsing dominates start-up:

  • Component loading (JSON-LD parse of all .jsonld) took ~9.2 s of a ~17.5 s cold boot.
  • 52.6 % of all active on-CPU time was inside jsonld-context-parser (ContextParser.parse + its term-expansion/IRI-validation callees), plus ~10 % GC driven by the same per-file allocations.
  • Runtime instrumentation confirmed the root cause: Components.js constructs a new PrefetchedDocumentLoader per file (RdfParser), and jsonld-streaming-parser constructs a new ContextParser per ParsingContext, so the caches never survive across files. During one boot the shared componentsjs ^5.0.0 context was normalized ~979 times and the @solid/community-server ^7.0.0 context ~594 times — once per file that references them.

What this PR does

It threads one shared, (optionally) cache-bearing ContextParser — carrying a single PrefetchedDocumentLoader and a single raw-document cache — through the whole component/config load instead of building new ones per file:

  • RdfParser.createSharedContextParser(...) builds the shared parser. It attaches a normalized-context cache when the installed jsonld-context-parser exposes one (feature-detected, so it stays compatible with versions that predate the cache).
  • RdfParserOptions gains an optional contextParser, forwarded to the JSON-LD parser via the contextParser parser option (honoured by a jsonld-streaming-parser that supports injecting a context parser; ignored by older versions, in which case the per-file documentLoader fallback preserves today's behaviour exactly).
  • ComponentsManagerBuilder creates the shared parser once and hands it to both the ComponentRegistry and ConfigRegistry (and, transitively, to imported files via RdfStreamIncluder).

The change is backwards compatible and fully covered — the existing suite stays green: 865 tests, 34 suites, 100 % coverage (yarn test), and eslint is clean on the touched files.

Sibling refreshes this integrates with

The full win depends on two upstream refreshes (both driven by @jeswr); this PR is the Components.js-side integration for them:

  1. jsonld-context-parser — opt-in normalized-context cache: feat: Add an opt-in normalized-context cache rubensworks/jsonld-context-parser.js#85
  2. jsonld-streaming-parser — inject a shared context parser (contextParser option): branch feat/context-parser-parameter-refresh on @jeswr's fork (PR pending).

With those installed, createSharedContextParser automatically attaches the cache and the streaming parser reuses the shared parser. Without them, this PR is a safe no-op-to-small-win refactor.

Benchmark (before/after) and an important finding

Measured end-to-end through the real RdfParser (rdf-parse → comunica → jsonld-streaming-parser → jsonld-context-parser) with the two refreshes installed, parsing 447 documents that each reference the CSS ^7.0.0 context (= [componentsjs ^5.0.0 ctx, 525-key object]), with a distinct baseIRI per file as in a real boot (best of 6 passes):

variant ms / 447 files ms / file vs before
before (per-file loader + context parser) 2011 4.498 1.00×
shared parser, cache disabled 1909 4.272 1.05×
shared parser, cache enabled 1978 4.426 1.02×

Output quads are identical before/after.

Finding — the shared cache does not yet help a real boot, and this is the actionable result: over 447 files the normalized-context cache recorded 0 hits / 447 misses. jsonld-streaming-parser calls contextParser.parse(context, { baseIRI, parentContext: { '@base': baseIRI, … } }) with the per-file baseIRI, and #85 keys its cache on the full (context, options, parentContext) — so every file gets a distinct key. The expensive work (term expansion, IRI validation, container hashing) on the external well-known contexts is actually baseIRI-independent (external contexts drop @base), but #85 only memoises the top-level parse() result after the per-file base is applied, and routes the recursive external-context normalization around the cache.

For scale, an isolated micro-benchmark of that exact parse call shows the prize if the cache could hit: with a fixed baseIRI the shared cache is ~313× faster per file (3.4 ms → 0.011 ms); with distinct baseIRIs it is ~1× (0 hits).

Recommendation for #85: cache the normalized external context by URL (i.e., memoise the baseIRI-independent load()+normalize of a remote/prefetched context, before applyBaseEntry), rather than only the base-applied top-level result. That is what turns this Components.js plumbing into the ~50 %-of-boot-CPU win the profile points at. (I also verified that pre-normalizing and serving the contexts from the loader is not a safe shortcut: it bakes one baseIRI into the ~523 nested scoped-context @base markers, which are legitimately per-file.)

Notes for the reviewer

  • This draft branch is based on the fork's current master (v6.0.1) because the CI token in the agent's environment lacks the workflow scope needed to sync the fork up to upstream master. It merges cleanly into master; the only extra lines in the diff are pre-existing 6.0.1→master modernizations on the touched files. @jeswr can rebase onto upstream master to remove that noise before readying.
  • No dependency versions are bumped here (the code feature-detects), so nothing breaks against the currently published deps. When Prepend to list param #85 and the streaming-parser refresh are released, their minimum versions should be bumped in package.json to actually enable the win.

🤖 Generated with Claude Code

Review timing: This draft was prepared with Claude; I (@jeswr) will personally review it before it progresses. I'm currently batching a lot of work in flight, so expect active review Wednesday–Friday (8–10 July).

Components.js constructs a fresh PrefetchedDocumentLoader and (indirectly,
via jsonld-streaming-parser) a fresh ContextParser for every .jsonld file.
As a result the well-known @contexts are re-loaded and re-normalized once
per file: profiling a Community Solid Server boot showed >50% of active CPU
inside jsonld-context-parser, with the componentsjs ^5.0.0 context alone
normalized ~979 times.

This threads a single, optionally cache-bearing ContextParser (carrying one
shared PrefetchedDocumentLoader and raw-document cache) through the whole
component/config load:

- RdfParser.createSharedContextParser(...) builds the shared parser, attaching
  a normalized-context cache when the installed jsonld-context-parser exposes
  one (feature-detected, so older versions keep working).
- RdfParserOptions gains an optional `contextParser`, forwarded to the JSON-LD
  parser as the `contextParser` option (honoured by a jsonld-streaming-parser
  that supports injecting a context parser; ignored otherwise). A per-file
  document loader is still provided as a fallback, preserving prior behaviour.
- ComponentsManagerBuilder creates the shared parser once and hands it to both
  the ComponentRegistry and ConfigRegistry (and, transitively, to imported
  files via RdfStreamIncluder).

The change is backwards compatible and fully covered; the existing suite stays
green (865 tests, 100% coverage).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant