explorer: spike a progressive heatmap layer as filter-honest alternative to cluster mode #233

Idea

Add a third visualization mode alongside cluster (H3 dots) and point (individual samples): a progressively-refined heatmap that's regenerated per filter change, giving filter-honest density visualization at any zoom.

Surfaced during the 2026-05-22 session that produced #230 / #231 / #232. The motivating problem: cluster-mode H3 dominant_source lies about facet filters — a cell with 1,200 SESAR-soil samples + 8,000 OpenContext-not-soil samples shows as a red OpenContext dot even when the user has filtered to material=soil. A heatmap regenerated per filter change naturally avoids this — the dots are generated from filtered data rather than pre-baked.

Why this is interesting

Three problems converge:

1. Cluster-mode filter dishonesty — see briefing for material=soil example. Cluster H3 summaries don't account for facet filters; the #facetNote is meant to apologize for this, and is bugged on URL load too.
2. Point-mode density artifacts (explorer: dense point overlap saturates to yellow, looks like Smithsonian dots #231) — at high zoom on dense sites, hundreds of translucent dots stack and saturate to spurious yellow/orange.
3. No "where's the density of my filtered data" view exists. Cluster shows where data is. Point shows individual records. Neither answers "where is the soil I'm filtering for?"

A progressive heatmap, derived per pan/zoom/filter change from a DuckDB-WASM query against the wide parquet, addresses all three.

Approach

Cesium has no built-in heatmap primitive. Three implementation paths:

For an MVP spike: (1) is the right starting point. It composes with the existing DuckDB-WASM + wide-parquet stack with minimal new infrastructure.

Progressive refinement pattern

The "progressive" part is the interesting trick — show something in <100 ms, then keep improving while the user sits still:

DuckDB-WASM TABLESAMPLE N PERCENT makes the sampling trivial. Cancellation reuses the existing facetCountsReqId / requestId patterns.

-- Pass N
SELECT latitude, longitude FROM read_parquet('${wide_url}')
  TABLESAMPLE ${pctForPassN} PERCENT
  WHERE lat BETWEEN ${s} AND ${n} AND lng BETWEEN ${w} AND ${e}
    ${sourceFilterSQL()}
    ${facetFilterSQL()}
    -- optionally: ${searchFilterSQL()} once #164 lands

What you'd get

A new toggle alongside cluster/point: "Heatmap." When active:

• Cluster's #facetNote apology goes away — heatmap IS the filter-aware density view.
• Point mode's saturation bug (explorer: dense point overlap saturates to yellow, looks like Smithsonian dots #231) becomes moot at world/regional zoom — heatmap supplants point-mode at high counts.
• A clean visual answer to "where is the soil I'm filtering for?" / "where is the pottery search matching?"
• Works as the natural "honest cluster" path (option C1 in the design briefing).

What's hard

1. Kernel sigma calibration. Screen-pixel-stable kernel vs world-meter-stable kernel give very different impressions; need to pick one (or expose as a toggle).
2. Color ramp. Avoid re-creating explorer: dense point overlap saturates to yellow, looks like Smithsonian dots #231's saturation bug — heatmap.js gradients are well-tested but need tuning per dataset density.
3. Cache invalidation. When the user changes filters, every cached heatmap for every viewport-hash is now stale. Reasonable LRU + bounded memory.
4. First-pass coldness. Even the 1% sample is a fresh parquet scan. If users pan fast, every pan triggers a fresh scan. Debounce 250-500 ms before kicking off the first pass.

Effort estimate

A custom WebGL primitive (option 3) would be a separate multi-week project, deferred until the MVP proves the value.

Out of scope (for this issue)

• Replacing cluster or point mode (heatmap is a third mode, not a replacement)
• The H3 generation pipeline (separate repo)
• Anything that requires server-side compute

Acceptance for the spike

• Toggleable "Heatmap" mode in the explorer UI.
• On moveEnd + any filter change, renders a filter-aware density heatmap of in-viewport samples.
• Visible within ~250 ms (coarse), refines for ~3 s, cached after.
• Cancels cleanly when user pans/types again.

Cross-refs

• explorer: halo crescents / disappearing dots over hilly terrain (root cause not yet identified) #185 — depth-test-distance / point-rendering quirks that motivate looking past PointPrimitives
• Explorer: unify click semantics — click opens detail card, external link is one-hop-away #226 — most recent point-mode UX work
• explorer: facet counts — remaining work after #157/#226 #230 — facet-counts remaining work (touches the same "what does a number mean" question)
• explorer: dense point overlap saturates to yellow, looks like Smithsonian dots #231 — point-overlap saturation (heatmap would supplant this rendering path at dense zoom)
• explorer: search-results '50+' label should show the real count #232 — search results "50+" (related: how do we represent the actual size of filtered data)
• The 2026-05-22 design briefing on filter semantics (also captured in ~/dev-journal/projects/isamples-facets.md)

Disposition

This is a possible-idea / spike request, not yet a commitment. File it, sit with it, decide later whether to allocate the week. Closing as "won't do" is a fine outcome if the existing cluster+point model is good enough once #231 and the #facetNote-on-URL-load bugs are fixed.