Skip to content

spec: Content Analytics Mode (persist/read-only) for issue #37521 - #37540

Open
jcastro-dotcms wants to merge 7 commits into
mainfrom
issue-37521-content-analytics-mode
Open

jcastro-dotcms wants to merge 7 commits into
mainfrom
issue-37521-content-analytics-mode

Conversation

@jcastro-dotcms

Copy link
Copy Markdown
Member

Summary

Spec for issue #37521 — replaces epic #37349's environment-based cross-instance analytics access model with a simpler, per-instance Analytics Mode dropdown (Read & Write / Read Only) in the Content Analytics app config.

This is PR 1 of 2 per the Spec-Kit flow — spec only, nothing else. Please review for: is this the right problem, is it scoped correctly, are the success criteria measurable, is anything out of scope missing.

Refs #37521

Test plan

N/A — this PR carries only spec.md, no code changes.

Refs #37521

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@claude

claude Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Claude finished @jcastro-dotcms's task in 52s —— View job


Spec Review — Content Analytics Mode (#37521)

Task list

  • Read the spec (specs/37521-content-analytics-mode/spec.md)
  • Review problem framing, scope, measurability, and open-questions
  • Address the two reviewer comments from @freddyDOTCMS
  • Post review

This PR carries only spec.md (231 lines, no code), so this is a spec-quality review — problem fit, scope, measurable criteria, and unresolved gaps — not a code review.


Overall

The spec is well-structured and unusually rigorous for a draft: prioritized user stories with independent tests, an explicit load-bearing assumption called out as such (lines 215–224), and success criteria that acknowledge the shared-dataset measurement constraint (SC-003). The scope reversal of #37349 / #37407 (required environment → optional) is stated clearly with the ClickHouse column left in place unused. Good.

A few things worth resolving before /speckit-plan.

Gaps / clarifications

  • 🟡 spec.md:111-113 — Experiment lifecycle vs. data collection. @freddyDOTCMS's comment is correct: the edge case says a running Experiment on a Read-Only instance "stops collecting new experiment result data," but it never states what happens to the experiment's status. Does the experiment stay RUNNING (just starved of new data), or is it paused/ended? These are very different outcomes for a customer looking at an experiment that appears active but silently accumulates no results. Recommend adding an explicit statement — likely "the experiment's status is unchanged; it remains running but receives no new result data" — as both an edge case and, ideally, an acceptance scenario under a requirement.

  • 🟡 spec.md:130-138 / spec.md:215-224 — Enforcement point (defense in depth). @freddyDOTCMS raises the key implementation concern: where is Read-Only actually enforced? The spec's model is instance-side gating of the single ingest hand-off, and it explicitly says "the infrastructure itself requires no new rejection logic" (lines 217–218). That's a reasonable MVP boundary, but note the tension: with no server-side/infrastructure check, Read-Only is only as strong as the assumption that every collection path routes through the instance. The spec already flags this as the load-bearing assumption and requires /speckit-plan to re-trace every ingest path — good — but consider whether the plan should also evaluate a cheap belt-and-suspenders check at the ingest/proxy endpoint (freddy's "we should check in the endpoint too"), so a future client path that bypasses the instance-side gate fails safe rather than silently leaking. At minimum, FR-004 should name which instance-side component performs the gating so the enforcement point is unambiguous going into the plan.

  • 🟡 spec.md:139-145 — FR-005 propagation window vs. FR-004 guarantee. FR-005 accepts a brief cluster-propagation window where a sibling node hasn't yet seen the mode change. During that window a node still in "Read & Write" will keep emitting events after the admin saved "Read Only." Acceptance Scenario 1 (line 39–41, "stops sending... from that point forward") reads as absolute and doesn't acknowledge this. Recommend reconciling the two — either scope the "point forward" guarantee to "per node, once the change has propagated" or explicitly note the propagation window as expected in the acceptance scenario, so it isn't later filed as a defect.

  • 🟡 spec.md:98-103 — In-flight event edge case is measurement-relevant. The edge case correctly says already-queued/in-flight events aren't recalled. SC-003 asserts "zero new analytics events" while Read-Only is active. If any batching/queueing exists on the instance, "zero" and "in-flight not recalled" can conflict at the switch boundary. Consider wording SC-003's measurement to start after the queue has drained, so the criterion is actually verifiable.

Nits

  • spec.md:124 — FR-002a is a sub-lettered requirement; consider promoting it to its own FR number for cleaner traceability in tasks.md.
  • The spec references ADR-0022 (line 205) as an example the plan "will formally consult" — fine, just confirm it exists in platform-adrs so the plan hook doesn't stall.

None of these block the spec as PR 1 — they're items to resolve in clarification or carry explicitly into /speckit-plan. The two reviewer comments (experiment status, endpoint enforcement) are the most important to fold in before sign-off.
· issue-37521-content-analytics-mode

jcastro-dotcms and others added 2 commits September 14, 2026 12:01
- FR-002a: default new (never-before-configured) instances to Read & Write too
- Edge case: define "from that point forward" for in-flight/queued events at switch time
- FR-004: scope "any analytics events" to Content Analytics ingest traffic only

Refs #37521

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- FR-004/Assumptions: name the single instance-side ingest hand-off (verified
  in EventAnalyticsProxyResource + the analytics SDK's config.server+ANALYTICS_ENDPOINT)
  as the one gate that covers both server-rendered auto-tracking and headless/SPA
  SDK traffic, since neither posts directly to the Content Analytics infrastructure.
  Removes the apparent contradiction between FR-004 and the old enforcement wording.
- Edge case + Legacy Considerations: Experiments (A/B testing) reads results via CubeJS
  against the same Content Analytics dataset (confirmed in ExperimentsAPIImpl /
  CubeJSClientFactoryImpl) - Read Only halting ingest is accepted to also halt new
  experiment result collection on that instance, not treated as a gap.

Refs #37521

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…findings

- FR-005: clarify cluster-wide propagation uses the same existing cache-invalidation
  mechanism every other Content Analytics app config field already relies on (verified
  via AppsAPIImpl.invalidateCache() / CacheLocator.getAppsCache()) — no new mechanism,
  brief propagation delay is expected and acceptable.
- Assumptions: flag the "events always route through the instance first" claim as the
  one load-bearing assumption the whole feature depends on; require /speckit-plan to
  re-trace it against the actual code rather than carry it forward unverified.
- US1: add acceptance scenario 4 exercising the headless/SPA SDK path specifically,
  not just server-rendered tracking.
- SC-002/SC-003: add concrete measurement methods; SC-003 measured instance-side since
  the shared dataset can't attribute events to a specific instance (per FR-007).

Refs #37521

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread specs/37521-content-analytics-mode/spec.md
Comment thread specs/37521-content-analytics-mode/spec.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants