Recalibrate the UTF8 caches every N spans instead of per span

[dougqh]

What Does This Do

Reduces the CPU cost of the UTF8 caches
The caches were designed to be recalibrated periodically, but they don't need to recalibrate on every span

Motivation

Reduces the biggest CPU cost in the background threads
GenerationalUtf8Cache.recalibrate was 18.3% of background thread CPU activity

Additional Notes

From Claude...

TraceMapperV0_4 recalibrated the UTF8 caches (TAG_CACHE, VALUE_CACHE) on every span. recalibrate() is O(cacheSize) — it decays every entry's score, purges cold ones, and retunes the promotion threshold — so driven per-span it became the largest single consumer of serializer-thread CPU.

This gates it to once every RECALIBRATE_SPAN_INTERVAL (512) spans, via a span counter and a pure, unit-tested shouldRecalibrate() predicate.

Evidence

Baseline profile — Spring PetClinic multi-endpoint under JMeter, JFR execution sampling, background (dd-trace-processor) thread:

  18.3%  GenerationalUtf8Cache.recalibrate   ← #1 hotspot on the serializer thread
  15.1%  FlushingBuffer.put
  10.9%  StringLatin1.hashCode
  ...

Measured A/B — same-base build (the only diff is this commit), same workload, JFR:

dd-trace-processor recalibrate share:
  baseline (per-span):   348 / 1898 samples =  18.3%   ← #1
  change  (every 512):     3 / 1775 samples =   0.2%   ← eliminated

• recalibrate drops out of the serializer's hot methods entirely.
• The serializer thread gets ~6.5% lighter (1898 → 1775 samples) while serving ~3% more requests — more spans serialized for less CPU.
• Throughput moved +3% (3.10M → 3.20M JMeter samples), but that's directional only: one uncounterbalanced run on a roomy -Xmx1g box, which is the regime where freed background CPU is least likely to convert. On a core-bound or CPU-limited-container host, more of it should land as throughput.

Why it's safe

• Cache correctness is unaffected — recalibrate only decays/retunes; it never changes the bytes getUtf8 returns.
• Cache effectiveness is preserved — the caches adapt to shifts in tag-value cardinality on a timescale far longer than a single span, so recalibrating every 512 spans still re-tunes in a fraction of a second at real span rates. Per-span was never the intended cadence.

Impact

A background serializer-thread CPU reduction, alloc-neutral. It converts to application-thread throughput/latency under core contention (busy hosts, CPU-limited containers where the cgroup quota throttles the whole process) and protects GC/JIT headroom where cores are spare.

🤖 Generated with Claude Code

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 20.00%
• Overall Coverage: 57.15% (-0.03%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 996c42e | Docs | Datadog PR Page | Give us feedback!}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.02 s	13.92 s	[+0.1%; +1.4%] (maybe worse)
startup:insecure-bank:tracing:Agent	12.94 s	13.03 s	[-1.3%; -0.1%] (maybe better)
startup:petclinic:appsec:Agent	16.86 s	16.56 s	[+0.8%; +2.9%] (maybe worse)
startup:petclinic:iast:Agent	16.93 s	16.98 s	[-1.0%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	16.71 s	16.85 s	[-1.9%; +0.3%] (no difference)
startup:petclinic:sca:Agent	16.93 s	16.78 s	[-0.1%; +1.9%] (no difference)
startup:petclinic:tracing:Agent	16.10 s	15.60 s	[-1.2%; +7.7%] (no difference)

Commit: 996c42e6 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[datadog-official]

More details

Cache recalibration interval logic is sound. The PR gates recalibrate() calls to once per 512 spans via an atomic counter and power-of-two bitmask, reducing serializer-thread CPU from 18.3% to 0.2% without affecting cache correctness—recalibrate() only tunes efficiency, never changes the bytes returned. Thread-safe atomic increment, correct bitmask arithmetic, proper null-guard when caches are disabled, and comprehensive unit tests all validate the change.

Was this helpful? React 👍 or 👎

_{📊 Validated against 5 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit ea5f842 · What is Autotest? · Any feedback? Reach out in #autotest}