Skip to content

Add ConcurrentHashtable (perf toolbox) - #11675

Open
dougqh wants to merge 36 commits into
masterfrom
feat/concurrent-hashtable
Open

Add ConcurrentHashtable (perf toolbox)#11675
dougqh wants to merge 36 commits into
masterfrom
feat/concurrent-hashtable

Conversation

@dougqh

@dougqh dougqh commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

What Does This Do?

Adds a concurrent version of Hashtable. ConcurrentHashtable (like Hashtable) is parameterized on its entry type.

Motivation

Parameterizing on entry type allows the dd-trace-java Hashtable-s to excel in use cases where regular Map-s don't fit or have high overhead including...

  • high arity keys - Hashtable-s don't require constructing a composite key to perform a lookup
  • primitive values - custom Entry type can include one or more primitive fields
  • volatile / atomic values - custom Entry type can use volatile fields, atomic updaters, etc as appropriate
  • metadata - Entry can carry extra information used to drive eviction policies, etc

Additional Notes

  • Adds ConcurrentHashtable with D1 (single-key) and D2 (composite-key) inner classes, mirroring the Hashtable API with concurrent access guarantees
  • Lock-free get / getOrCreate fast path via AtomicReferenceArray volatile bucket reads plus a volatile chain-next pointer; synchronized only on miss, with a double-checked re-read under the lock
  • Shared mechanics (sizeFor, bucketIndex, bucket, unlink, removeIf, drain, clear, forEach) are public static building blocks over a caller-owned AtomicReferenceArray — the same "static functions over a caller-owned array" shape as Hashtable (see how AggregateTable uses Hashtable). Callers that need primitive/higher-arity keys or their own lock strategy subclass Entry directly and drive the table with these; D1/D2 are the batteries-included wrappers over that spine
  • Entry-typed factories createFixedBuckets(Class<TEntry> entryClass, int capacity) on all three: ConcurrentHashtable.createFixedBuckets hands back the raw AtomicReferenceArray<TEntry> for the caller-owned path, while D1/D2.createFixedBuckets return a D1/D2 instance. The entry class anchors K/K1/K2/TEntry inference and keeps the family symmetric with Hashtable/FlatHashtable — though the AtomicReferenceArray spine is type-erased, so (unlike FlatHashtable) the class isn't consumed for allocation
  • D2.get(K1, K2) and D2.getOrCreate(K1, K2, creator) accept key parts directly — no composite key object allocated for the lookup, unlike ConcurrentHashMap<Pair<K1,K2>, V> where EA must conservatively treat the key as escaping even on hits (ownership-transfer contract)
  • Adds ConcurrentHashtableD1Test and ConcurrentHashtableD2Test (JUnit 5), including a concurrency correctness test verifying exactly one entry is created under 16 racing threads
  • Adds three JMH benchmarks (@Threads(8), all threads hitting a shared table): ThreadSafeMapD1Benchmark (D1 vs ConcurrentHashMap vs ConcurrentSkipListMap), ThreadSafeMapD2Benchmark (D2 and a raw caller-owned-array arm vs ConcurrentHashMap with a Key2 wrapper vs ConcurrentSkipListMap), and ThreadSafeMapCounterBenchmark (D1 + AtomicLongFieldUpdater inline counter vs ConcurrentHashMap + AtomicLong/LongAdder)

Test plan

  • ./gradlew :internal-api:test --tests "datadog.trace.util.ConcurrentHashtable*" — all tests pass
  • ./gradlew :internal-api:jmhCompileGeneratedClasses — benchmarks compile clean
  • Run the ThreadSafeMap* benchmarks locally to validate get/getOrCreate throughput advantage over CHM and CSLM

🤖 Generated with Claude Code

dougqh and others added 2 commits June 18, 2026 11:44
…y tables

Mirrors Hashtable's D1/D2 API with concurrent access guarantees: lock-free
get via AtomicReferenceArray volatile reads, synchronized getOrCreate with
double-checked re-read on miss. Eliminates composite key object allocation
on hot read paths — the same structural advantage Hashtable.D2 has over
HashMap<Pair<K1,K2>,V>, but thread-safe.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t class; add D2 benchmark

Extract bucketIndex and forEach into ConcurrentHashtable.Support, mirroring the
Hashtable.Support pattern. Add ConcurrentHashtableD2Benchmark comparing get and
getOrCreate throughput against ConcurrentHashMap and ConcurrentSkipListMap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
Patch Coverage: 93.75%
Overall Coverage: 58.93% (+0.12%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 81b84bf | Docs | View more details | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.97 s 13.89 s [-0.1%; +1.3%] (no difference)
startup:insecure-bank:tracing:Agent 12.92 s 12.98 s [-1.3%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 16.87 s 16.79 s [-0.3%; +1.3%] (no difference)
startup:petclinic:iast:Agent 16.95 s 17.03 s [-1.2%; +0.3%] (no difference)
startup:petclinic:profiling:Agent 16.09 s 16.84 s [-8.7%; -0.2%] (maybe better)
startup:petclinic:sca:Agent 16.73 s 16.54 s [+0.1%; +2.2%] (maybe worse)
startup:petclinic:tracing:Agent 16.34 s 16.17 s [+0.2%; +1.9%] (maybe worse)

Commit: 81b84bf7 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

… ConcurrentHashtable

Two gaps filled per-dimension (D1 and D2):
- Chain collision: force multiple entries into the same bucket (CollidingKey
  with fixed hashCode for D1; pigeonhole via 2-bucket table for D2) and verify
  all entries are reachable after concurrent inserts.
- Concurrent distinct keys: 16 threads each insert a unique key simultaneously,
  verifying final size and that every key is retrievable — exercises concurrent
  inserts to different buckets, which the single-shared-key test does not cover.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
dougqh and others added 15 commits June 22, 2026 22:13
…htable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ThreadSafeCounterBenchmarks

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…enchmark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ectly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…adSafeMapD2Benchmark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaced by the ThreadSafeMap{D1,D2,Counter}Benchmark split.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give ConcurrentHashtable its own entry hierarchy (Entry / D1.Entry / D2.Entry)
with a volatile next pointer, independent of the single-threaded Hashtable. The
volatile chain pointer lets a chain splice under the write lock be observed by
lock-free readers, which makes removal safe:

  - remove(key)        unlink a single entry
  - removeIf(predicate) sweep the whole table under one lock
  - drain(sink)        read-and-reset: remove every entry, handing each to a
                       caller-supplied accumulator (Consumer + context-passing
                       BiConsumer overload) -- the flush/publish primitive
  - clear()            empty the table

Removed entries keep their own next pointer intact so an in-flight reader can
still traverse forward. Migrates the ThreadSafeMap* benchmarks to the new
entry base. Adds single-threaded and concurrent removal tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lookups reuse the interned KEYS/SOURCE_* instances used to populate the table,
so they exercise the == identity fast path — deliberate and realistic for the
tracer (keys are typically interned tag-name constants), not an oversight.
Clarifies so it isn't misread against the equals()-path numbers elsewhere.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… shape

Flatten the nested Support class onto the ConcurrentHashtable namespace
(static fns over a caller-owned AtomicReferenceArray, mirroring FlatHashtable)
and type the bucket arrays AtomicReferenceArray<TEntry> so the unchecked casts
on the bucket read paths disappear.

- createFixedBuckets(entryClass, capacity) factories on ConcurrentHashtable
  (returns the raw spine), D1, and D2 (return a D1/D2); D1(int)/D2(int) ctors
  are now private. entryClass is a symmetry + type-inference anchor here (the
  AtomicReferenceArray spine is erased, so it isn't consumed for allocation the
  way FlatHashtable's E[] is).
- key()/key1()/key2() accessors on D1.Entry/D2.Entry to match Hashtable
  post-#12044.
- Context-passing forEach/drain overloads use <C> for the context type param.
- Double-checked-locking + lock-striping recipes moved to the class Javadoc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move D1/D2 tests and the ThreadSafeMap{Counter,D1,D2} benchmarks off the
removed public ctors / Support class onto createFixedBuckets and the flattened
ConcurrentHashtable.* static fns. The D2 benchmark's raw-array custom-entry arm
now drives a typed AtomicReferenceArray<SupportEntry>.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh

dougqh commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5829238fde

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +607 to +609
buckets.set(i, null);
for (TEntry e = head; e != null; e = e.next()) {
sink.accept(e);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep drain size consistent when sinks throw

If a caller's drain sink throws after this bucket has been cleared, the D1/D2 wrappers exit before their trailing size.set(0), leaving the drained entries unreachable while size() still reports the old count. Since drain is documented as a flush/publish primitive, a failed sink can leave subsequent logic believing stale entries remain; update the count as entries are detached or defer detaching until the sink succeeds.

Useful? React with 👍 / 👎.

@dougqh dougqh Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My decision but wording by Claude...

Documented the contract in e1b33c6 rather than changing behavior. This only bites when the sink throws mid-drain — a caller-bug path where the flush is already half-published with no rollback, so an accurate size() doesn't rescue the caller anyway. Making it exact would mean per-entry size bookkeeping in the deliberately size-agnostic static primitive, which isn't worth the cost on an error-only path. The Javadoc on both the D1/D2 drain wrappers and the static drain now states the sink must not throw.

Comment thread internal-api/src/jmh/java/datadog/trace/util/ThreadSafeMapD2Benchmark.java Outdated
@dougqh dougqh changed the title feat(util): add ConcurrentHashtable with lock-free D1/D2 composite-key tables Add ConcurrentHashtable with lock-free D1/D2 composite-key tables (perf toolbox) Jul 29, 2026
The bucket AtomicReferenceArray is the per-table write monitor, obtained
via getWriteLock(buckets) (opaque accessor, single source of truth) so
callers never hardcode what to synchronize on. Reads (bucket/forEach) stay
lock-free; whole-table mutators (removeIf/drain/clear) self-lock; the
single-slot write primitives (insertHeadEntry/unlink) are caller-locked and
assert Thread.holdsLock(getWriteLock(buckets)) under -ea.

insertHeadEntry mirrors Hashtable's insert helper so custom tables publish
entries without touching the chain pointer directly; Entry.setNext is
demoted to package-private accordingly while next() stays public for
lock-free chain walks.

Adapts ThreadSafeMapD2Benchmark call sites to the new API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
@dougqh
dougqh requested a review from a team as a code owner July 29, 2026 13:32
@dougqh
dougqh requested a review from PerfectSlayer July 29, 2026 13:32
@dd-octo-sts

dd-octo-sts Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@dd-octo-sts dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Jul 29, 2026

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The new concurrent tables preserved the existing Hashtable behavior under adversarial execution: null keys, collision-heavy chains, removal and drain, composite null parts, and 32-thread create/remove churn all produced the expected results. Full Gradle tests could not start because the wrapper requires unavailable network access and the installed Gradle requires JDK 25, so repository-level test execution remains unverified.

Was this helpful? React 👍 or 👎

📊 Validated against 4 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit 40bcc02 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@dougqh dougqh changed the title Add ConcurrentHashtable with lock-free D1/D2 composite-key tables (perf toolbox) Add ConcurrentHashtable (perf toolbox) Jul 29, 2026
dougqh and others added 5 commits July 29, 2026 10:22
The D1/D2 wrappers were well covered but the caller-owned-array path — the
static building blocks that back custom tables (primitive/higher-arity keys)
— had no direct tests. Adds ConcurrentHashtableStaticsTest, which drives a
hand-written primitive-int-key table (IntTable) through the documented
lock-free-read / locked-write recipe.

Covers sizeFor, createFixedBuckets, getWriteLock, bucketIndex, both bucket
and insertHeadEntry overloads, unlink (head/middle/tail), and the static
removeIf/drain/drain-with-context/clear/forEach primitives. Also asserts the
Thread.holdsLock guards on insertHeadEntry/unlink fire when called without
the write lock (guarded by an -ea check), plus exactly-once and lock-free
reader-safety races driven entirely through the statics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex flagged that if a drain sink throws part-way, already-detached entries
are gone while size() still reports the pre-drain count. Rather than add
per-entry size bookkeeping to a path that only matters when the caller is
already in error (a throwing sink is a half-published flush with no
rollback), document that the sink must not throw — on the D1/D2 drain
wrappers and the static drain primitive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh dougqh added comp: core Tracer core tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements labels Jul 29, 2026
dougqh and others added 2 commits July 29, 2026 12:39
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…GuardedBy

D1 and D2 are thread-safe (lock-free reads, locked writes), so mark them
@threadsafe at the type level. The hand-written mutating building blocks
insertHeadEntry and unlink require the caller to hold the table write monitor
(they already assert Thread.holdsLock(getWriteLock(buckets))); make that
precondition static/tooling-visible with @GuardedBy("getWriteLock(buckets)").

Deliberately leave the final, individually-thread-safe buckets/size fields
unannotated: reads are lock-free by design, so @GuardedBy there would
misdescribe the contract.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PerfectSlayer
PerfectSlayer requested review from a team and removed request for a team August 20, 2026 07:18

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: FAIL

The int overload reads a primitive key hash as a raw bucket index. Valid negative or large hashes can cause an array index error.

Open Bits AI session

🤖 Datadog Autotest · Commit 4a20ba8 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@Nullable
public static <TEntry extends Entry> TEntry bucket(
@Nonnull AtomicReferenceArray<TEntry> buckets, int index) {
return buckets.get(index);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Do not read int hashes as bucket indexes

Custom tables with primitive int keys can fail during valid reads or inserts.

Assertion details
  • Input: A custom primitive-key table passes an int hash to bucket or insertHeadEntry.
  • Expected: The helper must mask an int hash before it reads or writes a bucket.
  • Actual: Java selects the int overload. This overload reads the hash as a raw array index. Negative hashes and hashes that are not less than the bucket count cause IndexOutOfBoundsException.

Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

dougqh and others added 9 commits August 20, 2026 15:00
An int-typed key hash calling the overloaded bucket(buckets, hash) or
insertHeadEntry(buckets, hash, entry) binds to the int-index overload
instead of widening to long, treating the raw hash as an array index.
Split into distinct bucketAt/insertHeadEntryAt (index-based) and
bucketFor/insertHeadEntryFor (hash-based) so there's no overload to
mis-resolve.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…htable

# Conflicts:
#	internal-api/src/jmh/java/datadog/trace/util/ThreadSafeMapBenchmark.java
Mirrors the same guard added to Hashtable.insertHeadEntryAt. Here it
also catches reinserting an already-unlinked entry: unlink()
deliberately leaves next intact so in-flight lock-free readers can
keep traversing, so overwriting it via a reinsert would corrupt that
traversal.
Bundles buckets + a cursor-based SizeManager into a State<TEntry>,
threaded through D1/D2 as tryGetOrCreateOrEvict(OrNull) so callers can
cap table size and evict on overflow. Renames createFixedBuckets ->
createCapped and getOrCreate -> tryGetOrCreate(OrNull) to reflect the
capacity-aware contract. Adds unit test coverage for SizeManager's
reserve/evict/reset behavior and the D1/D2 eviction paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors Hashtable.insertReserved: splices a fully-built entry into an
already-reserved slot (from tryReserve()/tryReserveOrEvict) without
double-counting. Not used by D1/D2, whose creator is fallible and so
increments only after a successful link; documented as the contrast.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@bric3 bric3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current head. I found one blocking concurrency issue and added notes for the three SpotBugs findings currently failing check_base.


Disclaimer: it's Codex points.

public static <TEntry extends Entry> boolean tryReserveOrEvict(
@Nonnull State<TEntry> state, @Nonnull Predicate<? super TEntry> evictable) {
synchronized (getWriteLock(state)) {
return state.sizeManager.tryReserveOrEvict(state.buckets, evictable);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep reservation and insertion in one critical section

tryReserveOrEvict releases the write lock before its caller can invoke insertReserved. A concurrent clear(state) or drain(state, ...) can therefore acquire the lock in between and reset the SizeManager; the later insertReserved links an entry without restoring the count. I reproduced this with capacity 1: reserve -> clear -> insert reports size=0 with one live entry, and a second reserve/insert leaves two live entries at capacity 1. Please provide an atomic reserve-and-insert operation, or require one outer synchronized (getWriteLock(state)) spanning both operations with the entry built beforehand.

@GuardedBy("getWriteLock(buckets)")
public void reset() {
size.set(0);
cursor = 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SpotBugs reports AT_STALE_THREAD_WRITE_OF_PRIMITIVE for this cursor write. The access appears intentionally protected by the table write lock, but SpotBugs cannot model the dynamic getWriteLock(buckets) guard. If that locking contract is intentional, a targeted @SuppressFBWarnings with that justification would keep the non-volatile cursor while allowing :internal-api:spotbugsMain to pass.

for (TEntry e = buckets.get(i); e != null; e = e.next()) {
if (evictable.test(e)) {
unlink(buckets, i, prev, e);
cursor = i;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the second AT_STALE_THREAD_WRITE_OF_PRIMITIVE report on cursor. It is reached from the eviction path documented as write-lock guarded, so this looks like the same analyzer limitation rather than evidence that the field must be volatile. Please suppress it narrowly with the lock-based justification if that contract is confirmed.

}
}
}
cursor = 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reset is the third AT_STALE_THREAD_WRITE_OF_PRIMITIVE finding. As with the other cursor writes, evictAll is documented and called under the table write lock. A targeted suppression is preferable if the plain field is deliberate; otherwise check_base continues to fail on :internal-api:spotbugsMain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants