Skip to content

feat(stats)!: per-field cardinality limits#2211

Open
Eldolfin wants to merge 39 commits into
mainfrom
oscarld/stats-per-key-cardinality-limit
Open

feat(stats)!: per-field cardinality limits#2211
Eldolfin wants to merge 39 commits into
mainfrom
oscarld/stats-per-key-cardinality-limit

Conversation

@Eldolfin

@Eldolfin Eldolfin commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Implement per-key cardinality limits to stats computation.

Also:

  • fix overflow key peer_tags: [] -> [tracer_blocked_value]
  • fix encode_grouped_stats meta with empty value: key: -> key (breaking change ?)

Motivation

CSS spec compliance.

Additional Notes

Todo:

How to test the change?

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 3394 documentation warning(s) found

📦 libdd-data-pipeline-ffi - 1316 warning(s)

📦 libdd-data-pipeline - 1205 warning(s)

📦 libdd-trace-stats - 873 warning(s)


Updated: 2026-07-22 11:06:47 UTC | Commit: d850de6 | missing-docs job results

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jul 8, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 97.45%
Overall Coverage: 74.71% (+0.07%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 85c73e9 | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 8 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-data-pipeline-ffi - 3 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:78:1
   │
78 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   │   └── libdd-data-pipeline-ffi v37.0.0
                     │   │   ├── libdd-data-pipeline-ffi v37.0.0 (*)
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-data-pipeline-ffi v37.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── (dev) libdd-data-pipeline-ffi v37.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-common-ffi v37.0.0
                     │   │   └── libdd-data-pipeline-ffi v37.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:226:1
    │
226 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   │   └── libdd-data-pipeline-ffi v37.0.0
      │   │   ├── libdd-data-pipeline-ffi v37.0.0 (*)
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-data-pipeline-ffi v37.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── (dev) libdd-data-pipeline-ffi v37.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-common-ffi v37.0.0
      │   │   └── libdd-data-pipeline-ffi v37.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── libdd-data-pipeline-ffi v37.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:296:1
    │
296 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0
                  └── libdd-data-pipeline-ffi v37.0.0

advisories FAILED, bans ok, sources ok

📦 libdd-data-pipeline - 3 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:75:1
   │
75 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:218:1
    │
218 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:287:1
    │
287 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0

advisories FAILED, bans ok, sources ok

📦 libdd-trace-stats - 2 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:198:1
    │
198 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-07-22 11:08:01 UTC | Commit: d850de6 | dependency-check job results

@Eldolfin

Eldolfin commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from a5a4786 to 2845fa8 Compare July 8, 2026 12:50
@Eldolfin
Eldolfin changed the base branch from main to oscarld/fix-css-obfuscation-logic July 8, 2026 12:51
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from 2845fa8 to fca0f65 Compare July 8, 2026 12:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5a4786925

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-data-pipeline/src/trace_exporter/builder.rs
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 88.18 MB 88.29 MB +.12% (+112.58 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.13 MB 8.13 MB +0% (+48 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.93 MB 10.93 MB +.03% (+3.46 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 99.46 MB 99.57 MB +.11% (+112.24 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.59 MB 26.62 MB +.08% (+24.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.30 MB 191.40 MB +.05% (+104.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 1.04 GB 1.04 GB --.03% (-427.41 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.66 MB 8.67 MB +.10% (+9.00 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.36 MB 25.39 MB +.12% (+32.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 50.55 MB 50.61 MB +.10% (+54.83 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.16 MB 23.19 MB +.09% (+23.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 195.94 MB 196.12 MB +.08% (+176.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 1.03 GB 1.03 GB +.05% (+563.78 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.68 MB 6.69 MB +.15% (+10.50 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.26 MB 27.30 MB +.11% (+32.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.13 MB 48.18 MB +.11% (+54.26 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 78.64 MB 78.74 MB +.12% (+102.13 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.05 MB 9.07 MB +.17% (+16.06 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 94.14 MB 94.24 MB +.10% (+105.43 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.02 MB 11.03 MB +.12% (+14.60 KB) 🔍

@Eldolfin
Eldolfin marked this pull request as ready for review July 8, 2026 15:15
@Eldolfin
Eldolfin requested review from a team as code owners July 8, 2026 15:15

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d82ebfd172

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-data-pipeline-ffi/src/trace_exporter.rs
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from efe280f to 83043a4 Compare July 10, 2026 12:14
@pr-commenter

pr-commenter Bot commented Jul 10, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-07-22 11:15:54

Comparing candidate commit 85c73e9 in PR branch oscarld/stats-per-key-cardinality-limit with baseline commit 63ecad5 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Benchmark execution time: 2026-07-22 11:12:05

Comparing candidate commit 85c73e9 in PR branch oscarld/stats-per-key-cardinality-limit with baseline commit 63ecad5 in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 2 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:concentrator/add_spans_to_concentrator

  • 🟥 execution_time [+990.116µs; +997.265µs] or [+11.515%; +11.598%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 85c73e9 1784718237 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.283ms 56.504ms ± 0.124ms 56.479ms ± 0.066ms 56.551ms 56.740ms 56.872ms 57.073ms 1.05% 1.213 2.413 0.22% 0.009ms 1 200
trace_buffer/1_senders/10us_delay throughput 15769.168op/s 15928.229op/s ± 34.879op/s 15935.121op/s ± 18.731op/s 15952.447op/s 15973.038op/s 15987.114op/s 15990.556op/s 0.35% -1.193 2.337 0.22% 2.466op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.021ms 50.080ms ± 0.032ms 50.079ms ± 0.010ms 50.086ms 50.144ms 50.180ms 50.194ms 0.23% 1.049 2.030 0.06% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17930.602op/s 17971.420op/s ± 11.435op/s 17971.568op/s ± 3.576op/s 17977.009op/s 17988.800op/s 17990.569op/s 17992.300op/s 0.12% -1.043 2.016 0.06% 0.809op/s 1 200
trace_buffer/1_senders/no_delay execution_time 330.631µs 337.342µs ± 4.750µs 336.770µs ± 1.424µs 338.242µs 341.906µs 351.201µs 387.016µs 14.92% 6.230 59.225 1.40% 0.336µs 1 200
trace_buffer/1_senders/no_delay throughput 2325486.484op/s 2668397.901op/s ± 34746.640op/s 2672447.993op/s ± 11347.974op/s 2683476.961op/s 2704779.908op/s 2716002.716op/s 2722068.859op/s 1.86% -5.364 47.070 1.30% 2456.958op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 56.560ms 56.936ms ± 0.249ms 56.870ms ± 0.142ms 57.053ms 57.463ms 57.638ms 57.779ms 1.60% 1.057 0.664 0.44% 0.018ms 1 200
trace_buffer/2_senders/10us_delay throughput 31153.056op/s 31615.263op/s ± 137.497op/s 31651.395op/s ± 79.298op/s 31711.122op/s 31783.049op/s 31817.237op/s 31824.751op/s 0.55% -1.037 0.612 0.43% 9.723op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.020ms 50.111ms ± 0.036ms 50.108ms ± 0.019ms 50.123ms 50.185ms 50.232ms 50.282ms 0.35% 1.426 3.287 0.07% 0.003ms 1 200
trace_buffer/2_senders/1us_delay throughput 35798.077op/s 35920.236op/s ± 25.852op/s 35922.762op/s ± 13.310op/s 35938.331op/s 35948.635op/s 35963.938op/s 35985.499op/s 0.17% -1.419 3.259 0.07% 1.828op/s 1 200
trace_buffer/2_senders/no_delay execution_time 846.988µs 899.560µs ± 20.549µs 899.125µs ± 14.365µs 913.469µs 928.859µs 947.963µs 995.087µs 10.67% 0.814 3.130 2.28% 1.453µs 1 200
trace_buffer/2_senders/no_delay throughput 1808886.584op/s 2002001.298op/s ± 45044.657op/s 2001947.008op/s ± 31818.697op/s 2034128.671op/s 2077265.170op/s 2095067.107op/s 2125177.160op/s 6.16% -0.535 2.088 2.24% 3185.138op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.144ms 57.459ms ± 0.151ms 57.446ms ± 0.102ms 57.565ms 57.715ms 57.792ms 57.908ms 0.81% 0.285 -0.334 0.26% 0.011ms 1 200
trace_buffer/4_senders/10us_delay throughput 62167.092op/s 62653.371op/s ± 164.028op/s 62667.828op/s ± 111.754op/s 62763.238op/s 62902.881op/s 62953.152op/s 62999.088op/s 0.53% -0.272 -0.346 0.26% 11.599op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.106ms 50.168ms ± 0.039ms 50.159ms ± 0.020ms 50.182ms 50.251ms 50.284ms 50.330ms 0.34% 1.363 1.989 0.08% 0.003ms 1 200
trace_buffer/4_senders/1us_delay throughput 71528.419op/s 71758.500op/s ± 55.863op/s 71771.156op/s ± 27.910op/s 71795.734op/s 71823.776op/s 71838.975op/s 71847.844op/s 0.11% -1.358 1.971 0.08% 3.950op/s 1 200
trace_buffer/4_senders/no_delay execution_time 1.964ms 2.209ms ± 0.089ms 2.210ms ± 0.060ms 2.269ms 2.355ms 2.405ms 2.428ms 9.89% -0.079 -0.186 4.01% 0.006ms 1 200
trace_buffer/4_senders/no_delay throughput 1482405.697op/s 1632038.351op/s ± 66041.214op/s 1628979.554op/s ± 43790.760op/s 1674369.221op/s 1743118.476op/s 1799155.815op/s 1833250.816op/s 12.54% 0.304 0.021 4.04% 4669.819op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.547ms 57.858ms ± 0.218ms 57.797ms ± 0.112ms 57.952ms 58.286ms 58.424ms 59.095ms 2.25% 1.651 4.731 0.38% 0.015ms 1 200
trace_buffer/8_senders/10us_delay throughput 121837.346op/s 124445.083op/s ± 465.222op/s 124573.156op/s ± 241.288op/s 124775.588op/s 124964.013op/s 125080.673op/s 125115.069op/s 0.44% -1.607 4.435 0.37% 32.896op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.158ms 50.283ms ± 0.054ms 50.271ms ± 0.025ms 50.298ms 50.377ms 50.470ms 50.535ms 0.53% 1.637 3.796 0.11% 0.004ms 1 200
trace_buffer/8_senders/1us_delay throughput 142474.976op/s 143189.195op/s ± 153.220op/s 143223.596op/s ± 70.492op/s 143284.460op/s 143357.822op/s 143389.107op/s 143547.286op/s 0.23% -1.627 3.753 0.11% 10.834op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.795ms 6.518ms ± 0.141ms 6.541ms ± 0.084ms 6.614ms 6.684ms 6.731ms 6.765ms 3.42% -1.322 3.200 2.15% 0.010ms 1 200
trace_buffer/8_senders/no_delay throughput 1064269.721op/s 1105230.024op/s ± 24610.632op/s 1100671.469op/s ± 14168.195op/s 1116419.834op/s 1151267.996op/s 1178832.978op/s 1242482.335op/s 12.88% 1.570 4.621 2.22% 1740.234op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.487ms; 56.521ms] or [-0.030%; +0.030%] None None None
trace_buffer/1_senders/10us_delay throughput [15923.395op/s; 15933.063op/s] or [-0.030%; +0.030%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.075ms; 50.084ms] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/1us_delay throughput [17969.835op/s; 17973.004op/s] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/no_delay execution_time [336.684µs; 338.001µs] or [-0.195%; +0.195%] None None None
trace_buffer/1_senders/no_delay throughput [2663582.351op/s; 2673213.451op/s] or [-0.180%; +0.180%] None None None
trace_buffer/2_senders/10us_delay execution_time [56.901ms; 56.970ms] or [-0.061%; +0.061%] None None None
trace_buffer/2_senders/10us_delay throughput [31596.208op/s; 31634.319op/s] or [-0.060%; +0.060%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.106ms; 50.116ms] or [-0.010%; +0.010%] None None None
trace_buffer/2_senders/1us_delay throughput [35916.653op/s; 35923.819op/s] or [-0.010%; +0.010%] None None None
trace_buffer/2_senders/no_delay execution_time [896.712µs; 902.408µs] or [-0.317%; +0.317%] None None None
trace_buffer/2_senders/no_delay throughput [1995758.542op/s; 2008244.055op/s] or [-0.312%; +0.312%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.439ms; 57.480ms] or [-0.036%; +0.036%] None None None
trace_buffer/4_senders/10us_delay throughput [62630.638op/s; 62676.104op/s] or [-0.036%; +0.036%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.163ms; 50.174ms] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/1us_delay throughput [71750.758op/s; 71766.242op/s] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/no_delay execution_time [2.197ms; 2.222ms] or [-0.557%; +0.557%] None None None
trace_buffer/4_senders/no_delay throughput [1622885.674op/s; 1641191.028op/s] or [-0.561%; +0.561%] None None None
trace_buffer/8_senders/10us_delay execution_time [57.827ms; 57.888ms] or [-0.052%; +0.052%] None None None
trace_buffer/8_senders/10us_delay throughput [124380.608op/s; 124509.558op/s] or [-0.052%; +0.052%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.276ms; 50.291ms] or [-0.015%; +0.015%] None None None
trace_buffer/8_senders/1us_delay throughput [143167.960op/s; 143210.430op/s] or [-0.015%; +0.015%] None None None
trace_buffer/8_senders/no_delay execution_time [6.498ms; 6.537ms] or [-0.299%; +0.299%] None None None
trace_buffer/8_senders/no_delay throughput [1101819.227op/s; 1108640.821op/s] or [-0.309%; +0.309%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 85c73e9 1784718237 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.075µs 15.427µs ± 0.210µs 15.402µs ± 0.131µs 15.544µs 15.701µs 15.877µs 16.771µs 8.88% 2.273 11.618 1.36% 0.015µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.398µs; 15.456µs] or [-0.189%; +0.189%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 85c73e9 1784718237 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.567ms 9.593ms ± 0.017ms 9.589ms ± 0.011ms 9.602ms 9.622ms 9.633ms 9.682ms 0.97% 1.378 4.170 0.18% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.590ms; 9.595ms] or [-0.024%; +0.024%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 85c73e9 1784718237 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.058µs 1.075µs ± 0.005µs 1.074µs ± 0.004µs 1.080µs 1.082µs 1.083µs 1.083µs 0.84% -0.561 -0.031 0.51% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.074µs; 1.076µs] or [-0.070%; +0.070%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 63ecad5 1784664248 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.299ms 56.494ms ± 0.111ms 56.482ms ± 0.064ms 56.535ms 56.669ms 56.874ms 57.162ms 1.20% 1.940 8.036 0.20% 0.008ms 1 200
trace_buffer/1_senders/10us_delay throughput 15744.829op/s 15930.884op/s ± 31.070op/s 15934.307op/s ± 18.089op/s 15952.934op/s 15971.410op/s 15977.109op/s 15986.099op/s 0.33% -1.903 7.797 0.19% 2.197op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.019ms 50.074ms ± 0.033ms 50.074ms ± 0.013ms 50.085ms 50.140ms 50.193ms 50.197ms 0.25% 1.276 2.772 0.07% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17929.202op/s 17973.432op/s ± 11.790op/s 17973.312op/s ± 4.812op/s 17981.274op/s 17988.966op/s 17991.723op/s 17993.040op/s 0.11% -1.270 2.752 0.07% 0.834op/s 1 200
trace_buffer/1_senders/no_delay execution_time 329.538µs 336.774µs ± 5.482µs 336.022µs ± 1.579µs 337.907µs 341.362µs 346.943µs 400.679µs 19.24% 8.157 90.911 1.62% 0.388µs 1 200
trace_buffer/1_senders/no_delay throughput 2246188.104op/s 2673045.008op/s ± 38767.212op/s 2678395.755op/s ± 12646.507op/s 2690065.330op/s 2706369.717op/s 2724976.998op/s 2731099.875op/s 1.97% -6.952 72.185 1.45% 2741.256op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 56.682ms 57.028ms ± 0.197ms 57.010ms ± 0.126ms 57.141ms 57.382ms 57.591ms 57.613ms 1.06% 0.737 0.260 0.35% 0.014ms 1 200
trace_buffer/2_senders/10us_delay throughput 31242.733op/s 31563.811op/s ± 109.034op/s 31573.646op/s ± 69.914op/s 31641.758op/s 31711.409op/s 31741.238op/s 31756.037op/s 0.58% -0.719 0.223 0.34% 7.710op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.036ms 50.110ms ± 0.032ms 50.106ms ± 0.018ms 50.122ms 50.176ms 50.213ms 50.226ms 0.24% 1.108 1.224 0.06% 0.002ms 1 200
trace_buffer/2_senders/1us_delay throughput 35838.285op/s 35920.696op/s ± 23.171op/s 35924.035op/s ± 13.242op/s 35937.765op/s 35947.905op/s 35953.568op/s 35974.102op/s 0.14% -1.105 1.214 0.06% 1.638op/s 1 200
trace_buffer/2_senders/no_delay execution_time 859.520µs 905.544µs ± 17.666µs 905.827µs ± 11.879µs 917.959µs 928.170µs 936.598µs 1000.178µs 10.42% 0.438 3.355 1.95% 1.249µs 1 200
trace_buffer/2_senders/no_delay throughput 1799680.357op/s 1988503.249op/s ± 38570.496op/s 1987135.166op/s ± 26078.978op/s 2012205.213op/s 2054941.231op/s 2083796.380op/s 2094192.107op/s 5.39% -0.165 2.276 1.93% 2727.346op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.145ms 57.437ms ± 0.121ms 57.438ms ± 0.074ms 57.508ms 57.623ms 57.779ms 57.812ms 0.65% 0.097 0.198 0.21% 0.009ms 1 200
trace_buffer/4_senders/10us_delay throughput 62271.182op/s 62677.967op/s ± 132.248op/s 62676.243op/s ± 81.270op/s 62759.238op/s 62890.158op/s 62986.035op/s 62997.603op/s 0.51% -0.084 0.187 0.21% 9.351op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.090ms 50.169ms ± 0.037ms 50.162ms ± 0.015ms 50.178ms 50.247ms 50.282ms 50.410ms 0.49% 2.462 9.891 0.07% 0.003ms 1 200
trace_buffer/4_senders/1us_delay throughput 71414.406op/s 71756.881op/s ± 53.121op/s 71766.996op/s ± 20.826op/s 71786.317op/s 71813.503op/s 71823.233op/s 71870.974op/s 0.14% -2.449 9.791 0.07% 3.756op/s 1 200
trace_buffer/4_senders/no_delay execution_time 2.013ms 2.304ms ± 0.094ms 2.300ms ± 0.061ms 2.369ms 2.454ms 2.493ms 2.557ms 11.15% -0.098 -0.005 4.07% 0.007ms 1 200
trace_buffer/4_senders/no_delay throughput 1408131.175op/s 1565400.240op/s ± 64433.691op/s 1565193.364op/s ± 41192.817op/s 1603789.968op/s 1670255.972op/s 1736726.599op/s 1788557.480op/s 14.27% 0.353 0.320 4.11% 4556.150op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.671ms 57.914ms ± 0.199ms 57.846ms ± 0.098ms 58.038ms 58.292ms 58.503ms 58.537ms 1.20% 1.165 0.619 0.34% 0.014ms 1 200
trace_buffer/8_senders/10us_delay throughput 122998.870op/s 124324.146op/s ± 424.849op/s 124468.816op/s ± 211.453op/s 124640.156op/s 124775.504op/s 124819.302op/s 124847.190op/s 0.30% -1.152 0.579 0.34% 30.041op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.201ms 50.277ms ± 0.046ms 50.266ms ± 0.025ms 50.300ms 50.368ms 50.425ms 50.468ms 0.40% 1.397 2.685 0.09% 0.003ms 1 200
trace_buffer/8_senders/1us_delay throughput 142665.377op/s 143207.402op/s ± 130.395op/s 143237.082op/s ± 70.795op/s 143294.657op/s 143360.515op/s 143407.645op/s 143423.703op/s 0.13% -1.390 2.654 0.09% 9.220op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.302ms 6.329ms ± 0.176ms 6.355ms ± 0.099ms 6.459ms 6.526ms 6.569ms 6.584ms 3.61% -1.756 5.488 2.78% 0.012ms 1 200
trace_buffer/8_senders/no_delay throughput 1093568.395op/s 1138629.095op/s ± 33575.764op/s 1133016.020op/s ± 17629.388op/s 1149672.359op/s 1208409.470op/s 1237387.944op/s 1357873.033op/s 19.85% 2.197 8.852 2.94% 2374.165op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.479ms; 56.510ms] or [-0.027%; +0.027%] None None None
trace_buffer/1_senders/10us_delay throughput [15926.578op/s; 15935.190op/s] or [-0.027%; +0.027%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.069ms; 50.078ms] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/1us_delay throughput [17971.798op/s; 17975.066op/s] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/no_delay execution_time [336.014µs; 337.533µs] or [-0.226%; +0.226%] None None None
trace_buffer/1_senders/no_delay throughput [2667672.245op/s; 2678417.771op/s] or [-0.201%; +0.201%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.001ms; 57.055ms] or [-0.048%; +0.048%] None None None
trace_buffer/2_senders/10us_delay throughput [31548.700op/s; 31578.922op/s] or [-0.048%; +0.048%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.106ms; 50.115ms] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/1us_delay throughput [35917.485op/s; 35923.908op/s] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/no_delay execution_time [903.096µs; 907.992µs] or [-0.270%; +0.270%] None None None
trace_buffer/2_senders/no_delay throughput [1983157.749op/s; 1993848.749op/s] or [-0.269%; +0.269%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.420ms; 57.454ms] or [-0.029%; +0.029%] None None None
trace_buffer/4_senders/10us_delay throughput [62659.638op/s; 62696.295op/s] or [-0.029%; +0.029%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.164ms; 50.175ms] or [-0.010%; +0.010%] None None None
trace_buffer/4_senders/1us_delay throughput [71749.519op/s; 71764.243op/s] or [-0.010%; +0.010%] None None None
trace_buffer/4_senders/no_delay execution_time [2.291ms; 2.317ms] or [-0.565%; +0.565%] None None None
trace_buffer/4_senders/no_delay throughput [1556470.351op/s; 1574330.130op/s] or [-0.570%; +0.570%] None None None
trace_buffer/8_senders/10us_delay execution_time [57.886ms; 57.941ms] or [-0.048%; +0.048%] None None None
trace_buffer/8_senders/10us_delay throughput [124265.266op/s; 124383.026op/s] or [-0.047%; +0.047%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.270ms; 50.283ms] or [-0.013%; +0.013%] None None None
trace_buffer/8_senders/1us_delay throughput [143189.331op/s; 143225.474op/s] or [-0.013%; +0.013%] None None None
trace_buffer/8_senders/no_delay execution_time [6.304ms; 6.353ms] or [-0.386%; +0.386%] None None None
trace_buffer/8_senders/no_delay throughput [1133975.817op/s; 1143282.373op/s] or [-0.409%; +0.409%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 63ecad5 1784664248 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 14.880µs 15.315µs ± 0.165µs 15.307µs ± 0.115µs 15.427µs 15.613µs 15.745µs 15.858µs 3.60% 0.384 0.484 1.07% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.292µs; 15.338µs] or [-0.149%; +0.149%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 63ecad5 1784664248 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.569ms 8.599ms ± 0.020ms 8.593ms ± 0.012ms 8.609ms 8.642ms 8.657ms 8.664ms 0.82% 1.139 0.940 0.23% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.596ms; 8.602ms] or [-0.031%; +0.031%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 63ecad5 1784664248 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.070µs 1.087µs ± 0.007µs 1.089µs ± 0.004µs 1.092µs 1.098µs 1.099µs 1.100µs 0.99% -0.433 -0.589 0.63% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.087µs; 1.088µs] or [-0.088%; +0.088%] None None None

Base automatically changed from oscarld/fix-css-obfuscation-logic to main July 10, 2026 16:26
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot requested review from a team as code owners July 10, 2026 16:26
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot requested review from mabdinur and removed request for a team July 10, 2026 16:26

@VianneyRuhlmann VianneyRuhlmann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, although I think we should make sure that this is shipped in the same release as the telemetry PR

Comment thread libdd-data-pipeline-ffi/src/trace_exporter.rs
@gh-worker-ownership-write-b05516
gh-worker-ownership-write-b05516 Bot removed the request for review from a team July 21, 2026 09:55
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Jul 21, 2026
…2170)

# What does this PR do?

- Add additional metric tags to the stats aggregation key. For each configured key, the matching span meta tag is included in the key and populates `ClientGroupedStats.additional_metric_tags`.
- Limit the number of keys for `additional_metric_tags` to 4, not configurable.
- Limit the maximum length of individual additional_metric_tags values to 200 characters, not configurable. Values exceeding this are replaced with `tracer_blocked_value`.

# Motivation

- Allow Serverless Compatibility Layer to send additional metric tags on trace stats. See [RFC](https://datadoghq.atlassian.net/wiki/spaces/APM/pages/6482919540/PENDING+Span-Derived+Primary+Tags+-+V1).
- DataDog/serverless-components#140 to follow.
- https://datadoghq.atlassian.net/browse/SVLS-8787

# Additional Notes

Additional metric tags still need to be implemented in libdd-data-pipeline. Omitted from this PR since the goal of this PR is to support additional metric tags tags for Serverless without affecting other teams.

https://github.com/DataDog/libdatadog/blob/42e3296aa3fa76749caf5d96b5bbf1ff2b1913b6/libdd-data-pipeline/src/trace_exporter/stats.rs#L146


Per-field cardinality limit for additional metric tags will be implemented in a following PR, #2211.
- Limit the number of distinct aggregation entries per bucket that carry additional metric tags to 100 by default, configurable via `additional_metric_tags_max_entries`. Once exceeded, new entries have their tag values masked to `tracer_blocked_value`.

---

Implementing additional metric tags instead of span derived primary tags. See previous PR for span derived primary tags - #1815.

# How to test the change?

- Unit tests
- Deploy to Azure Functions with the following environment variables and validate that additional tags appear on trace metrics
  * `DD_TAGS: custom.primary:val,workload:azure_functions`
  * `DD_TRACE_EXPERIMENTAL_FEATURES_ENABLED: true`
  * `DD_TRACE_STATS_ADDITIONAL_TAGS: custom.primary,workload`

<img width="685" height="411" alt="Screenshot 2026-07-06 at 3 39 47 PM" src="https://github.com/user-attachments/assets/b91495b6-11aa-4051-922f-63655666c4e5" />


Co-authored-by: duncan.harvey <duncan.harvey@datadoghq.com>
@Eldolfin

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2906b45c52

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/mod.rs
@Eldolfin Eldolfin changed the title feat(stats)!: per-key cardinality limits feat(stats)!: per-field cardinality limits Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants