Skip to content

FFL-3439: Attribute PHP flag evaluations to the application - #4368

Draft
vjfridge wants to merge 2 commits into
masterfrom
vickie/FFL-3439/php-flagevaluation-application-identity
Draft

vjfridge wants to merge 2 commits into
masterfrom
vickie/FFL-3439/php-flagevaluation-application-identity

Conversation

@vjfridge

@vjfridge vjfridge commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

Motivation

PHP emits flagevaluation track EVP events, but its current libdatadog
dependency does not attach the application's identity. This adopts merged
libdatadog #2716 so events
identify the right application and do not combine evaluations from different
applications. Implements FFL-3439
and follows PHP #4288.

Changes and key decisions

  • Pin libdatadog to merged commit 3c17d86ddbbc2e4067b0a5680af38b5276810eb9;
    regenerate the required Cargo dependencies and C bindings.
  • Keep PHP's per-evaluation application context empty. The sidecar supplies
    the application already registered on the connection, before aggregation.
  • Attach service, environment, and application version to the batch, plus
    context.dd.service on ordinary events. Environment/version stay off the
    individual events; protected/degraded events retain their privacy rules.
  • Verify default identity and identity changes before a single flush. Earlier
    evaluations keep their original identity, even when they use the same flag.
  • Match the updated native debugger API: captured string size is now a number.
    Update PHP's assignment and the existing regression test accordingly.

Blast radius

  • No new PHP setting, worker, queue, FFI call, or per-evaluation identity copy.
    Flag results, privacy rules, exposures, and SDK-origin headers keep their
    existing behavior.
  • Compared with current PHP master, the native update also includes upstream
    debugger-size, OTLP resource-attribute, remote-config state, and regex-facade
    changes, plus the v48 release commit. The debugger change requires the small
    PHP compatibility adjustment above; this is more than an FFE-only update.
  • Linux ARM64 validation covers FFE and shared debugger, remote-config,
    telemetry, and sidecar paths. Other supported platforms remain CI coverage.

Testing

Validated the source tree committed as
6ed17113c
against PHP master b89e238e4, using separately built local SDK artifacts.

  • PHP 8.2 NTS / Linux ARM64: optimized builds pass on both branches;
    FFE/native/OpenFeature tests 34 pass, 3 skip and shared-path tests
    56 pass, 5 skip on each. Skips require other services, extensions,
    PHP versions, ZTS, or Windows.
  • System-tests: all five FFE files forced on, 55/55 pass, versus
    51/55 on master. The four baseline failures lack application attribution.
    Absent/false/true consent pass, with protected payloads checked for raw PII.
  • Stress accounting: both builds account for all 12,000 evaluations.
    Master delivered 11,955 + 45 reported drops; this branch delivered 11,846 +
    154 reported drops. This checks accounting, not equivalent performance or
    lossless delivery.
  • ffe-dogfooding: native API and OpenFeature each ready; both builds emit
    36 evaluations and 36 OTel metrics, with identical flag responses.
    All new events have the expected service; origin remains dd-trace-php /
    1.27.0. Live consent was enabled; system-tests cover absent/false consent.
  • Request reuse: eight evaluations across two persistent PHP servers retain
    the correct service/environment/version through identity changes and resets.

Frozen harnesses: system-tests 429e9e5
and ffe-dogfooding 2b662f8.

Remaining before merge

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

Update libdatadog to the FFL-3438 implementation and require application attribution in native and OpenFeature wire tests. Keep per-evaluation FFI context empty and align required inherited dependencies and generated bindings.

Co-Authored-By: Codex <noreply@openai.com>
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

Your PR has warnings. Please review the issues below.

🚦 4 Pipeline jobs failed

DDCI - DataDog/dd-trace-php | dsm-throughput

View more details · View in DDCI

DDCI - DataDog/dd-trace-php | publish docker image for system tests

View more details · View in DDCI

DataDog/apm-reliability/dd-trace-php | dsm-throughput

View more details · View in GitLab

View all 4 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 79.99% (-0.01%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6ed1711 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Benchmarks [ tracer ]

Benchmark execution time: 2026-11-06 17:59:10

Comparing candidate commit 6ed1711 in PR branch vickie/FFL-3439/php-flagevaluation-application-identity with baseline commit b89e238 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 1 performance regressions! Performance is the same for 220 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:MessagePackSerializationBench/benchMessagePackSerialization

  • 🟥 execution_time [+3.023µs; +3.877µs] or [+2.238%; +2.870%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:ComposerTelemetryBench/benchTelemetryParsing

  • unstable execution_time [-1144.901ns; +344.901ns] or [-8.006%; +2.412%]

gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/libdatadog that referenced this pull request Nov 6, 2026
…2716)

# What does this PR do?

Attach application identity to `flagevaluation` track EVP events in the sidecar, using the application already registered on the connection.

# Motivation

[PHP #4288](DataDog/dd-trace-php#4288) intentionally sends empty application context instead of repeating service, environment, and application version on every evaluation. Until the sidecar fills those values, evaluations lack application attribution and different applications can share aggregation buckets.

This implements [FFL-3438](https://datadoghq.atlassian.net/browse/FFL-3438), following the [application-identity review discussion](DataDog/dd-trace-php#4288 (comment)).

# Changes and key decisions

- When **all three batch-context fields are empty**, copy service, environment, and application version from the connection's current application **before aggregation**. Evaluations already queued for application A stay attributed to A if the connection switches to B before flushing.
- When the application has a non-empty service name, fill missing or empty `context.dd.service` on non-degraded events. Keep explicit event-level service values, and never add per-event environment/version or restore context to degraded events.
- Preserve explicitly supplied batch context, including partially populated context. If no application is registered, keep today's unattributed delivery rather than dropping the evaluation.
- Check application service, environment, and version before copying them. If any exceeds 256 characters, keep the evaluations unattributed and preserve their counts. A debug log explains the skip without printing application values.
- Submissions explicitly routed through `enqueue_actions_for_service` do not inherit the connection's application identity.
- Reuse the existing application-update ordering and reconnect replay. No new queue, worker, configuration, or IPC fields.

# Blast radius

- Behavior changes only for empty-context `flagevaluation` submissions with a registered application.
- Privacy consent, targeting-key hashing, truncation/degradation rules, and SDK-origin headers are unchanged.
- Exposures, OTel metrics, tracing, and transport/reconnection behavior are unchanged. No FFI signature or wire-layout changes.
- PHP must adopt the native release and validate the integration in [FFL-3439](https://datadoghq.atlassian.net/browse/FFL-3439) before releasing this functionality to customers.

# How to test the change?

- **Latest native validation (`2703c8ac8`):** 174 sidecar tests pass on Rust 1.91.1 / Linux ARM64; compilation, formatting, strict Clippy, and structural lint checks pass. macOS/Windows runtime coverage remains with CI.
- **Regression coverage:** application changes before flush, separate connections, request-end clearing, explicit context, pending application replay, 256/257-character boundaries including Unicode, and the service-routing exclusion. Real IPC → HTTP tests check attribution, protected/consented/degraded payloads, raw PII canaries, and unchanged SDK-origin headers.
- **C interface checks at [06e7952](06e79521e8ca196d0620e96bc21d409080a73cbd):** 220 sidecar/FFI tests passed with one existing skip, four doctests passed, and 13 C/C++ examples passed with two runner-defined skips. Generated C headers include both submission functions' attribution documentation.
- **Earlier PHP/dogfooding validation:** [PHP #4368](DataDog/dd-trace-php#4368) used native `58b7380` and PHP `e9e88caf2`, before the later native fixes. On Linux ARM64 / PHP 8.2 NTS, 23 PHPT tests passed with three environment-dependent skips; the two-worker request-reuse probe attributed 8/8 evaluations correctly. Dogfooding matched master with 36/36 successful evaluations and 36 OTel metrics, unchanged results, and correct application service tags. Live configuration enabled full consent.
- **Earlier system-tests on those same revisions:** 51/53 passed versus 48/53 on master, including all three consent modes. Remaining failures were the context-bounds fixture rejecting the service tag ([FFL-3407](https://datadoghq.atlassian.net/browse/FFL-3407)) and shared-sender backpressure during the high-volume count test on both builds. These results do not validate the latest native revision or establish equivalent stress performance.

- [ ] Before releasing the PHP functionality, update [PHP #4368](DataDog/dd-trace-php#4368) to the merged libdatadog version and rerun PHP validation, system-tests, and ffe-dogfooding ([FFL-3439](https://datadoghq.atlassian.net/browse/FFL-3439)).


[FFL-3438]: https://datadoghq.atlassian.net/browse/FFL-3438?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[FFL-3439]: https://datadoghq.atlassian.net/browse/FFL-3439?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Co-authored-by: vickie.fridge <vickie.fridge@datadoghq.com>
Pin libdatadog to merged #2716, regenerate native bindings and Cargo dependencies, and retain the application-identity regression tests. Adapt debugger capture size to the native numeric field and update its existing regression expectation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant