Add execution instrumentation for Karate v2

[daniel-mohedano]

What Does This Do

• Implements execution instrumentation for the new Karate v2
• This instrumentation is responsible for all features related to modifying the execution of tests:
  • Early Flake Detection
  • Auto Test Retries
  • Flaky Test Management Policies

Motivation

Karate v2 is a complete ground-up rewrite of the framework. Because of this, the original karate-1.0 module cannot instrument it. This PR builds upon the changes introduced in #11923

Additional Notes

Most LOC are related to instrumentation tests' span fixtures.

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: SDTEST-3816

[daniel-mohedano]

@codex review

[cit-pr-commenter-54b7da]

Test Environment - sbt-scalatest

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[datadog-datadog-us1-prod]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 69.73% (+12.53%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 969df18 | Docs | Datadog PR Page | Give us feedback!}

[cit-pr-commenter-54b7da]

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-36.15	37.15	$\color{green}{\blacktriangledown}$ -73.30	36.42	$\color{green}{\blacktriangledown}$ -72.57	37/119
agentless	-36.74	36.42	$\color{green}{\blacktriangledown}$ -73.16	36.42	$\color{green}{\blacktriangledown}$ -73.16	37/119
agentlessCodeCoverage	-33.34	45.38	$\color{green}{\blacktriangledown}$ -78.72	44.48	$\color{green}{\blacktriangledown}$ -77.82	37/119
agentlessLineCoverage	-18.46	74.82	$\color{green}{\blacktriangledown}$ -93.28	74.82	$\color{green}{\blacktriangledown}$ -93.28	36/118

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - pass4s

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	22.02	21.65	$\color{red}{\blacktriangle}$ +0.37	21.65	$\color{red}{\blacktriangle}$ +0.37	38/126
agentless	20.36	18.82	$\color{red}{\blacktriangle}$ +1.54	18.82	$\color{red}{\blacktriangle}$ +1.54	36/122
agentlessCodeCoverage	20.44	19.99	$\color{red}{\blacktriangle}$ +0.45	19.99	$\color{red}{\blacktriangle}$ +0.45	36/121
agentlessLineCoverage	30.20	29.82	$\color{red}{\blacktriangle}$ +0.38	29.82	$\color{red}{\blacktriangle}$ +0.38	35/120

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	86.88	87.80	$\color{green}{\blacktriangledown}$ -0.92	87.80	$\color{green}{\blacktriangledown}$ -0.92	39/121
agentless	79.60	81.05	$\color{green}{\blacktriangledown}$ -1.45	81.05	$\color{green}{\blacktriangledown}$ -1.45	39/120
agentlessCodeCoverage	96.19	97.04	$\color{green}{\blacktriangledown}$ -0.85	95.12	$\color{red}{\blacktriangle}$ +1.07	39/118
agentlessLineCoverage	111.48	111.62	$\color{green}{\blacktriangledown}$ -0.14	111.62	$\color{green}{\blacktriangledown}$ -0.14	38/117

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-10.31	9.54	$\color{green}{\blacktriangledown}$ -19.85	9.54	$\color{green}{\blacktriangledown}$ -19.85	36/43

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d130935321

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.94 s	13.97 s	[-0.9%; +0.3%] (no difference)
startup:insecure-bank:tracing:Agent	13.00 s	13.05 s	[-1.2%; +0.4%] (no difference)
startup:petclinic:appsec:Agent	16.55 s	16.83 s	[-6.0%; +2.7%] (no difference)
startup:petclinic:iast:Agent	16.89 s	16.89 s	[-0.9%; +0.9%] (no difference)
startup:petclinic:profiling:Agent	16.51 s	16.45 s	[-4.2%; +4.9%] (no difference)
startup:petclinic:sca:Agent	16.89 s	16.68 s	[+0.2%; +2.4%] (maybe worse)
startup:petclinic:tracing:Agent	15.91 s	16.00 s	[-1.7%; +0.5%] (no difference)

Commit: 969df18e · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[cit-pr-commenter-54b7da]

Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	96.82	95.12	$\color{red}{\blacktriangle}$ +1.70	93.23	$\color{red}{\blacktriangle}$ +3.59	42/129
agentless	91.13	89.58	$\color{red}{\blacktriangle}$ +1.55	89.58	$\color{red}{\blacktriangle}$ +1.55	41/126
agentlessCodeCoverage	100.92	99.00	$\color{red}{\blacktriangle}$ +1.92	99.00	$\color{red}{\blacktriangle}$ +1.92	41/124
agentlessLineCoverage	100.87	99.00	$\color{red}{\blacktriangle}$ +1.87	99.00	$\color{red}{\blacktriangle}$ +1.87	39/122

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	21.48	19.20	$\color{red}{\blacktriangle}$ +2.28	19.20	$\color{red}{\blacktriangle}$ +2.28	40/126
agentless	18.48	18.82	$\color{green}{\blacktriangledown}$ -0.34	18.82	$\color{green}{\blacktriangledown}$ -0.34	38/124
agentlessCodeCoverage	21.63	22.54	$\color{green}{\blacktriangledown}$ -0.91	22.09	$\color{green}{\blacktriangledown}$ -0.46	38/122
agentlessLineCoverage	45.01	44.48	$\color{red}{\blacktriangle}$ +0.53	44.48	$\color{red}{\blacktriangle}$ +0.53	38/127

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-5.18	16.04	$\color{green}{\blacktriangledown}$ -21.22	16.04	$\color{green}{\blacktriangledown}$ -21.22	35/115
agentless	-10.41	9.73	$\color{green}{\blacktriangledown}$ -20.14	9.73	$\color{green}{\blacktriangledown}$ -20.14	35/116
agentlessCodeCoverage	-8.25	13.67	$\color{green}{\blacktriangledown}$ -21.92	13.40	$\color{green}{\blacktriangledown}$ -21.65	36/115
agentlessLineCoverage	8.14	32.95	$\color{green}{\blacktriangledown}$ -24.81	32.95	$\color{green}{\blacktriangledown}$ -24.81	34/114

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-1.75	9.16	$\color{green}{\blacktriangledown}$ -10.91	13.13	$\color{green}{\blacktriangledown}$ -14.88	37/124
agentless	0.76	12.12	$\color{green}{\blacktriangledown}$ -11.36	17.03	$\color{green}{\blacktriangledown}$ -16.27	37/123
agentlessCodeCoverage	93.33	77.88	$\color{red}{\blacktriangle}$ +15.45	86.07	$\color{red}{\blacktriangle}$ +7.26	37/123
agentlessLineCoverage	113.17	130.99	$\color{green}{\blacktriangledown}$ -17.82	136.34	$\color{green}{\blacktriangledown}$ -23.17	36/122

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d130935321

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[datadog-datadog-us1-prod]

More details

All critical Karate v2 API contracts verified against the actual karate-core-2.0.9.jar: ScenarioResult.scenario is private final Scenario (ByteBuddy @Advice.FieldValue access works), StepResult.skipped(Step, long) exists, ScenarioRuntime.getScenario()/getFeatureRuntime() are public, and FeatureResult.addScenarioResult() receives the final retry result via the @Advice.Return(readOnly=false) override. The retry loop's CallDepthThreadLocalMap correctly prevents recursive re-entry while allowing called-scenario advice to pass through harmlessly. No behavioral regressions found.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit d130935 · What is Autotest? · Any feedback? Reach out in #autotest}

[cit-pr-commenter-54b7da]

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-36.94	13.13	$\color{green}{\blacktriangledown}$ -50.07	12.87	$\color{green}{\blacktriangledown}$ -49.81	31/115
agentless	-37.70	11.88	$\color{green}{\blacktriangledown}$ -49.58	12.12	$\color{green}{\blacktriangledown}$ -49.82	31/114
agentlessCodeCoverage	-35.97	15.41	$\color{green}{\blacktriangledown}$ -51.38	15.11	$\color{green}{\blacktriangledown}$ -51.08	31/114
agentlessLineCoverage	-33.06	19.20	$\color{green}{\blacktriangledown}$ -52.26	19.20	$\color{green}{\blacktriangledown}$ -52.26	31/115

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[daniel-mohedano]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-07-15 10:19:53 UTC ℹ️ Start processing command /merge

---

2026-07-15 10:20:04 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

---

2026-07-15 11:35:09 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in master is approximately 2h (p90).

---

2026-07-15 12:33:18 UTC ℹ️ MergeQueue: This merge request was merged