[debugger] Split method capture-expressions into parameter/local tests; enable Ruby by p-datadog · Pull Request #7309 · DataDog/system-tests

p-datadog · 2026-07-14T17:30:50Z

Motivation

Enable Ruby capture-expressions system tests, bringing Ruby DI toward parity with Java / Python / .NET / Node.js / Go. dd-trace-rb adds the feature in DataDog/dd-trace-rb#5845 (merged; ships in 2.38.0).

Changes

Enable ST for Ruby
Split method test into local and method param tests

Ruby does not implement local variable capture for method probes, therefore to have method probe test coverage for Ruby the existing method probe test was split into separate tests for params and locals. For Ruby method params and line probes are enabled, method locals is skipped.

Coverage by language

The four capture-expression tests:

Parameter — Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions (method probe captures inputValue)
Local — ...::test_method_local_capture_expressions (method probe captures localValue/testStruct)
Complex — ...::test_complex_capture_expressions (method probe: getmember/index over testStruct)
Line — Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions (line probe)

Language	Parameter	Local	Complex	Line
Java (spring-boot*)	✅ v1.38.0	✅ v1.38.0	✅ v1.38.0	✅ v1.38.0
.NET	❌ missing_feature	❌ missing_feature	❌ missing_feature	❌ missing_feature
PHP (≠ laravel11x, symfony7x)	✅ v1.19.0	✅ v1.19.0	✅ v1.19.0	❌ missing_feature
Python	❌ missing_feature	❌ missing_feature	❌ missing_feature	❌ missing_feature
Node.js (express4/-ts/5, fastify, uds-express4)	❌ missing_feature	❌ missing_feature	❌ missing_feature	✅ ~v5.87
Go	✅ v2.2.3	✅ v2.2.3	❌ index unsupported	✅ v2.2.3
Ruby (rails72/rails80/uds-rails)	✅ v2.38.0-dev	❌ locals not captured	❌ locals not captured	✅ v2.38.0-dev

CI coverage → jobs

Each capture-expression test runs in the DEBUGGER_PROBES_SNAPSHOT scenario (tests/debugger/test_debugger_capture_expressions.py), executed as a step inside the per-language End-to-end jobs. An enabled (✅) cell runs only on the weblogs listed; the other weblogs deselect it via the manifest. Jobs below are from run 30382091891.

Green cells	Enabled weblogs (manifest)	End-to-end jobs
Java — Parameter, Local, Complex, Line	spring-boot, spring-boot-jetty, spring-boot-openliberty, spring-boot-payara, spring-boot-undertow, spring-boot-wildfly, uds-spring-boot	`System Tests (java, prod) / End-to-end #1..#3 / <weblog>`
PHP — Parameter, Local, Complex	all php weblogs except laravel11x, symfony7x — apache-mod-7.0..8.2 (incl. -zts), php-fpm-7.0..8.5	`System Tests (php, dev/prod) / End-to-end #1..#2 / <weblog>`
Node.js — Line	express4, express4-typescript, express5, fastify, uds-express4	`System Tests (nodejs, dev/prod) / End-to-end #1..#2 / <weblog>`
Go — Parameter, Local, Line	chi, echo, gin, net-http, net-http-orchestrion, uds-echo	`System Tests (golang, dev/prod) / End-to-end #1..#3 / <weblog>`
Ruby — Parameter, Line	rails72, rails80, uds-rails	`System Tests (ruby, dev/prod) / End-to-end #1..#2 / <weblog>`

All ✅ tests of one language execute inside the same per-weblog job (one DEBUGGER_PROBES_SNAPSHOT run contains all four test methods). Variant coverage (dev/prod) reflects run 30382091891; Java ran prod only in that run.

github-actions · 2026-07-14T17:31:22Z

CODEOWNERS have been resolved as:

tests/debugger/utils/probes/probe_capture_expressions_method_parameter.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/probes/probe_capture_expressions_method_parameter_ruby.json  @DataDog/debugger @DataDog/system-tests-core
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
tests/debugger/test_debugger_capture_expressions.py                     @DataDog/debugger @DataDog/system-tests-core
tests/debugger/test_debugger_probe_budgets.py                           @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils.py                                                 @DataDog/debugger @DataDog/system-tests-core
utils/build/docker/ruby/rails72/config/routes.rb                        @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails80/config/routes.rb                        @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/shared/rails/app/controllers/debugger_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core

datadog-prod-us1-6 · 2026-07-14T17:31:46Z

Tests

✨ Unblock PR with BitsAI

⚠️ Warnings

🚦 16 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #2 / apache-mod-7.0-zts 2

🧪 1 Test failed

tests.test_telemetry.Test_Telemetry.test_app_heartbeats_delays[apache-mod-7.0-zts] from system_tests_suite

AssertionError: Heartbeat sent too fast: 1.4746974736842107s. It should be sent every 2s
assert 1.4746974736842107 &gt; 1.5

self = &lt;tests.test_telemetry.Test_Telemetry object at 0x7fd48d11c9e0&gt;

    @features.telemetry_heart_beat_collected
    def test_app_heartbeats_delays(self):
        &#34;&#34;&#34;Check for telemetry heartbeat are not sent too fast/slow, regarding DD_TELEMETRY_HEARTBEAT_INTERVAL
        There are a lot of reason for individual heartbeats to be sent too slow/fast, and the subsequent ones
        to be sent too fast/slow so the RFC says that it must not drift. So we will check the average delay
...

Testing the test | System Tests (php, dev) / End-to-end #2 / apache-mod-7.2 2

🧪 1 Test failed

tests.test_telemetry.Test_Telemetry.test_app_heartbeats_delays[apache-mod-7.2] from system_tests_suite

AssertionError: Heartbeat sent too fast: 1.474720052631579s. It should be sent every 2s
assert 1.474720052631579 &gt; 1.5

self = &lt;tests.test_telemetry.Test_Telemetry object at 0x7f7371fa40e0&gt;

    @features.telemetry_heart_beat_collected
    def test_app_heartbeats_delays(self):
        &#34;&#34;&#34;Check for telemetry heartbeat are not sent too fast/slow, regarding DD_TELEMETRY_HEARTBEAT_INTERVAL
        There are a lot of reason for individual heartbeats to be sent too slow/fast, and the subsequent ones
        to be sent too fast/slow so the RFC says that it must not drift. So we will check the average delay
...

Testing the test | System Tests (php, dev) / End-to-end #2 / apache-mod-7.2-zts 2

🧪 1 Test failed

tests.test_telemetry.Test_Telemetry.test_app_heartbeats_delays[apache-mod-7.2-zts] from system_tests_suite

AssertionError: Heartbeat sent too fast: 1.4743944736842105s. It should be sent every 2s
assert 1.4743944736842105 &gt; 1.5

self = &lt;tests.test_telemetry.Test_Telemetry object at 0x7fc3580a8da0&gt;

    @features.telemetry_heart_beat_collected
    def test_app_heartbeats_delays(self):
        &#34;&#34;&#34;Check for telemetry heartbeat are not sent too fast/slow, regarding DD_TELEMETRY_HEARTBEAT_INTERVAL
        There are a lot of reason for individual heartbeats to be sent too slow/fast, and the subsequent ones
        to be sent too fast/slow so the RFC says that it must not drift. So we will check the average delay
...

View all 16 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 9d9dbe9 | Docs | Datadog PR Page | Give us feedback!}

…method probes) Ruby DI does not capture local variables in method probes, so Test_Debugger_Method_Capture_Expressions (which references the expression method's locals inputValue/localValue/testStruct) cannot pass. Verified against dd-trace-rb#5845: all captures come back as NilClass/isNull and the complex index expressions are not emitted. The line-probe test captures locals at the probe line and passes, so Test_Debugger_Line_Capture_Expressions stays enabled. Mirrors the existing gating of Test_Debugger_PII_Redaction::test_pii_redaction_method_full.

Method probes in dd-trace-rb capture method arguments, not method-body locals. Add a helper method that receives inputValue as a keyword argument so a method probe can capture it via a capture expression. Covers rails72, rails80, and uds-rails (built from rails80).

Ruby method probes capture method arguments but not method-body locals, so the combined method test could not pass. Split it: - test_method_argument_capture_expressions: captures inputValue. Ruby targets the new /debugger/expression/args endpoint (inputValue is a keyword argument); other languages use /debugger/expression where inputValue is already a method parameter (Java/.NET/PHP) or local. - test_method_local_capture_expressions: captures localValue/testStruct. Enable the argument test for Ruby (rails72/rails80/uds-rails) at v2.38.0-dev; keep the local and complex tests missing_feature (local capture in method probes not implemented). Verified against dd-trace-rb#5845: argument test captures inputValue={String,testValue}.

The split method tests had a per-language conditional (Ruby used a separate /debugger/expression/args endpoint). Remove it: make Ruby's probed 'expression' method receive inputValue as a passed keyword argument via a 'run_expression' wrapper action, so a method probe captures it. Now both split tests run unconditionally on /debugger/expression for every language: - test_method_parameter_capture_expressions: asserts inputValue (a method parameter in Java/.NET/PHP; a captured local in Python/Go/Node; the passed kwarg in Ruby). - test_method_local_capture_expressions: asserts localValue/testStruct (method-body locals). Ruby method probes do not capture locals, so this stays missing_feature for Ruby. Also fixes a line-shift regression from the previous commit: inserting the args helper after line 82 pushed expression_operators and the other expression-language methods off their hardcoded lines (102/122/162/192/233) in method_and_language_to_line_number, breaking the Ruby expression-language line-probe tests. run_expression is placed in existing padding so all line-locked positions are preserved. Removes the /debugger/expression/args route and probe_capture_expressions_method_args.json. Verified against merged dd-trace-rb#5845 (datadog 2.38.0.dev, master 97cc6f6), rails72: parameter + line capture-expression tests pass; local + complex skip (missing_feature); 12 expression-language line tests pass.

datadog-prod-us1-6

More details

The split preserves the debugger endpoint response and keeps Ruby line probes anchored to the intended line 82; parameter, local, and complex capture cases are activated or skipped according to the documented Ruby capability. No diff-only behavioral regression was reproduced. No additional tests recommended: the existing scenarios already cover the changed request and capture wiring.

Was this helpful? React 👍 or 👎

_{📊 Validated against 3 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 917d0db · What is Autotest? · Any feedback? Reach out in #autotest}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 917d0db4e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

The line-probe test derived its probe line from two sources: the constant 71 baked into probe_capture_expressions_line.json, and a Ruby-only borrow of the "Expression" entry in method_and_language_to_line_number. Reusing "Expression" was correct for Ruby only by coincidence and would be wrong for Node.js (Expression=82, but the capture-expressions line probe fires at 71), which is why the borrow was guarded to `language == "ruby"`. Add a dedicated "CaptureExpressionsLine" entry with the correct per-language in-scope line (java/nodejs/golang=71, ruby=82) and have the test read from it for every language. Languages that do not run the test (python/dotnet/php, all missing_feature) fall back via `or None` to the JSON default, preserving their existing xfail setup path (including PHP's string-keyed line->method conversion). Enabled languages keep their current line values, so no behavioral change: java/nodejs/golang stay at 71, ruby stays at 82. Verified: format.sh (mypy, ruff, yamllint, shellcheck, node linters) passes. End-to-end debugger scenario not run locally.

Address review comment: the method parameter capture-expression test reused the method fixture that also requests localValue and testStruct, which Ruby method probes do not support. - Added tests/debugger/utils/probes/probe_capture_expressions_method_parameter.json with only inputValue - Updated tests/debugger/test_debugger_capture_expressions.py to use the parameter-only fixture for test_method_parameter_capture_expressions Verification: - ./format.sh passed - python3 -m json.tool tests/debugger/utils/probes/probe_capture_expressions_method_parameter.json passed - Targeted Ruby run was started but interrupted before completion Co-Authored-By: Claude <noreply@anthropic.com>

Address local verification failure: Ruby method probes expose positional arguments to capture expressions as arg1, arg2, ... rather than source parameter names. - Added a Ruby-specific method-parameter capture fixture using arg1 - Kept the generic fixture using inputValue for the other tracers - Changed the Ruby weblog helper method to pass inputValue as a positional argument Verification: - ./format.sh passed - python3 -m json.tool passed for both parameter capture fixtures - TEST_LIBRARY=ruby WEBLOG_VARIANT=rails72 ./run.sh DEBUGGER_PROBES_SNAPSHOT -F tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions passed - TEST_LIBRARY=ruby WEBLOG_VARIANT=rails72 ./run.sh DEBUGGER_PROBES_SNAPSHOT -F tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions passed - TEST_LIBRARY=ruby WEBLOG_VARIANT=rails80 ./run.sh DEBUGGER_PROBES_SNAPSHOT -F tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions passed - TEST_LIBRARY=ruby WEBLOG_VARIANT=rails80 ./run.sh DEBUGGER_PROBES_SNAPSHOT -F tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions passed - TEST_LIBRARY=ruby WEBLOG_VARIANT=uds-rails ./run.sh DEBUGGER_PROBES_SNAPSHOT -F tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions passed - TEST_LIBRARY=ruby WEBLOG_VARIANT=uds-rails ./run.sh DEBUGGER_PROBES_SNAPSHOT -F tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions tests/debugger/test_debugger_capture_expressions.py::Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions passed Co-Authored-By: Claude <noreply@anthropic.com>

The Go system-probe rejects integer `where.lines` values: an integer line number makes the probe never reach INSTALLED status. The budget _setup already stringified (`[str(n) for n in lines]`) but the capture-expressions _setup assigned the raw list[int] from method_and_language_to_line_number. On main the capture-expressions line test passed only because it used lines=None (keeping the probe JSON's string "71"). Once this branch switched setup_log_line_capture_expressions to an explicit int line list, all 12 golang End-to-end jobs failed test_log_line_capture_expressions with "Probes did not reach INSTALLED status" (other tracers tolerate int lines; golang's system-probe does not). Extract the where-rewrite into BaseDebuggerTest._apply_line_probe_where so both _setup methods share one implementation that always stringifies. Behavior is preserved for budgets (callers only pass non-None line lists). Verified: python syntax (py_compile). Not run locally: mypy/ruff (venv bootstrap unavailable) and the golang e2e scenario; CI validates both.

Copilot

Pull request overview

Enables Ruby debugger capture-expression system tests (dd-trace-rb ≥ 2.38.0) and refactors the existing method-probe capture-expression coverage by splitting it into separate method-parameter vs method-local tests to account for Ruby’s lack of method-local capture.

Changes:

Updated the Ruby Rails debugger endpoint so /debugger/expression exercises a method with an explicit argument (via a new run_expression action calling expression(inputValue)).
Split the method capture-expressions test into dedicated “parameter” and “local” tests, adding Ruby-specific probe DSL (arg1) for method-argument capture.
Added a dedicated line-number mapping (CaptureExpressionsLine) and enabled Ruby execution in the Ruby manifest for the supported Rails variants.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
utils/build/docker/ruby/shared/rails/app/controllers/debugger_controller.rb	Adds `run_expression` wrapper and changes `expression` to accept `inputValue` as an argument to support method-argument capture.
utils/build/docker/ruby/rails80/config/routes.rb	Routes `/debugger/expression` to `run_expression`.
utils/build/docker/ruby/rails72/config/routes.rb	Routes `/debugger/expression` to `run_expression`.
tests/debugger/utils/probes/probe_capture_expressions_method_parameter.json	New probe config focused on capturing `inputValue`.
tests/debugger/utils/probes/probe_capture_expressions_method_parameter_ruby.json	Ruby-specific probe capturing `arg1` but naming it `inputValue` for consistent assertions.
tests/debugger/utils.py	Adds `CaptureExpressionsLine` mapping to support language-specific line probe placement.
tests/debugger/test_debugger_capture_expressions.py	Splits method capture-expression tests into parameter/local variants and sets explicit line probe placement.
manifests/ruby.yml	Enables Ruby capture-expression tests for Rails 7.2/8.0/UDS Rails, while marking unsupported Ruby method-local/complex cases as missing_feature.

Comments suppressed due to low confidence (1)

tests/debugger/test_debugger_capture_expressions.py:222

method_and_language_to_line_number returns a list[int], but _setup is annotated as taking lines: list[str] | None (and probe JSON uses string line numbers). Passing ints works in practice but can trip static type checks and makes the where.lines representation inconsistent. Consider normalizing to list[str] at this call site.


        # Build expected captures with validation functions
        expected_captures = {}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Clearer name: the method rewrites each probe's `where` block to target the given source lines. No behavior change.

[ruby] Enable capture expressions system tests

5ba56db

p-ddsign added 5 commits July 14, 2026 13:36

[ruby] Gate capture expressions system tests at v2.38.0-dev

c66e549

p-datadog changed the title ~~[ruby] Enable capture expressions system tests~~ [debugger] Split method capture-expressions into parameter/local tests; enable Ruby Jul 14, 2026

Set tracer version to 2.38.0 for capture-expressions tests

917d0db

p-datadog marked this pull request as ready for review July 22, 2026 16:50

p-datadog requested review from a team as code owners July 22, 2026 16:50

datadog-prod-us1-6 Bot reviewed Jul 22, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 22, 2026

View reviewed changes

Comment thread tests/debugger/test_debugger_capture_expressions.py Outdated

Comment thread manifests/ruby.yml

p-ddsign and others added 3 commits July 22, 2026 14:35

jpbempel approved these changes Jul 23, 2026

View reviewed changes

nccatoni approved these changes Jul 23, 2026

View reviewed changes

Merge branch 'main' into ddsign/ruby-capture-expressions

909208f

p-datadog requested a review from Copilot July 27, 2026 20:24

Copilot started reviewing on behalf of p-datadog July 27, 2026 20:25 View session

Copilot AI reviewed Jul 27, 2026

View reviewed changes

p-ddsign and others added 2 commits July 27, 2026 16:33

[debugger] Rename line-probe where helper to _rewrite_where_for_lines

3694fee

Clearer name: the method rewrites each probe's `where` block to target the given source lines. No behavior change.

Merge branch 'main' into ddsign/ruby-capture-expressions

9d9dbe9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[debugger] Split method capture-expressions into parameter/local tests; enable Ruby - #7309

[debugger] Split method capture-expressions into parameter/local tests; enable Ruby#7309
p-datadog wants to merge 14 commits into
mainfrom
ddsign/ruby-capture-expressions

p-datadog commented Jul 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-6 Bot commented Jul 14, 2026 •

edited by datadog-datadog-us1-prod Bot

Loading

Uh oh!

datadog-prod-us1-6 Bot left a comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

p-datadog commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Coverage by language

CI coverage → jobs

Uh oh!

github-actions Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-prod-us1-6 Bot commented Jul 14, 2026 • edited by datadog-datadog-us1-prod Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

datadog-prod-us1-6 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

p-datadog commented Jul 14, 2026 •

edited

Loading

github-actions Bot commented Jul 14, 2026 •

edited

Loading

datadog-prod-us1-6 Bot commented Jul 14, 2026 •

edited by datadog-datadog-us1-prod Bot

Loading