Add Node.js experiment setup instructions by mehulsonowal · Pull Request #39963 · DataDog/documentation

mehulsonowal · 2026-09-15T16:49:49Z

What does this PR do? What is the motivation?

Update the Agent Observability Experiments documentation to include Node.js instructions alongside Python and organize the setup content under Guides.

Add Python/Node.js selectors for setup and experiment code snippets.
Document Node.js tracer initialization, datasets, tasks, evaluators, summary evaluators, OpenTelemetry spans, and experiment runs.
Move the setup and Experiment topic pages into the Guides section.
Update navigation, links, and legacy aliases.

Merge readiness

Ready for merge

Testing

git diff --check
Local Hugo preview verified for the updated routes.

AI assistance

Additional notes

github-actions · 2026-09-15T16:56:32Z

Preview links (active after the `build_preview` check completes)

New or renamed files

https://docs-staging.datadoghq.com/docs/nodejs-experiments-setup-signed/llm_observability/guide/experiments

Removed or renamed files (these should redirect)

https://docs-staging.datadoghq.com/docs/nodejs-experiments-setup-signed/llm_observability/improve/experiments/setup

Renamed files

Modified Files

buraizu · 2026-09-15T17:03:58Z

Created DOCS-15714 for the editorial review.

domalessi

Left some feedback! Let me know when this is ready for another look.

domalessi

A bit more tweaking and then we should be good to go!

domalessi · 2026-09-25T21:19:24Z

+
+{{< tabs >}}
+{{% tab "Python" %}}
+Supply both a Datadog API key and application key. Pass `api_key` and `app_key` to `LLMObs.enable()`, or set the `DD_API_KEY` and `DD_APP_KEY` environment variables:


Suggested change

Supply both a Datadog API key and application key. Pass `api_key` and `app_key` to `LLMObs.enable()`, or set the `DD_API_KEY` and `DD_APP_KEY` environment variables:

Pass `api_key` and `app_key` to `LLMObs.enable()`, or set the `DD_API_KEY` and `DD_APP_KEY` environment variables:

domalessi · 2026-09-25T21:19:24Z

+{{% /tab %}}
+
+{{% tab "Node.js" %}}
+Set `DD_API_KEY` and `DD_APP_KEY` in your environment, then initialize `dd-trace` in your application entrypoint. The Experiments client uses these environment variables for authentication:


Suggested change

Set `DD_API_KEY` and `DD_APP_KEY` in your environment, then initialize `dd-trace` in your application entrypoint. The Experiments client uses these environment variables for authentication:

Set `DD_API_KEY` and `DD_APP_KEY` in your environment. The Experiments client uses these variables for authentication. Then choose one of the following initialization methods.

**In-code initialization**

Initialize `dd-trace` in your application entrypoint:

domalessi · 2026-09-25T21:19:24Z

+- `projectName` identifies the project that contains your datasets and experiments. If omitted, it defaults to `default-project`.
+- `mlApp` identifies the LLM application used for Agent Observability traces. Neither `mlApp` nor `service` determines the Experiments project name.
+
+For command-line setup, include both your API key and application key:


Suggested change

For command-line setup, include both your API key and application key:

**Command-line initialization**

Alternatively, initialize `dd-trace` from the command line instead of calling `require('dd-trace').init(...)` in your application. Set the configuration and authentication environment variables when starting your application:

domalessi · 2026-09-25T21:19:24Z

+
+{{< tabs >}}
+{{% tab "Python" %}}
+Set `agentless_enabled` to `False` (the default) to enable APM trace correlation:


Suggested change

Set `agentless_enabled` to `False` (the default) to enable APM trace correlation:

APM trace correlation requires `agentless_enabled=False`, which is the default:

domalessi · 2026-09-25T21:19:24Z

+{{% /tab %}}
+
+{{% tab "Node.js" %}}
+Set `agentlessEnabled` to `false` (the default) to enable APM trace correlation:


Suggested change

Set `agentlessEnabled` to `false` (the default) to enable APM trace correlation:

APM trace correlation requires `agentlessEnabled: false`, which is the default. If you used the in-code initialization example in the setup section, change `agentlessEnabled` from `true` to `false`:

domalessi · 2026-09-25T21:19:24Z

+
+Datadog supports the following evaluator return types:
+
+- **Boolean**: returns true or false


Suggested change

- **Boolean**: returns true or false

- **Boolean**: Returns true or false

domalessi · 2026-09-25T21:19:24Z

+For the class-based approach using `BaseSummaryEvaluator`, see the [Evaluation Developer Guide](/llm_observability/investigate/evaluations/evaluation_developer_guide).
+


Move this reference under the Class-based summary evaluators heading, as suggested there.

Suggested change

For the class-based approach using `BaseSummaryEvaluator`, see the [Evaluation Developer Guide](/llm_observability/investigate/evaluations/evaluation_developer_guide).

domalessi · 2026-09-25T21:19:24Z

+
+Summary evaluator functions can take a list of any non-null type as `inputs` (string, number, Boolean, object, or array); `outputs` and `expected_outputs` can be lists of any type. `evaluators_results` is a dictionary of lists of results from evaluators, keyed by the name of the evaluator function.
+
+#### Class-based summary evaluators


Suggested change

#### Class-based summary evaluators

#### Class-based summary evaluators

For details on implementing `BaseSummaryEvaluator`, see the [Evaluation Developer Guide](/llm_observability/investigate/evaluations/evaluation_developer_guide/).

domalessi · 2026-09-25T21:19:24Z

+
+{{< tabs >}}
+{{% tab "Python" %}}
+For custom tracing, use the [Python tracing decorators](/llm_observability/instrument/custom_instrumentation?tab=decorators#trace-an-llm-application). For automatic instrumentation, see the [supported Python frameworks](/llm_observability/instrument/auto_instrumentation?tab=python).


The custom_instrumentation URL redirects to the SDK page, where the trace-an-llm-application anchor no longer exists. Use reference-style links with both definitions inside this Python tab.

Suggested change

For custom tracing, use the [Python tracing decorators](/llm_observability/instrument/custom_instrumentation?tab=decorators#trace-an-llm-application). For automatic instrumentation, see the [supported Python frameworks](/llm_observability/instrument/auto_instrumentation?tab=python).

For custom tracing, use the [Python tracing decorators][2]. For automatic instrumentation, see the [supported Python frameworks][3].

[2]: /llm_observability/instrument/sdk/?tab=python#manual-instrumentation

[3]: /llm_observability/instrument/auto_instrumentation?tab=python

domalessi · 2026-09-25T21:19:24Z

+
+1. Install the Agent Observability SDK:
+
+{{< tabs >}}


Nest the setup tabs inside their numbered steps.

mehulsonowal · 2026-10-09T16:52:05Z

Superseded by #40594. The replacement PR contains the finalized documentation changes in a single GitHub-verified commit authored and signed as mehulsonowal.

docs: reorganize Agent Observability experiments docs

e8644c1

mehulsonowal requested a review from a team as a code owner September 15, 2026 16:49

Merge origin/master into docs/nodejs-experiments-setup-signed

ff95fe2

github-actions Bot added Architecture Everything related to the Doc backend Guide Content impacting a guide labels Sep 15, 2026

buraizu added the editorial review Waiting on a more in-depth review label Sep 15, 2026 — with ddtool CLI

domalessi reviewed Sep 22, 2026

View reviewed changes

docs: address experiments guide review feedback

10e978d

domalessi reviewed Sep 25, 2026

View reviewed changes

Comment thread hugo/content/en/llm_observability/guide/experiments.md Outdated

Apply suggestion from @domalessi

11ce0e9

domalessi reviewed Sep 25, 2026

View reviewed changes

Comment thread hugo/content/en/llm_observability/guide/experiments.md

Apply suggestion from @domalessi

547fb85

mehulsonowal mentioned this pull request Oct 9, 2026

[DOCS-15714] Add Node.js experiment setup instructions #40594

Open

1 task

mehulsonowal closed this Oct 9, 2026

	Supply both a Datadog API key and application key. Pass `api_key` and `app_key` to `LLMObs.enable()`, or set the `DD_API_KEY` and `DD_APP_KEY` environment variables:
	Pass `api_key` and `app_key` to `LLMObs.enable()`, or set the `DD_API_KEY` and `DD_APP_KEY` environment variables:

-Set `DD_API_KEY` and `DD_APP_KEY` in your environment, then initialize `dd-trace` in your application entrypoint. The Experiments client uses these environment variables for authentication:
+Set `DD_API_KEY` and `DD_APP_KEY` in your environment. The Experiments client uses these variables for authentication. Then choose one of the following initialization methods.
+**In-code initialization**
+Initialize `dd-trace` in your application entrypoint:

-For command-line setup, include both your API key and application key:
+**Command-line initialization**
+Alternatively, initialize `dd-trace` from the command line instead of calling `require('dd-trace').init(...)` in your application. Set the configuration and authentication environment variables when starting your application:

	Set `agentless_enabled` to `False` (the default) to enable APM trace correlation:
	APM trace correlation requires `agentless_enabled=False`, which is the default:

	Set `agentlessEnabled` to `false` (the default) to enable APM trace correlation:
	APM trace correlation requires `agentlessEnabled: false`, which is the default. If you used the in-code initialization example in the setup section, change `agentlessEnabled` from `true` to `false`:


		Datadog supports the following evaluator return types:

		- Boolean: returns true or false

		For the class-based approach using `BaseSummaryEvaluator`, see the [Evaluation Developer Guide](/llm_observability/investigate/evaluations/evaluation_developer_guide).


		Summary evaluator functions can take a list of any non-null type as `inputs` (string, number, Boolean, object, or array); `outputs` and `expected_outputs` can be lists of any type. `evaluators_results` is a dictionary of lists of results from evaluators, keyed by the name of the evaluator function.

		#### Class-based summary evaluators

-For custom tracing, use the [Python tracing decorators](/llm_observability/instrument/custom_instrumentation?tab=decorators#trace-an-llm-application). For automatic instrumentation, see the [supported Python frameworks](/llm_observability/instrument/auto_instrumentation?tab=python).
+For custom tracing, use the [Python tracing decorators][2]. For automatic instrumentation, see the [supported Python frameworks][3].
+[2]: /llm_observability/instrument/sdk/?tab=python#manual-instrumentation
+[3]: /llm_observability/instrument/auto_instrumentation?tab=python

Conversation

mehulsonowal commented Sep 15, 2026

What does this PR do? What is the motivation?

Merge readiness

Testing

AI assistance

Additional notes

Uh oh!

github-actions Bot commented Sep 15, 2026

Preview links (active after the build_preview check completes)

New or renamed files

Removed or renamed files (these should redirect)

Renamed files

Modified Files

Uh oh!

buraizu commented Sep 15, 2026

Uh oh!

domalessi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

domalessi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mehulsonowal commented Oct 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Preview links (active after the `build_preview` check completes)