[gengnamstyle]

Changes

• Mount the host's /etc/passwd read-only at /host/etc/passwd for process collection on Agent >= 7.86.0 (including 7.86.0+ prereleases, matching other -0 minimum versions), preserving image-local accounts for IPC. Like other version-gated features, the version comes from the node Agent image override (spec.override.nodeAgent.image) or the Operator's default Agent version (also used when the override sets no image name or tag, for example only pullSecrets); older versions, tags without a full X.Y.Z version (7, latest), custom tags, and digest-only images retain /etc/passwd. Original agent implementation pr [CXP-3881] Resolve process users from host passwd datadog-agent#55789
• Only the mount path changes: process containers keep the existing passwd volume (hostPath: /etc/passwd), and security-agent/system-probe still mount it at /etc/passwd. No new passwd volume is added. GKE Autopilot will need a separately named host-passwd volume because Warden matches allowlisted volumeMounts by name; that comes with the Autopilot follow-up. Talos strips passwd and os-release-file.
• GKE Autopilot is gated: it keeps /etc/passwd, and the WorkloadAllowlist synchronizer is unchanged (default v1.0.7). Enabling Autopilot (v1.0.8 default, syncing v1.0.7 during rollout, allowlist-pin handling) moves to a follow-up PR here [CXP-3978][operator] Mount host passwd at /host/etc/passwd on GKE Autopilot #3766 once Google publishes v1.0.8 here: https://gke-ap-allowlist-review.googlesource.com/c/Datadog/+/20280
• Always mount the host's /etc/os-release read-only at /host/etc/os-release (volume os-release-file) in the core agent and process-agent, from the node Agent's default volumes, like the Helm chart. The Agent sets HOST_ETC=/host/etc whenever that directory exists (which the relocated passwd now creates) and then reads the host OS from there. Every GKE WorkloadAllowlist version (v1.0.0 to v1.0.8) already allows it, and the Windows transform strips it with the other Linux hostPaths. User-visible effects on Operator upgrade: the node Agent DaemonSet rolls once, and host metadata reports the node's OS instead of the Agent image's OS on every Agent version. More details on this in this comment here: [CXP-3978] Migrate Operator process passwd mount + mount /etc/os-release by default #3645 (comment)
• Preserve Talos/Windows restrictions, security/group mounts, and explicit environment settings. Keep shared image helpers unchanged. Add a k8s_suite e2e subtest for the new mount, and a GKE Autopilot e2e step checking that a 7.86 Agent keeps /etc/passwd.

Validation

go test ./internal/controller/datadogagent/..., go vet, gofmt, and go vet -tags=e2e on the e2e suite pass; docs regenerated.

E2e tests were updated + added as well.

Runtime check on local kind (v0.29.0, Kubernetes 1.33.1) with a placeholder API key. Before installing the Agent, a user that exists only in the node's passwd (cxp-host-user, uid 31001) starts a sleep on the node. On 7.86 the user is missing from the container's /etc/passwd, so the process check resolving it proves usernames come from /host/etc/passwd. Outputs are trimmed; the agent container never turns Ready with the placeholder key, so checks ran once the pod was Running. The Operator runs from this branch.

kind create cluster --name cxp-3881 --kubeconfig /tmp/cxp-kind-kubeconfig
KC="--kubeconfig /tmp/cxp-kind-kubeconfig --context kind-cxp-3881"
docker exec cxp-3881-control-plane useradd -u 31001 -M -s /usr/sbin/nologin cxp-host-user
docker exec -d cxp-3881-control-plane setsid su -s /bin/sh cxp-host-user -c 'exec sleep 86400'
kubectl kustomize config/crd | kubectl $KC apply --server-side --force-conflicts -f -
kubectl $KC create namespace cxp-3881-operator
kubectl $KC -n cxp-3881-operator create secret generic datadog-secret --from-literal=api-key=placeholder
KUBECONFIG=/tmp/cxp-kind-kubeconfig POD_NAMESPACE=cxp-3881-operator go run ./cmd/main.go --enable-leader-election=false --operatorMetricsEnabled=false --logEncoder=console > operator.log 2>&1 &
sed 's/TAG/7.85.0/' dda.yaml | kubectl $KC -n cxp-3881-operator apply -f -        # a
sed 's/TAG/7.86.0-rc.6/' dda.yaml | kubectl $KC -n cxp-3881-operator apply -f -   # b

dda.yaml

apiVersion: datadoghq.com/v2alpha1
kind: DatadogAgent
metadata:
  name: datadog
  namespace: cxp-3881-operator
spec:
  global:
    credentials:
      apiSecret:
        secretName: datadog-secret
        keyName: api-key
    kubelet:
      tlsVerify: false
  features:
    liveProcessCollection:
      enabled: true
  override:
    nodeAgent:
      image:
        tag: TAG

POD=$(kubectl $KC -n cxp-3881-operator get pods -l app.kubernetes.io/instance=datadog-agent -o name | head -1)
kubectl $KC -n cxp-3881-operator get ds datadog-agent -o json | jq -r '.spec.template.spec as $s | $s.volumes[] | select(.hostPath.path=="/etc/passwd") | .name as $v | "\($v) -> " + ([$s.containers[] | .name as $c | .volumeMounts[] | select(.name==$v) | "\($c):\(.mountPath)"] | join(", "))'
kubectl $KC -n cxp-3881-operator exec $POD -c agent -- getent passwd dd-agent
kubectl $KC -n cxp-3881-operator exec $POD -c agent -- getent passwd cxp-host-user
kubectl $KC -n cxp-3881-operator exec $POD -c agent -- env DD_LOG_LEVEL=off agent processchecks process --json | jq -c '.processes[] | select(.command.args==["sleep","86400"]) | {pid, user: {name: .user.name, uid: .user.uid}}'
kubectl $KC -n cxp-3881-operator exec $POD -c agent -- env DD_LOG_LEVEL=off agent status -j | jq -r .hostinfo.platform
kubectl $KC -n cxp-3881-operator exec $POD -c agent -- sh -c '. /host/etc/os-release; echo $ID'

Case	Volumes mounted by	dd-agent in container	cxp-host-user in container	Process check user	platform
a 7.85.0	passwd: agent /etc/passwd	not found (the original bug)	resolves	cxp-host-user	ubuntu (Agent image; no os-release mount)
b 7.86.0-rc.6	passwd: agent /host/etc/passwd; os-release-file: agent /host/etc/os-release	resolves	not found	cxp-host-user	debian = node os-release ID

No host-passwd volume is rendered.

With the default os-release mount (re-run on kind at ceff214):

Case	os-release-file mounted in	passwd mounted by	Agent platform
7.85.0	agent	agent /etc/passwd	debian 12 (was ubuntu before the default mount)
7.86.0-rc.6	agent	agent /host/etc/passwd	debian 12
7.85.0, live process and process discovery off	agent	none	debian 12
7.86.0-rc.6 + NPM with agent.datadoghq.com/cnm-direct-send-enabled: "false" (separate process-agent)	agent, process-agent	agent /host/etc/passwd	debian 12
7.59.0 (live process in process-agent)	agent, process-agent	process-agent /etc/passwd	debian 12

The host-only user's process resolved to cxp-host-user in whichever container runs live process, and no case rendered host-passwd or an unmounted volume.

Output: a) 7.85.0

image: registry.datadoghq.com/agent:7.85.0
volume passwd (hostPath /etc/passwd) -> agent:/etc/passwd
--- getent passwd dd-agent
command terminated with exit code 2
--- getent passwd cxp-host-user
cxp-host-user:x:31001:31001::/home/cxp-host-user:/usr/sbin/nologin
--- agent processchecks process --json (host process)
{"pid":961,"args":["sleep","86400"],"user":{"name":"cxp-host-user","uid":31001}}
--- hostinfo.platform / os-release ID
ubuntu (no /host/etc/os-release)

Output: b) 7.86.0-rc.6

image: registry.datadoghq.com/agent:7.86.0-rc.6
volume passwd (hostPath /etc/passwd) -> agent:/host/etc/passwd
volume os-release-file (hostPath /etc/os-release) -> agent:/host/etc/os-release
--- getent passwd dd-agent
dd-agent:x:100:0::/nonexistent:/usr/sbin/nologin
--- getent passwd cxp-host-user
command terminated with exit code 2
--- agent processchecks process --json (host process)
{"pid":961,"args":["sleep","86400"],"user":{"name":"cxp-host-user","uid":31001}}
--- hostinfo.platform / os-release ID
debian / debian

Output: agent status process section (b)

Process Component
  Enabled Checks: [process rtprocess service_discovery]
Collector
  Last collection time: 2026-11-11 15:59:44
  Number of processes: 43
  Number of submission errors: 9   # placeholder API key

Why one volume works

Plain Kubernetes accepts one hostPath volume mounted at two paths, so 7.86+ process containers reuse passwd at /host/etc/passwd while system-probe/security-agent keep it at /etc/passwd. Only GKE Autopilot's Warden needs a separately named volume, which comes with the Autopilot follow-up.

kubectl $KC create namespace cxp-3881-test
kubectl $KC -n cxp-3881-test apply --dry-run=server -f passwd-reuse.yaml
kubectl $KC -n cxp-3881-test apply -f passwd-reuse.yaml
kubectl $KC -n cxp-3881-test exec passwd-reuse -- grep passwd /proc/mounts

passwd-reuse.yaml

apiVersion: v1
kind: Pod
metadata:
  name: passwd-reuse
  namespace: cxp-3881-test
spec:
  containers:
  - name: c
    image: busybox:1.36.1
    command: ["sleep", "3600"]
    volumeMounts:
    - name: passwd
      mountPath: /etc/passwd
      readOnly: true
    - name: passwd
      mountPath: /host/etc/passwd
      readOnly: true
  volumes:
  - name: passwd
    hostPath:
      path: /etc/passwd

Output: volume reuse

namespace/cxp-3881-test created
pod/passwd-reuse created (server dry run)
pod/passwd-reuse created
NAME           READY   STATUS    RESTARTS   AGE
passwd-reuse   1/1     Running   0          2s
passwd /etc/passwd readOnly=true
passwd /host/etc/passwd readOnly=true
overlay /etc/passwd overlay ro,relatime,...
overlay /host/etc/passwd overlay ro,relatime,...

For the Autopilot follow-up: GKE Autopilot (sandbox, GKE 1.37) against allowlist v1.0.8 admitted Agent 7.86.0-rc.2 at /host/etc/passwd from a host-passwd volume; Agent 7.85.0 and a v1.0.7 allowlist pin kept /etc/passwd. Commands are in ddoghq/datadog-gke-workload-allowlist#29.

Merge

• No allowlist dependency: GKE Autopilot keeps /etc/passwd in this PR. The Autopilot follow-up depends on Google publishing allowlist v1.0.8 (PR #29).
• Coordinated with Helm passwd migration (PR #3009).

Known limitation

• Mount selection does not consider experimental per-container image annotations or container renames, the same gap other version-gated features have. If they change the Agent version, the mount can disagree with the running image; the effect is limited to process usernames (an older Agent with /host/etc/passwd falls back to image-local users, and a 7.86+ Agent with /etc/passwd behaves as before this change).
• The Agent process resolver change does not cover Gohai. Moving the core-Agent mount can make Gohai report image-local usernames or numeric UIDs instead of host usernames; that remains separate follow-up work.

Note: existing os-release reporting issue

The Operator has never mounted the host's /etc/os-release in the agent container, so without a /host/etc mount the Agent reads the Agent image's /etc/os-release and reports that OS (for example ubuntu 24.04) in host metadata instead of the node's. We have seen Operator-deployed Agents on Amazon Linux 2023 nodes report ubuntu 24.04 while Helm-deployed Agents on the same OS report amazon 2023. Mounting it by default (above) fixes this for every Agent version, matching Helm. More details in this comment here: #3645 (comment)

🤖 Generated with Claude Code

[datadog-datadog-prod-us1-2]

 

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 57.90% (+0.04%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a5c9b46 | Docs | Give us feedback!}

[gengnamstyle]

Now that /host/etc exists with our passwd mount, we also need to mount the os-release location because the agent will use /host/etc for mounted paths when the directory exists as the agent sets HOST_ETC=/host/etc https://github.com/DataDog/datadog-agent/blob/8113ee2796ecc8b0c89301aebb339e8fdbb29c1a/pkg/config/setup/config.go#L372-L385

This is then also used to read host OS files from there.

Before this PR the Operator's agent container had nothing under /host/etc, so mounting passwd at /host/etc/passwd is what turns HOST_ETC on. Therefore, we must mount host /etc/os-release there too keeps host metadata's platform populated (gopsutil).

Helm's agent container already mounts it https://github.com/DataDog/helm-charts/blob/9eccc9bc6d0f325c7807f4dafcc15b1019cb11b1/charts/datadog/templates/_container-host-release-volumemounts.yaml#L1-L13

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-11T18:25:41.061876Z	345e224	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 345e2244a9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[daniel-taf]

I think having this here seems odd since this mount actually has nothing to do with the liveprocess feature. Shouldn't the os-release file just always be mounted at HOST_ETC if this is what helm does? It won't break the processes features.

[gengnamstyle]

Yeah I agree having it here is kind of odd since we would be mounting it only if these features are enabled.

The os-release file would be searched for at HOST_ETC but it won't be mounted there.

https://github.com/shirou/gopsutil/blob/v4.26.9/host/host_linux.go#L270-L275

So if we don't actually explicitly mount it then we could see issues with the libraries reading there.

Something that is interesting is that os-release is not mounted by default in the operator at all it appears

datadog-operator/internal/controller/datadogagent/component/agent/default.go

Lines 899 to 912 in 5b4a186

	func volumeMountsForCoreAgent() []corev1.VolumeMount {
	return []corev1.VolumeMount{
	common.GetVolumeMountForLogs(),
	common.GetVolumeMountForAuth(false),
	common.GetVolumeMountForInstallInfo(),
	common.GetVolumeMountForConfig(),
	common.GetVolumeMountForProc(),
	common.GetVolumeMountForCgroups(),
	common.GetVolumeMountForDogstatsdSocket(false),
	common.GetVolumeMountForRuntimeSocket(true),
	common.GetVolumeMountForRunPath(),
	common.GetVolumeMountForTmp(),
	}
	}

What this mean is that when the agent reads the the os-release file it may read the agent's own container image os-release file instead of the host's node which I believe has been an existing bug on the operator for a while.

Ideally, it seems like the operator should by default mount this but this seems like something that should be a followup.

My agent pointed me to these 2 hosts internally (one installed with the operator and one with helm and they report different os's despite being on the same type of amazon node.

https://app.datadoghq.com/infrastructure/hosts?groupSortCol=-count&hostType=all-hosts&pageIndex=0&pageSize=20&scope=host%3Ai-0871fa2efc7273a0d%20OR%20host%3Ai-01742189e05967468&sortCol=-cpu_pct_unified

[daniel-taf]

To rephrase, I think the mount should be added by default by the Operator, in the spot you linked:

datadog-operator/internal/controller/datadogagent/component/agent/default.go

Lines 899 to 912 in 5b4a186

	func volumeMountsForCoreAgent() []corev1.VolumeMount {
	return []corev1.VolumeMount{
	common.GetVolumeMountForLogs(),
	common.GetVolumeMountForAuth(false),
	common.GetVolumeMountForInstallInfo(),
	common.GetVolumeMountForConfig(),
	common.GetVolumeMountForProc(),
	common.GetVolumeMountForCgroups(),
	common.GetVolumeMountForDogstatsdSocket(false),
	common.GetVolumeMountForRuntimeSocket(true),
	common.GetVolumeMountForRunPath(),
	common.GetVolumeMountForTmp(),
	}
	}

. If its not a huge lift and someone from container-platform confirms its the right thing to do. I think we should just do it now, or at least in the same release.

[tbavelier]

Thanks for the detailed PR and the validation notes. I'm requesting changes for the default-version tests that will break on the next Agent bump, and for the duplicated os-release volume. Details are in the inline comments.

• Tests: the default-version tests fail as soon as images.AgentLatestVersion reaches 7.86 (inline on process_passwd_test.go).

• Duplicated os-release volume with CWS and the private action runner: reuse the new os-release-file volume instead (inline on common/const.go).

• Nits: the e2e platform comparison and the Windows branch in ProcessPasswdMountPath (inline).

• PR description: "Talos strips passwd and os-release-file" is out of date. Since ceff214, Talos only strips passwd, which is correct. I checked on a live Talos v1.15.0-alpha.0 QEMU cluster:

  • /etc/os-release exists on the node as a regular file (ID=talos).
  • A pod with the same untyped hostPath mounted read-only at /host/etc/os-release runs fine and reads the host file.
  • A hostPath for /etc/passwd fails with failed to mkdir "/etc/passwd": mkdir /etc/passwd: read-only file system, so the passwd strip is still required.

  Could you update that line?

[tbavelier]

These cases hard-code the legacy path for the default Agent version, as do the default cases in the liveprocess and processdiscovery feature tests (for example liveprocess/feature_test.go:51, which expects common.PasswdMountPath).

When the automated "Update default Agent to 7.86.x" PR bumps images.AgentLatestVersion, ProcessPasswdMountPath starts returning /host/etc/passwd and these tests fail. I checked locally by setting AgentLatestVersion = "7.86.0". TestProcessPasswdMountPath, Test_liveProcessFeature_Configure and Test_processDiscoveryFeature_Configure all fail. The bump PRs already regenerate the golden files, but they don't update hand-written tests, so the bump PR would be blocked.

Could the expected path for the default cases come from images.AgentLatestVersion? For example, a small test helper that returns HostPasswdMountPath when the default is >= 7.86.0-0. The other option is to pin the version these tests use, the way nodeAgentVersion already takes defaultVersion.

[tbavelier]

This duplicates SystemProbeOSReleaseDir* (L139-141): same hostPath /etc/os-release, same mountPath /host/etc/os-release, but a different volume name (host-osrelease). With CWS or the private action runner enabled, the pod now declares two hostPath volumes for the same host file. That leaves two sets of constants to keep in sync.

Now that os-release-file is a default node Agent volume, I'd remove SystemProbeOSReleaseDir* and have CWS and the private action runner reuse the new helpers:

// cws/feature.go:352 and privateactionrunner/feature.go:379
osReleaseVol, osReleaseVolMount := common.GetVolumeForOSRelease(), common.GetVolumeMountForOSRelease()

The volume manager merges volumes with the same name, so the pod ends up with a single os-release-file volume. The core agent, process-agent, system-probe and the private action runner all share it, and it matches the Helm volume name. Autopilot is unaffected because CWS and the private action runner are Rejected there (feature/provider_support.go).

I tried this locally:

• Only the two Talos golden files change: the host-osrelease volume disappears and system-probe's mount is renamed to os-release-file.
• go test ./internal/controller/... ./pkg/... passes after updating the CWS and private action runner tests to the new constants.
• One assertion in process_passwd_integration_test.go needs to skip system-probe. The check that the process features add no os-release mount now sees the os-release-file mount that CWS adds to system-probe.

[tbavelier]

Nit: this compares the raw ID from the node's os-release with the Agent's hostinfo.platform, but the two aren't always the same string.

Only os-release is mounted under /host/etc. There is no debian_version, redhat-release, system-release and so on, so gopsutil uses its os-release branch. That branch returns ID unchanged except for one rename: amzn → amazon (gopsutil v4.26.9, internal/common/common_linux.go:328). On an Amazon Linux node (AL2 or AL2023), this assertion compares amzn with amazon and fails even though the mount works.

The k8s_suite only runs on kind today, where nodes are Debian (ID=debian), so it passes. It would break if the suite were reused on EKS nodes. You could apply the same amzn → amazon mapping before comparing. Or assert what the test is really after: that the platform comes from the node and not from the Agent image, i.e. it differs from the image's own os-release ID.

[tbavelier]

Nit: this branch never has an effect. ApplyWindowsPodTransformation (component/agent/windows.go:633) already drops every mount whose path starts with /, so on Windows the passwd mount is removed whichever path is returned here. It's fine to keep as documentation, but you could also drop it and its test cases.