Skip to content

fix: guard against non-string options.host in http instrumentation - #78

Draft
RaphaelManke wants to merge 1 commit into
masterfrom
sup-1457-lambda-extension-unhandled-typeerror-in-getabsoluteurl
Draft

fix: guard against non-string options.host in http instrumentation#78
RaphaelManke wants to merge 1 commit into
masterfrom
sup-1457-lambda-extension-unhandled-typeerror-in-getabsoluteurl

Conversation

@RaphaelManke

Copy link
Copy Markdown
Contributor

Fixes the crash in SUP-1457: instrumented Lambda invocations dying with

TypeError: u.indexOf is not a function
    at t.getAbsoluteUrl (file:///opt/init.mjs:2:275912)
    at t.getOutgoingRequestAttributes (file:///opt/init.mjs:2:279285)
    at Object.request (file:///opt/init.mjs:2:263159)
    at <anonymous> (/var/lib/graphql.js:13:23)

Cause

@opentelemetry/instrumentation-http reads options.host and calls .indexOf() / .match() on it without checking that it is a string:

let host = reqUrlObject.host || reqUrlObject.hostname || headers.host || 'localhost';
if (host.indexOf(':') === -1 && port && ) {      // <-- throws

Node's ClientRequest derives the target from options.hostname first and ignores options.host entirely whenever hostname is set (it overwrites it internally). So a non-string host sitting next to a valid hostname is completely inert for Node — the request works, and has worked for as long as the calling code existed — but it throws inside the instrumentation. Because getOutgoingRequestAttributes() is called outside any safeExecuteInTheMiddle(), the error propagates synchronously out of the caller's http.request() and takes the whole invocation with it.

Verified against the published packages: every version from 0.50.0 through 0.221.0 is affected (0.50.0, 0.54.2, 0.55.0, 0.57.0, 0.200.0, 0.203.0, 0.205.0, 0.207.0–0.210.0, 0.215.0, 0.219.0, 0.220.0, 0.221.0). Not a regression, and bumping the dependency is not a fix. Reported upstream as open-telemetry/opentelemetry-js#6967.

No off-the-shelf client produces this shape — I probed 24 stacks (graphql-request, Apollo, urql, node-fetch, cross-fetch, axios, got, superagent, needle, the AWS Smithy handler, aws4-signed AppSync requests, …) and they all pass host as a string or set only hostname. It comes from hand-rolled request-option builders.

Fix

A makeHostSafe() guard at the top of the ignoreOutgoingRequestHook this distro already configures. That hook is the right seam: upstream calls it with the same optionsParsed object before both crash sites, optionsParsed is a fresh copy so mutating it never touches the caller's object, and upstream already wraps the hook in safeExecuteInTheMiddle() so a bug in our guard cannot crash an invocation either.

  • non-string host + usable hostname → drop host (behaviour-preserving; Node ignores it anyway)
  • non-string host, no usable hostname → skip instrumenting, so Node raises its own ERR_INVALID_ARG_TYPE instead of us masking it with a TypeError of our own. These options are invalid for Node with or without the layer; we get out of the way rather than inventing a different failure.

The guard is a no-op once #6967 ships, and is harmless to leave in place until then.

Tests

httpInstrumentation.test.ts previously had one assertion (module name). It now exercises the real upstream wrapper over the real Dash0 config and asserts on exported spans.

Before the fix: 4 failed, 2 passed — three shapes with TypeError: host.indexOf is not a function, plus the no-hostname case failing because our TypeError replaced Node's error. After: 6 passed.

case before after
host is a URL object TypeError: host.indexOf traced, correct http.url / net.peer.name / net.peer.port
host is a plain object TypeError: host.indexOf traced, correct attributes
host is a number TypeError: host.indexOf traced, correct attributes
host non-string, no hostname TypeError: host?.match Node's own ERR_INVALID_ARG_TYPE
host is a string (control) traced traced, unchanged

One thing to know if you write tests here later: instrumentation.enable() does nothing under jest, because require-in-the-middle cannot hook jest's module registry — requests run completely unpatched and the tests pass silently without testing anything. These tests instead apply the instrumentation's own patch through the public getModuleDefinitions(), which runs the genuine _outgoingRequestFunction.

Verification

  • npx jest --no-cache → 137 passed, 12 suites
  • npm run build → succeeds; guard confirmed present in the minified dist/init.mjs
  • attribute assertions use old semconv keys (http.url, net.peer.name), since nothing in the repo sets OTEL_SEMCONV_STABILITY_OPT_IN

Notes, out of scope for this PR

  • getOutgoingRequestAttributes() running outside safeExecuteInTheMiddle() means any future upstream defect in attribute building can still kill an invocation. A real safety net needs the pristine http.request captured in bootstrap.ts before instrumentation registers, so an outer wrapper can fall back to it. Suggested upstream too.
  • npm install in opt/node fails on a fresh clone: the file:../../build/opentelemetry-instrumentation-aws-sdk.tgz dependency is resolved before its own preinstall builds it, so scripts/build-aws-sdk-tarball.sh has to be run manually first.
  • The FileSpanExporter tests write test-spans-*.json into opt/node/ without cleaning up, and those paths are not gitignored.

instrumentation-http reads `options.host` and calls .indexOf() / .match() on
it without checking that it is a string, outside of any safeExecuteInTheMiddle.
Node resolves the target from `options.hostname` first and ignores
`options.host` entirely whenever `hostname` is set, so a non-string `host` is
inert for Node but throws `TypeError: host.indexOf is not a function` inside
the instrumentation -- killing the caller's request and the whole invocation.

Sanitize the options in the ignoreOutgoingRequestHook we already configure:
drop an unused non-string `host` when `hostname` is usable, and skip
instrumenting the request entirely when it is not, so Node reports its own
error instead of us masking it with ours.

Reported upstream as
open-telemetry/opentelemetry-js#6967 -- every
published version from 0.50.0 to 0.221.0 is affected, so upgrading the
dependency is not an option.

Tests drive the real upstream wrapper over the real Dash0 config by applying
the instrumentation's own patch via getModuleDefinitions(); enable() is a
no-op under jest because require-in-the-middle cannot hook jest's module
registry, which would let the requests run unpatched and pass silently.

Refs SUP-1457
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants