Implement the Python SDK with full spec conformance#85
Merged
Conversation
Add a conformance battery for the DispatchContext -> RequestContext -> ExchangeContext promotion chain: - assert all three links subclass CallContext and implement the context-manager protocol, and each carries an InstrumentationContext; - assert promotion threads a single shared InstrumentationContext (and the per-hop Request) by reference through the chain rather than copying it; - assert registration into the ContextStore is a side effect of promotion, not of construction; - assert a with-block evicts the context from the store on exit, including when the block body raises; - audit that no context class defines __del__, so garbage collection is never the eviction mechanism. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…riants Turn the workspace's packaging invariants into mechanical gates rather than conventions reviewers have to remember: - tools/check_namespace_packages.py fails if any distribution ships an __init__.py at dexpace/, dexpace/sdk/, or dexpace/sdk/http/. A single stray one turns an implicit namespace directory into a regular package and breaks PEP 420 merging for the other four distributions, sometimes only under certain install orders. - tools/check_mit_headers.py fails if any .py file (src or tests) is missing the two-line MIT licence header, tolerating an optional leading shebang. - tools/smoke_wheel_import.py imports core plus the stdlib adapter and asserts the shared dexpace.sdk namespace merges across two separately built wheels and that both ship their py.typed marker (PEP 561). A new "packaging" CI job runs the two static checks, then builds the core and stdlib wheels and smoke-imports them in a fresh venv, so namespace merging and py.typed shipping are verified against installed wheels rather than only the editable workspace layout. Tests seed a violation in a temporary tree and assert each gate trips, and assert the real repository stays clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The `LoggingPolicy` did not capture request bodies at all, and the `Loggable*` body wrappers default their `max_capture_bytes` ceiling to ~2 GiB — effectively unbounded, which is the wrong default for a logging preview. Give `LoggingPolicy` a `preview_bytes` knob defaulting to 8 KiB. The policy now wraps the outgoing body in a `LoggableRequestBody` capped at that size and emits a `request_body_preview` field on the response log line. The wrapper's additive tap means the full payload still streams to the transport uncut; only the diagnostic preview truncates. The preview is decoded charset-aware (falling back to UTF-8) with `errors="replace"`, so binary or mislabelled bodies never break the logging path. The snapshot is bounded to `preview_bytes` at emission time as well, so a caller-supplied wrapper with a larger ceiling still yields a small preview. The wrapper ceiling stays large and independently configurable — only the logging policy defaults to the tighter bound. Also document, on `default_async_pipeline`, that the async stack has no wire logging yet: there is no async logging policy, so async pipelines emit no request/response lines or body preview. Correlation ids still propagate, and an async logging policy is planned. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tomic The staged pipeline builders let a second occupant slip into a singleton (pillar) stage through the insert/reload path, silently clobbering the incumbent, and could leave partial mutations behind when an edit failed midway. - Reject a second occupant of a pillar stage on every mutation path (append, prepend, replace, insert, bulk reload), with an error naming the stage and the incumbent policy type. `_reload` now validates single occupancy across the whole flattened list before touching builder state. - Reject relocating a shipped singleton to a different stage via `replace`: a pillar incumbent may only be swapped for a policy declaring that stage. - Make cross-stage `replace` atomic by placing the new policy before removing the old, so a rejected pillar occupancy leaves the incumbent intact. - Add a `batch()` context manager that applies a group of edits as an all-or-nothing transaction, rolling back on failure so a failing batch leaves no partial mutation and any previously built pipeline untouched. - Document the single-threaded contract on the async builder. Applies to both `StagedPipelineBuilder` and `AsyncStagedPipelineBuilder`. Regenerates the public-surface baseline for the new `batch()` method. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Introduce `TypeRef`, a thin wrapper around a runtime type witness (e.g. `list[Pet]`) that the codec can decode against directly. Subscripted generics are real, introspectable objects at runtime, so the element type flows through to decoding instead of being reconstructed from fragile string annotations. `TypeRef` validates its witness eagerly at construction: - bare type parameters (`TypeVar` / `ParamSpec` / `TypeVarTuple`) are rejected, since they name no concrete type; - unparameterized generic aliases (`typing.List`, `typing.Dict`, ...) are rejected, since they carry no element type; - PEP 695 `type X = list[Pet]` aliases are resolved through their lazy `__value__` (following alias-to-alias chains) so the concrete type is pinned at construction rather than deferred. Each rejection raises `TypeError` naming the offending object. `Codec.decode` now accepts `type[T] | TypeRef[T]`, unwrapping a `TypeRef` to its resolved witness while preserving the return type. The module docstring documents why witnesses must be passed as live objects at the call site: `from __future__ import annotations` makes written annotations strings at runtime, so they cannot be harvested as witnesses. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Introduce a `@pytest.mark.req("SOME-ID")` marker (registered centrally in the
root pytest config so it is warning-free under `-W error`) and a reusable audit
that cross-references collected markers against a requirement index.
tools/traceability_audit.py provides the machinery: a generic RequirementIndex,
an `audit` that reports a requirement -> covering-test matrix and flags
MUST-level requirements with no covering test, and JSON/CSV matrix renderers. A
thin pytest plugin collects the markers at collection time and writes the matrix
artifact on every run (gitignored `build/` by default).
The audit is milestone-scoped: a MUST requirement is only a gap once its
subsystem is explicitly enabled, so the audit never fails on the whole
still-being-assembled spec at once. It also accepts an optional not-applicable
set (the seam a future deviations ledger plugs into) so intentionally-skipped
requirements never count as gaps.
The requirement index shipped here is a small, clearly-marked placeholder that
proves the mechanism end-to-end; it is meant to be replaced by the real index
without touching the audit or plugin.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add tools/parity_check.py, a verifier that confirms an async twin of a sync module has not semantically drifted from its counterpart. It normalizes both files' ASTs through a closed, declared token map -- stripping await, collapsing async def / async for / async with onto their sync node shapes, and renaming a declared set of identifiers (and exact name-as-string references such as __all__ entries) -- then compares the resulting trees structurally. The comparison is on AST structure, not source text, so comments and formatting are ignored while any residual difference after normalization surfaces as drift. Normalization never uses fuzzy matching: an identifier the map does not name is compared as-is, so an accidental rename is caught rather than smoothed over. Two modes: blocking (drift fails, for future generated pairs whose contract is AST identity) and report-only (drift is written to a diff artifact but never fails, for the existing hand-written policy twins whose correctness rests on import-linter contracts instead). Ships a small library API (check_parity, check_parity_sources, normalize_source) plus a CLI. Tests cover a clean fixture pair that passes, a seeded one-line drift that fails in blocking mode and is merely reported in report-only mode, the token map's handling of asyncio.sleep, async for/with and protocol method renames, and a report-only sweep over the real policy twins. The set_date twin is certified as an exact transliteration under the default token map. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Distinguish the two validation regimes an HTTP header model needs. The default constructor stays inbound-lenient: it tolerates obs-text (bytes 0x80-0xFF) so response headers decoded as Latin-1 by a transport round-trip unchanged, per RFC 7230's rule that received messages accept obs-text. A new `Headers.outbound` constructor is outbound-strict: it additionally rejects obs-text and other non-field-content bytes, so generated request headers never emit characters forbidden on the wire. Both paths continue to reject the CR / LF / NUL bytes that enable header injection. Harden name folding to be ASCII-invariant. `_normalize` now validates the original name against an ASCII token set before lower-casing, instead of lower-casing first. This rejects non-ASCII names that a naive `str.lower()` would otherwise fold into a valid ASCII token (for example U+212A KELVIN SIGN folding to "k"), and guarantees the fold never triggers locale-style special-casing such as the Turkish dotless-i. Add a conformance battery covering canonical lower-case storage, fold idempotence, equality-iff-fold-equality, ordered multi-values, removal via a value-less set, the strict/lenient obs-text split, injection rejection, error-message hygiene (values never echoed, control chars in names escaped), and the pre-folded `HttpHeaderName` hot path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add property-based tests pinning the algebraic invariants of the existing parsers and codecs: - Headers: name-fold idempotence and equality-is-fold-equality - MediaType: parse/render round-trip and of() normalization idempotence - QueryParams/Url: encode/parse inversion and structural round-trip - Status: totality over ints and equality by numeric code - SSE parser and iter_jsonl: identical output under arbitrary chunk re-splits - UrlRedactor: totality (never raises, always returns str) on arbitrary input Register capped, deterministic Hypothesis profiles in the core test conftest so the default run stays fast and reproducible (bounded example count, no per-example deadline, fixed seed), with a larger "nightly" profile selectable via HYPOTHESIS_PROFILE for deeper runs. Add hypothesis as a dev dependency. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…descriptor Complete the Configuration surface and the clock/date utilities it anchors: - Configuration lookup gains an injectable property source as a third layer (override > env > property > default). The property source is keyed by a dotted-lowercase normalisation of the name (MAX_RETRY_ATTEMPTS -> max.retry.attempts); the new get_raw accessor queries it verbatim. Typed accessors (get_int/get_bool/get_duration) resolve through the full layer stack. Empty env/property values are treated as absent. - Add with_override for copy-on-write derivation (the derived config shares the source's env and property seams) and a process-wide default slot (default_configuration / set_default_configuration, last-write-wins). - Reject a None configuration key with a clear TypeError, and document the builder's single-threaded contract. - Add BuildDescriptor, which resolves the distribution name and version from importlib.metadata and falls back to a non-blank "unknown" sentinel when metadata is unavailable (zipapp/frozen installs) or blank. - Add CancellationToken to the clock seam: an interruptible, cancellable sleep backed by threading.Event that rejects negative/NaN durations, complementing the existing lenient Clock.sleep. - Add a shared, pure RFC 1123 HTTP-date module (format_http_date / parse_http_date) with canonical GMT rendering and tolerant multi-format parsing, for reuse by date-aware components. Adds a precedence matrix, a duration-grammar corpus, a hermetic seam-substitution suite, and a metadata-less descriptor-fallback test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… under shared trace ids
ContextStore keyed entries by trace id alone. When a single traced operation
fans out into concurrent calls, those calls legitimately share a trace id, so
the second registration overwrote the first and closing either evicted the
survivor — silent context loss under concurrency, with a race-shaped repro.
Register under `{trace_id}:{span_id}:{n}` instead, where `n` is drawn from a
process-wide `itertools.count()` under the store lock. The increment never
assumes `next()` is atomic, so the guarantee holds on free-threaded CPython
(PEP 703). The key is stamped when a call is registered and carried forward, so
a call's request and exchange tiers share one slot and a single close still
clears both. The trace id stays the leading, colon-delimited key segment, and a
new `contexts_for_trace` prefix scan returns every live context for a trace id.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ClientLogger previously exposed only eager `info`/`error`/... helpers with a simple logfmt renderer. This adds a fluent `LogEvent` builder (`at_info().event(...).add(...).log(...)`) with the guarantees a structured SDK logger needs: - Disabled levels return a single shared inert event, so the disabled path allocates nothing and is identity-stable. - Each active event emits at most once, guarded by an explicit threading.Lock rather than relying on the GIL, so the latch holds under free-threaded CPython. - Rendering is total: a value whose `__str__` raises yields a placeholder instead of propagating; `None` renders as the literal `null`; values are truncated to a configurable length with primitives exempt. - Fields fold in with a clear precedence: per-event fields override the logger's global context, which overrides the ambient diagnostic context (trace/span ids by default, controlled by an allow-list); each key appears once. The prior merge order let the diagnostic context override global context, which is now corrected. - The reserved `event` tag is emitted exactly once (empty clears it); a field colliding with it warns once per logger. Emission failures are caught and re-emitted as an `http.instrumentation.error` diagnostic so a broken sink never crashes the caller, while exceptions that must surface still do. The existing eager helpers are preserved and now route through the builder. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Introduce a checked-in, byte-exact fixture corpus that will serve as the single source of truth for the wire behavior of several HTTP seams: the URL/query codec, redirect Location resolution, raw-query pagination splices, Server-Sent Event streams, Retry-After pacing, and RFC 1123 dates. Later work on those codecs/parsers points its conformance checks at these vectors so behavior stays byte-for-byte stable. Fixtures are stored as raw byte files (organized per family under data/), read in binary mode via importlib.resources, and surfaced through a small loader that turns each vector into a frozen, slotted record. Binary loading is deliberate: text-mode reads would apply universal-newline translation and corrupt the CR / LF / CRLF SSE streams. A scoped .gitattributes marks the data tree as raw bytes so git never applies EOL normalization to the fixtures either. Coverage pins the tricky edges: strict RFC 3986 query rendering (space to %20, "+" to %2B, order-sensitive, value-less flag as present empty string) with lenient parse as its exact inverse; redirect splices that preserve percent-encoding verbatim, drop userinfo, and keep IPv6 brackets; verbatim raw-query pagination splices; SSE newline variants, BOM stripping, sticky id, and retry fields; Retry-After pacing including the reject cases float() wrongly accepts (nan, inf, 1_000) versus the strict digit pre-screen, past-date flooring to zero as distinct from a malformed value yielding None, and the 365-day clamp; and RFC 1123 canonical rendering plus tolerant parsing of informational weekdays and zone aliases. Each family is tagged with a requirement-ID-style label so a later traceability tool can consume it, and the corpus lives in a single self-contained directory so it can be lifted into a standalone TCK distribution later. The accompanying test only validates that the corpus loads and is internally self-consistent (round-tripping where a vector claims it) via stdlib oracles; it does not implement the production codecs. A non-blocking, opt-in furl-upgrade canary lane is stubbed so the corpus can later be re-run against a newer furl. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…resh Rework bearer-token acquisition around a dedicated single-flight cache so a burst of concurrent requests triggers exactly one token fetch, and give the async policy a proactive three-zone refresh strategy. - InMemoryTokenCache reads are now wait-free: get() loads an immutable published dict snapshot and looks the key up without taking a lock, while set()/clear() publish a fresh dict under the lock. A concurrent reader sees the old or new snapshot in full, never a torn one, without relying on the GIL for atomic dict ops. - New _SyncSingleFlight / _AsyncSingleFlight coordinators own the refresh path. Sync uses double-checked locking; async serves fresh tokens immediately, serves a still-valid token inside the 30s margin while kicking off a non-fatal background refresh, and blocks only on a hard-expired token. - The async blocking fetch is a single shared task awaited through asyncio.shield, so one waiter's cancellation propagates to that waiter alone and never aborts the fetch the others depend on. - A provider that returns None or an already-expired token now raises instead of caching a bad token, and a throwing provider propagates without being cached so the next request retries it. - Shared zone-classification and fetched-token validation live in a pure _refresh module so the two coordinators cannot drift apart. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The URL redactor parsed strings with furl's throwing parser wrapped in a broad except. Exotic inputs could still escape as exceptions on the logging path, and a parse failure surfaced an opaque "REDACTED:unparseable" marker. It also dropped the entire fragment, discarding useful plain anchors, and offered no protection against a literal '?' inside a fragment being re-read as a query separator. Replace the furl round-trip with a self-contained, total splitter that never raises: it peels the fragment before the query (so a '?' in the fragment stays put), validates scheme/host/port itself, and returns a fixed "[malformed url]" string for anything it cannot parse. Fragments now scrub key=value pairs (which can carry OAuth implicit-flow tokens) while preserving plain anchors. Userinfo, query allow-listing, path, and atomic multi-value handling are unchanged in intent. Two emitters rendered URLs without going through the shared redactor: the tracing policy stamped the unredacted URL onto the url.full span attribute, and Request.__repr__ (the default dataclass repr) rendered the URL in full — both leaking query-string secrets. Route both through the one UrlRedactor so credentials and tokens are stripped consistently wherever a URL is surfaced; the logging policy already used it. Add a conformance battery covering userinfo, query allow-listing, atomic multi-value redaction, fragment scrubbing, component preservation, the no-spurious-'?' rule, and malformed handling, plus a seeded fuzz sweep asserting the redactor never throws and a used-by-all check that repr, logging, and tracing all route through the shared redactor. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Status(code) previously rejected any integer outside the 100..599 band, raising ValueError. That discarded otherwise-usable responses: a broken or vendor-only status code (or a non-standard code from a proxy) could never be surfaced as a Status, so callers lost the live response before any policy could react. Synthesize an UNKNOWN_<code> pseudo-member for any integer, so lookup is total and equality stays by code. Only a non-integer lookup still raises; the band helpers (is_success/is_redirect/...) report no band for a code outside 100..599. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
type/subtype/parameters were public dataclass fields, so an instance built through the raw constructor could hold un-normalized state and compare unequal to an equivalent MediaType built via of()/parse(). Move the normalized parts into unexported fields exposed through read-only properties, and normalize in __post_init__ so every construction path — the raw constructor included — yields the lower-cased type/subtype and sorted, lower-cased-key parameters. Public accessor names and behavior are unchanged; the surface baseline is regenerated to record the three accessors. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add focused batteries for ETag (RFC 7232 strong/weak comparison, the obs-text and DEL character rules, and the distinction between dataclass equality and weak comparison), HttpRange (wire-format shapes, the end property, and construction validation), RequestConditions (multi-tag and wildcard rendering, empty-list rejection, all-field rendering, and HTTP date normalization), and Protocol parse round-trips. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolved conflict in test_promotion_chain.py: PY-M1-01's CTX-4 key-format fix (counter-suffixed keys) landed in parallel with PY-M3-01's new certification tests, which were written against the old trace-id-only key. Kept all tests from both branches; rewrote the PY-M3-01 tests that looked up ContextStore by bare trace id to use the promoted context's store_key instead.
PY-M8-02 certified the stdlib transports but, unlike the httpx/requests/ aiohttp certifications, put its harness (stdlib_transport.py) inside core's shared transport_conformance/ dir and registered it from the shared conftest — coupling core's test suite to the stdlib adapter package and making the shared battery no longer transport-agnostic. That would have broken PY-M10-01's TCK extraction, whose whole premise is a transport-neutral battery that certifies ANY adapter through the harness seam and depends on NONE. Normalized to match the other four adapters: moved the harness to packages/dexpace-sdk-http-stdlib/tests/stdlib_conformance_harness.py (unique module name, so no mypy collision with aiohttp's conformance_harness.py) and its registration into the stdlib package's own tests/conftest.py. Kept every general improvement PY-M8-02 made to the SHARED battery — the four new honest capability-flag opt-outs and the shutdown() teardown hook on the harness ABC, plus the skip-guards in test_sync_contract.py — since those benefit all adapters (defaults preserve existing behavior). The shared battery now registers only the in-memory reference and imports no adapter, so it lifts into a standalone TCK cleanly. Verified: all six harness params (in-memory, stdlib-urllib, stdlib-asyncio, httpx, requests, aiohttp) still run and pass; 3255 passed, 0 failed; mypy/ruff/format/lint-imports all clean.
…ck package Add a sixth, test-only workspace distribution, dexpace-sdk-tck, that houses the shared transport-agnostic conformance battery, the golden wire-exactness byte corpus, and the adapter-certification seam, so any transport adapter can be certified against one shared spec. - Move the battery (harness ABCs, in-memory reference adapter, sync/async contract tests, registration conftest) and the golden corpus (loader plus raw byte fixtures) out of dexpace-sdk-core's test tree and into the installed dexpace.sdk.tck namespace package. The harness ABCs and corpus now ship as library code: adapters import them from dexpace.sdk.tck, and the corpus loads via importlib.resources as package data (zip-safe, byte-exact, verified to ship in the built wheel). - Rewire every consumer with no behaviour change: the five transport-adapter packages register their harnesses against dexpace.sdk.tck.harness, and core's query-codec / SSE / retry-pacing / redirect / RFC 1123 tests load the corpus from dexpace.sdk.tck.golden_corpus. - Enable pytest consider_namespace_packages so the battery collects under its full dexpace.sdk.tck.* name and shares one registry; each adapter conftest puts its own test directory on sys.path so its bare harness module still resolves regardless of import mode. - Wire the new package into the uv workspace, ruff, mypy (library modules stay under --strict; only the kit's own test bodies get the standard test-code relaxations), and import-linter (the kit stays off furl). Add self-tests proving the corpus loads from the installed package and that the kit's import graph touches no private core module. The kit is runnable as `pytest --pyargs dexpace.sdk.tck`, and the full workspace suite continues to certify all six harnesses (in-memory, stdlib-urllib, stdlib-asyncio, httpx, requests, aiohttp) against the battery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
…h workspace member)
The httpx transport-battery test test_malformed_inbound_header_dropped_obs_ text_preserved fails on Python 3.14 (a supported CI-matrix version). Root cause, reproduced on 3.14.5: httpx's own cookie-jar extraction inside client.send() builds an email.message.Message from the response headers (via response.info()), and Python 3.14 tightened email.validate_header_name to reject an invalid header name. The deliberately-malformed 'Bad Name' header this facet programs therefore makes httpx raise ValueError from its cookie machinery BEFORE the adapter's response adaptation (and its defensive per-header drop) ever runs. httpx genuinely cannot deliver a malformed-name header to the adapter on 3.14+ — a real, verified capability limitation of the httpx transport, not an adapter defect and not a test to wave off as 'environmental'. Fix, using the capability-flag mechanism PY-M8-02 built for exactly this: gate supports_defensive_inbound_headers to on the httpx harness, so the facet runs with full coverage on 3.13 and earlier (where httpx tolerates the header and the adapter's drop is exercised) and skips honestly on 3.14+. Also fixed a latent gap PY-M8-02 left: it added the supports_defensive_inbound_headers skip guard to the SYNC contract test but not its ASYNC twin, so the async test ignored the flag — added the matching guard. Verified: 3.14 full battery 0 failed (facet skips for httpx sync+async); 3.13 facet still runs and passes; full 3.13 suite 3259 passed.
… gate Stand up the release automation and the CI that keeps the workspace's forward-compatibility promise mechanical. - version-skew job: the newest adapters plus the TCK conformance battery and the generated petstore canary run against both the highest and the lowest-direct dependency resolution -- core's compatible-release floor and each transport's declared minimum -- across Python 3.12-3.14. This turns "a generated SDK keeps working against the oldest supported core" into a passing pipeline. - free-threaded job: runs the lock-sensitive core suites (single-use body once-guards, single-flight token cache, ContextStore cap-drain, emit-once logger latch, SSE cross-thread close) on a GIL-disabled CPython 3.13t/3.14t, so the explicit-lock claims get a real free-threaded runtime rather than only code review. - version-source job + tools/check_version_train.py: every published package shares one real version across its pyproject and uv.lock, and a built wheel carries that version rather than a build-time placeholder. - release.yml: tag-triggered (and dry-run-able) publish via PyPI trusted publishing with PEP 740 attestations and reproducible builds (SOURCE_DATE_EPOCH, fixed hash seed, deterministic wheel ordering). The adapters and TCK already pin dexpace-sdk-core>=0.1,<0.2; the skew matrix makes that constraint's guarantee observable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
…rsion-source gate
… set Adds four consumer-facing extension guides under docs/ — writing a transport (HttpClient), a serde, a pagination strategy, and a response handler — each with a contract checklist keyed to real requirement IDs and ending in the command that certifies an implementation (the shared TCK battery for transports, the matching conformance suite for the others). Adds docs/codegen-target.md: the normative description of the Tier-1 codegen layer a code generator targets — Operation/OperationInput/assemble, the ServiceCore/AsyncServiceCore executors, StatusErrorMap, ServiceDefaults with merge_config/merge_async_config, and the tiered auth resolver — cross-checked against the codegen package's public surface, with the petstore example as the compiling witness. Refreshes the existing docs for current behaviour: the async pipeline now ships per-attempt wire logging and a tracing span at parity with the sync default (the stale "async ships without logging" deviation is removed); documents the configurable retryable-status set, the default-stack redirect asymmetry, the sync cooperative-cancellation limit, and the deliberate RuntimeError shape for double-consuming a single-use body. Adds a test that compiles every python snippet in the docs and resolves its dexpace imports against the real API, so the prose cannot drift from the code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
TracingPolicy's module docstring claimed the per-attempt OpenTelemetry span machinery is sync-only with no async counterpart. AsyncTracingPolicy opens the same per-attempt span with identical semantic-convention attributes and per-request events, and is wired into default_async_pipeline. Correct the docstring to describe the async twin.
The traceability audit shipped against a 15-item placeholder index. Vendor the full requirement index — a machine-readable projection of the product spec's consolidated normative requirement appendix — as tools/requirement_index.json, generated by tools/build_requirement_index.py, and load it as the audit's real index via build_plugin_from_env(). The placeholder is retained for the audit's own machinery unit tests. With the real index in place the audit now measures true coverage: 22 of 532 MUST-level requirements currently carry a covering test marker.
Cross-reference the test suite against the consolidated requirement index by tagging each covering test with @pytest.mark.req(<id>). Replaces the earlier ad-hoc/placeholder marker ids with the real Appendix C ids and adds markers for previously-unannotated covered requirements, taking MUST-level coverage from 22 to 395 of 532. Behaviour is unchanged; these are traceability annotations only.
…ility gate Record the 97 MUST-level requirements that the Python port either does not implement (Java/JVM idioms, the removed byte-stream provider seam, the Builder pattern) or satisfies through a documented different mechanism, as a machine-readable projection in docs/traceability-na-ids.txt. Support an inline '# justification' comment per id so the id and its rationale stay on one line. With these accounted, 40 MUST-level requirements remain as genuine gaps.
The Server-Sent Events parser diverged from the WHATWG event-stream grammar in several ways a conformant client depends on: - The event name defaulted to "message" and surfaced even when a block sent no event: field. The parser now surfaces an omitted event name as absent (None); the EventSource-style "message" default is applied at the typed-adapter / connection consumer layer instead. - The parser persisted a sticky last-event-id across events. It is now single-pass: each event carries only the id present in its own block. Carrying an id forward for reconnection remains the reconnecting connection's job. - Dispatch was data-gated, so id-only, retry-only, and comment-only blocks emitted nothing. Dispatch is now permissive: any tracked field (data/event/id/comment/retry) set in a block emits an event, while a block with no field set (pure blank lines) is skipped. Comment text is captured on the event, an id containing a NUL is ignored entirely, and a present-but-empty field (e.g. "event:") is recorded as "" and counts as a field seen, distinct from an absent field. Connection lifecycle: - Opening a stream over a successful response that has no body now fails loudly rather than being treated as an immediately-empty stream that reconnects forever. - A release failure on an automatic clean-terminal path (clean end-of-stream or a DONE sentinel, with no error in flight) is reported out-of-band and swallowed so it cannot discard already-delivered events; an explicit close() still propagates its release failure. The golden wire-exactness corpus and its loader are updated to pin the corrected parse (absent event name, per-block id, captured comment). The conformance ledger records that auto-reconnection and Last-Event-ID continuity are provided by the separable, opt-in SseConnection/AsyncSseConnection value-add layer rather than the core parser. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
…rage Retry-safety (method idempotency plus body replayability) is now decided once, independently of retryability, and applied to every retry candidate. A body-less non-idempotent request such as a bare POST is no longer retried on a connect-phase transport error that never reached the server: the connect path previously skipped the idempotency check the read path already enforced, so the two phases disagreed. Transport-family and custom error types that advertise retryability through the structural capability now participate in retry decisions via the shared cause-chain predicate rather than a concrete-type match, so a caller's own error type can opt into retries without editing the classifier. Status- carrying errors are excluded from that path; their eligibility remains the configured status set. The baked retryability flag on protocol errors now derives from a single canonical classifier that treats 408, 429, and every 5xx except 501 and 505 as retryable, so exotic 5xx codes (506/507/508/510/511) report retryable instead of silently non-retryable. The retry policy's configurable eligibility set is unchanged and remains a curated subset of that rule. The pacing parser gains the retry-after-ms and x-ms-retry-after-ms millisecond-delta headers, composed into the fixed precedence Retry-After (delta-seconds then HTTP-date), retry-after-ms, x-ms-retry-after-ms, then X-RateLimit-Reset; the rate-limit-reset jitter now spans [100%, 120%]. RetryPolicy validates its scalar configuration at construction: retry counts must be non-negative, durations finite and non-negative, and the jitter fraction within [0.0, 1.0]; collection settings stay defensively copied. Adds regression coverage for fatal non-SDK errors (MemoryError and RecursionError) propagating unretried and untrailed, and for a single policy instance staying stateless under concurrent and repeated use. The total- timeout budget remains an always-on knob with an effectively unbounded 7-day default, modelled on the reference client; that choice is recorded in the conformance ledger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
…e controls Harden how the sync and async redirect policies handle credentials across a redirect chain, and expose two configuration points a caller previously could not reach. - Strip the Authorization header before every reissue, including a same-origin hop and the 303 GET rebuild. Re-stamping a credential for a known origin is the auth layer's job: a downstream auth policy re-applies it for a same-origin reissue, so ordinary authed redirects still carry it while a bare caller-set header no longer leaks past a redirect on its own. - Judge cross-origin against the original seed request origin rather than the immediately preceding hop, and additionally strip the origin-scoped Cookie and Proxy-Authorization headers on a cross-origin reissue. A same-origin sub-redirect that lands back on a foreign host can no longer re-expose them. - Reject an HTTPS-to-HTTP scheme downgrade by default (the hop is not followed and the current response is handed back). A new allow_scheme_downgrade opt-in permits it and logs a warning; the check runs on each hop transition and credential stripping still applies. - Add an optional should_redirect predicate that fully overrides the built-in follow decision. It receives an immutable RedirectCondition snapshot — the current response, the count of redirects already followed, and the insertion-ordered visited URIs including the current request's — so it cannot mutate the live cycle-detection state. Both policies share one per-hop decision helper, so the behaviour is identical on the sync and async pillars. The executable specs that previously pinned the missing behaviour as expected failures now assert it directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
…ce-id flavours
Extend the observability layer with the missing pieces of the structured
logging and tracing contract:
- URL-value redaction for header values: `UrlRedactor.redact_header_value`
redacts a parseable absolute value like a request URL, and for a relative or
unparseable value keeps the path while dropping any query or fragment (which
can carry an OAuth code, a pre-signed signature, or an implicit-flow token),
appending a fixed "?***" marker; a value with neither is returned verbatim.
- A shared `HeaderRedactor` used by both the sync and async logging policies so
the rules cannot drift: header logging is gated by a name allow-list of
diagnostic, non-credential headers; a disallowed name is either marked
REDACTED or omitted; URL-valued headers (Location / Content-Location) route
their value through the URL redactor.
- Selectable HTTP logging granularity via `HttpLogDetailLevel`
(none / headers / body): none emits no request/response events, headers omits
body capture, body adds a bounded preview. Span lifecycle and metrics are
owned by the tracing policy and run regardless of this level.
- A stable structured field vocabulary on the log events: http.request.method,
url.full (always redacted), http.response.status_code,
http.response.duration_ms, and http.{request,response}.header.* fields; a
failure now emits an http.response event carrying error.type and the cause.
- Trace-id generation flavours on `TraceId.generate`: W3C (32 lowercase hex),
Datadog (64-bit unsigned decimal, never the reserved zero), and no-op (always
the all-zero invalid sentinel).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
Several HTTP model invariants were only partially enforced or deferred to
the transport; this closes the gaps at the point of construction.
- Request: reject a body on TRACE and CONNECT at construction, not just
GET/HEAD. Both are body-less by classification (TRACE forbids an entity
body; CONNECT's body has tunnel-specific meaning), so a payload on either
is now a construction error rather than a wire-time surprise, and
with_method drops the body when downgrading to them.
- Method/retry: introduce a single idempotent-method set
{GET, HEAD, OPTIONS, PUT, DELETE} and derive the default retry method
allow-list from it, so the allow-list and the inherent replay-safety gate
can no longer diverge. This drops the stray TRACE entry the old allow-list
carried.
- MediaType: reject control characters (C0 except HTAB, plus DEL) and
non-ASCII bytes in the type, subtype, and every parameter key/value at
construction, using the same predicate as outbound header-value
validation. A media type can now always be emitted as a Content-Type
header without a late rejection, and a CR/LF injection is refused where the
type is built instead of at the header/multipart boundary.
- FileRequestBody: validate fail-fast at construction — the file must exist
and be a regular file, and the offset (and offset+count) must fit the size
captured then. content_length now reports the exact upload size from that
captured size rather than re-stat-ing lazily.
- Response parse latch: memoize a thrown decode failure and re-raise the same
error on every later parse, instead of re-reading the already-consumed
single-use body and surfacing a different error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
A file-backed request body declares its Content-Length from the size it stats at construction, but the file can be truncated afterwards. The read path read to EOF and returned silently, so a shrunk file produced a body shorter than the declared length — a Content-Length mismatch that would put a truncated request on the wire. Bound each transfer to the declared content length and, if EOF arrives before delivering it, raise naming transferred-of-total instead of finishing quietly. Reads are now capped at content_length() for both the read-to-EOF sentinel and an explicit count, so a grown file no longer over-runs the declared length either. Add coverage for the short-write detection (rest-of-file and explicit count) and for the full construction fail-fast contract (existence, regular-file, offset/count window). Update the fresh-handle ownership test to mutate the file in place at the same length, which still proves no descriptor or bytes are cached without depending on a length change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
Configuration gains a `without_override` derivation that peels a single key back off the override layer copy-on-write, so a later lookup for that key falls through to env -> property -> default exactly as if the override had never been set. Removal never forces the key to resolve to None, removing an absent key is a harmless no-op, and the derived configuration shares the source's env and property seams like `with_override`. The NO_PROXY parser now honours a backslash escape before a literal comma and follows a well-defined split -> drop-empty -> unescape -> trim order. An escaped separator survives as a literal in the emitted token, zero-length fragments are dropped, and because the drop precedes the trim a whitespace-only fragment is retained as an empty token rather than removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
…tency StagedPipelineBuilder and its async twin gain append_all/prepend_all for adding a batch of policies in one call. append_all preserves the batch's iteration order within each stage; prepend_all prepends each element individually and therefore reverses the batch order. Both apply all-or-nothing, and the ordering asymmetry is documented on the methods and the module so callers get predictable results. Re-installing the same policy instance onto an already-occupied pillar is now an idempotent no-op. Reference identity, not value equality, distinguishes re-adding the incumbent object (a silent no-op, no duplication) from a distinct second occupant, which still raises a descriptive error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
The conformance fixes add public API — retryable-status classifier helpers, millisecond Retry-After parsing, redirect credential controls, configuration override removal, trace-id flavours, and pipeline batch-add helpers. Refresh the committed surface baseline to match.
Point the blocking traceability audit at the real consolidated requirement index's full subsystem set so the gate fails CI on any uncovered MUST-level requirement that is not ledgered as not-applicable or a documented deviation.
Add docs/conformance-ledger.md — a machine-generated record mapping all 645 normative requirements to their coverage state (test-covered, deviation, or not-applicable) with rationale, and link it from docs/deviations.md. This is the executed-and-archived conformance record: 505 test-covered, 43 deviation, 55 n/a, zero unaccounted MUSTs.
The logging policies defaulted their detail level to BODY, so simply raising the SDK's stdlib logger to INFO would immediately emit request/response log events including an up-to-8 KiB request/response body preview. Bodies routinely carry credentials and PII, so this made the wire log a leak that a caller could trigger without ever asking for body logging. Default `detail_level` to `NONE` in both `LoggingPolicy` and `AsyncLoggingPolicy`. At NONE no http.request/http.response log event is emitted; a caller must explicitly opt into HEADERS (events plus allow-listed headers) or BODY (adds the bounded body preview). Span lifecycle and metrics are owned by the tracing policy and are unaffected by this level, so tracing and metrics still run on every request. The default pipelines wire the policies with the default, so they now log nothing until a caller opts in. Tests that assert log emission now pass an explicit detail level matching what they exercise (HEADERS for url/header assertions, BODY for body-preview assertions); a new regression test pins that a default-constructed policy emits no wire log event at INFO while a BODY-level one does. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5
aiohttp 3.9 is the supported floor, but it does not run on Python 3.14: 3.14 rejects break/continue/return inside a finally block, a pattern aiohttp used until 3.13, so aiohttp 3.9 raises SyntaxError on import there. Split the floor by interpreter — keep >=3.9 below 3.14 and require >=3.13 (the first release with a 3.14 wheel) on 3.14 and newer — so the oldest-supported-dependency resolution resolves to a working aiohttp on every supported Python.
test_noop_metric_calls_retain_no_memory measures a tracemalloc allocation delta to prove a no-op metric call retains nothing. Under a coverage/line tracer the tracer allocates alongside the code under test, so the delta no longer isolates a per-call leak and the assertion flakes. Skip it when a tracer is active (detecting both settrace-based tracers and coverage's sys.monitoring path on 3.12+); it still runs and asserts the invariant in ordinary, untraced test runs.
…plit The adapter maps aiohttp's connect-vs-read timeout classes (ConnectionTimeoutError / SocketTimeoutError), which aiohttp introduced in 3.10; against the previous >=3.9 floor a lowest-direct resolution pulled aiohttp 3.9, where those attributes do not exist and the adapter's error mapping raised AttributeError. Raise the pre-3.14 floor to 3.10 (the 3.14 floor stays at 3.13).
mypy reports the extra-positional-argument call as a call-arg error, so the suppression on the SdkError over-application test must name [call-arg] rather than [misc]; the stale [misc] code read as an unused ignore under --strict.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This builds out the Python SDK as the counterpart to the Java SDK: an HTTP-client
toolkit (abstractions, models, and pipelines — not a bundled HTTP client), with
transport plugged in through a narrow
HttpClient/AsyncHttpClientseam. Thearchitecture mirrors the Java shape — immutable HTTP models, staged pipeline
policies, a context promotion chain, and a Tier-1 codegen layer — but the public
API is Pythonic: frozen dataclasses instead of builders,
Protocols instead ofinterface-plus-impl modules, context managers for deterministic cleanup, and
native
bytes/BytesIO/BinaryIOin place of a pluggable I/O seam.It is a
uv-managed workspace of six distributions sharing thedexpace.sdk.*namespace: the dependency-free
coretoolkit (its only runtime dependency isfurl), four reference transport adapters (stdlib, httpx, aiohttp, requests), anda transport conformance kit (TCK) that certifies any adapter against a shared
behavioural battery.
What's included
Request/Response/Headers/MediaType/
Url/Status/ETag/HttpRange, case-insensitive header handling, andtyped, replayable-or-single-use request/response bodies with logging taps.
Pipeline/AsyncPipelinewith staged policies: redirect,idempotency, retry, set-date, client-identity, logging, and tracing, assembled by
a staged builder, plus the context promotion chain and a thread-safe
ContextStore.redirect handling with strict credential hygiene, bearer/basic/key credentials,
and Basic/Digest challenge handling with a token cache.
pagers (sync and async), an SSE parser and connection layer, tracing/metrics SPIs
with URL and header redaction, and a layered configuration model.
each certified by the shared conformance battery (
pytest --pyargs dexpace.sdk.tck).petstore example.
Conformance
Every normative requirement in the consolidated requirement index (645 across 19
subsystems) is accounted for: covered by a test tagged with its requirement id,
satisfied by a documented deviation, or recorded as not-applicable to the Python
port (the Java/JVM-specific idioms and the deliberately-omitted byte-stream seam).
A blocking CI gate fails the build on any uncovered MUST-level requirement that is
neither tested nor ledgered, and
docs/conformance-ledger.mdarchives the fullmapping. Security-relevant behaviours are covered explicitly — cross-origin
credential stripping on redirect, HTTPS→HTTP downgrade denial, retry safety for
non-idempotent methods, and default-off body logging.
Quality gates
mypy --strictclean;rufflint + format clean; import-linter contracts kept.lock-sensitive suites, forward-compatibility skew checks, a single-version-source
gate, and reproducible build settings.
py.typedshipped for downstream typing.Notes
No version tag or package publish is included here — that remains a separate,
deliberate release step.
🤖 Generated with Claude Code
https://claude.ai/code/session_01RZRRWesJkVFymrxxHGZde5