Skip to content

Add native contract-generated ordinary API client - #47

Draft
kvz wants to merge 38 commits into
mainfrom
sdk-contract
Draft

kvz wants to merge 38 commits into
mainfrom
sdk-contract

Conversation

@kvz

@kvz kvz commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Why

Prove canonical-contract generation in a second native language and deliver real workflows through
that new surface, without implementing the scenarios through legacy SDK adapters.

Scope

  • Add an opt-in contract package covering 37 ordinary operations and three tus bindings, with
    navigable domain types, semantic unions, exact integers and omission/null distinctions.
  • Deliver wait, cancel, fixed-size ReaderAt upload and saved-session resume. Validate the uploader,
    metadata, digest and actual server offset. Keep upload buffers owned until HTTP body closure.
  • Bound safe reads and recovery, respect backoff, and never retry creation or cancellation.
  • Return REQUEST_ABORTED as a finite, unsuccessful wait result. Explicit cancellation still
    reaches the owner; a subsequent aborted status cannot hide a failed DELETE.
  • Preserve proxy prefixes, caller contexts and credential isolation. Document that trusted origins
    are additive and the upload deadline includes hashing, persistence and transfer.
  • Run shared upload/resume acceptance through the new public layer; Go resume is no longer skipped.
    Smart CDN remains a local signing helper. Keep Go 1.15 and the 8 MB generated-source cap.

Existing-API behavior changes / release notes

The existing CreateSignedSmartCDNUrl now follows the shared canonical signing vectors: path
escaping, UTF-16 key ordering and millisecond expiration can change generated URLs/signatures.
Callers comparing or caching previously generated strings must account for that correction.
Legacy WaitForAssembly also preserves caller cancellation/deadline identity and continues polling
ASSEMBLY_REPLAYING rather than returning it as complete. Public API shapes remain compatible;
these are intentional, tested behavior changes, not a claim of unchanged behavior.

Verification and remaining gates

  • Regenerated from committed producer f13c20fe24, digest
    a72c65901795052a96036a2fa6beed7ac1885299fb3d6dfec8a011e2efc6f819.
  • Native/shared race tests, example tests/build, opt-in import guard and go vet ./... pass.
    Contract/example race tests and vet pass again after documentation edits.
  • 49ccde560e passes all five CI versions (Go 1.15, 1.20, 1.24, 1.25 and 1.26).
    Documentation follow-up 9af9601a29 is pushed and exact-head CI is being monitored.
  • Opus findings are triaged. The full council attempt is incomplete because its Codex reviewer
    and arbiter hit provider quota; this is not multi-model approval.
  • API2's refreshed native source pins and real local API2/tusd canaries remain a producer gate.
    Prior passing runtime receipts do not establish acceptance of this newer finite-outcome head.

The historical timed Opus reader completed all five live tasks on a35909efb8, including a real
process restart and HEAD-confirmed resume. It did not test subsequent fixes. No new live credentialed
tests were run for this follow-up.

Boundaries

Fixed-size ReaderAt inputs only; no deferred lengths, concatenation or non-seekable streams.
Synchronous caller-owned readers and callbacks must return promptly. Local failure does not prove
remote cleanup. Session URLs are private. SSE and Webhook receivers remain separate.
Empty unrelated tus metadata and deterministic multipart ordering are tracked interoperability
follow-ups before releasing this draft; they were not silently declared verified.

All three SDK PRs remain draft: do not merge, release or deploy.
Canonical record: API2 docs/prompts/2026-07-09-handover-sdks-branch-restructure.md.
Native checklist: docs/prompts/2026-09-29-contract-finality.md.

Companions: https://github.com/transloadit/api2/pull/9252 · transloadit/node-sdk#517.

@codecov-commenter

codecov-commenter commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.33363% with 329 lines in your changes missing coverage. Please review.
✅ Project coverage is 4.97%. Comparing base (b567a3e) to head (9af9601).

Files with missing lines Patch % Lines
contract/transport.go 74.52% 76 Missing and 44 partials ⚠️
contract/workflows.go 77.99% 59 Missing and 53 partials ⚠️
examples/contract-workflow/main.go 2.10% 93 Missing ⚠️
transloadit.go 93.10% 2 Missing ⚠️
wait.go 60.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #47       +/-   ##
==========================================
- Coverage   82.60%   4.97%   -77.64%     
==========================================
  Files           6      10        +4     
  Lines         345   32357    +32012     
==========================================
+ Hits          285    1609     +1324     
- Misses         32   30505    +30473     
- Partials       28     243      +215     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kvz

kvz commented Sep 25, 2026

Copy link
Copy Markdown
Member Author

Verification footprint for 0ad7504039190ca004bfad76d48cdca9be1a6d01 against b567a3eefef5ce3e74767ba239cb00cbaf33ac2c:

1,040 authored additions; 55,282 generated additions; 1 deletion. Generated classifications and destructive regeneration procedure are recorded in the shared API2 receipt and maintainer guide. Generated lines are not a claim of native/runtime proof; both SDK CI and API2's local-native acceptance passed separately.

Raw canonical numstat (additions, deletions, path):

4	0	.gitattributes
1	1	Makefile
34	0	README.md
39316	0	contract/client_generated.go
15894	0	contract/coverage.json
200	0	contract/live_test.go
45	0	contract/manifest.json
461	0	contract/transport.go
332	0	contract/transport_test.go
27	0	contract/wire-vectors.json
8	0	transloadit.go

@kvz
kvz marked this pull request as draft September 25, 2026 13:27
@kvz
kvz marked this pull request as ready for review September 25, 2026 15:25
@kvz

kvz commented Sep 25, 2026 •

Copy link
Copy Markdown
Member Author

Final source footprint and review receipt

The approved first-party SDK slice is implemented. Existing SDK entrypoints remain intact; the new namespaces are experimental. No SDK release, deployment or merge was performed.

Repository Base Final source head Authored additions Generated additions Deletions Files
API2 c449a7b1f871e12e30a2a544fce1739187d83cf8 8e601e6ffd274fb9f7c00032a3e2b30b3cd0f3ae 2,993 398 143 27
Node SDK 34970b60c770a34c3dc227ad9ecb1ec9283ecd77 30cb8807dab7b8098ba8eb5c53635c84e08b98cc 799 29,488 2 15
Go SDK b567a3eefef5ce3e74767ba239cb00cbaf33ac2c 1897a050f4ba658787abbd9e113c2468c03f0ba0 1,363 84,283 1 11
Total 5,155 114,169 146 53

Measured with the canonical text/Myers/no-renames/no-indent-heuristic numstat command from the living document. Generated ownership: API2's generated contract; each SDK's client/manifest/coverage/vectors; Node's existing legacy-package README/package projection. All native transport, tests, documentation and receipt additions count as authored. The conservative historical authored lower bound is now 59,141, not a fully reconciled program total or an additional footprint authorization. The last API2 increment is the targeted cold-artifact test-budget correction and its CI receipt; production and generated SDK bytes are unchanged.

All council findings have dispositions and fail-first evidence in API2's repodocs/prompts/2026-09-25-sdk-contract.md (six producer, five Go, two Node rounds). Final guards are deliberately narrow and fail closed. Full yarn check, exact-byte generation checks, native wire/race/vet/example checks, actual Go 1.15 tests and both devdock SDK suites pass.

All three exact-head CI runs are now green: API2 36161865722, Node 36136692611, and Go 36153479049. The prescribed API2 watcher finished with all checks passed on September 25 at 17:34 UTC. The generator suite passed in 2.8 minutes under coverage and the actual native SDK server canary in 30.5 seconds. API2 changed-line coverage is 93.62% (514/549); the unchanged 80% gate passed. No review threads were open at this final checkpoint.

The green API2 run also reports two already nonblocking cloud-image .flaky.vitest.ts failures: Fal's visual-diff artifact upload and SVG generation's upstream 503. Their sources, fixtures, thresholds and classification are unchanged. No new test skip or failure waiver was added. The preceding related SDK regression timeout was fixed and passed, not classified as unrelated. All three PRs are ready for review; no merge, SDK release or deployment was performed.

Before landing: API2 main has since advanced by one Slack-workflow/test commit, 8e5c2e0dfe8a2e2f963324a7ec9140179945c165 (#9258). API2 is mergeable but behind and still needs review; integrate and validate that main update before merging. The exact-head CI and footprint receipt above remain unchanged. Node and Go are cleanly mergeable.

Producer first, then consumers:

@kvz
kvz marked this pull request as draft September 28, 2026 15:40
@kvz

kvz commented Sep 28, 2026

Copy link
Copy Markdown
Member Author

Type-identity acceptance receipt, September 28, 22:24 UTC.

  • Exact head 3729aa43116d8deca6d04b5020390e6ad3bf9e02 passes
    CI run 36488758965 on all five
    versions: Go 1.15, 1.20, 1.24, 1.25 and 1.26. The watcher exited successfully.
  • Local contract race tests, vet, examples and exact generated-byte checks pass. API2's hash-pinned
    native canary uses the migrated source-owned domain types against owned local resources. The
    final commit only clarifies generated multipart support, so the native source pin remains valid.
  • Council's final Go finding was the corrected README wording. The later Node review identified an
    additional cleanup/routing gap in both examples: cancellation must safely follow the returned
    owning uploader and verify a terminal response. This is a documented next-slice merge blocker,
    not waived by green type tests or CI. API2's living document records the source evidence and
    multi-uploader acceptance requirements.

The type-identity cleanup is accepted locally and in CI. This does not establish generated-client
workflow parity. The PR stays draft; nothing was merged, released or deployed.

https://github.com/transloadit/api2/pull/9252
transloadit/node-sdk#517
#47

@kvz

kvz commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Wait/cancel acceptance receipt (2026-09-29)

Final candidate: 33ef9475e9de70eafdab4b137dd586a7aca91f92. Draft, unmerged, no release.

  • Exact-head CI is green on Go 1.15, 1.20, 1.24, 1.25 and 1.26.
  • Contract/example race tests, shared lifecycle cases, vet and example builds pass after review. Generated output matches the producer byte-for-byte.
  • The final functional revision 54bde4cac0ee8370a1976d40e6c8c0a5aada8416 is hash-pinned in API2 adeaef40ba89b05652eac0809126d2c794b9f265. Its four fresh local suites pass, with zero flakes, including completion/cancellation against actual API2 runtime.
  • Final review is triaged. Accepted findings have fail-first regressions, including twelve host-case/default-port/proxy cases for both wait and cancel. The remaining P3 metadata observation for responses over 128 MiB is intentionally not converted to retriable ResponseError: retrying such bodies would undermine the resource-safety guard. A distinct non-retriable status-bearing size-limit error is optional future work. Three explanatory code-comment lines are the entire difference from the pinned functional revision; no runtime or generated-code change followed the review.

This proves bounded wait/cancel through generated operations, not all workflow parity. It does not claim two complete distributed uploader processes, a new blind-reader test, tus/upload/resume support, a release or a deployment. The latest API2 remote CI is still running and must be reported separately.

Producer: https://github.com/transloadit/api2/pull/9252. Node: transloadit/node-sdk#517.

@kvz

kvz commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Upload/resume acceptance receipt, September 29

This closes the upload/resume slice's historical pending review/check items in the canonical living
document and repodocs/prompts/2026-09-27-sdk-dx.md. It is not merge, release or deployment approval.
All three PRs remain draft.

Repository Exact candidate CI
API2 49824420400c6947a39a260c314ef26a1d95d608 Green: API2, attempt 2, Utils, CRM routing, Statuspage
Node 88ec0f37a201ceff422696f2e6fedf8b4527ee26 Green: all 11 checks
Go ee28c68dd85d3878c0affffaa429714628581edc Green: all five Go versions

Local acceptance

  • Full API2 and Node yarn check pass. Node's focused native/shared suites pass 213 tests.
  • Both packed Node package names pass strict installed-consumer compilation, including their
    contract entrypoints and shipped example.
  • Go go vet ./..., native/example race tests and shared workflow race tests pass.
  • Canonical contract/OpenAPI and both generated SDKs pass exact-byte check mode.
  • All six post-main pinned devdock suites pass: workflow paths, Companion compatibility,
    dispatch-registry, SDK generation, shared SDK workflows and actual local API2/tusd native canaries.
    The owned clone-7 test container is stopped afterward; its persistent volume is retained.

Main a4f9e02f4e landed while the preceding CI was running. The one conflict was in workflow tests:
retain both SDK Go-toolchain guards and remove only the obsolete vendored Companion hydration
tests, matching main's published-package migration. Generated contract/SDK bytes are unchanged.
Superseded API2 runs 36593626095 and 36590500301 were canceled after the new merged candidate
was pushed; cancellation is not a passing CI result. Node/Go commits did not change.

API2 attempt 1 stopped in the Core suite before the API2 suites: 161/162 Core suites passed and
core/test/unit/alphalib/net.vitest.ts failed its negative TCP-readiness assertion because the
promise resolved instead of rejecting. Both the test and core/alphalib/net.ts, plus Core's package
and lockfile, are unchanged from main. Their Git blob IDs are respectively
c70734526c6c8f1bb2d5c56c7aa54ed4bd347d0a and 3b77eee8fc5c4d9a358ae78901e20b672830c7f7.
The fixture closes an ephemeral listener, then assumes that port stays unavailable. A disposable
devdock reproducer that binds a second listener in that gap triggers the identical assertion;
the original isolated suite passes. This verifies the unrelated fixture race, not which process
held the port in CI. The probe is retained only under ignored local evidence and the container is
stopped. No assertion was weakened. Only the failed CI job was retried on the unchanged commit;
attempt 2 confirms all 162 executed Core suites pass, including this TCP test (one non-test fixture
is skipped), and the complete workflow is green. Stabilizing this inherited negative-port fixture
is a separate follow-up. The intermediate report is retained under
tmp/sdk-tus.vVDVAO/api2-attempt2-results.json, generated at 2026-09-29T16:36:07.172Z.

The final API2 report records 1,821 passed suites, 19 passing suites already marked flaky, one
non-blocking flaky failure and 67 skips. Every SDK-specific suite (models, command, generation,
shared workflows and native runtime) passes without a flaky disposition, as do dispatch-registry,
workflow paths and Companion compatibility. The remaining flaky failure is the pre-marked
cloud_ai/image_generate/provider-fal.flaky.vitest.ts image comparison: difference 0.0903001
against threshold 0.05. Its test, comparison helper and fixture are unchanged from main. No image
fixture, threshold, skip or flakiness marker was changed to obtain green.

GitHub tested merge 17c47d1b16d8c10dd4eee9082aa7090a8cfff8b5, whose parents are main
a4f9e02f4e and candidate 4982442040. Its tree is identical to the candidate's
df00ef5a2bf3f98ec09f1296f34c04184a0c26a3. The prescribed watcher reports six passing checks,
zero failing and zero pending at 17:19:21Z. All checkouts are clean; no owned devdock remains.

Review disposition

The last API2 council found no issues. All accepted native council findings are reproduced and
fixed, with fail-first regressions. The final small dispositions are not claimed as another clean
council run: processing failure does not prevent read-only confirmation of finished file transfer;
receipt URLs share capability normalization/admission; ownerless aborted cancellation retains its
unconfirmed outcome; response-body cleanup cannot discard HTTP retry/backoff metadata; configured
proxy prefixes cannot be bypassed without explicit bare-origin admission. Applicable fixes are
mirrored. Earlier fixes cover request deadlines, partial/lost PATCH responses, stopped-state write
prevention, Go buffer ownership and bounded response handling. No creation/cancel write retry.

Independent consumer evidence and limits

Both fresh Opus 5.5 readers completed all five tasks, with successful compiles and actual new-process
resume after 65,536/147,052 bytes, HEAD confirmation and no replacement POST or legacy workflow
fallback. Go completed live tasks in about 2m48s and Node in 3m10s. Neutral reports were sealed
before separate hypothetical Rauch/DHH assessments.

Those readers tested Node 50b659d5ac and Go a35909efb8, not subsequent fixes. Two automated runs
are not a human usability study or exhaustive race proof. Node's checksum/dimension observations
were not all assertions; Go recorded SHA-256 without an independent expected checksum. Shared
exact-byte vectors and local runtime tests are separate evidence. The cleanup-reconciliation
canary simulates only HEAD 404 and reads the matching receipt from real API2; it does not execute
the actual cleanup scheduler. Owned Templates were deleted, Assemblies completed/canceled and
secret checkpoints removed; results expire normally. No more live-reader writes under that budget.

Reader-driven example/transport clarification is included. Follow-ups, not implemented here:
public persisted-session parser reusing the native validator, more semantic source-owned Go names,
an honest typed Template-content view and separated consumer/maintainer guidance. Do not narrow
general Template JSON or erase null/omission distinctions merely to simplify generated types.

Companions: https://github.com/transloadit/api2/pull/9252 · transloadit/node-sdk#517 · #47.

@kvz

kvz commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Exact-head CI receipt: 9af9601a29b5fccbe3e2f86769adfd3b700d613d is green in run 36627589552, all five supported Go versions passing. Contract/example race tests and go vet ./... also pass after the documentation follow-up. This closes the CI gate, not the quota-blocked full council or the refreshed producer runtime-canary gate. Nothing merged or released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants