Skip to content

fix(generate): send falsy optional values as present and match variant tags by own property - #178

Closed
marc0olo wants to merge 1 commit into
fix/conversion-property-positionsfrom
fix/conversion-encoding
Closed

marc0olo wants to merge 1 commit into
fix/conversion-property-positionsfrom
fix/conversion-encoding

Conversation

@marc0olo

@marc0olo marc0olo commented Sep 16, 2026 •

Copy link
Copy Markdown
Member

Part of the naming and encoding chain.

Behaviour

This PR changes what goes on the wire — a fix rather than an API change: no type changes, and a canister relying on the old behaviour was relying on a bug. No BREAKING CHANGE footer for that reason.

you write you mean before after
{ count: 0n } present, value 0 [] — absent [0n]
{ count: undefined } absent [] []
decoding any variant a tag named constructor, toString, valueOf, … matched every value and won decodes correctly
a candid type named Object shadowed the global the conversions call escaped — Object_; a method of that name is unchanged

An opt nat carries two independent things: whether there is a value, and what it is. 0 is a perfectly good nat, so present with value 0 is not absent — but the old code answered the first question by testing the second for truthiness. Setting a field to 0n and omitting it produced identical bytes, so 0, false and "" could not be sent at all. A canister that was receiving nothing for those fields now receives what the caller asked to send.

null counts as absent alongside undefined, symmetric with decoding, which yields null for a bare opt and undefined for a record field. opt null stays ambiguous — present-with-null and absent are indistinguishable in any JavaScript mapping, as before.

Snapshot churn is 42 lines over 7 files, all of it the optional-field encoding (the example fixture repeated across the wasm-generate matrix). Variants whose tags are not inherited property names are untouched.

Why

Both were well-formed TypeScript that computed the wrong answer, which is why parsing, typechecking and snapshots all passed over them. Optional fields were tested for truthiness (value.count ? … : …). Variant tags were matched with in, which walks the prototype chain, so a tag named constructor matched whatever the value held and the first such tag won outright.

Reviewer note. in does double duty: runtime test and the narrowing TypeScript uses to discriminate the variant union. Replacing it with Object.hasOwn outright costs the member access inside each arm and the exhaustiveness of the last one, and churns every snapshot. So an own-property test is conjoined only for the twelve names in gets wrong — redundant at runtime, since in is implied by it — and the last arm asserts never only where a conjunction was needed. It is spelled Object.prototype.hasOwnProperty.call(value, name) rather than Object.hasOwn, which is ES2022: the wrapper ships into someone else's build, bundlers do not polyfill built-in methods, and going through Object.prototype also survives a decoded object carrying its own hasOwnProperty. Object is escaped through MODULE_NAMES rather than KEYWORDS: the keyword table also shapes method keys, so it would have renamed a method named Object while the wire call kept the name.

Testing

tests/conversion-runtime.test.ts runs the generated conversions instead of reading them — nothing else in the suite would have caught either bug, since the output was valid, typechecked and wrong. The tests fail against the previous generator, with expected [] to deeply equal [ 0n ] and expected 'toString' to be 'plain'. They cover the three encoding paths separately — record field, variant payload and bare opt argument — and execute the conversion of a candid type named Object, which is what proves the escaped declaration leaves the global reachable.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Moderate unresolved issues remain around variant payload coverage and shared Object identifier escaping.

Pull request overview

Fixes generated TypeScript conversions so falsy optional values remain present and variant decoding ignores inherited properties.

Changes:

  • Uses explicit presence checks for optional values.
  • Adds own-property validation for variant tags.
  • Adds runtime regression coverage and updates snapshots.
File summaries
File Reviewed change
tests/snapshots/wasm-generate/typescript/example/example.ts.snapshot Updated generated optional encoding snapshot.
tests/snapshots/wasm-generate/typescript-root-export/example/example.ts.snapshot Updated generated optional encoding snapshot.
tests/snapshots/wasm-generate/root-export/example/example.ts.snapshot Updated generated optional encoding snapshot.
tests/snapshots/wasm-generate/no-root-export/example/example.ts.snapshot Updated generated optional encoding snapshot.
tests/snapshots/generate/variant_payload_tags/variant_payload_tags.ts.snapshot Updated variant payload encoding snapshot.
tests/snapshots/generate/example/example.ts.snapshot Updated optional encoding snapshot.
tests/snapshots/generate/conversion_encoding/declarations/conversion_encoding.did.js.snapshot Updated generated JavaScript declarations.
tests/snapshots/generate/conversion_encoding/declarations/conversion_encoding.did.d.ts.snapshot Updated generated TypeScript declarations.
tests/snapshots/generate/conversion_encoding/conversion_encoding.ts.snapshot Updated generated conversion wrapper snapshot.
tests/generate.test.ts Registers the conversion encoding fixture.
tests/conversion-runtime.test.ts Executes regression cases; variant payload coverage for present 0n and absent values remains needed.
tests/assets/conversion_encoding.did Adds optional and variant regression types.
src/core/generate/rs/src/bindings/typescript_native/utils.rs Adds Object to shared keyword protection, which risks renaming generated APIs and should be separated from property-name escaping.
src/core/generate/rs/src/bindings/typescript_native/conversion_functions_generator.rs Corrects optional encoding and variant tag handling.
Review details

Suppressed comments (2)

src/core/generate/rs/src/bindings/typescript_native/conversion_functions_generator.rs:948

  • The new runtime regression test exercises optional primitive fields on a record, but not this separate discriminated-variant payload path. A variant { payload : opt nat } could therefore regress to the old truthiness check while the suite still passes; please execute the generated variant conversion with a present 0n payload and assert that it emits [0n] (and retains the absent case).
                                test: Box::new(self.field_is_present(
                                    param_name,
                                    &field_name,
                                    field_access.clone(),
                                )),

src/core/generate/rs/src/bindings/typescript_native/utils.rs:404

  • Adding Object to this shared KEYWORDS list also feeds get_ident_guarded, which is used for generated service method/interface names (compile_wrapper.rs:441 and new_typescript_native_types.rs:1003). A valid Candid method named Object (and a service basename object) therefore changes from Object to Object_, while the actor call still targets .Object; this introduces a generated API/name change not described by the PR. Please keep the global protection needed by Object.hasOwn separate from property-name escaping, or reference the global without reserving Object.
    "Object",
  • Files reviewed: 14/14 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 4102730 to 3510769 Compare September 16, 2026 13:38
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 3510769 to 0e9e2d3 Compare September 16, 2026 13:54
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 0e9e2d3 to ed3d25b Compare September 16, 2026 14:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved blocking issues were identified.

Review details
  • Files reviewed: 20/20 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from ea0e20c to 3a03234 Compare September 17, 2026 14:59
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 3a03234 to 51f61c0 Compare September 17, 2026 15:01
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 51f61c0 to e2c6fa9 Compare September 18, 2026 07:06
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from e2c6fa9 to 23a7692 Compare September 18, 2026 07:54
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 23a7692 to 2f40324 Compare September 18, 2026 07:55
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 2f40324 to 6d1c760 Compare September 18, 2026 08:14
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 6d1c760 to 563cd09 Compare September 18, 2026 09:07
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 563cd09 to 41e0ed7 Compare September 18, 2026 09:51
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 41e0ed7 to 7c3a069 Compare September 18, 2026 10:55
@marc0olo marc0olo changed the title fix(generate): encode optional fields and variant tags correctly fix(generate): send falsy optional values as present and match variant tags by own property Sep 18, 2026
@marc0olo
marc0olo force-pushed the fix/conversion-encoding branch from 7c3a069 to 9f9c853 Compare September 18, 2026 12:20
@marc0olo
marc0olo requested a lite review from Copilot September 18, 2026 13:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical unresolved variant-tag matching issue remains for inherited properties outside the hard-coded list.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 20/20 changed files
  • Comments generated: 1
  • Review effort level: Lite

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved review issues remain, and the regression coverage addresses the corrected behavior.

Review details
  • Files reviewed: 20/20 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@marc0olo

Copy link
Copy Markdown
Member Author

Force-pushed as part of a stack-wide rebase. This PR's own diff is unchanged — nothing to re-review here.

Only #175, #187 and #188 changed content in this round: a nested option reached through a type alias took the single-option path, so opt Inner with type Inner = opt Cfg was declared and converted unlike the opt opt Cfg it is. #187 fixes it for standalone types and record fields, #188 for variant payloads, and #175 corrects a doc comment that promised more than it did.

Comparing this PR's commit patch before and after the rebase, with context and ignoring index lines, gives a byte-identical diff.

…t tags by own property

Two conversions were well-formed TypeScript that did the wrong thing, which is why
parsing, typechecking and snapshots all passed over them.

An optional record field was encoded by testing the value for truthiness, so a
present `opt nat` of `0`, an `opt bool` of `false` and an `opt text` of `""` were all
put on the wire as absent. The test is now against `undefined` and `null`.

A variant tag was matched with `in`, which walks the prototype chain — so a tag named
`constructor`, `toString` or any of the ten other names every object inherits matched
whatever the value actually held, and the first such tag won. An own-property test is
conjoined for exactly those names, spelled `Object.prototype.hasOwnProperty.call` so
that it needs no ES2022 `Object.hasOwn` and survives a decoded object carrying its own
`hasOwnProperty`. `in` is kept elsewhere, and kept alongside, because
TypeScript discriminates the variant union with it: replacing it outright costs the
member access inside each arm and the exhaustiveness of the last one, which is why the
last arm asserts `never` only where a conjunction was needed.

`tests/conversion-runtime.test.ts` runs the generated conversions rather than reading
them. Nothing else in the suite would have caught either bug — the output looked
correct in every respect except what it computed. Both tests fail against the previous
generator.

A standalone `opt T` is tested the same way. It is decoded as `T | null` while the
record path yields `undefined` for an absent field, so a value passed from one to the
other has to read as absent either way.

`Object` joins the names a *declaration* must avoid, since this is the first generated
code to depend on the global: a candid type or a `.did` basename of that name would
shadow it for the whole module and make that call throw. It is deliberately not
added to the reserved-word list, which method names also consult — a method named
`Object` must keep its name.

An optional field named after an inherited member is guarded the same way. Reading
`value.toString` off an object that omits the field yields
`Object.prototype.toString`, so a presence test alone reports the inherited function
as a value and encodes it.
@marc0olo

Copy link
Copy Markdown
Member Author

Superseded by #197 for the optional-value encoding (0, false and "" sent as present). The own-property matching of inherited tag names (constructor, toString) is dropped as not realistic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants