Skip to content

[FEATURE](codegen) Carry field defaults through the IR into the Markdown reference, and refuse default_factory - #759

Open
Seth Fitzsimmons (sethfitz) wants to merge 4 commits into
mainfrom
carry-default-through-ir
Open

Seth Fitzsimmons (sethfitz) wants to merge 4 commits into
mainfrom
carry-default-through-ir

Conversation

@sethfitz

@sethfitz Seth Fitzsimmons (sethfitz) commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Codegen renderers can read each field's declared default: a target that supports defaults can emit one, and a target that does not can report that it dropped one. The Markdown reference is the first to emit one, as a note on the field's row. A model that declares default_factory fails extraction with an error naming the field, where before the factory was discarded silently.

FieldSpec, the codegen IR's per-field record, is (name, shape, description, is_required, is_optional). Extraction reads FieldInfo.default and FieldInfo.default_factory, uses them to settle one question — whether the field is required — and discards the values. Nothing downstream can emit a field's default, report that it had to drop one, or refuse one.

This carries the value, renders it in Markdown, and refuses the one form of it no target can render.

FieldSpec.default, with UNDEFINED exported from extraction.specs for the absent case. UNDEFINED is Pydantic's own PydanticUndefined under a shorter name, not a second sentinel: a consumer reads spec.default is UNDEFINED without importing pydantic_core to ask the IR a question, the two values meaning "no default" cannot drift apart, and copy.deepcopy and pickle both return the same object, where a freshly minted singleton returns a different one.

A declared default of None is kept as None, distinct from absence: None is legal, and collapsing the two leaves a consumer unable to tell them apart. = None is also how a Pydantic field is declared optional, and nothing in the source separates that from an intended null default, so the IR reports what it finds. Pydantic's own model_json_schema() does the same, emitting "default": null.

Omitable[T] installs Field(default=MISSING) to get JSON Schema omissibility instead of Pydantic nullability. MISSING is machinery for "this key may be absent", not a value anyone declared, so it normalizes to UNDEFINED — and Feature.bbox and Feature.id are both Omitable, inherited by every feature model, so without that normalization every feature type in the schema would claim two defaults it does not have.

default_factory is refused at extraction, naming the model and field. #695 forbids it outright, because a callable cannot be rendered into any representation but Python. The other two things the extractor could do are both silent: invoking the factory freezes one sample of a value meant to be produced per instance, and the frozen sample is indistinguishable from a declared literal; recording it as "no default" hides a declared default from every consumer. Refusing is the only option the author ever sees.

The Markdown reference shows a non-null default as an italic Default: note in the field's description cell, ahead of any constraint notes. It goes on every row, including the dot-notation rows expanded inline from a sub-model, so a reader who never opens the sub-model's page still sees it. A default of None gets no note: that is the optional spelling above, and the (optional) qualifier in the type column already says it. An enum member shows its value, the form a record carries.

No field in the schema declares a factory, so nothing that exists today is refused. The published reference does not change either: every declared default in the schema is = None on an optional field, and #695 forbids any other. All 139 generated pages are identical before and after this branch.

This does not warn or fail on a declared default. Enforcing #695's policy belongs downstream, in a schema validator distinct from a data validator, and carrying the value is what makes it possible.

Reference

  1. Closes Carry field defaults through the extraction IR #758.
  2. [ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default #695 — the no-non-null-default policy, which forbids default_factory outright. This carries the value so a target can see it; it does not reintroduce defaults to the schema.
  3. [ENHANCEMENT](codegen) Decide support-or-reject for Pydantic Field kwargs the extractor ignores or crashes on #640 — the support-or-reject frame for Field() kwargs. [ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default #695 settled default as policy; this settles it in the extractor, and settles default_factory alongside it.
  4. [FEATURE](codegen) Add a stac-table-columns codegen format #724 — the stac-table-columns renderer, which reads FieldSpec.default to report defaults it cannot carry.
  5. [FEATURE] Carry deprecation through extraction and render it in the Markdown reference #674 — the sibling carrier for deprecation, which adds is_deprecated to the same record.

Checklist

  1. Add relevant examples. — N/A: no schema change; this is codegen behaviour, exercised by inline test fixtures rather than by published examples.
  2. Add relevant counterexamples. — N/A, same reason.
  3. Update any counterexamples that became obsolete. — N/A: no published example or counterexample changes meaning here.
  4. Update in-schema documentation using plain English written in complete sentences, if an update is required. — N/A: no schema field changed.
  5. Update Docusaurus documentation, if an update is required. — Not required. The generated reference is unchanged, because the schema declares no non-null default.
  6. Review change with Overture technical writer to ensure any advanced documentation needs will be taken care of, unless the change is trivial and would not affect the documentation. — Not required: the reference gains a note format that no Overture field triggers.

Documentation website

Docs preview for this PR.

Extraction read FieldInfo for description and requiredness but dropped
the declared default, so anything needing to know a field declares one
-- a renderer, or a check enforcing #695's policy -- had to re-walk
Pydantic and redo the unwrapping extraction already does.

FieldSpec now carries `default`, and specs.py exports UNDEFINED for the
absent case. UNDEFINED is Pydantic's own PydanticUndefined re-exported
under a shorter name rather than a second sentinel: a consumer reads
`spec.default is UNDEFINED` without reaching past the IR into
pydantic_core to ask the IR a question, the two values meaning "no
default" cannot drift apart, and copy.deepcopy and pickle both return
the same object, where a freshly minted singleton comes back as a
different one.

A declared default of None stays None. It is legal, and collapsing it
into the absent case would leave a consumer unable to tell the two
apart. `= None` is also how a Pydantic field is declared optional; the
IR reports what it finds and leaves that judgement to the consumer,
because nothing in the source distinguishes the two.

Omitable[T] is Field(default=MISSING) -- machinery for JSON Schema
omissibility, not a value anyone declared -- so MISSING normalizes to
UNDEFINED. Feature.bbox and Feature.id are both Omitable and every
feature model inherits them, so without the normalization every feature
type would claim two defaults it does not have.

`default` is optional on FieldSpec, so no construction site changes.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
A field declaring `default_factory` had three possible fates and the IR
took the worst one silently. Invoking the factory freezes one sample of
a value meant to be produced per instance, and the frozen sample is
indistinguishable from a declared literal. Recording it as "no default"
-- what the carrier did until now -- hides a declared default from every
consumer, including any check looking for declared defaults. #695
forbids default_factory outright for the same reason a callable cannot
be rendered into Markdown, a PySpark expression or JSON Schema.
Refusing is the only one of the three the author can see.

So extraction now raises, naming the model and field. This is #640's
shape applied to one more kwarg: decide support-or-reject, and reject
legibly rather than dropping in silence.

Nothing in the schema declares a factory today, so this refuses nothing
that exists. The extraction tests that used `Field(default_factory=list)`
did so incidentally, to give a recursive forward-ref model an empty
list; both drop the default entirely, since what they test is
forward-ref resolution.

`_is_field_required` no longer consults default_factory: the refusal
runs first in the same loop, so a factory cannot reach it.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz Seth Fitzsimmons (sethfitz) added the change type - minor 🤏 Minor schema change. See https://lf-overturemaps.atlassian.net/wiki/x/GgDa label Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

🗺️ Schema reference docs preview is live!

🌍 Preview https://staging.overturemaps.org/schema/pr/759/schema/index.html
🕐 Updated Sep 23, 2026 00:07 UTC
📝 Commit 2550142
🔧 env SCHEMA_PREVIEW true

Note

♻️ This preview updates automatically with each push to this PR.

FieldSpec carries each field's declared default, but the Markdown
renderer never read it, so a model declaring `level: int = 0` documented
the field as optional without saying what an absent value means.

A non-null default now renders as an italic `Default:` note in the
field's description cell, ahead of any constraint notes, on every row of
the field table -- including the dot-notation rows expanded inline from
a sub-model, where a reader who never opens the sub-model's page still
needs it.

A default of None gets no note. `= None` is how a Pydantic field is
declared optional, and the `(optional)` qualifier in the type column
already says so -- a note would repeat it on every optional row. Enum
members show their value, the form a record carries, and an empty string
shows as `""` rather than the blank cell an example table uses for it.

The note helper is renamed `_annotate_notes`, since a default is not a
constraint.

The published reference does not change: #695 forbids non-null defaults,
and every default the schema declares is None. Generating all 139 pages
before and after this commit produces identical output.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz Seth Fitzsimmons (sethfitz) changed the title [FEATURE](codegen) Carry field defaults through the extraction IR and refuse default_factory [FEATURE](codegen) Carry field defaults through the IR into the Markdown reference, and refuse default_factory Sep 23, 2026

This branch was successfully deployed

1 active deployment
staging — 25501420 Deployed Sep 23, 2026 by sethfitz via Deploy #571
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change type - minor 🤏 Minor schema change. See https://lf-overturemaps.atlassian.net/wiki/x/GgDa

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Carry field defaults through the extraction IR

1 participant