Carry field defaults through the extraction IR #758

Type

Task

Scope

Multi-theme or Platform

Skillset

engineering

Description

The gap

FieldSpec, the codegen IR's per-field record, is (name, shape, description, is_required, is_optional). Extraction reads FieldInfo.default and FieldInfo.default_factory only to decide is_required (extraction/model_extraction.py), then discards both. Nothing downstream can emit a field's default, report that it had to drop one, or refuse one: extraction discards the value.

Why the IR should carry it, given #695

The argument against: #695 adopts the policy that an Overture schema field carries no non-null default, and the models already comply, so there is nothing worth carrying. That is true of this repo's models and not true of the IR.

1. = None is a declared default, and a target that can express it needs it. Pydantic's own model_json_schema() emits "default": null for optional fields. A renderer whose target has a notion of a default -- the Markdown reference, for one -- can only show it if the IR carries it.
2. A target that cannot express a default should say so rather than fall silent. The stac-table-columns renderer in [FEATURE](codegen) Add a stac-table-columns codegen format #724 logs every capability a STAC Column Object cannot carry. It can only log a dropped default if the IR carries one.
3. The IR is not Overture-only. Anything registered in the overture.models entry-point group is extracted by the same code, including extensions and third-party schemas built on overture-schema-system. [ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default #695 binds this repo's models; it does not make the extractor's blindness correct for everyone else's.

Proposed

Carry default on FieldSpec. Export UNDEFINED from extraction.specs for the absent case, so a consumer reads spec.default is UNDEFINED rather than importing pydantic_core to ask the IR a question. A declared default of None is kept as None and stays distinct from absence: None is legal, and collapsing the two leaves a consumer unable to tell them apart. = None is also how a Pydantic field is declared optional -- the IR reports the declared default it finds and leaves that judgement to the consumer, because nothing in the source distinguishes the two.

Omitable[T] installs Field(default=MISSING) to get JSON Schema omissibility instead of Pydantic nullability. MISSING is machinery for "this key may be absent", not a value anyone declared, so it normalizes to UNDEFINED. Feature.bbox and Feature.id are both Omitable and every feature model inherits them, so without that normalization every feature type would claim two defaults it does not have.

Refuse default_factory at extraction, naming the model and field. #695 forbids it outright, on the grounds that a callable cannot be rendered into any representation but Python. A field declaring one has three possible fates and the extractor currently takes the worst: invoking the factory freezes one sample of a value meant to be produced per instance, and the frozen sample is indistinguishable from a declared literal; recording it as "no default" hides a declared default from every consumer, including whatever is looking for one. Refusing is the only one of the three the author can see. This is #640's shape applied to one more kwarg -- decide support-or-reject, and reject legibly rather than dropping in silence.

No field in the schema declares a factory today, so this refuses nothing that exists.

Out of scope

• Warning or failing on a declared default, which is how [ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default #695's policy would be enforced. Carrying the value is what makes that possible; it is a separate change.
• Deprecation, which needs the same kind of carrier and has its own issue ([FEATURE] Carry deprecation through extraction and render it in the Markdown reference #674).

Reference

1. [ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default #695 -- the no-non-null-default policy. This carries the value so a target can see it; it does not reintroduce defaults to the schema.
2. [ENHANCEMENT](codegen) Decide support-or-reject for Pydantic Field kwargs the extractor ignores or crashes on #640 -- the support-or-reject frame for Field() kwargs. [ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default #695 settled default as policy; this settles it in the extractor, and settles default_factory alongside it.
3. [FEATURE](codegen) Add a stac-table-columns codegen format #724 -- the stac-table-columns renderer, the first consumer.
4. [FEATURE] Carry deprecation through extraction and render it in the Markdown reference #674 -- the sibling carrier for deprecation.