From a6b0ee382a2fdf8ac3c612819d533f78c0195d10 Mon Sep 17 00:00:00 2001 From: Sebastian Werner Date: Mon, 17 Aug 2026 11:56:57 +0200 Subject: [PATCH] feat(oxc-reference): produce primitive literals independently --- README.md | 7 +- ...teral-candidates-from-go-graph-identity.md | 4 +- ...te-rust-semantics-against-the-go-oracle.md | 4 +- ...rimitive-literals-independently-in-rust.md | 84 + docs/adr/README.md | 17 +- docs/evidence/README.md | 11 +- docs/porting/migration-contract.md | 22 +- internal/oxc_reference/README.md | 44 +- .../occurrence-consumer/src/candidate.rs | 230 +-- .../occurrence-consumer/src/conformance.rs | 1388 +++++++---------- .../occurrence-consumer/src/contract.rs | 4 +- .../occurrence-consumer/src/inspector.rs | 3 - .../crates/occurrence-consumer/src/lib.rs | 162 +- .../src/primitive_producer.rs | 1091 +++++++++++++ .../primitive-literal-independent/README.md | 6 + .../primitive-literal-independent/case.json | 94 ++ .../src/primitives.ts | 25 + .../tsconfig.json | 8 + rfcs/0001-semantic-facts-kernel.md | 45 +- 19 files changed, 1966 insertions(+), 1283 deletions(-) create mode 100644 docs/adr/0019-compute-primitive-literals-independently-in-rust.md create mode 100644 internal/oxc_reference/crates/occurrence-consumer/src/primitive_producer.rs create mode 100644 internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/README.md create mode 100644 internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/case.json create mode 100644 internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/src/primitives.ts create mode 100644 internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/tsconfig.json diff --git a/README.md b/README.md index 0f24b3561..c7cf05b31 100644 --- a/README.md +++ b/README.md @@ -12,10 +12,11 @@ plus response-local symbol, alias, and declaration provenance. An isolated Rust/OXC consumer exercises that contract as a reference and migration harness; TypeScript 7's Go checker remains the semantic oracle. -The current [migration decision](docs/adr/0016-port-occurrence-attachment-before-semantic-categories.md) +The current [migration decision](docs/adr/0019-compute-primitive-literals-independently-in-rust.md) and [measured evidence](docs/evidence/ts7-oxc-spike-2026-08-17.json) approve no -Rust semantic replacement. Primitive/literal record construction is the next -candidate for independent Rust output and differential comparison. The +Rust semantic replacement. An independent Rust/OXC primitive/literal producer +now runs under exact differential comparison while Go remains the production +fallback and semantic authority. The compatibility boundary and current exclusions are tracked in the [migration contract](docs/porting/migration-contract.md), and durable implementation choices are recorded in [ADRs](docs/adr/README.md). diff --git a/docs/adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md b/docs/adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md index 082c2c4c2..d7a4e5dae 100644 --- a/docs/adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md +++ b/docs/adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md @@ -1,10 +1,10 @@ # ADR-0017: Project primitive/literal candidates from Go graph identity -- Status: accepted +- Status: superseded - Date: 2026-08-17 - Deciders: TypeScript Semantic Kernel maintainers - Supersedes: — -- Superseded by: — +- Superseded by: ADR-0019 ## Context diff --git a/docs/adr/0018-gate-rust-semantics-against-the-go-oracle.md b/docs/adr/0018-gate-rust-semantics-against-the-go-oracle.md index 541f917b2..02a2c01ee 100644 --- a/docs/adr/0018-gate-rust-semantics-against-the-go-oracle.md +++ b/docs/adr/0018-gate-rust-semantics-against-the-go-oracle.md @@ -1,10 +1,10 @@ # ADR-0018: Gate Rust semantics against the Go oracle -- Status: accepted +- Status: superseded - Date: 2026-08-17 - Deciders: TypeScript Semantic Kernel maintainers - Supersedes: — -- Superseded by: — +- Superseded by: ADR-0019 ## Context diff --git a/docs/adr/0019-compute-primitive-literals-independently-in-rust.md b/docs/adr/0019-compute-primitive-literals-independently-in-rust.md new file mode 100644 index 000000000..cbbf46225 --- /dev/null +++ b/docs/adr/0019-compute-primitive-literals-independently-in-rust.md @@ -0,0 +1,84 @@ +# ADR-0019: Compute primitive/literal candidates independently in Rust + +- Status: accepted +- Date: 2026-08-17 +- Deciders: TypeScript Semantic Kernel maintainers +- Supersedes: ADR-0017 and ADR-0018 +- Superseded by: — + +## Context + +ADR-0017 defined structured primitive/literal candidate records, but projected +them from the graph already produced by the Go checker. ADR-0018 added an exact +shadow comparison, but that comparison could only verify the projection. It +could not establish that Rust/OXC derived the semantic answer independently. + +The first independent slice must preserve the existing shadow boundary. It +must not change the TS7 producer protocol, affect external consumers, route +production traffic to Rust, or weaken Go's role as semantic authority and +fallback. + +## Decision + +The isolated OXC workspace contains a version-2 primitive/literal producer that +accepts project source, selections, capabilities, and budgets. It parses and +builds OXC semantics, resolves selected identifiers through OXC references and +symbol declarations, and derives the following forms without reading a Go +`TypeGraph`: + +- boolean, string, number, and bigint primitives; +- boolean, string, number, and bigint literal values; +- null, undefined, and void; +- unions whose members are all in the supported set; and +- contextual primitive annotations for supported literal initializers. + +The producer owns response-local type interning, emits actual, contextual, +widened, apparent, and declared roots in fixed order, and retains explicit +complete, unsupported, and truncated states. Boxed apparent types are outside +this narrow category and therefore remain named unsupported/truncated records +instead of being approximated. Unsupported type syntax is also explicit. +Recoverable OXC parser or semantic diagnostics mark every selected fact in that +file as recovered and therefore incomplete, matching the producer's file-level +recovery contract without reading Go diagnostics. + +`run-conformance.sh` invokes the Go oracle and the Rust producer independently. +Only the comparator receives both outputs. It requires exact occurrence and OXC +node mapping, compares the five view roots and structured records recursively, +and validates response-local identity with a bijection rather than requiring +the two producers to allocate equal ID strings. Repeated Rust output must be +byte-identical. + +The executable shadow threshold is at least 15 complete supported records, +1,000,000 parts-per-million structured agreement, and zero semantic, +transport, or mapping differences. The tagged baseline contains 11 facts: 10 +supported facts plus one named unsupported object, 50 compared roots with 50 +identity matches, and 11 exact OXC mappings. Expected apparent-type truncation +and unsupported states remain visible outside the supported denominator. + +This producer remains an internal shadow implementation. The Go checker stays +the semantic authority and production fallback. No TS7 producer protocol, +external consumer, capability negotiation, or production routing changes are +part of this decision. + +## Consequences + +- Primitive/literal conformance now measures independently computed Rust/OXC + semantics instead of a Go-graph projection. +- The narrow category has deterministic source-to-record and graph-identity + evidence, including contextual literals, literal unions, unsupported forms, + and budget truncation. +- Graph inspection continues to describe Go responses; it no longer embeds a + misleading projected Rust candidate. +- Passing this shadow gate does not itself authorize a production authority + switch. Broader project loading, recovery, inference, flow, object, generic, + and callable semantics remain Go-authoritative. +- Any future authority switch requires a separate decision, maintained Go + fallback, supported-matrix CI, and evidence for the production integration. + +## References + +- [RFC 0001](../../rfcs/0001-semantic-facts-kernel.md) +- [ADR-0016](0016-port-occurrence-attachment-before-semantic-categories.md) +- [ADR-0017](0017-project-primitive-literal-candidates-from-go-graph-identity.md) +- [ADR-0018](0018-gate-rust-semantics-against-the-go-oracle.md) +- [Issue #45](https://github.com/swernerx/typescript-semantic-kernel/issues/45) diff --git a/docs/adr/README.md b/docs/adr/README.md index 12bf807c8..7c3033827 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -24,8 +24,9 @@ decision is recorded in a successor ADR that links to the record it replaces. | [0014](0014-isolate-the-oxc-reference-consumer.md) | Accepted | Isolate the OXC reference consumer | | [0015](0015-attach-semantic-facts-without-expanding-graph-identity.md) | Accepted | Attach semantic facts without expanding graph identity | | [0016](0016-port-occurrence-attachment-before-semantic-categories.md) | Accepted | Port occurrence attachment before semantic categories | -| [0017](0017-project-primitive-literal-candidates-from-go-graph-identity.md) | Accepted | Project primitive/literal candidates from Go graph identity | -| [0018](0018-gate-rust-semantics-against-the-go-oracle.md) | Accepted | Gate Rust semantics against the Go oracle | +| [0017](0017-project-primitive-literal-candidates-from-go-graph-identity.md) | Superseded | Project primitive/literal candidates from Go graph identity | +| [0018](0018-gate-rust-semantics-against-the-go-oracle.md) | Superseded | Gate Rust semantics against the Go oracle | +| [0019](0019-compute-primitive-literals-independently-in-rust.md) | Accepted | Compute primitive/literal candidates independently in Rust | ## Current migration decision @@ -36,9 +37,9 @@ Rust/OXC consumer remains a reference and migration harness; the evidence does not establish compiler equivalence, production readiness, or a performance advantage. -[ADR-0018](0018-gate-rust-semantics-against-the-go-oracle.md) runs that first -Rust-owned primitive/literal candidate over the complete Go corpus and enforces -exact supported structured agreement in shadow CI. The candidate is still a -projection of the Go-produced graph, not an independent producer. Passing the -shadow gate therefore leaves primitive/literal construction Go-authoritative; -no semantic category is currently approved for replacement. +[ADR-0019](0019-compute-primitive-literals-independently-in-rust.md) runs an +independent Rust/OXC primitive/literal producer over the tagged shared corpus +and enforces exact structured agreement in shadow CI. It owns its type graph +and receives no Go semantic graph input. Passing the gate still leaves +primitive/literal construction Go-authoritative in production; no semantic +category is currently approved for replacement. diff --git a/docs/evidence/README.md b/docs/evidence/README.md index 71377f52b..e8725e5bc 100644 --- a/docs/evidence/README.md +++ b/docs/evidence/README.md @@ -26,8 +26,9 @@ count is zero. The checked record supports the migration decision in [ADR-0016](../adr/0016-port-occurrence-attachment-before-semantic-categories.md): occurrence identity and attachment plumbing may be ported behind the Go oracle. -[ADR-0017](../adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md) -subsequently adds a structured primitive/literal Rust candidate over that -Go-produced graph. It is not an approved replacement or an independent -producer. The TypeScript 7 Go checker remains authoritative until independent -Rust output passes the differential compatibility gates. +[ADR-0019](../adr/0019-compute-primitive-literals-independently-in-rust.md) +subsequently adds a structured primitive/literal producer that derives its +answer from source and OXC semantics without receiving the Go graph. It passes +an exact shadow differential gate but is not an approved production +replacement. The TypeScript 7 Go checker remains authoritative and available +as the production fallback. diff --git a/docs/porting/migration-contract.md b/docs/porting/migration-contract.md index 7c0907bed..d6d17f35c 100644 --- a/docs/porting/migration-contract.md +++ b/docs/porting/migration-contract.md @@ -88,16 +88,14 @@ source remains Go-only because the current OXC parse fails; its facts are reported as consumer differences rather than omitted. ADR-0016 selects occurrence identity and attachment plumbing as the first safe -mechanical port. ADR-0017 adds the first Rust-owned primitive/literal candidate -record over the Go-produced graph so future comparisons can use structured -kinds, literal values, union edges, roots, and states. It is not an independent -producer. ADR-0018 adds the corpus-wide shadow gate and exact compatibility -threshold, but passing it does not transfer semantic authority. No category may -replace Go until independent Rust output completes ADR-0018's replacement -checklist with no completeness/state downgrade, new unsupported form, or -unexplained mapping gap. Project loading, resolution, binding, symbols, -inference, contextual/widened types, overloads, generic instantiation, -narrowing, and recovery remain Go-authoritative. +mechanical port. ADR-0019 replaces the earlier Go-graph projection with a +narrow independent Rust/OXC primitive/literal producer. The corpus-wide shadow +gate compares its five roots, structured graph identity, states, mapping, and +truncation against Go at an exact threshold. Passing it does not transfer +semantic authority or alter production routing. Project loading, resolution, +binding, symbols, inference beyond the supported contextual literals, +overloads, generic instantiation, narrowing, recovery, and all production +fallback behavior remain Go-authoritative. ## Gate ladder @@ -116,8 +114,8 @@ narrowing, and recovery remain Go-authoritative. 8. `./internal/oxc_reference/run-evidence.sh --output ` produces stable repeated observations and passes ADR-0016's occurrence/attachment gate. 9. `./internal/oxc_reference/run-conformance.sh --output ` emits the - deterministic ADR-0018 report and fails on any unexplained semantic or - transport mismatch while retaining mapping, unsupported, and budget cases. + deterministic ADR-0019 report and fails on any semantic, transport, or + mapping mismatch while retaining named unsupported and budget cases. ## Upstream synchronization diff --git a/internal/oxc_reference/README.md b/internal/oxc_reference/README.md index 4941cb8d5..212a728f5 100644 --- a/internal/oxc_reference/README.md +++ b/internal/oxc_reference/README.md @@ -60,15 +60,18 @@ Run the deterministic Go-versus-Rust shadow conformance gate with: --output /tmp/ts7-rust-conformance.json ``` -The command runs the same corpus through the Go oracle and the version-1 Rust -primitive/literal candidate. Its JSON compares fact identity, all five roots, -response-local graph identity, structured payloads, diagnostics, unsupported -and error states, and truncation. Every entry is classified as `semantic`, -`transport`, `mapping`, `unsupported`, or `budget`. Unexplained semantic or -transport differences fail the command; expected unsupported/budget cases and -the known recovery-file mapping gap remain separately reported. The gate is a -shadow comparison and does not change the producer or production routing. See -[ADR-0018](../../docs/adr/0018-gate-rust-semantics-against-the-go-oracle.md). +The command runs the same corpus through the Go oracle and the version-2 Rust +primitive/literal candidate. Rust derives its records independently from the +project source and OXC semantic nodes; the Go graph is supplied only to the +comparator. Its JSON compares fact identity, exact OXC mapping, all five roots, +response-local graph identity through a Go-to-Rust ID bijection, structured +payloads, recovery, unsupported states, and truncation. Every entry is +classified as `semantic`, `transport`, `mapping`, `unsupported`, or `budget`. +Unexplained +semantic, transport, or mapping differences fail the command; named +unsupported and budget cases remain separately reported. The gate is a shadow +comparison and does not change the producer or production routing. See +[ADR-0019](../../docs/adr/0019-compute-primitive-literals-independently-in-rust.md). The first test suite applies the Rust implementation of the portable contract to every shared JSON fixture in `internal/occurrencemap/testdata/v1` and checks @@ -94,14 +97,11 @@ diagnostics remain visible. Depth, node, and edge budgets are consumer-local guards and do not rewrite or reinterpret producer records. See [ADR-0015](../../docs/adr/0015-attach-semantic-facts-without-expanding-graph-identity.md). -Each inspection also contains the internal version-1 primitive/literal Rust -candidate documented by -[ADR-0017](../../docs/adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md). -It records the occurrence, fact status, all five roots, structured primitive or -literal values, ordered union member TypeIDs, and explicit candidate/source -states. Candidate records intentionally omit display text. The shared -`primitive-literal-candidate` canonical fixture is round-tripped by Go and -attached, inspected, and asserted here through OXC. +Graph inspection remains a consumer of the Go response and does not synthesize +Rust candidate records. The independent primitive/literal producer lives beside +the inspector and is exercised only by focused tests and the shadow conformance +runner. It accepts source/project inputs, owns its response-local type identity, +and never receives the Go graph used as the oracle. ## Migration boundary @@ -115,11 +115,11 @@ negotiation in Go. The intended migration sequence is: explicit and met. Occurrence identity and attachment plumbing is the first approved mechanical -port category. The primitive/literal projection is the first implemented -semantic candidate record, but it consumes Go-produced graph facts and does -not constitute an independent producer. All semantic answers remain -Go-authoritative until an independent Rust producer completes ADR-0018's -replacement checklist. +port category. Primitive/literal construction is now independently implemented +for the narrow, tagged conformance corpus and satisfies the shadow threshold in +ADR-0019. This does not transfer semantic authority: the implementation is not +wired into production, Go remains the fallback, and broader TypeScript +semantics remain Go-authoritative. Adding a projection does not transfer semantic authority. New parser-boundary normalizations require a shared fixture and the versioned portable contract. diff --git a/internal/oxc_reference/crates/occurrence-consumer/src/candidate.rs b/internal/oxc_reference/crates/occurrence-consumer/src/candidate.rs index a3074711d..a682af51c 100644 --- a/internal/oxc_reference/crates/occurrence-consumer/src/candidate.rs +++ b/internal/oxc_reference/crates/occurrence-consumer/src/candidate.rs @@ -1,22 +1,18 @@ -use std::collections::{BTreeMap, BTreeSet}; - use serde::Serialize; use crate::{ contract::Occurrence, - facts::{ - EntityState, GraphIssue, OccurrenceTypeFacts, TypeGraph, TypeId, TypeKind, TypeView, - TypeViewState, - }, + facts::{TypeId, TypeView, TypeViewState}, }; -pub const PRIMITIVE_LITERAL_CANDIDATE_VERSION: u32 = 1; +pub const PRIMITIVE_LITERAL_CANDIDATE_VERSION: u32 = 2; #[derive(Clone, Debug, Eq, PartialEq, Serialize)] #[serde(rename_all = "camelCase")] pub struct PrimitiveLiteralCandidate { pub candidate_version: u32, pub occurrence: Occurrence, + pub oxc_node_id: Option, pub fact: CandidateFactStatus, pub roots: Vec, pub types: Vec, @@ -52,31 +48,26 @@ pub enum CandidateState { #[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd, Serialize)] #[serde(rename_all = "kebab-case")] pub enum CandidateReason { - EmptyUnion, - MissingLiteralDetails, - SourceError, - SourceTruncated, - SourceUnsupported, - UnsupportedLiteralKind, - UnsupportedTypeKind, + ApparentTypeOutsideCategory, + OxcParseOrSemanticError, + SelectionUnmapped, + TypeBudgetExceeded, + UnsupportedExpression, + UnsupportedTypeForm, } #[derive(Clone, Debug, Eq, PartialEq, Serialize)] #[serde(rename_all = "camelCase")] pub struct CandidateTypeRecord { pub id: TypeId, - pub source_kind: TypeKind, - pub source_state: EntityState, pub candidate_state: CandidateState, #[serde(skip_serializing_if = "Option::is_none")] pub semantic: Option, #[serde(skip_serializing_if = "Vec::is_empty")] pub reasons: Vec, - #[serde(skip_serializing_if = "Vec::is_empty")] - pub issues: Vec, } -#[derive(Clone, Debug, Eq, PartialEq, Serialize)] +#[derive(Clone, Debug, Eq, Ord, PartialEq, PartialOrd, Serialize)] #[serde(tag = "kind", rename_all = "kebab-case")] pub enum CandidateSemantic { Primitive { @@ -95,7 +86,7 @@ pub enum CandidateSemantic { }, } -#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)] +#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd, Serialize)] #[serde(rename_all = "lowercase")] pub enum PrimitiveKind { Boolean, @@ -104,7 +95,7 @@ pub enum PrimitiveKind { Bigint, } -#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)] +#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd, Serialize)] #[serde(rename_all = "lowercase")] pub enum LiteralKind { Boolean, @@ -113,7 +104,7 @@ pub enum LiteralKind { Bigint, } -#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)] +#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd, Serialize)] #[serde(rename_all = "lowercase")] pub enum NullLikeKind { Null, @@ -130,194 +121,13 @@ pub struct CandidateSummary { pub error: usize, } -impl PrimitiveLiteralCandidate { - pub fn build(graph: &TypeGraph, facts: &OccurrenceTypeFacts) -> Self { - let roots = facts - .roots() - .into_iter() - .map(|root| CandidateRoot { - view: root.view, - state: root.state, - type_id: root.type_id.cloned(), - }) - .collect::>(); - let mut builder = CandidateBuilder::new(graph); - for root in &roots { - if let Some(type_id) = &root.type_id { - builder.visit(type_id); - } +impl CandidateSummary { + pub fn add(&mut self, state: CandidateState) { + match state { + CandidateState::Complete => self.complete += 1, + CandidateState::Truncated => self.truncated += 1, + CandidateState::Unsupported => self.unsupported += 1, + CandidateState::Error => self.error += 1, } - let types = builder.records.into_values().collect::>(); - let summary = types - .iter() - .fold(CandidateSummary::default(), |mut summary, record| { - match record.candidate_state { - CandidateState::Complete => summary.complete += 1, - CandidateState::Truncated => summary.truncated += 1, - CandidateState::Unsupported => summary.unsupported += 1, - CandidateState::Error => summary.error += 1, - } - summary - }); - Self { - candidate_version: PRIMITIVE_LITERAL_CANDIDATE_VERSION, - occurrence: facts.occurrence(), - fact: CandidateFactStatus { - complete: facts.complete, - recovered: facts.recovered, - truncated: facts.truncated, - }, - roots, - types, - summary, - } - } -} - -struct CandidateBuilder<'a> { - graph: &'a TypeGraph, - records: BTreeMap, - visiting: BTreeSet, -} - -impl<'a> CandidateBuilder<'a> { - fn new(graph: &'a TypeGraph) -> Self { - Self { - graph, - records: BTreeMap::new(), - visiting: BTreeSet::new(), - } - } - - fn visit(&mut self, id: &TypeId) -> CandidateState { - if let Some(record) = self.records.get(id) { - return record.candidate_state; - } - if !self.visiting.insert(id.clone()) { - return CandidateState::Unsupported; - } - - let source = self - .graph - .type_record(id) - .expect("validated fact and type edges reference existing graph types"); - let mut reasons = BTreeSet::new(); - let mut child_state = CandidateState::Complete; - let semantic = match source.type_kind { - TypeKind::Boolean => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Boolean, - }), - TypeKind::String => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::String, - }), - TypeKind::Number => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Number, - }), - TypeKind::Bigint => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Bigint, - }), - TypeKind::Null => Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Null, - }), - TypeKind::Undefined => Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Undefined, - }), - TypeKind::Void => Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Void, - }), - TypeKind::Literal => match &source.literal { - Some(literal) => match literal_kind(&literal.kind) { - Some(kind) => Some(CandidateSemantic::Literal { - literal: kind, - value: literal.value.clone(), - }), - None => { - reasons.insert(CandidateReason::UnsupportedLiteralKind); - None - } - }, - None => { - reasons.insert(CandidateReason::MissingLiteralDetails); - None - } - }, - TypeKind::Union => { - if source.members.is_empty() { - reasons.insert(CandidateReason::EmptyUnion); - } - for member in &source.members { - child_state = merge_state(child_state, self.visit(member)); - } - Some(CandidateSemantic::Union { - members: source.members.clone(), - }) - } - _ => { - reasons.insert(CandidateReason::UnsupportedTypeKind); - None - } - }; - - let source_state = match source.state { - EntityState::Complete if source.truncated => { - reasons.insert(CandidateReason::SourceTruncated); - CandidateState::Truncated - } - EntityState::Complete => CandidateState::Complete, - EntityState::Truncated => { - reasons.insert(CandidateReason::SourceTruncated); - CandidateState::Truncated - } - EntityState::Unsupported => { - reasons.insert(CandidateReason::SourceUnsupported); - CandidateState::Unsupported - } - EntityState::Error => { - reasons.insert(CandidateReason::SourceError); - CandidateState::Error - } - }; - let structural_state = if semantic.is_some() - && !reasons.contains(&CandidateReason::EmptyUnion) - && !reasons.contains(&CandidateReason::MissingLiteralDetails) - && !reasons.contains(&CandidateReason::UnsupportedLiteralKind) - { - child_state - } else { - CandidateState::Unsupported - }; - let candidate_state = merge_state(source_state, structural_state); - let record = CandidateTypeRecord { - id: id.clone(), - source_kind: source.type_kind, - source_state: source.state, - candidate_state, - semantic, - reasons: reasons.into_iter().collect(), - issues: source.issues.clone(), - }; - self.visiting.remove(id); - self.records.insert(id.clone(), record); - candidate_state - } -} - -fn literal_kind(kind: &str) -> Option { - match kind { - "boolean" => Some(LiteralKind::Boolean), - "string" => Some(LiteralKind::String), - "number" => Some(LiteralKind::Number), - "bigint" => Some(LiteralKind::Bigint), - _ => None, - } -} - -fn merge_state(left: CandidateState, right: CandidateState) -> CandidateState { - use CandidateState::{Complete, Error, Truncated, Unsupported}; - match (left, right) { - (Error, _) | (_, Error) => Error, - (Truncated, _) | (_, Truncated) => Truncated, - (Unsupported, _) | (_, Unsupported) => Unsupported, - (Complete, Complete) => Complete, } } diff --git a/internal/oxc_reference/crates/occurrence-consumer/src/conformance.rs b/internal/oxc_reference/crates/occurrence-consumer/src/conformance.rs index 588e231f6..aec81b914 100644 --- a/internal/oxc_reference/crates/occurrence-consumer/src/conformance.rs +++ b/internal/oxc_reference/crates/occurrence-consumer/src/conformance.rs @@ -4,27 +4,28 @@ use std::{ io::{BufReader, Write}, path::{Path, PathBuf}, process::{Command, Stdio}, - sync::Arc, }; -use oxc_allocator::Allocator; use serde::{Deserialize, Serialize}; use crate::{ candidate::{ - CandidateFactStatus, CandidateReason, CandidateRoot, CandidateSemantic, CandidateState, - CandidateSummary, CandidateTypeRecord, LiteralKind, NullLikeKind, - PRIMITIVE_LITERAL_CANDIDATE_VERSION, PrimitiveKind, PrimitiveLiteralCandidate, + CandidateSemantic, CandidateState, CandidateSummary, CandidateTypeRecord, LiteralKind, + NullLikeKind, PRIMITIVE_LITERAL_CANDIDATE_VERSION, PrimitiveKind, + PrimitiveLiteralCandidate, }, - contract::{DiagnosticCode, Occurrence}, + contract::{Occurrence, Span}, facts::{ EntityState, OccurrenceTypeFacts, ProducerBudgetReport, SemanticSnapshot, TypeGraph, TypeId, TypeKind, TypeViewState, }, - oxc::OxcConsumer, + primitive_producer::{ + IndependentPrimitiveLiteralOutput, PrimitiveLiteralSelection, PrimitiveProducerLimits, + produce_primitive_literals, + }, }; -pub const CONFORMANCE_SCHEMA_VERSION: u32 = 1; +pub const CONFORMANCE_SCHEMA_VERSION: u32 = 2; #[derive(Clone, Debug, Eq, PartialEq, Serialize)] #[serde(rename_all = "camelCase")] @@ -46,7 +47,7 @@ pub struct CompatibilityThreshold { pub required_supported_compatibility_ppm: u64, pub max_unexplained_semantic_differences: usize, pub max_unexplained_transport_differences: usize, - pub mapping_differences_block_shadow_gate: bool, + pub max_unexplained_mapping_differences: usize, pub unsupported_and_budget_differences_are_expected: bool, } @@ -54,9 +55,11 @@ pub struct CompatibilityThreshold { #[serde(rename_all = "camelCase")] pub struct ConformanceCase { pub name: String, + pub independent_scope: bool, pub facts: usize, - pub diagnostics: DiagnosticComparison, - pub budgets: BudgetComparison, + pub repeated_rust_output_equal: bool, + pub go_oracle: GoOracleEvidence, + pub rust_producer: RustProducerEvidence, pub roots: RootCoverage, pub mapping: MappingCoverage, pub candidate_states: CandidateSummary, @@ -68,18 +71,18 @@ pub struct ConformanceCase { #[derive(Clone, Copy, Debug, Default, Eq, PartialEq, Serialize)] #[serde(rename_all = "camelCase")] -pub struct DiagnosticComparison { - pub go_oracle_count: u32, - pub rust_observed_count: u32, - pub matched: bool, +pub struct GoOracleEvidence { + pub diagnostic_count: u32, + pub budgets: ProducerBudgetReport, } #[derive(Clone, Copy, Debug, Default, Eq, PartialEq, Serialize)] #[serde(rename_all = "camelCase")] -pub struct BudgetComparison { - pub go_oracle: ProducerBudgetReport, - pub rust_observed: ProducerBudgetReport, - pub matched: bool, +pub struct RustProducerEvidence { + pub diagnostic_count: usize, + pub max_type_nodes: usize, + pub type_nodes_used: usize, + pub truncated: bool, } #[derive(Clone, Copy, Debug, Default, Eq, PartialEq, Serialize)] @@ -100,7 +103,6 @@ pub struct MappingCoverage { pub facts: usize, pub mapped: usize, pub unmapped: usize, - pub multiply_mapped: usize, pub failed_files: usize, } @@ -177,6 +179,8 @@ struct CorpusManifest { project: String, capabilities: Vec, #[serde(default)] + coverage: Vec, + #[serde(default)] budgets: ProducerBudgetRequest, selections: Vec, } @@ -212,26 +216,6 @@ struct ProducerSelection { end: usize, } -struct RustCandidateRun { - diagnostic_count: u32, - budgets: ProducerBudgetReport, - candidates: Vec, -} - -impl RustCandidateRun { - fn from_snapshot(snapshot: &SemanticSnapshot) -> Self { - Self { - diagnostic_count: snapshot.diagnostic_count, - budgets: snapshot.budgets, - candidates: snapshot - .facts() - .iter() - .map(|facts| PrimitiveLiteralCandidate::build(snapshot.graph(), facts)) - .collect(), - } - } -} - pub fn run_conformance( tsfacts_binary: &Path, corpus_root: &Path, @@ -246,32 +230,48 @@ pub fn run_conformance( for case_directory in sorted_case_directories(&corpus_root)? { let manifest = read_manifest(&case_directory)?; let request = build_request(&case_directory, &manifest)?; - let source_files = request + let snapshot = run_go_oracle(&tsfacts_binary, &case_directory, &request)?; + let selections = request .selections .iter() - .map(|selection| selection.file.clone()) - .collect::>() - .into_iter() - .collect::>(); - let snapshot = Arc::new(run_go_oracle(&tsfacts_binary, &case_directory, &request)?); - let candidate_run = RustCandidateRun::from_snapshot(&snapshot); + .map(|selection| { + Ok(PrimitiveLiteralSelection { + file: selection.file.clone(), + span: Span { + start: u32::try_from(selection.start) + .map_err(|_| "selection start exceeds u32")?, + end: u32::try_from(selection.end) + .map_err(|_| "selection end exceeds u32")?, + }, + }) + }) + .collect::, String>>()?; + let limits = PrimitiveProducerLimits { + max_type_nodes: usize::try_from(manifest.budgets.max_type_nodes).unwrap_or(usize::MAX), + }; + let first = produce_primitive_literals(&case_directory, &selections, limits)?; + let repeated = produce_primitive_literals(&case_directory, &selections, limits)?; + let repeated_equal = serde_json::to_vec(&first).ok() == serde_json::to_vec(&repeated).ok(); + let independent_scope = manifest + .coverage + .iter() + .any(|item| item == "primitive-literals-independent"); cases.push(compare_case( manifest.name, - &case_directory, - &source_files, + independent_scope, &snapshot, - candidate_run, - )?); + first, + repeated_equal, + )); } let summary = summarize(&cases); let threshold = compatibility_threshold(); let passes = threshold_passes(&summary, threshold); - Ok(ConformanceReport { schema_version: CONFORMANCE_SCHEMA_VERSION, - gate_kind: "go-vs-rust-semantic-conformance", - candidate: "primitive-literal-v1", + gate_kind: "go-vs-independent-rust-semantic-conformance", + candidate: "independent-primitive-literal-v2", shadow_only: true, threshold, cases, @@ -282,30 +282,29 @@ pub fn run_conformance( fn compatibility_threshold() -> CompatibilityThreshold { CompatibilityThreshold { - minimum_supported_records: 7, + minimum_supported_records: 15, required_supported_compatibility_ppm: 1_000_000, max_unexplained_semantic_differences: 0, max_unexplained_transport_differences: 0, - mapping_differences_block_shadow_gate: false, + max_unexplained_mapping_differences: 0, unsupported_and_budget_differences_are_expected: true, } } fn threshold_passes(summary: &ConformanceSummary, threshold: CompatibilityThreshold) -> bool { - let semantic = summary - .differences_by_category - .get("semantic") - .copied() - .unwrap_or_default(); - let transport = summary - .differences_by_category - .get("transport") - .copied() - .unwrap_or_default(); summary.supported_records >= threshold.minimum_supported_records && summary.supported_compatibility_ppm >= threshold.required_supported_compatibility_ppm - && semantic <= threshold.max_unexplained_semantic_differences - && transport <= threshold.max_unexplained_transport_differences + && difference_count(summary, "semantic") <= threshold.max_unexplained_semantic_differences + && difference_count(summary, "transport") <= threshold.max_unexplained_transport_differences + && difference_count(summary, "mapping") <= threshold.max_unexplained_mapping_differences +} + +fn difference_count(summary: &ConformanceSummary, category: &str) -> usize { + summary + .differences_by_category + .get(category) + .copied() + .unwrap_or_default() } fn run_go_oracle( @@ -344,619 +343,525 @@ fn run_go_oracle( fn compare_case( name: String, - case_directory: &Path, - source_files: &[String], - snapshot: &Arc, - candidate_run: RustCandidateRun, -) -> Result { - let mut differences = Vec::new(); - let mut roots = RootCoverage::default(); - let mut candidate_states = CandidateSummary::default(); - let mut supported_records = 0; - let mut matched_supported_records = 0; - - compare_scalar( - &mut differences, - DifferenceCategory::Transport, - "diagnostic-count-mismatch", - None, - "diagnostics.count", - (snapshot.diagnostic_count, candidate_run.diagnostic_count), - "Rust must preserve the Go oracle diagnostic count", - ); - compare_scalar( - &mut differences, - DifferenceCategory::Transport, - "budget-report-mismatch", - None, - "budgets", - (snapshot.budgets, candidate_run.budgets), - "Rust must preserve the Go oracle budget report", - ); - if snapshot.budgets.truncated { - differences.push(expected_difference( - DifferenceCategory::Budget, - "producer-budget-truncated", + independent_scope: bool, + snapshot: &SemanticSnapshot, + output: IndependentPrimitiveLiteralOutput, + repeated_equal: bool, +) -> ConformanceCase { + let mut comparison = Comparison::new(snapshot, &output); + if !repeated_equal { + comparison.differences.push(mismatch( + DifferenceCategory::Transport, + "nondeterministic-rust-output", None, - "budgets.truncated", - json_value(&true), - json_value(&true), - "the corpus intentionally exercised the Go producer budget", + "rustProducer", + json_value(&"first output"), + json_value(&"different repeated output"), + "the independent Rust producer must be byte-stable across repeated runs", )); } - if snapshot.facts().len() != candidate_run.candidates.len() { - differences.push(mismatch( - DifferenceCategory::Transport, - "fact-count-mismatch", + if independent_scope { + comparison.compare_scoped_facts(); + } else { + comparison.differences.push(expected_difference( + DifferenceCategory::Unsupported, + "case-outside-independent-slice", + None, + "case.coverage", None, - "facts", - json_value(&snapshot.facts().len()), - json_value(&candidate_run.candidates.len()), - "Rust must emit one candidate per Go fact", + None, + "this pre-existing corpus case remains outside the selected primitive/literal slice", )); } - for (fact_index, (facts, candidate)) in snapshot - .facts() - .iter() - .zip(&candidate_run.candidates) - .enumerate() - { - let compared = compare_fact( - snapshot.graph(), - fact_index, - facts, - candidate, - &mut differences, - &mut roots, - &mut candidate_states, - ); - supported_records += compared.0; - matched_supported_records += compared.1; + if output.truncated { + comparison.differences.push(expected_difference( + DifferenceCategory::Budget, + "rust-type-budget-truncated", + None, + "rustProducer.truncated", + json_value(&snapshot.budgets.truncated), + json_value(&true), + "the independent Rust response-local type budget was exhausted", + )); } + comparison.finish(name, independent_scope, repeated_equal) +} - let mapping = compare_mapping(case_directory, source_files, snapshot, &mut differences)?; - sort_differences(&mut differences); - let supported_compatibility_ppm = ratio_ppm(matched_supported_records, supported_records); - Ok(ConformanceCase { - name, - facts: snapshot.facts().len(), - diagnostics: DiagnosticComparison { - go_oracle_count: snapshot.diagnostic_count, - rust_observed_count: candidate_run.diagnostic_count, - matched: snapshot.diagnostic_count == candidate_run.diagnostic_count, - }, - budgets: BudgetComparison { - go_oracle: snapshot.budgets, - rust_observed: candidate_run.budgets, - matched: snapshot.budgets == candidate_run.budgets, - }, - roots, - mapping, - candidate_states, - supported_records, - matched_supported_records, - supported_compatibility_ppm, - differences, - }) +struct Comparison<'a> { + snapshot: &'a SemanticSnapshot, + output: &'a IndependentPrimitiveLiteralOutput, + records: BTreeMap, + roots: RootCoverage, + mapping: MappingCoverage, + candidate_states: CandidateSummary, + bijection: TypeBijection, + compared_pairs: BTreeSet<(TypeId, TypeId)>, + supported_records: usize, + matched_supported_records: usize, + differences: Vec, } -fn compare_mapping( - case_directory: &Path, - source_files: &[String], - snapshot: &Arc, - differences: &mut Vec, -) -> Result { - let mut coverage = MappingCoverage { - facts: snapshot.facts().len(), - ..MappingCoverage::default() - }; - for file in source_files { - let path = case_directory.join(file); - let source = fs::read_to_string(&path) - .map_err(|error| format!("mapping: read {}: {error}", path.display()))?; - let allocator = Allocator::default(); - let mut consumer = match OxcConsumer::parse(&allocator, file, &source) { - Ok(consumer) => consumer, - Err(error) => { - coverage.failed_files += 1; - for (fact_index, facts) in snapshot - .facts() - .iter() - .enumerate() - .filter(|(_, facts)| facts.file == *file) +impl<'a> Comparison<'a> { + fn new(snapshot: &'a SemanticSnapshot, output: &'a IndependentPrimitiveLiteralOutput) -> Self { + let mut records = BTreeMap::new(); + let mut differences = Vec::new(); + let mut candidate_states = CandidateSummary::default(); + let mut seen_states = BTreeMap::new(); + for candidate in &output.candidates { + for record in &candidate.types { + if let Some(previous) = records.insert(record.id.clone(), record) + && previous != record { differences.push(mismatch( - DifferenceCategory::Mapping, - "consumer-file-failed", - Some((fact_index, facts.occurrence())), - "mapping", - json_value(&"mapped Go fact"), - json_value(&"consumer file failed"), - &error, + DifferenceCategory::Transport, + "inconsistent-response-local-record", + None, + &format!("types[{}]", record.id.as_str()), + json_value(previous), + json_value(record), + "one Rust response-local TypeID must identify one immutable record", )); } - continue; + seen_states + .entry(record.id.clone()) + .or_insert(record.candidate_state); } - }; - let report = consumer - .attach_file(Arc::clone(snapshot)) - .map_err(|error| format!("mapping: attach {file}: {error}"))?; - coverage.mapped += report.summary.mapped as usize; - coverage.unmapped += report.summary.unmapped as usize; - coverage.multiply_mapped += report.summary.multiply_mapped as usize; - for diagnostic in report.diagnostics { - let facts = &snapshot.facts()[diagnostic.fact_index]; - let (code, explanation) = match diagnostic.code { - DiagnosticCode::Unmapped => ( - "unmapped-fact", - "the OXC syntax projection did not map this Go fact", - ), - DiagnosticCode::MultiplyMapped => ( - "multiply-mapped-fact", - "the OXC syntax projection mapped this Go fact ambiguously", - ), + } + for state in seen_states.into_values() { + candidate_states.add(state); + } + Self { + snapshot, + output, + records, + roots: RootCoverage::default(), + mapping: MappingCoverage { + facts: output.candidates.len(), + ..MappingCoverage::default() + }, + candidate_states, + bijection: TypeBijection::default(), + compared_pairs: BTreeSet::new(), + supported_records: 0, + matched_supported_records: 0, + differences, + } + } + + fn compare_scoped_facts(&mut self) { + let mut candidates = self + .output + .candidates + .iter() + .map(|candidate| (candidate.occurrence.clone(), candidate)) + .collect::>(); + for (fact_index, facts) in self.snapshot.facts().iter().enumerate() { + let occurrence = facts.occurrence(); + let Some(candidate) = candidates.remove(&occurrence) else { + self.mapping.unmapped += 1; + self.differences.push(mismatch( + DifferenceCategory::Mapping, + "missing-rust-candidate", + Some((fact_index, occurrence)), + "mapping", + json_value(&"one candidate"), + json_value(&"none"), + "every selected Go fact must map to one independent OXC candidate", + )); + continue; }; - differences.push(mismatch( + if candidate.oxc_node_id.is_some() { + self.mapping.mapped += 1; + } else { + self.mapping.unmapped += 1; + self.differences.push(mismatch( + DifferenceCategory::Mapping, + "missing-oxc-node", + Some((fact_index, occurrence.clone())), + "oxcNodeId", + json_value(&"typed OXC NodeId"), + None, + "independent facts must originate from an exact OXC semantic node", + )); + } + self.compare_fact(fact_index, facts, candidate); + } + for (occurrence, _) in candidates { + self.mapping.unmapped += 1; + self.differences.push(mismatch( DifferenceCategory::Mapping, - code, - Some((diagnostic.fact_index, facts.occurrence())), + "extra-rust-candidate", + None, "mapping", - json_value(&"one node"), - json_value(&format!("{} nodes", diagnostic.candidates.len())), - explanation, + json_value(&"no extra candidate"), + json_value(&occurrence), + "the Rust producer must not invent facts outside the pinned request", )); } } - Ok(coverage) -} -fn compare_fact( - graph: &TypeGraph, - fact_index: usize, - facts: &OccurrenceTypeFacts, - candidate: &PrimitiveLiteralCandidate, - differences: &mut Vec, - root_coverage: &mut RootCoverage, - state_summary: &mut CandidateSummary, -) -> (usize, usize) { - let occurrence = facts.occurrence(); - let context = Some((fact_index, occurrence.clone())); - compare_scalar( - differences, - DifferenceCategory::Transport, - "candidate-version-mismatch", - context.clone(), - "candidateVersion", - ( - PRIMITIVE_LITERAL_CANDIDATE_VERSION, - candidate.candidate_version, - ), - "the shadow candidate version must be explicit and stable", - ); - compare_scalar( - differences, - DifferenceCategory::Transport, - "fact-identity-mismatch", - context.clone(), - "occurrence", - (occurrence, candidate.occurrence.clone()), - "file, UTF-8 span, and syntax kind identify the compared fact", - ); - let expected_fact = CandidateFactStatus { - complete: facts.complete, - recovered: facts.recovered, - truncated: facts.truncated, - }; - compare_scalar( - differences, - DifferenceCategory::Transport, - "fact-state-mismatch", - context.clone(), - "fact", - (expected_fact, candidate.fact), - "complete, recovered, and truncated fact states must survive transport", - ); - if facts.truncated { - differences.push(expected_difference( - DifferenceCategory::Budget, - "fact-truncated", - context.clone(), - "fact.truncated", - json_value(&true), - json_value(&candidate.fact.truncated), - "the Go oracle marked this fact as truncated", - )); - } - - let expected_roots = facts - .roots() - .into_iter() - .map(|root| CandidateRoot { - view: root.view, - state: root.state, - type_id: root.type_id.cloned(), - }) - .collect::>(); - root_coverage.compared_roots += expected_roots.len(); - if expected_roots.len() == 5 { - root_coverage.facts_with_all_five_views += 1; - } - for root in &expected_roots { - match root.state { - TypeViewState::Available => root_coverage.available += 1, - TypeViewState::SameAsActual => root_coverage.same_as_actual += 1, - TypeViewState::Inapplicable => root_coverage.inapplicable += 1, - TypeViewState::Unavailable => root_coverage.unavailable += 1, - } - } - for (index, expected) in expected_roots.iter().enumerate() { - let actual = candidate.roots.get(index); - if actual == Some(expected) { - root_coverage.identity_matches += 1; - } else { - differences.push(mismatch( + fn compare_fact( + &mut self, + fact_index: usize, + facts: &OccurrenceTypeFacts, + candidate: &PrimitiveLiteralCandidate, + ) { + let context = Some((fact_index, facts.occurrence())); + if candidate.candidate_version != PRIMITIVE_LITERAL_CANDIDATE_VERSION { + self.differences.push(mismatch( DifferenceCategory::Transport, - "root-identity-mismatch", + "candidate-version-mismatch", context.clone(), - &format!("roots[{index}]"), - json_value(expected), - actual.and_then(json_value), - "all actual, contextual, widened, apparent, and declared roots retain state and response-local TypeID", + "candidateVersion", + json_value(&PRIMITIVE_LITERAL_CANDIDATE_VERSION), + json_value(&candidate.candidate_version), + "the independent candidate version must be explicit and stable", )); } - } - if candidate.roots.len() != expected_roots.len() { - differences.push(mismatch( - DifferenceCategory::Transport, - "root-count-mismatch", - context.clone(), - "roots", - json_value(&expected_roots.len()), - json_value(&candidate.roots.len()), - "the candidate must expose exactly five ordered type views", - )); - } + let actual_record = self.snapshot.graph().type_record(facts.actual()); + let actual_supported = actual_record.is_some_and(|record| { + oracle_type_supported(self.snapshot.graph(), record, &mut BTreeSet::new()) + }); + if !actual_supported { + self.differences.push(expected_difference( + DifferenceCategory::Unsupported, + "actual-type-outside-primitive-literal-slice", + context, + "roots[actual]", + actual_record.map(oracle_record_value), + candidate.roots.first().and_then(json_value), + "the fixture intentionally retains an out-of-category selection", + )); + return; + } - let expected_records = OracleProjection::build(graph, facts); - let actual_records = candidate - .types - .iter() - .map(|record| (record.id.clone(), record)) - .collect::>(); - let actual_summary = - summarize_states(candidate.types.iter().map(|record| record.candidate_state)); - state_summary.complete += actual_summary.complete; - state_summary.truncated += actual_summary.truncated; - state_summary.unsupported += actual_summary.unsupported; - state_summary.error += actual_summary.error; - let expected_summary = summarize_states( - expected_records - .values() - .map(|record| record.candidate_state), - ); - compare_scalar( - differences, - DifferenceCategory::Transport, - "candidate-summary-mismatch", - context.clone(), - "summary", - (expected_summary, candidate.summary), - "candidate state totals must account for every projected TypeID", - ); + let expected_fact = (facts.complete, facts.recovered, facts.truncated); + let actual_fact = ( + candidate.fact.complete, + candidate.fact.recovered, + candidate.fact.truncated, + ); + if expected_fact != actual_fact { + self.differences.push(mismatch( + DifferenceCategory::Transport, + "fact-state-mismatch", + context.clone(), + "fact", + json_value(&expected_fact), + json_value(&actual_fact), + "complete, recovered, and truncated state must be independently reproduced", + )); + } - let mut supported = 0; - let mut matched_supported = 0; - for (id, expected) in &expected_records { - match expected.candidate_state { - CandidateState::Complete => { - supported += 1; - } - CandidateState::Truncated => { - differences.push(expected_difference( - DifferenceCategory::Budget, - "candidate-type-truncated", - context.clone(), - &format!("types[{}]", id.as_str()), - json_value(expected), - actual_records - .get(id) - .and_then(|record| json_value(*record)), - "the source graph or a reachable union member exhausted its budget", - )); + let expected_roots = facts.roots(); + self.roots.compared_roots += expected_roots.len(); + if candidate.roots.len() == 5 { + self.roots.facts_with_all_five_views += 1; + } + if candidate.roots.len() != expected_roots.len() { + self.differences.push(mismatch( + DifferenceCategory::Transport, + "root-count-mismatch", + context.clone(), + "roots", + json_value(&expected_roots.len()), + json_value(&candidate.roots.len()), + "the independent candidate must expose exactly five ordered views", + )); + } + for (index, expected) in expected_roots.iter().enumerate() { + match expected.state { + TypeViewState::Available => self.roots.available += 1, + TypeViewState::SameAsActual => self.roots.same_as_actual += 1, + TypeViewState::Inapplicable => self.roots.inapplicable += 1, + TypeViewState::Unavailable => self.roots.unavailable += 1, } - CandidateState::Unsupported => { - differences.push(expected_difference( - DifferenceCategory::Unsupported, - "candidate-type-unsupported", + let Some(actual) = candidate.roots.get(index) else { + continue; + }; + if expected.view != actual.view || expected.state != actual.state { + self.differences.push(mismatch( + DifferenceCategory::Transport, + "root-view-state-mismatch", context.clone(), - &format!("types[{}]", id.as_str()), - json_value(expected), - actual_records - .get(id) - .and_then(|record| json_value(*record)), - "this graph form is outside primitive/literal-v1 or is explicitly unsupported", + &format!("roots[{index}]"), + json_value(&(expected.view, expected.state)), + json_value(&(actual.view, actual.state)), + "all five type-view states must be independently reproduced", )); + continue; } - CandidateState::Error => { - differences.push(expected_difference( - DifferenceCategory::Unsupported, - "candidate-source-error", + match (expected.type_id, actual.type_id.as_ref()) { + (None, None) => self.roots.identity_matches += 1, + (Some(go_id), Some(rust_id)) => { + if self.compare_type(go_id, rust_id, fact_index, facts) { + self.roots.identity_matches += 1; + } + } + _ => self.differences.push(mismatch( + DifferenceCategory::Transport, + "root-identity-presence-mismatch", context.clone(), - &format!("types[{}]", id.as_str()), - json_value(expected), - actual_records.get(id).and_then(|record| json_value(*record)), - "the Go oracle exposed an error-state type which remains ineligible for replacement", - )); + &format!("roots[{index}].typeId"), + expected.type_id.and_then(json_value), + actual.type_id.as_ref().and_then(json_value), + "available and same-as-actual roots must retain response-local identity", + )), } } - let Some(actual) = actual_records.get(id) else { - differences.push(mismatch( + } + + fn compare_type( + &mut self, + go_id: &TypeId, + rust_id: &TypeId, + fact_index: usize, + facts: &OccurrenceTypeFacts, + ) -> bool { + let context = Some((fact_index, facts.occurrence())); + if !self.bijection.bind(go_id, rust_id) { + self.differences.push(mismatch( DifferenceCategory::Transport, - "missing-candidate-type", - context.clone(), - &format!("types[{}]", id.as_str()), - json_value(expected), - None, - "the candidate omitted a response-local graph identity reachable through its supported projection", + "response-local-identity-mismatch", + context, + &format!("types[{}]", go_id.as_str()), + json_value(go_id), + json_value(rust_id), + "Go and Rust TypeIDs may differ in spelling but must form one response-wide bijection", )); - continue; + return false; + } + let pair = (go_id.clone(), rust_id.clone()); + if !self.compared_pairs.insert(pair) { + return true; + } + let Some(go_record) = self.snapshot.graph().type_record(go_id) else { + return false; }; - let exact = compare_type_record(differences, context.clone(), id, expected, actual); - if expected.candidate_state == CandidateState::Complete && exact { - matched_supported += 1; + if !oracle_type_supported(self.snapshot.graph(), go_record, &mut BTreeSet::new()) { + return true; } - } - for (id, actual) in actual_records { - if !expected_records.contains_key(&id) { - differences.push(mismatch( + self.supported_records += 1; + let Some(rust_record) = self.records.get(rust_id).copied() else { + self.differences.push(mismatch( DifferenceCategory::Transport, - "unexpected-candidate-type", - context.clone(), - &format!("types[{}]", id.as_str()), + "missing-rust-type-record", + context, + &format!("types[{}]", rust_id.as_str()), + Some(oracle_record_value(go_record)), None, - json_value(actual), - "the candidate fabricated a graph identity outside the Go-rooted projection", + "every Rust root and union member must resolve in the same response", )); - } - } - (supported, matched_supported) -} - -fn compare_type_record( - differences: &mut Vec, - context: Option<(usize, Occurrence)>, - id: &TypeId, - expected: &CandidateTypeRecord, - actual: &CandidateTypeRecord, -) -> bool { - let before = differences.len(); - for (field, left, right) in [ - ("id", json_value(&expected.id), json_value(&actual.id)), - ( - "sourceKind", - json_value(&expected.source_kind), - json_value(&actual.source_kind), - ), - ( - "sourceState", - json_value(&expected.source_state), - json_value(&actual.source_state), - ), - ( - "issues", - json_value(&expected.issues), - json_value(&actual.issues), - ), - ] { - if left != right { - differences.push(mismatch( - DifferenceCategory::Transport, - "type-source-mismatch", - context.clone(), - &format!("types[{}].{field}", id.as_str()), - left, - right, - "source graph kind, state, issues, and identity must survive the Rust boundary", + return false; + }; + if rust_record.candidate_state != CandidateState::Complete { + self.differences.push(mismatch( + DifferenceCategory::Semantic, + "supported-type-downgraded", + context, + &format!("types[{}]", go_id.as_str()), + Some(oracle_record_value(go_record)), + json_value(rust_record), + "a complete in-category Go type may not become unsupported or truncated in Rust", )); + return false; } + let matched = self.compare_semantic(go_record, rust_record, fact_index, facts); + if matched { + self.matched_supported_records += 1; + } + matched } - for (field, left, right) in [ - ( - "candidateState", - json_value(&expected.candidate_state), - json_value(&actual.candidate_state), - ), - ( - "semantic", - json_value(&expected.semantic), - json_value(&actual.semantic), - ), - ( - "reasons", - json_value(&expected.reasons), - json_value(&actual.reasons), - ), - ] { - if left != right { - differences.push(mismatch( + + fn compare_semantic( + &mut self, + go_record: &crate::facts::TypeRecord, + rust_record: &CandidateTypeRecord, + fact_index: usize, + facts: &OccurrenceTypeFacts, + ) -> bool { + let context = Some((fact_index, facts.occurrence())); + let matched = match (&go_record.type_kind, &rust_record.semantic) { + (TypeKind::Boolean, Some(CandidateSemantic::Primitive { primitive })) => { + *primitive == PrimitiveKind::Boolean + } + (TypeKind::String, Some(CandidateSemantic::Primitive { primitive })) => { + *primitive == PrimitiveKind::String + } + (TypeKind::Number, Some(CandidateSemantic::Primitive { primitive })) => { + *primitive == PrimitiveKind::Number + } + (TypeKind::Bigint, Some(CandidateSemantic::Primitive { primitive })) => { + *primitive == PrimitiveKind::Bigint + } + (TypeKind::Null, Some(CandidateSemantic::NullLike { null_like })) => { + *null_like == NullLikeKind::Null + } + (TypeKind::Undefined, Some(CandidateSemantic::NullLike { null_like })) => { + *null_like == NullLikeKind::Undefined + } + (TypeKind::Void, Some(CandidateSemantic::NullLike { null_like })) => { + *null_like == NullLikeKind::Void + } + (TypeKind::Literal, Some(CandidateSemantic::Literal { literal, value })) => { + go_record.literal.as_ref().is_some_and(|go_literal| { + literal_wire_name(*literal) == go_literal.kind && *value == go_literal.value + }) + } + (TypeKind::Union, Some(CandidateSemantic::Union { members })) => { + if go_record.members.len() != members.len() { + false + } else { + go_record + .members + .iter() + .zip(members) + .all(|(go_member, rust_member)| { + self.compare_type(go_member, rust_member, fact_index, facts) + }) + } + } + _ => false, + }; + if !matched { + self.differences.push(mismatch( DifferenceCategory::Semantic, "primitive-literal-semantic-mismatch", - context.clone(), - &format!("types[{}].{field}", id.as_str()), - left, - right, - "structured primitive/literal payload, state, and reasons must equal the Go oracle projection", + context, + &format!("types[{}].semantic", go_record.id.as_str()), + Some(oracle_record_value(go_record)), + json_value(rust_record), + "structured primitive, literal, null-like, and union payloads must match", )); } + matched + } + + fn finish( + mut self, + name: String, + independent_scope: bool, + repeated_equal: bool, + ) -> ConformanceCase { + sort_differences(&mut self.differences); + ConformanceCase { + name, + independent_scope, + facts: self.snapshot.facts().len(), + repeated_rust_output_equal: repeated_equal, + go_oracle: GoOracleEvidence { + diagnostic_count: self.snapshot.diagnostic_count, + budgets: self.snapshot.budgets, + }, + rust_producer: RustProducerEvidence { + diagnostic_count: self.output.diagnostics.len(), + max_type_nodes: self.output.limits.max_type_nodes, + type_nodes_used: self.output.type_nodes_used, + truncated: self.output.truncated, + }, + roots: self.roots, + mapping: self.mapping, + candidate_states: self.candidate_states, + supported_records: self.supported_records, + matched_supported_records: self.matched_supported_records, + supported_compatibility_ppm: ratio_ppm( + self.matched_supported_records, + self.supported_records, + ), + differences: self.differences, + } } - differences.len() == before } -struct OracleProjection<'a> { - graph: &'a TypeGraph, - records: BTreeMap, - visiting: BTreeSet, +#[derive(Default)] +struct TypeBijection { + go_to_rust: BTreeMap, + rust_to_go: BTreeMap, } -impl<'a> OracleProjection<'a> { - fn build( - graph: &'a TypeGraph, - facts: &OccurrenceTypeFacts, - ) -> BTreeMap { - let mut projection = Self { - graph, - records: BTreeMap::new(), - visiting: BTreeSet::new(), - }; - for root in facts.roots().into_iter().filter_map(|root| root.type_id) { - projection.visit(root); +impl TypeBijection { + fn bind(&mut self, go: &TypeId, rust: &TypeId) -> bool { + if self + .go_to_rust + .get(go) + .is_some_and(|existing| existing != rust) + || self + .rust_to_go + .get(rust) + .is_some_and(|existing| existing != go) + { + return false; } - projection.records + self.go_to_rust.insert(go.clone(), rust.clone()); + self.rust_to_go.insert(rust.clone(), go.clone()); + true } +} - fn visit(&mut self, id: &TypeId) -> CandidateState { - if let Some(record) = self.records.get(id) { - return record.candidate_state; - } - if !self.visiting.insert(id.clone()) { - return CandidateState::Unsupported; - } - let source = self - .graph - .type_record(id) - .expect("validated fact roots and union edges resolve"); - let mut reasons = BTreeSet::new(); - let mut child_state = CandidateState::Complete; - let semantic = match source.type_kind { - TypeKind::Boolean => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Boolean, - }), - TypeKind::String => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::String, - }), - TypeKind::Number => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Number, - }), - TypeKind::Bigint => Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Bigint, - }), - TypeKind::Null => Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Null, - }), - TypeKind::Undefined => Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Undefined, - }), - TypeKind::Void => Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Void, - }), - TypeKind::Literal => match &source.literal { - Some(literal) => match literal.kind.as_str() { - "boolean" => Some(CandidateSemantic::Literal { - literal: LiteralKind::Boolean, - value: literal.value.clone(), - }), - "string" => Some(CandidateSemantic::Literal { - literal: LiteralKind::String, - value: literal.value.clone(), - }), - "number" => Some(CandidateSemantic::Literal { - literal: LiteralKind::Number, - value: literal.value.clone(), - }), - "bigint" => Some(CandidateSemantic::Literal { - literal: LiteralKind::Bigint, - value: literal.value.clone(), - }), - _ => { - reasons.insert(CandidateReason::UnsupportedLiteralKind); - None - } - }, - None => { - reasons.insert(CandidateReason::MissingLiteralDetails); - None - } - }, - TypeKind::Union => { - if source.members.is_empty() { - reasons.insert(CandidateReason::EmptyUnion); - } - for member in &source.members { - child_state = merge_state(child_state, self.visit(member)); - } - Some(CandidateSemantic::Union { - members: source.members.clone(), - }) - } - _ => { - reasons.insert(CandidateReason::UnsupportedTypeKind); - None - } - }; - let source_state = match source.state { - EntityState::Complete if source.truncated => { - reasons.insert(CandidateReason::SourceTruncated); - CandidateState::Truncated - } - EntityState::Complete => CandidateState::Complete, - EntityState::Truncated => { - reasons.insert(CandidateReason::SourceTruncated); - CandidateState::Truncated - } - EntityState::Unsupported => { - reasons.insert(CandidateReason::SourceUnsupported); - CandidateState::Unsupported - } - EntityState::Error => { - reasons.insert(CandidateReason::SourceError); - CandidateState::Error +fn oracle_type_supported( + graph: &TypeGraph, + record: &crate::facts::TypeRecord, + visiting: &mut BTreeSet, +) -> bool { + if record.state != EntityState::Complete || !record.complete || record.truncated { + return false; + } + match record.type_kind { + TypeKind::Boolean + | TypeKind::String + | TypeKind::Number + | TypeKind::Bigint + | TypeKind::Null + | TypeKind::Undefined + | TypeKind::Void => true, + TypeKind::Literal => record.literal.as_ref().is_some_and(|literal| { + matches!( + literal.kind.as_str(), + "boolean" | "string" | "number" | "bigint" + ) + }), + TypeKind::Union => { + if !visiting.insert(record.id.clone()) { + return false; } - }; - let structural_state = if semantic.is_some() - && !reasons.contains(&CandidateReason::EmptyUnion) - && !reasons.contains(&CandidateReason::MissingLiteralDetails) - && !reasons.contains(&CandidateReason::UnsupportedLiteralKind) - { - child_state - } else { - CandidateState::Unsupported - }; - let candidate_state = merge_state(source_state, structural_state); - let record = CandidateTypeRecord { - id: id.clone(), - source_kind: source.type_kind, - source_state: source.state, - candidate_state, - semantic, - reasons: reasons.into_iter().collect(), - issues: source.issues.clone(), - }; - self.visiting.remove(id); - self.records.insert(id.clone(), record); - candidate_state + let supported = !record.members.is_empty() + && record.members.iter().all(|member| { + graph + .type_record(member) + .is_some_and(|record| oracle_type_supported(graph, record, visiting)) + }); + visiting.remove(&record.id); + supported + } + _ => false, } } -fn merge_state(left: CandidateState, right: CandidateState) -> CandidateState { - use CandidateState::{Complete, Error, Truncated, Unsupported}; - match (left, right) { - (Error, _) | (_, Error) => Error, - (Truncated, _) | (_, Truncated) => Truncated, - (Unsupported, _) | (_, Unsupported) => Unsupported, - (Complete, Complete) => Complete, +fn literal_wire_name(kind: LiteralKind) -> &'static str { + match kind { + LiteralKind::Boolean => "boolean", + LiteralKind::String => "string", + LiteralKind::Number => "number", + LiteralKind::Bigint => "bigint", } } -fn summarize_states(states: impl Iterator) -> CandidateSummary { - states.fold(CandidateSummary::default(), |mut summary, state| { - match state { - CandidateState::Complete => summary.complete += 1, - CandidateState::Truncated => summary.truncated += 1, - CandidateState::Unsupported => summary.unsupported += 1, - CandidateState::Error => summary.error += 1, - } - summary +fn oracle_record_value(record: &crate::facts::TypeRecord) -> serde_json::Value { + let literal = record.literal.as_ref().map(|literal| { + serde_json::json!({ + "kind": literal.kind, + "value": literal.value, + }) + }); + serde_json::json!({ + "id": record.id.as_str(), + "typeKind": record.type_kind, + "members": record.members, + "literal": literal, + "state": record.state, + "complete": record.complete, + "truncated": record.truncated, }) } @@ -980,13 +885,12 @@ fn summarize(cases: &[ConformanceCase]) -> ConformanceSummary { .or_default() += 1; if difference.expected { summary.expected_differences += 1; - } - if !difference.expected - && matches!( - difference.category, - DifferenceCategory::Semantic | DifferenceCategory::Transport - ) - { + } else if matches!( + difference.category, + DifferenceCategory::Semantic + | DifferenceCategory::Transport + | DifferenceCategory::Mapping + ) { summary.blocking_differences += 1; } } @@ -996,43 +900,6 @@ fn summarize(cases: &[ConformanceCase]) -> ConformanceSummary { summary } -fn sort_differences(differences: &mut [ConformanceDifference]) { - differences.sort_by(|left, right| { - left.category - .cmp(&right.category) - .then_with(|| left.fact_index.cmp(&right.fact_index)) - .then_with(|| left.path.cmp(&right.path)) - .then_with(|| left.code.cmp(&right.code)) - .then_with(|| value_sort_key(&left.go_oracle).cmp(&value_sort_key(&right.go_oracle))) - .then_with(|| { - value_sort_key(&left.rust_candidate).cmp(&value_sort_key(&right.rust_candidate)) - }) - }); -} - -fn compare_scalar( - differences: &mut Vec, - category: DifferenceCategory, - code: &str, - context: Option<(usize, Occurrence)>, - path: &str, - values: (T, T), - explanation: &str, -) { - let (expected, actual) = values; - if expected != actual { - differences.push(mismatch( - category, - code, - context, - path, - json_value(&expected), - json_value(&actual), - explanation, - )); - } -} - fn mismatch( category: DifferenceCategory, code: &str, @@ -1042,10 +909,9 @@ fn mismatch( rust_candidate: Option, explanation: &str, ) -> ConformanceDifference { - let (fact_index, occurrence) = match context { - Some((index, occurrence)) => (Some(index), Some(occurrence)), - None => (None, None), - }; + let (fact_index, occurrence) = context + .map(|(index, occurrence)| (Some(index), Some(occurrence))) + .unwrap_or((None, None)); ConformanceDifference { category, code: code.to_owned(), @@ -1085,6 +951,20 @@ fn json_value(value: &impl Serialize) -> Option { serde_json::to_value(value).ok() } +fn sort_differences(differences: &mut [ConformanceDifference]) { + differences.sort_by(|left, right| { + left.category + .cmp(&right.category) + .then_with(|| left.fact_index.cmp(&right.fact_index)) + .then_with(|| left.path.cmp(&right.path)) + .then_with(|| left.code.cmp(&right.code)) + .then_with(|| value_sort_key(&left.go_oracle).cmp(&value_sort_key(&right.go_oracle))) + .then_with(|| { + value_sort_key(&left.rust_candidate).cmp(&value_sort_key(&right.rust_candidate)) + }) + }); +} + fn value_sort_key(value: &Option) -> String { serde_json::to_string(value).unwrap_or_default() } @@ -1101,7 +981,7 @@ fn category_name(category: DifferenceCategory) -> &'static str { fn ratio_ppm(numerator: usize, denominator: usize) -> u64 { if denominator == 0 { - 1_000_000 + 0 } else { (numerator as u64).saturating_mul(1_000_000) / denominator as u64 } @@ -1169,101 +1049,10 @@ fn build_request<'a>( #[cfg(test)] mod tests { - use std::{fs, io::BufReader, path::Path}; - - use serde::Deserialize; - use super::*; - #[derive(Deserialize)] - #[serde(rename_all = "camelCase")] - struct ClassificationFixture { - description: String, - cases: Vec, - } - - #[derive(Deserialize)] - #[serde(rename_all = "camelCase")] - struct ClassificationCase { - mutation: String, - expected_category: String, - expected_code: String, - } - - #[test] - fn canonical_candidate_has_only_expected_unsupported_and_budget_differences() { - let snapshot = canonical_snapshot(); - let run = RustCandidateRun::from_snapshot(&snapshot); - let case = compare_without_mapping(&snapshot, run); - - assert_eq!(case.roots.facts_with_all_five_views, 4); - assert_eq!(case.roots.compared_roots, 20); - assert_eq!(case.roots.identity_matches, 20); - assert!(case.supported_records > 0); - assert_eq!(case.supported_records, case.matched_supported_records); - assert_eq!(case.supported_compatibility_ppm, 1_000_000); - assert!(case.differences.iter().any(|difference| { - difference.category == DifferenceCategory::Unsupported && difference.expected - })); - assert!(case.differences.iter().any(|difference| { - difference.category == DifferenceCategory::Budget && difference.expected - })); - assert!(case.differences.iter().all(|difference| { - difference.expected - || !matches!( - difference.category, - DifferenceCategory::Semantic | DifferenceCategory::Transport - ) - })); - } - #[test] - fn semantic_and_transport_mutations_are_blocking_and_deterministic() { - let snapshot = canonical_snapshot(); - let fixture = classification_fixture(); - assert!(!fixture.description.is_empty()); - for fixture_case in fixture.cases { - let mut first = RustCandidateRun::from_snapshot(&snapshot); - match fixture_case.mutation.as_str() { - "fact-identity" => { - first.candidates[0].occurrence.syntax_kind = "KindMutated".to_owned(); - } - "root-identity" => { - first.candidates[0].roots[0].type_id = Some(TypeId("type:999".to_owned())); - } - "semantic-payload" => { - let record = first.candidates[0] - .types - .iter_mut() - .find(|record| record.candidate_state == CandidateState::Complete) - .expect("fixture has a supported record"); - record.semantic = Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::String, - }); - } - unknown => panic!("unknown classification mutation {unknown:?}"), - } - let second = first.clone_for_test(); - let first_case = compare_without_mapping(&snapshot, first); - let second_case = compare_without_mapping(&snapshot, second); - assert_eq!( - serde_json::to_string(&first_case).expect("serialize first comparison"), - serde_json::to_string(&second_case).expect("serialize repeated comparison"), - "{}", - fixture_case.mutation - ); - assert!(first_case.differences.iter().any(|difference| { - category_name(difference.category) == fixture_case.expected_category - && difference.code == fixture_case.expected_code - && !difference.expected - })); - let summary = summarize(&[first_case]); - assert!(!threshold_passes(&summary, compatibility_threshold())); - } - } - - #[test] - fn empty_supported_denominator_cannot_pass_vacuously() { + fn empty_supported_denominator_never_passes() { let summary = ConformanceSummary { supported_compatibility_ppm: 1_000_000, ..ConformanceSummary::default() @@ -1271,78 +1060,17 @@ mod tests { assert!(!threshold_passes(&summary, compatibility_threshold())); } - fn canonical_snapshot() -> SemanticSnapshot { - let path = Path::new(env!("CARGO_MANIFEST_DIR")).join( - "../../../../internal/tsfacts/testdata/canonical/v0/primitive-literal-candidate.jsonl", - ); - let file = fs::File::open(&path) - .unwrap_or_else(|error| panic!("open {}: {error}", path.display())); - SemanticSnapshot::from_json_lines(BufReader::new(file)).expect("decode canonical fixture") - } - - fn classification_fixture() -> ClassificationFixture { - let path = Path::new(env!("CARGO_MANIFEST_DIR")) - .join("../../testdata/conformance/v1/classification.json"); - let source = - fs::read(&path).unwrap_or_else(|error| panic!("read {}: {error}", path.display())); - serde_json::from_slice(&source) - .unwrap_or_else(|error| panic!("decode {}: {error}", path.display())) - } - - fn compare_without_mapping( - snapshot: &SemanticSnapshot, - run: RustCandidateRun, - ) -> ConformanceCase { - let mut differences = Vec::new(); - let mut roots = RootCoverage::default(); - let mut candidate_states = CandidateSummary::default(); - let mut supported = 0; - let mut matched = 0; - for (index, (facts, candidate)) in snapshot.facts().iter().zip(&run.candidates).enumerate() - { - let compared = compare_fact( - snapshot.graph(), - index, - facts, - candidate, - &mut differences, - &mut roots, - &mut candidate_states, - ); - supported += compared.0; - matched += compared.1; - } - sort_differences(&mut differences); - ConformanceCase { - name: "fixture".to_owned(), - facts: snapshot.facts().len(), - diagnostics: DiagnosticComparison { - go_oracle_count: snapshot.diagnostic_count, - rust_observed_count: run.diagnostic_count, - matched: snapshot.diagnostic_count == run.diagnostic_count, - }, - budgets: BudgetComparison { - go_oracle: snapshot.budgets, - rust_observed: run.budgets, - matched: snapshot.budgets == run.budgets, - }, - roots, - mapping: MappingCoverage::default(), - candidate_states, - supported_records: supported, - matched_supported_records: matched, - supported_compatibility_ppm: ratio_ppm(matched, supported), - differences, - } - } - - impl RustCandidateRun { - fn clone_for_test(&self) -> Self { - Self { - diagnostic_count: self.diagnostic_count, - budgets: self.budgets, - candidates: self.candidates.clone(), - } - } + #[test] + fn response_local_identity_comparison_is_bijective() { + let mut bijection = TypeBijection::default(); + let go_one = TypeId("type:1".to_owned()); + let go_two = TypeId("type:2".to_owned()); + let rust_one = TypeId("type:8".to_owned()); + let rust_two = TypeId("type:9".to_owned()); + assert!(bijection.bind(&go_one, &rust_one)); + assert!(bijection.bind(&go_two, &rust_two)); + assert!(bijection.bind(&go_one, &rust_one)); + assert!(!bijection.bind(&go_one, &rust_two)); + assert!(!bijection.bind(&go_two, &rust_one)); } } diff --git a/internal/oxc_reference/crates/occurrence-consumer/src/contract.rs b/internal/oxc_reference/crates/occurrence-consumer/src/contract.rs index e2ba85cb7..2ca25b09a 100644 --- a/internal/oxc_reference/crates/occurrence-consumer/src/contract.rs +++ b/internal/oxc_reference/crates/occurrence-consumer/src/contract.rs @@ -4,14 +4,14 @@ use serde::{Deserialize, Serialize}; pub const CONTRACT_VERSION: u32 = 1; -#[derive(Clone, Copy, Debug, Deserialize, Eq, Hash, PartialEq, Serialize)] +#[derive(Clone, Copy, Debug, Deserialize, Eq, Hash, Ord, PartialEq, PartialOrd, Serialize)] #[serde(rename_all = "camelCase")] pub struct Span { pub start: u32, pub end: u32, } -#[derive(Clone, Debug, Deserialize, Eq, Hash, PartialEq, Serialize)] +#[derive(Clone, Debug, Deserialize, Eq, Hash, Ord, PartialEq, PartialOrd, Serialize)] #[serde(rename_all = "camelCase")] pub struct Occurrence { pub file: String, diff --git a/internal/oxc_reference/crates/occurrence-consumer/src/inspector.rs b/internal/oxc_reference/crates/occurrence-consumer/src/inspector.rs index 8df09eb3c..77c611a44 100644 --- a/internal/oxc_reference/crates/occurrence-consumer/src/inspector.rs +++ b/internal/oxc_reference/crates/occurrence-consumer/src/inspector.rs @@ -2,7 +2,6 @@ use std::collections::{BTreeSet, VecDeque}; use serde::Serialize; -use crate::candidate::PrimitiveLiteralCandidate; use crate::facts::{ EntityState, GraphIssue, GraphRef, OccurrenceTypeFacts, TypeGraph, TypeView, TypeViewState, }; @@ -30,7 +29,6 @@ impl Default for InspectorLimits { pub struct InspectionReport { pub fact: InspectedFactStatus, pub roots: Vec, - pub primitive_literal_candidate: PrimitiveLiteralCandidate, pub nodes: Vec, pub edges: Vec, pub diagnostics: Vec, @@ -254,7 +252,6 @@ impl<'a> GraphInspector<'a> { truncated: facts.truncated, }, roots, - primitive_literal_candidate: PrimitiveLiteralCandidate::build(self.graph, facts), summary: InspectionSummary { nodes: nodes.len(), edges: edges.len(), diff --git a/internal/oxc_reference/crates/occurrence-consumer/src/lib.rs b/internal/oxc_reference/crates/occurrence-consumer/src/lib.rs index 0567e6852..9f7ba3894 100644 --- a/internal/oxc_reference/crates/occurrence-consumer/src/lib.rs +++ b/internal/oxc_reference/crates/occurrence-consumer/src/lib.rs @@ -6,6 +6,7 @@ pub mod facts; pub mod fixture; pub mod inspector; pub mod oxc; +pub mod primitive_producer; #[cfg(test)] mod tests { @@ -20,10 +21,6 @@ mod tests { use oxc_allocator::Allocator; use crate::{ - candidate::{ - CandidateReason, CandidateSemantic, CandidateState, LiteralKind, NullLikeKind, - PrimitiveKind, - }, contract::{DiagnosticCode, correlate}, facts::{GraphRef, SemanticSnapshot, TypeViewState}, fixture::load_fixtures, @@ -144,152 +141,6 @@ mod tests { assert!(std::ptr::eq(first.graph(), second.graph())); } - #[test] - fn primitive_literal_candidate_is_structured_deterministic_and_state_preserving() { - let snapshot = load_canonical_snapshot("primitive-literal-candidate.jsonl"); - let source = fs::read_to_string(canonical_fixture_path("primitive-literal-candidate.ts")) - .expect("read primitive/literal candidate source"); - let allocator = Allocator::default(); - let mut consumer = - OxcConsumer::parse(&allocator, "src/primitive-literal-candidate.ts", &source) - .expect("parse primitive/literal candidate source"); - let report = consumer - .attach(snapshot) - .expect("attach primitive/literal candidate fixture"); - assert_eq!(report.summary.facts, 4); - assert_eq!(report.summary.mapped, 4); - - let inspections = report - .mappings - .iter() - .map(|mapping| { - let node = consumer - .node_for_fact(mapping.fact_index) - .expect("mapped candidate node"); - let attached = consumer - .type_facts_for_node(node) - .find(|facts| facts.fact_index == mapping.fact_index) - .expect("attached candidate facts"); - let first = attached.inspect(InspectorLimits::default()); - let second = attached.inspect(InspectorLimits::default()); - assert_eq!( - serde_json::to_string(&first.primitive_literal_candidate) - .expect("serialize first candidate"), - serde_json::to_string(&second.primitive_literal_candidate) - .expect("serialize second candidate") - ); - first - }) - .collect::>(); - - let primitives = &inspections[0].primitive_literal_candidate; - assert_eq!(primitives.roots.len(), 5); - assert!(matches!( - &candidate_type(primitives, "type:1").semantic, - Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Boolean - }) - )); - assert!(matches!( - &candidate_type(primitives, "type:2").semantic, - Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::String - }) - )); - assert!(matches!( - &candidate_type(primitives, "type:3").semantic, - Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Number - }) - )); - assert!(matches!( - &candidate_type(primitives, "type:4").semantic, - Some(CandidateSemantic::Primitive { - primitive: PrimitiveKind::Bigint - }) - )); - assert!(matches!( - &candidate_type(primitives, "type:5").semantic, - Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Null - }) - )); - - let literals = &inspections[1].primitive_literal_candidate; - for (id, kind, value) in [ - ("type:6", LiteralKind::String, "ready"), - ("type:7", LiteralKind::Boolean, "true"), - ("type:8", LiteralKind::Number, "42"), - ("type:9", LiteralKind::Bigint, "42"), - ] { - assert!(matches!( - &candidate_type(literals, id).semantic, - Some(CandidateSemantic::Literal { - literal, - value: actual - }) if *literal == kind && actual == value - )); - } - assert!(matches!( - &candidate_type(literals, "type:10").semantic, - Some(CandidateSemantic::NullLike { - null_like: NullLikeKind::Undefined - }) - )); - - let union = &inspections[2].primitive_literal_candidate; - assert!(matches!( - &candidate_type(union, "type:11").semantic, - Some(CandidateSemantic::Union { members }) - if members.iter().map(|id| id.as_str()).collect::>() - == vec!["type:6", "type:7", "type:8", "type:9", "type:5", "type:10"] - )); - assert_eq!(union.summary.complete, 7); - - let incomplete = &inspections[3].primitive_literal_candidate; - assert_eq!(incomplete.roots.len(), 5); - assert_eq!( - incomplete.roots[3].type_id.as_ref().map(|id| id.as_str()), - Some("type:12") - ); - assert_eq!(incomplete.roots[4].state, TypeViewState::Unavailable); - assert!(incomplete.roots[4].type_id.is_none()); - assert_eq!( - candidate_type(incomplete, "type:12").candidate_state, - CandidateState::Unsupported - ); - assert_eq!( - candidate_type(incomplete, "type:13").candidate_state, - CandidateState::Truncated - ); - assert_eq!( - candidate_type(incomplete, "type:14").candidate_state, - CandidateState::Truncated - ); - assert!( - candidate_type(incomplete, "type:12") - .reasons - .contains(&CandidateReason::SourceUnsupported) - ); - assert!( - candidate_type(incomplete, "type:14") - .reasons - .contains(&CandidateReason::SourceTruncated) - ); - assert_eq!(incomplete.summary.complete, 1); - assert_eq!(incomplete.summary.truncated, 2); - assert_eq!(incomplete.summary.unsupported, 1); - - let serialized = serde_json::to_string( - &inspections - .iter() - .map(|inspection| &inspection.primitive_literal_candidate) - .collect::>(), - ) - .expect("serialize structured primitive/literal candidates"); - assert!(!serialized.contains("\"display\"")); - } - #[test] fn inspector_is_deterministic_identity_preserving_and_complete_about_states() { let snapshot = load_canonical_snapshot("graph-inspector.jsonl"); @@ -511,17 +362,6 @@ mod tests { ) } - fn candidate_type<'a>( - candidate: &'a crate::candidate::PrimitiveLiteralCandidate, - id: &str, - ) -> &'a crate::candidate::CandidateTypeRecord { - candidate - .types - .iter() - .find(|record| record.id.as_str() == id) - .unwrap_or_else(|| panic!("candidate omitted {id}")) - } - fn minimal_snapshot(syntax_kinds: &[&str]) -> String { let mut output = String::from( "{\"record\":\"header\",\"schemaVersion\":1,\"offsetEncoding\":\"utf8-bytes\",\"capabilities\":[]}\n\ diff --git a/internal/oxc_reference/crates/occurrence-consumer/src/primitive_producer.rs b/internal/oxc_reference/crates/occurrence-consumer/src/primitive_producer.rs new file mode 100644 index 000000000..b669b7280 --- /dev/null +++ b/internal/oxc_reference/crates/occurrence-consumer/src/primitive_producer.rs @@ -0,0 +1,1091 @@ +use std::{ + collections::{BTreeMap, BTreeSet}, + fs, + path::Path, +}; + +use oxc_allocator::Allocator; +use oxc_ast::{ + AstKind, + ast::{Expression, TSLiteral, TSType, TSTypeName}, +}; +use oxc_parser::{ParseOptions, Parser}; +use oxc_semantic::{NodeId, Semantic, SemanticBuilder}; +use oxc_span::{GetSpan, SourceType}; +use serde::Serialize; + +use crate::{ + candidate::{ + CandidateFactStatus, CandidateReason, CandidateRoot, CandidateSemantic, CandidateState, + CandidateSummary, CandidateTypeRecord, LiteralKind, NullLikeKind, + PRIMITIVE_LITERAL_CANDIDATE_VERSION, PrimitiveKind, PrimitiveLiteralCandidate, + }, + contract::{Occurrence, Span}, + facts::{TypeId, TypeView, TypeViewState}, +}; + +pub const INDEPENDENT_PRIMITIVE_LITERAL_PRODUCER_VERSION: u32 = 1; + +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct PrimitiveLiteralSelection { + pub file: String, + pub span: Span, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct PrimitiveProducerLimits { + pub max_type_nodes: usize, +} + +impl Default for PrimitiveProducerLimits { + fn default() -> Self { + Self { + max_type_nodes: 4096, + } + } +} + +#[derive(Clone, Debug, Eq, PartialEq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct IndependentPrimitiveLiteralOutput { + pub producer_version: u32, + pub candidate_version: u32, + pub limits: PrimitiveProducerLimits, + pub type_nodes_used: usize, + pub truncated: bool, + pub diagnostics: Vec, + pub candidates: Vec, +} + +#[derive(Clone, Debug, Eq, PartialEq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct PrimitiveProducerDiagnostic { + pub file: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub span: Option, + pub code: String, + pub message: String, +} + +pub fn produce_primitive_literals( + project_root: &Path, + selections: &[PrimitiveLiteralSelection], + limits: PrimitiveProducerLimits, +) -> Result { + let mut sources = BTreeMap::new(); + for selection in selections { + if !sources.contains_key(&selection.file) { + let path = project_root.join(&selection.file); + let source = fs::read_to_string(&path) + .map_err(|error| format!("read project source {}: {error}", path.display()))?; + sources.insert(selection.file.clone(), source); + } + } + Ok(produce_primitive_literals_from_sources( + &sources, selections, limits, + )) +} + +pub fn produce_primitive_literals_from_sources( + sources: &BTreeMap, + selections: &[PrimitiveLiteralSelection], + limits: PrimitiveProducerLimits, +) -> IndependentPrimitiveLiteralOutput { + let mut producer = PrimitiveProducer::new(limits); + let mut candidates = vec![None; selections.len()]; + + for (file, source) in sources { + let selected = selections + .iter() + .enumerate() + .filter(|(_, selection)| selection.file == *file) + .collect::>(); + if selected.is_empty() { + continue; + } + producer.produce_file(file, source, &selected, &mut candidates); + } + + for (index, selection) in selections.iter().enumerate() { + if candidates[index].is_none() { + let message = if sources.contains_key(&selection.file) { + "selection was not produced" + } else { + "selection source was not supplied" + }; + producer.diagnostics.push(PrimitiveProducerDiagnostic { + file: selection.file.clone(), + span: Some(selection.span), + code: "source-unavailable".to_owned(), + message: message.to_owned(), + }); + candidates[index] = Some(producer.failure_candidate( + selection, + "KindUnknown", + CandidateState::Error, + CandidateReason::OxcParseOrSemanticError, + )); + } + } + + IndependentPrimitiveLiteralOutput { + producer_version: INDEPENDENT_PRIMITIVE_LITERAL_PRODUCER_VERSION, + candidate_version: PRIMITIVE_LITERAL_CANDIDATE_VERSION, + limits, + type_nodes_used: producer.interner.records.len(), + truncated: producer.interner.truncated, + diagnostics: producer.diagnostics, + candidates: candidates + .into_iter() + .map(|candidate| candidate.expect("every selection receives a candidate")) + .collect(), + } +} + +struct PrimitiveProducer { + interner: TypeInterner, + diagnostics: Vec, +} + +impl PrimitiveProducer { + fn new(limits: PrimitiveProducerLimits) -> Self { + Self { + interner: TypeInterner::new(limits.max_type_nodes), + diagnostics: Vec::new(), + } + } + + fn produce_file( + &mut self, + file: &str, + source: &str, + selections: &[(usize, &PrimitiveLiteralSelection)], + candidates: &mut [Option], + ) { + let allocator = Allocator::default(); + let source_type = match SourceType::from_path(Path::new(file)) { + Ok(source_type) => source_type, + Err(_) => { + self.fail_file( + file, + selections, + candidates, + "unsupported-source-type", + format!("unsupported source type for {file:?}"), + ); + return; + } + }; + let parsed = Parser::new(&allocator, source, source_type) + .with_options(ParseOptions { + preserve_parens: true, + ..ParseOptions::default() + }) + .parse(); + if parsed.panicked { + self.fail_file( + file, + selections, + candidates, + "oxc-parse-error", + "OXC parser panicked before it could produce a recoverable tree".to_owned(), + ); + return; + } + let mut recovered = !parsed.diagnostics.is_empty(); + if recovered { + self.diagnostics.push(PrimitiveProducerDiagnostic { + file: file.to_owned(), + span: None, + code: "oxc-parse-recovery".to_owned(), + message: format!( + "OXC recovered a syntax tree with {} parser diagnostic(s)", + parsed.diagnostics.len() + ), + }); + } + let built = SemanticBuilder::new_compiler() + .with_build_nodes(true) + .build(&parsed.program); + if !built.diagnostics.is_empty() { + recovered = true; + self.diagnostics.push(PrimitiveProducerDiagnostic { + file: file.to_owned(), + span: None, + code: "oxc-semantic-recovery".to_owned(), + message: format!( + "OXC built semantics with {} diagnostic(s)", + built.diagnostics.len() + ), + }); + } + let semantic = built.semantic; + + for (index, selection) in selections { + let matched = semantic + .nodes() + .iter_enumerated() + .find_map(|(node_id, node)| { + let kind = node.kind(); + let node_span = portable_span(kind.span()); + (node_span == selection.span) + .then(|| syntax_kind(kind).map(|syntax_kind| (node_id, kind, syntax_kind))) + .flatten() + }); + let Some((node_id, kind, syntax_kind)) = matched else { + self.diagnostics.push(PrimitiveProducerDiagnostic { + file: file.to_owned(), + span: Some(selection.span), + code: "selection-unmapped".to_owned(), + message: "no exact OXC semantic node matched the requested UTF-8 span" + .to_owned(), + }); + candidates[*index] = Some(self.failure_candidate( + selection, + "KindUnknown", + CandidateState::Unsupported, + CandidateReason::SelectionUnmapped, + )); + continue; + }; + candidates[*index] = Some(self.build_candidate( + selection, + node_id, + kind, + syntax_kind, + &semantic, + recovered, + )); + } + } + + fn fail_file( + &mut self, + file: &str, + selections: &[(usize, &PrimitiveLiteralSelection)], + candidates: &mut [Option], + code: &str, + message: String, + ) { + self.diagnostics.push(PrimitiveProducerDiagnostic { + file: file.to_owned(), + span: None, + code: code.to_owned(), + message, + }); + for (index, selection) in selections { + candidates[*index] = Some(self.failure_candidate( + selection, + "KindUnknown", + CandidateState::Error, + CandidateReason::OxcParseOrSemanticError, + )); + } + } + + fn build_candidate( + &mut self, + selection: &PrimitiveLiteralSelection, + node_id: NodeId, + kind: AstKind<'_>, + syntax_kind: String, + semantic: &Semantic<'_>, + recovered: bool, + ) -> PrimitiveLiteralCandidate { + let mut resolving = BTreeSet::new(); + let actual = infer_kind(kind, node_id, semantic, &mut resolving); + let subject = subject_kind(kind); + let contextual = if subject == SubjectKind::Literal { + contextual_shape(node_id, semantic, &mut resolving) + } else { + None + }; + let apparent = apparent_shape(&actual); + let root_shapes = [ + (TypeView::Actual, TypeViewState::Available, Some(actual)), + ( + TypeView::Contextual, + if contextual.is_some() { + TypeViewState::Available + } else { + TypeViewState::Unavailable + }, + contextual, + ), + (TypeView::Widened, TypeViewState::SameAsActual, None), + match apparent { + Some(apparent) => (TypeView::Apparent, TypeViewState::Available, Some(apparent)), + None => (TypeView::Apparent, TypeViewState::SameAsActual, None), + }, + ( + TypeView::Declared, + if subject == SubjectKind::Identifier { + TypeViewState::SameAsActual + } else { + TypeViewState::Inapplicable + }, + None, + ), + ]; + let actual_id = self.interner.intern(root_shapes[0].2.as_ref().unwrap()); + let mut roots = Vec::with_capacity(5); + for (view, state, shape) in root_shapes { + let type_id = match state { + TypeViewState::Available => shape.as_ref().map(|shape| self.interner.intern(shape)), + TypeViewState::SameAsActual => Some(actual_id.clone()), + TypeViewState::Inapplicable | TypeViewState::Unavailable => None, + }; + roots.push(CandidateRoot { + view, + state, + type_id, + }); + } + self.finish_candidate( + Occurrence { + file: selection.file.clone(), + span: selection.span, + syntax_kind, + }, + Some(u32::try_from(node_id.index()).expect("OXC NodeId exceeds u32")), + roots, + recovered, + ) + } + + fn failure_candidate( + &mut self, + selection: &PrimitiveLiteralSelection, + syntax_kind: &str, + state: CandidateState, + reason: CandidateReason, + ) -> PrimitiveLiteralCandidate { + let shape = SemanticShape::Unavailable { + key: format!( + "{}:{}:{}:{}", + selection.file, selection.span.start, selection.span.end, syntax_kind + ), + state, + reason, + }; + let actual = self.interner.intern(&shape); + let roots = [ + ( + TypeView::Actual, + TypeViewState::Available, + Some(actual.clone()), + ), + (TypeView::Contextual, TypeViewState::Unavailable, None), + ( + TypeView::Widened, + TypeViewState::SameAsActual, + Some(actual.clone()), + ), + ( + TypeView::Apparent, + TypeViewState::SameAsActual, + Some(actual.clone()), + ), + (TypeView::Declared, TypeViewState::Unavailable, None), + ] + .into_iter() + .map(|(view, state, type_id)| CandidateRoot { + view, + state, + type_id, + }) + .collect(); + self.finish_candidate( + Occurrence { + file: selection.file.clone(), + span: selection.span, + syntax_kind: syntax_kind.to_owned(), + }, + None, + roots, + false, + ) + } + + fn finish_candidate( + &self, + occurrence: Occurrence, + oxc_node_id: Option, + roots: Vec, + recovered: bool, + ) -> PrimitiveLiteralCandidate { + let types = self.interner.reachable_records(&roots); + let mut summary = CandidateSummary::default(); + for record in &types { + summary.add(record.candidate_state); + } + PrimitiveLiteralCandidate { + candidate_version: PRIMITIVE_LITERAL_CANDIDATE_VERSION, + occurrence, + oxc_node_id, + fact: CandidateFactStatus { + complete: !recovered + && summary.truncated == 0 + && summary.unsupported == 0 + && summary.error == 0, + recovered, + truncated: summary.truncated > 0, + }, + roots, + types, + summary, + } + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum SubjectKind { + Identifier, + Literal, + Other, +} + +fn subject_kind(kind: AstKind<'_>) -> SubjectKind { + match kind { + AstKind::IdentifierReference(_) | AstKind::BindingIdentifier(_) => SubjectKind::Identifier, + AstKind::BooleanLiteral(_) + | AstKind::NullLiteral(_) + | AstKind::NumericLiteral(_) + | AstKind::BigIntLiteral(_) + | AstKind::StringLiteral(_) => SubjectKind::Literal, + _ => SubjectKind::Other, + } +} + +fn infer_kind( + kind: AstKind<'_>, + node_id: NodeId, + semantic: &Semantic<'_>, + resolving: &mut BTreeSet<(u32, u32)>, +) -> SemanticShape { + match kind { + AstKind::BooleanLiteral(literal) => boolean_literal(literal.value), + AstKind::NullLiteral(_) => SemanticShape::NullLike(NullLikeKind::Null), + AstKind::NumericLiteral(literal) => { + SemanticShape::Literal(LiteralKind::Number, number_value(literal.value)) + } + AstKind::BigIntLiteral(literal) => { + SemanticShape::Literal(LiteralKind::Bigint, literal.value.to_string()) + } + AstKind::StringLiteral(literal) => { + SemanticShape::Literal(LiteralKind::String, literal.value.to_string()) + } + AstKind::IdentifierReference(identifier) => identifier + .reference_id + .get() + .and_then(|reference_id| semantic.scoping().get_reference(reference_id).symbol_id()) + .map(|symbol_id| { + infer_declaration( + semantic.symbol_declaration(symbol_id).kind(), + semantic, + resolving, + ) + }) + .unwrap_or_else(|| { + if identifier.name == "undefined" { + SemanticShape::NullLike(NullLikeKind::Undefined) + } else { + unsupported(node_id, CandidateReason::UnsupportedExpression) + } + }), + AstKind::BindingIdentifier(identifier) => identifier + .symbol_id + .get() + .map(|symbol_id| { + infer_declaration( + semantic.symbol_declaration(symbol_id).kind(), + semantic, + resolving, + ) + }) + .unwrap_or_else(|| unsupported(node_id, CandidateReason::UnsupportedExpression)), + AstKind::TSBooleanKeyword(_) => SemanticShape::Primitive(PrimitiveKind::Boolean), + AstKind::TSStringKeyword(_) => SemanticShape::Primitive(PrimitiveKind::String), + AstKind::TSNumberKeyword(_) => SemanticShape::Primitive(PrimitiveKind::Number), + AstKind::TSBigIntKeyword(_) => SemanticShape::Primitive(PrimitiveKind::Bigint), + AstKind::TSNullKeyword(_) => SemanticShape::NullLike(NullLikeKind::Null), + AstKind::TSUndefinedKeyword(_) => SemanticShape::NullLike(NullLikeKind::Undefined), + AstKind::TSVoidKeyword(_) => SemanticShape::NullLike(NullLikeKind::Void), + AstKind::TSLiteralType(literal) => infer_ts_literal(&literal.literal), + AstKind::TSUnionType(union) => canonical_union( + union + .types + .iter() + .map(|member| infer_ts_type(member, semantic, resolving)) + .collect(), + ), + _ => unsupported(node_id, CandidateReason::UnsupportedTypeForm), + } +} + +fn infer_declaration( + kind: AstKind<'_>, + semantic: &Semantic<'_>, + resolving: &mut BTreeSet<(u32, u32)>, +) -> SemanticShape { + match kind { + AstKind::VariableDeclarator(declarator) => declarator + .type_annotation + .as_ref() + .map(|annotation| infer_ts_type(&annotation.type_annotation, semantic, resolving)) + .or_else(|| { + declarator + .init + .as_ref() + .map(|expression| infer_expression(expression, semantic, resolving)) + }) + .unwrap_or_else(|| { + unavailable_span(declarator.span, CandidateReason::UnsupportedExpression) + }), + AstKind::TSTypeAliasDeclaration(alias) => { + let key = (alias.span.start, alias.span.end); + if !resolving.insert(key) { + return unavailable_span(alias.span, CandidateReason::UnsupportedTypeForm); + } + let shape = infer_ts_type(&alias.type_annotation, semantic, resolving); + resolving.remove(&key); + shape + } + _ => unavailable_span(kind.span(), CandidateReason::UnsupportedTypeForm), + } +} + +fn infer_ts_type( + r#type: &TSType<'_>, + semantic: &Semantic<'_>, + resolving: &mut BTreeSet<(u32, u32)>, +) -> SemanticShape { + match r#type { + TSType::TSBooleanKeyword(_) => SemanticShape::Primitive(PrimitiveKind::Boolean), + TSType::TSStringKeyword(_) => SemanticShape::Primitive(PrimitiveKind::String), + TSType::TSNumberKeyword(_) => SemanticShape::Primitive(PrimitiveKind::Number), + TSType::TSBigIntKeyword(_) => SemanticShape::Primitive(PrimitiveKind::Bigint), + TSType::TSNullKeyword(_) => SemanticShape::NullLike(NullLikeKind::Null), + TSType::TSUndefinedKeyword(_) => SemanticShape::NullLike(NullLikeKind::Undefined), + TSType::TSVoidKeyword(_) => SemanticShape::NullLike(NullLikeKind::Void), + TSType::TSLiteralType(literal) => infer_ts_literal(&literal.literal), + TSType::TSUnionType(union) => canonical_union( + union + .types + .iter() + .map(|member| infer_ts_type(member, semantic, resolving)) + .collect(), + ), + TSType::TSParenthesizedType(parenthesized) => { + infer_ts_type(&parenthesized.type_annotation, semantic, resolving) + } + TSType::TSTypeReference(reference) => match &reference.type_name { + TSTypeName::IdentifierReference(identifier) => identifier + .reference_id + .get() + .and_then(|reference_id| semantic.scoping().get_reference(reference_id).symbol_id()) + .map(|symbol_id| { + infer_declaration( + semantic.symbol_declaration(symbol_id).kind(), + semantic, + resolving, + ) + }) + .unwrap_or_else(|| { + unavailable_span(reference.span, CandidateReason::UnsupportedTypeForm) + }), + TSTypeName::QualifiedName(_) | TSTypeName::ThisExpression(_) => { + unavailable_span(reference.span, CandidateReason::UnsupportedTypeForm) + } + }, + _ => unavailable_span(r#type.span(), CandidateReason::UnsupportedTypeForm), + } +} + +fn infer_ts_literal(literal: &TSLiteral<'_>) -> SemanticShape { + match literal { + TSLiteral::BooleanLiteral(literal) => boolean_literal(literal.value), + TSLiteral::NumericLiteral(literal) => { + SemanticShape::Literal(LiteralKind::Number, number_value(literal.value)) + } + TSLiteral::BigIntLiteral(literal) => { + SemanticShape::Literal(LiteralKind::Bigint, literal.value.to_string()) + } + TSLiteral::StringLiteral(literal) => { + SemanticShape::Literal(LiteralKind::String, literal.value.to_string()) + } + TSLiteral::TemplateLiteral(_) | TSLiteral::UnaryExpression(_) => { + unavailable_span(literal.span(), CandidateReason::UnsupportedTypeForm) + } + } +} + +fn infer_expression( + expression: &Expression<'_>, + semantic: &Semantic<'_>, + resolving: &mut BTreeSet<(u32, u32)>, +) -> SemanticShape { + match expression { + Expression::BooleanLiteral(literal) => boolean_literal(literal.value), + Expression::NullLiteral(_) => SemanticShape::NullLike(NullLikeKind::Null), + Expression::NumericLiteral(literal) => { + SemanticShape::Literal(LiteralKind::Number, number_value(literal.value)) + } + Expression::BigIntLiteral(literal) => { + SemanticShape::Literal(LiteralKind::Bigint, literal.value.to_string()) + } + Expression::StringLiteral(literal) => { + SemanticShape::Literal(LiteralKind::String, literal.value.to_string()) + } + Expression::Identifier(identifier) => identifier + .reference_id + .get() + .and_then(|reference_id| semantic.scoping().get_reference(reference_id).symbol_id()) + .map(|symbol_id| { + infer_declaration( + semantic.symbol_declaration(symbol_id).kind(), + semantic, + resolving, + ) + }) + .unwrap_or_else(|| { + unavailable_span(identifier.span, CandidateReason::UnsupportedExpression) + }), + Expression::ParenthesizedExpression(parenthesized) => { + infer_expression(&parenthesized.expression, semantic, resolving) + } + Expression::TSAsExpression(assertion) => { + infer_ts_type(&assertion.type_annotation, semantic, resolving) + } + Expression::TSTypeAssertion(assertion) => { + infer_ts_type(&assertion.type_annotation, semantic, resolving) + } + Expression::TSSatisfiesExpression(satisfies) => { + infer_expression(&satisfies.expression, semantic, resolving) + } + _ => unavailable_span(expression.span(), CandidateReason::UnsupportedExpression), + } +} + +fn contextual_shape( + node_id: NodeId, + semantic: &Semantic<'_>, + resolving: &mut BTreeSet<(u32, u32)>, +) -> Option { + semantic.nodes().ancestor_kinds(node_id).find_map(|kind| { + let AstKind::VariableDeclarator(declarator) = kind else { + return None; + }; + let annotation = declarator.type_annotation.as_ref()?; + let shape = infer_ts_type(&annotation.type_annotation, semantic, resolving); + (!matches!(shape, SemanticShape::NullLike(NullLikeKind::Null))).then_some(shape) + }) +} + +fn apparent_shape(actual: &SemanticShape) -> Option { + let primitive = match actual { + SemanticShape::Primitive(primitive) => Some(*primitive), + SemanticShape::Literal(literal, _) => Some(match literal { + LiteralKind::Boolean => PrimitiveKind::Boolean, + LiteralKind::String => PrimitiveKind::String, + LiteralKind::Number => PrimitiveKind::Number, + LiteralKind::Bigint => PrimitiveKind::Bigint, + }), + SemanticShape::NullLike(_) + | SemanticShape::Union(_) + | SemanticShape::Unavailable { .. } => None, + }?; + Some(SemanticShape::Unavailable { + key: format!("apparent:{primitive:?}"), + state: CandidateState::Truncated, + reason: CandidateReason::ApparentTypeOutsideCategory, + }) +} + +fn canonical_union(mut members: Vec) -> SemanticShape { + members.sort_by_key(union_member_rank); + members.dedup(); + SemanticShape::Union(members) +} + +fn union_member_rank(member: &SemanticShape) -> u8 { + match member { + SemanticShape::Literal(LiteralKind::String, _) => 0, + SemanticShape::Literal(LiteralKind::Number, _) => 1, + SemanticShape::Literal(LiteralKind::Bigint, _) => 2, + SemanticShape::Literal(LiteralKind::Boolean, _) => 3, + SemanticShape::Primitive(PrimitiveKind::String) => 4, + SemanticShape::Primitive(PrimitiveKind::Number) => 5, + SemanticShape::Primitive(PrimitiveKind::Bigint) => 6, + SemanticShape::Primitive(PrimitiveKind::Boolean) => 7, + SemanticShape::NullLike(NullLikeKind::Null) => 8, + SemanticShape::NullLike(NullLikeKind::Undefined) => 9, + SemanticShape::NullLike(NullLikeKind::Void) => 10, + SemanticShape::Union(_) => 11, + SemanticShape::Unavailable { .. } => 12, + } +} + +fn boolean_literal(value: bool) -> SemanticShape { + SemanticShape::Literal(LiteralKind::Boolean, value.to_string()) +} + +fn number_value(value: f64) -> String { + if value.fract() == 0.0 { + format!("{value:.0}") + } else { + value.to_string() + } +} + +fn unsupported(node_id: NodeId, reason: CandidateReason) -> SemanticShape { + SemanticShape::Unavailable { + key: format!("node:{}", node_id.index()), + state: CandidateState::Unsupported, + reason, + } +} + +fn unavailable_span(span: oxc_span::Span, reason: CandidateReason) -> SemanticShape { + SemanticShape::Unavailable { + key: format!("span:{}:{}", span.start, span.end), + state: CandidateState::Unsupported, + reason, + } +} + +fn syntax_kind(kind: AstKind<'_>) -> Option { + Some( + match kind { + AstKind::IdentifierReference(_) + | AstKind::IdentifierName(_) + | AstKind::BindingIdentifier(_) => "KindIdentifier", + AstKind::NumericLiteral(_) => "KindNumericLiteral", + AstKind::BigIntLiteral(_) => "KindBigIntLiteral", + AstKind::StringLiteral(_) => "KindStringLiteral", + AstKind::BooleanLiteral(literal) if literal.value => "KindTrueKeyword", + AstKind::BooleanLiteral(_) => "KindFalseKeyword", + AstKind::NullLiteral(_) => "KindNullKeyword", + AstKind::TSBooleanKeyword(_) => "KindBooleanKeyword", + AstKind::TSStringKeyword(_) => "KindStringKeyword", + AstKind::TSNumberKeyword(_) => "KindNumberKeyword", + AstKind::TSBigIntKeyword(_) => "KindBigIntKeyword", + AstKind::TSNullKeyword(_) => "KindNullKeyword", + AstKind::TSUndefinedKeyword(_) => "KindUndefinedKeyword", + AstKind::TSVoidKeyword(_) => "KindVoidKeyword", + AstKind::TSLiteralType(_) => "KindLiteralType", + AstKind::TSUnionType(_) => "KindUnionType", + _ => return None, + } + .to_owned(), + ) +} + +fn portable_span(span: oxc_span::Span) -> Span { + Span { + start: span.start, + end: span.end, + } +} + +#[derive(Clone, Debug, Eq, Ord, PartialEq, PartialOrd)] +enum SemanticShape { + Primitive(PrimitiveKind), + Literal(LiteralKind, String), + NullLike(NullLikeKind), + Union(Vec), + Unavailable { + key: String, + state: CandidateState, + reason: CandidateReason, + }, +} + +struct TypeInterner { + max_type_nodes: usize, + ids: BTreeMap, + records: BTreeMap, + truncated: bool, + truncation_id: Option, +} + +impl TypeInterner { + fn new(max_type_nodes: usize) -> Self { + Self { + max_type_nodes, + ids: BTreeMap::new(), + records: BTreeMap::new(), + truncated: false, + truncation_id: None, + } + } + + fn intern(&mut self, shape: &SemanticShape) -> TypeId { + if let Some(id) = self.ids.get(shape) { + return id.clone(); + } + if self.records.len() >= self.max_type_nodes { + return self.truncation_id(); + } + let semantic = match shape { + SemanticShape::Primitive(primitive) => Some(CandidateSemantic::Primitive { + primitive: *primitive, + }), + SemanticShape::Literal(literal, value) => Some(CandidateSemantic::Literal { + literal: *literal, + value: value.clone(), + }), + SemanticShape::NullLike(null_like) => Some(CandidateSemantic::NullLike { + null_like: *null_like, + }), + SemanticShape::Union(members) => Some(CandidateSemantic::Union { + members: members.iter().map(|member| self.intern(member)).collect(), + }), + SemanticShape::Unavailable { .. } => None, + }; + if let Some(id) = self.ids.get(shape) { + return id.clone(); + } + if self.records.len() >= self.max_type_nodes { + return self.truncation_id(); + } + let id = TypeId(format!("type:{}", self.records.len() + 1)); + let (candidate_state, reasons) = match shape { + SemanticShape::Unavailable { state, reason, .. } => (*state, vec![*reason]), + _ => (CandidateState::Complete, Vec::new()), + }; + self.ids.insert(shape.clone(), id.clone()); + self.records.insert( + id.clone(), + CandidateTypeRecord { + id: id.clone(), + candidate_state, + semantic, + reasons, + }, + ); + id + } + + fn truncation_id(&mut self) -> TypeId { + self.truncated = true; + if let Some(id) = &self.truncation_id { + return id.clone(); + } + let id = TypeId(format!("type:{}", self.records.len() + 1)); + self.records.insert( + id.clone(), + CandidateTypeRecord { + id: id.clone(), + candidate_state: CandidateState::Truncated, + semantic: None, + reasons: vec![CandidateReason::TypeBudgetExceeded], + }, + ); + self.truncation_id = Some(id.clone()); + id + } + + fn reachable_records(&self, roots: &[CandidateRoot]) -> Vec { + let mut pending = roots + .iter() + .filter_map(|root| root.type_id.clone()) + .collect::>(); + let mut reachable = BTreeSet::new(); + while let Some(id) = pending.pop() { + if !reachable.insert(id.clone()) { + continue; + } + if let Some(CandidateTypeRecord { + semantic: Some(CandidateSemantic::Union { members }), + .. + }) = self.records.get(&id) + { + pending.extend(members.iter().cloned()); + } + } + reachable + .into_iter() + .filter_map(|id| self.records.get(&id).cloned()) + .collect() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const SOURCE: &str = r#" +type LiteralUnion = "ready" | 42 | true | 42n | null | undefined; +type Unsupported = { value: string }; +declare const union: LiteralUnion; +declare const booleanValue: boolean; +declare const stringValue: string; +declare const numberValue: number; +declare const bigintValue: bigint; +declare const unsupported: Unsupported; +const contextual: string = "context"; +union; +booleanValue; +stringValue; +numberValue; +bigintValue; +unsupported; +"#; + + #[test] + fn source_and_oxc_facts_produce_all_selected_semantics_deterministically() { + let sources = BTreeMap::from([("src/primitives.ts".to_owned(), SOURCE.to_owned())]); + let selections = [ + ("union", 1), + ("booleanValue", 1), + ("stringValue", 1), + ("numberValue", 1), + ("bigintValue", 1), + ("\"context\"", 0), + ("unsupported", 1), + ] + .into_iter() + .map(|(text, occurrence)| selection(SOURCE, text, occurrence)) + .collect::>(); + + let first = produce_primitive_literals_from_sources( + &sources, + &selections, + PrimitiveProducerLimits::default(), + ); + let repeated = produce_primitive_literals_from_sources( + &sources, + &selections, + PrimitiveProducerLimits::default(), + ); + assert_eq!( + serde_json::to_vec(&first).expect("serialize first output"), + serde_json::to_vec(&repeated).expect("serialize repeated output") + ); + assert!(first.diagnostics.is_empty()); + assert!( + first + .candidates + .iter() + .all(|candidate| { candidate.roots.len() == 5 && candidate.oxc_node_id.is_some() }) + ); + + let semantics = first + .candidates + .iter() + .flat_map(|candidate| &candidate.types) + .filter_map(|record| record.semantic.as_ref()) + .collect::>(); + for primitive in [ + PrimitiveKind::Boolean, + PrimitiveKind::String, + PrimitiveKind::Number, + PrimitiveKind::Bigint, + ] { + assert!(semantics.iter().any(|semantic| { + matches!(semantic, CandidateSemantic::Primitive { primitive: actual } if *actual == primitive) + })); + } + for literal in [ + LiteralKind::Boolean, + LiteralKind::String, + LiteralKind::Number, + LiteralKind::Bigint, + ] { + assert!(semantics.iter().any(|semantic| { + matches!(semantic, CandidateSemantic::Literal { literal: actual, .. } if *actual == literal) + })); + } + assert!(semantics.iter().any(|semantic| { + matches!( + semantic, + CandidateSemantic::NullLike { + null_like: NullLikeKind::Null + } + ) + })); + assert!(semantics.iter().any(|semantic| { + matches!( + semantic, + CandidateSemantic::NullLike { + null_like: NullLikeKind::Undefined + } + ) + })); + assert!(semantics.iter().any(|semantic| { + matches!(semantic, CandidateSemantic::Union { members } if members.len() == 6) + })); + assert_eq!(first.candidates.last().unwrap().summary.unsupported, 1); + } + + #[test] + fn response_local_type_budget_emits_one_explicit_truncation_sentinel() { + let sources = BTreeMap::from([("src/primitives.ts".to_owned(), SOURCE.to_owned())]); + let selections = vec![selection(SOURCE, "union", 1)]; + let output = produce_primitive_literals_from_sources( + &sources, + &selections, + PrimitiveProducerLimits { max_type_nodes: 2 }, + ); + assert!(output.truncated); + assert_eq!( + output + .candidates + .iter() + .flat_map(|candidate| &candidate.types) + .filter(|record| record + .reasons + .contains(&CandidateReason::TypeBudgetExceeded)) + .map(|record| record.id.as_str()) + .collect::>() + .len(), + 1 + ); + } + + #[test] + fn recoverable_oxc_diagnostics_mark_independent_facts_recovered() { + const RECOVERED_SOURCE: &str = "declare const value: null;\nvalue;\n\ + const duplicate = 1;\nconst duplicate = 2;\n"; + let sources = + BTreeMap::from([("src/recovered.ts".to_owned(), RECOVERED_SOURCE.to_owned())]); + let selections = vec![PrimitiveLiteralSelection { + file: "src/recovered.ts".to_owned(), + span: { + let selection = selection(RECOVERED_SOURCE, "value", 1); + selection.span + }, + }]; + let output = produce_primitive_literals_from_sources( + &sources, + &selections, + PrimitiveProducerLimits::default(), + ); + assert!( + output + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "oxc-semantic-recovery") + ); + assert!(output.candidates[0].fact.recovered); + assert!(!output.candidates[0].fact.complete); + assert!(!output.candidates[0].fact.truncated); + } + + fn selection(source: &str, text: &str, occurrence: usize) -> PrimitiveLiteralSelection { + let start = source + .match_indices(text) + .nth(occurrence) + .map(|(start, _)| start) + .unwrap(); + PrimitiveLiteralSelection { + file: "src/primitives.ts".to_owned(), + span: Span { + start: u32::try_from(start).unwrap(), + end: u32::try_from(start + text.len()).unwrap(), + }, + } + } +} diff --git a/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/README.md b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/README.md new file mode 100644 index 000000000..1561706a7 --- /dev/null +++ b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/README.md @@ -0,0 +1,6 @@ +# Independent primitive/literal producer + +Exercises the Rust/OXC primitive/literal producer against the Go oracle with +primitive annotations, literal expressions, a literal union, contextual +literals, null-like values, recoverable duplicate-declaration diagnostics, and +an out-of-category object type. diff --git a/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/case.json b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/case.json new file mode 100644 index 000000000..1b53b6e1b --- /dev/null +++ b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/case.json @@ -0,0 +1,94 @@ +{ + "name": "primitive-literal-independent", + "description": "Independent Rust/OXC primitive and literal construction", + "coverage": [ + "contextual-literals", + "literal-unions", + "null-like", + "primitive-literals-independent", + "primitive-literals-recovery", + "primitive-types", + "unsupported" + ], + "project": "tsconfig.json", + "capabilities": [ + "occurrence.type-views", + "types.core-composite" + ], + "budgets": { + "maxTypeNodes": 256, + "maxTypeDepth": 16 + }, + "selections": [ + { + "file": "src/primitives.ts", + "text": "literalUnion", + "occurrence": 1, + "proves": "literal union and response-local member identity" + }, + { + "file": "src/primitives.ts", + "text": "booleanPrimitive", + "occurrence": 1, + "proves": "boolean primitive" + }, + { + "file": "src/primitives.ts", + "text": "stringPrimitive", + "occurrence": 1, + "proves": "string primitive" + }, + { + "file": "src/primitives.ts", + "text": "numberPrimitive", + "occurrence": 1, + "proves": "number primitive" + }, + { + "file": "src/primitives.ts", + "text": "bigintPrimitive", + "occurrence": 1, + "proves": "bigint primitive" + }, + { + "file": "src/primitives.ts", + "text": "\"context\"", + "occurrence": 0, + "proves": "contextual string literal" + }, + { + "file": "src/primitives.ts", + "text": "7", + "occurrence": 0, + "proves": "contextual number literal" + }, + { + "file": "src/primitives.ts", + "text": "false", + "occurrence": 0, + "proves": "contextual boolean literal" + }, + { + "file": "src/primitives.ts", + "text": "7n", + "occurrence": 0, + "proves": "contextual bigint literal" + }, + { + "file": "src/primitives.ts", + "text": "null", + "occurrence": 1, + "proves": "contextual null-like value" + }, + { + "file": "src/primitives.ts", + "text": "unsupported", + "occurrence": 1, + "proves": "out-of-category type remains explicit" + } + ], + "expectations": { + "recovery": true, + "truncation": true + } +} diff --git a/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/src/primitives.ts b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/src/primitives.ts new file mode 100644 index 000000000..e48f194ad --- /dev/null +++ b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/src/primitives.ts @@ -0,0 +1,25 @@ +type LiteralUnion = "ready" | 42 | true | 42n | null | undefined; +type Unsupported = { value: string }; + +declare const literalUnion: LiteralUnion; +declare const booleanPrimitive: boolean; +declare const stringPrimitive: string; +declare const numberPrimitive: number; +declare const bigintPrimitive: bigint; +declare const unsupported: Unsupported; + +const contextualString: string = "context"; +const contextualNumber: number = 7; +const contextualBoolean: boolean = false; +const contextualBigint: bigint = 7n; +const contextualNull: null = null; + +literalUnion; +booleanPrimitive; +stringPrimitive; +numberPrimitive; +bigintPrimitive; +unsupported; + +const duplicate = 1; +const duplicate = 2; diff --git a/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/tsconfig.json b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/tsconfig.json new file mode 100644 index 000000000..201f6ded7 --- /dev/null +++ b/internal/semanticfacts/testdata/corpus/v0/primitive-literal-independent/tsconfig.json @@ -0,0 +1,8 @@ +{ + "compilerOptions": { + "strict": true, + "noEmit": true, + "target": "esnext" + }, + "files": ["src/primitives.ts"] +} diff --git a/rfcs/0001-semantic-facts-kernel.md b/rfcs/0001-semantic-facts-kernel.md index 4e00f1bb1..d12c5b309 100644 --- a/rfcs/0001-semantic-facts-kernel.md +++ b/rfcs/0001-semantic-facts-kernel.md @@ -264,21 +264,20 @@ general performance claim. This evidence approves mechanical porting of occurrence identity, correlation, response-global fact indexing, and side-table attachment behind the Go oracle. -It approves no semantic checker category for replacement. ADR-0017 now records -the first Rust-owned primitive/literal candidate over the Go-produced graph, -but decoding and projecting Go-produced roots is not semantic equivalence or an -independent producer. ADR-0016 defines the compatibility gates and lists the -semantic areas that remain Go-authoritative. - -Issue #41 implements the next shadow gate under ADR-0018. The real Go producer -and the Rust primitive/literal candidate run over the same six-case corpus; -the deterministic JSON comparison covers fact identity, all five type-view -roots, response-local graph identity, structured payloads, diagnostics, -unsupported/error states, and truncation. CI requires 1,000,000 ppm agreement -for complete in-category records and zero unexplained semantic or transport -differences. Mapping gaps and expected unsupported/budget cases remain -separately classified. This gate does not change the schema-v1 producer or -approve a semantic replacement. +It approves no semantic checker category for replacement. ADR-0019 now records +the first independent Rust/OXC primitive/literal producer and its exact shadow +comparison with Go. The producer is not wired into production, so this evidence +does not transfer semantic authority; ADR-0016 still lists the semantic areas +that remain Go-authoritative. + +Issue #45 replaces that projection with independent Rust/OXC computation under +ADR-0019. The real Go producer and Rust run over the same tagged corpus, while +only the comparator sees both outputs. Deterministic JSON comparison covers +fact identity, exact OXC mapping, all five type-view roots, response-local graph +identity, structured payloads, unsupported states, and truncation. CI requires +1,000,000 ppm agreement for at least 15 complete supported records and zero +semantic, transport, or mapping differences. This gate does not change the +schema-v1 producer, external consumers, or production authority. ## Considered alternatives @@ -348,11 +347,10 @@ spike demonstrate: Acceptance establishes the versioned boundary, not compiler equivalence, production readiness, or a Rust performance advantage. The TypeScript 7 Go -checker remains the semantic oracle. The first primitive/literal Rust candidate -is now a structured projection of the Go graph under ADR-0017 and passes the -ADR-0018 shadow threshold. An independent producer and the complete ADR-0018 -replacement checklist are still required before any semantic category can be -approved for replacement. +checker remains the semantic oracle. The primitive/literal candidate is now +computed independently from source and OXC semantics under ADR-0019 and passes +its exact shadow threshold. It is not wired into production, and no semantic +category is approved for replacement. ## Open questions @@ -361,9 +359,9 @@ approved for replacement. - Is source range plus syntax category sufficient for all relevant OXC nodes? - Which portions of the type graph require exact structure in version 1, and which may initially be represented as opaque or truncated? -- Which category-specific evidence will first satisfy ADR-0018's exact - compatibility threshold and replacement checklist using independent Rust - semantic output? +- What production-integration evidence and rollout boundary would be required + before independently computed primitive/literal output could leave shadow + mode while preserving Go as a fallback? - Should the eventual public name be `tsfacts`, `TypeScript Semantic Kernel`, or another name that avoids implying an alternate TypeScript language? @@ -376,3 +374,4 @@ approved for replacement. - [ADR-0016: Port occurrence attachment before semantic categories](../docs/adr/0016-port-occurrence-attachment-before-semantic-categories.md) - [ADR-0017: Project primitive/literal candidates from Go graph identity](../docs/adr/0017-project-primitive-literal-candidates-from-go-graph-identity.md) - [ADR-0018: Gate Rust semantics against the Go oracle](../docs/adr/0018-gate-rust-semantics-against-the-go-oracle.md) +- [ADR-0019: Compute primitive/literal candidates independently in Rust](../docs/adr/0019-compute-primitive-literals-independently-in-rust.md)