Repository navigation
feat(cube-model): Cube exporter for FR-044 reporting (Plan 4) - #416
Merged
Merged
Conversation
The plan for the cube-model reference helper: the mapping contract per spec section 5 with a golden fixture per row, the rollup rules, name and escaping rules, the canonical golden, and a live lane against a pinned Cube instance. The Cube shapes were executed against Cube v1.7.43 and Postgres 16 before writing; @spine and @default rows are provisional on the zero-rows-and-defaults plan.
Move q, ref, literal, FILTER_OP_SQL and cond (report-ddl-emit) and resolveReportFilter, segmentClause, declared, andOf and temporalOf (extract-report-spec) into projection/report-sql.ts so a second consumer can reuse the one filter-to-SQL translator. No behaviour change: the existing goldens pass unedited. extract-report-spec still exports temporalOf.
…R-044) buildCubeModel turns a loaded model's dimension.*, measure.* and segment.filter members into plain Cube data: cubes, joins, dimensions, measures and segments (plan 4 Tables A to E and G). Rollups, report scope segments and Cube views are typed but left empty for later tasks. - cube/cube-model-spec.ts: the CubeModel data types. - cube/build-cube-model.ts: which cubes exist (vocabulary, join-target and alias cubes), joins, @via members, and the ordering Table H fixes. - cube/cube-members.ts: Table C dimension types and Table D measures, split out so the builder stays near its size budget. - cube/cube-names.ts: Table G name rules and member/cube collisions. - cube/cube-sql.ts: the Cube renderer (brace escaping, Jinja raw wrap). - cube/cube-errors.ts: CubeModelError and the seven ERR_CUBE_* codes. @Of, @via, segments and filters resolve through the same functions the report view lowering uses, and a join's ON columns come from the view's own hop walk. report-sql.ts gains an optional SqlRenderer on ref() and cond(); the default writes exactly what it wrote before, and every projection golden passes unedited. The Task 1 review notes are fixed there too: the local ref no longer shadows ref(), cond's errors name report-sql, and the header describes the module rather than the move. extract-report-spec.ts exports viaHopError for reuse. The canonical fitness model builds to Table H's Program, Week and Asset cubes (without rollups and the scope segment).
…(FR-044) Task 2 review, fix round 1. - A composite identity.reference joins on every column pair, ANDed in position order. A reference whose @fields count differs from the key it references is ERR_CUBE_UNMAPPABLE_JOIN (new code), naming the cube, the reference and both field lists. walkViaPath and view SQL are unchanged; extract-view-spec only exports joinColumnFor and a new hopReferenceIdentity for the full @fields list. - A hop the view walk refuses while joins are emitted (an ambiguous to-one relationship) is ERR_CUBE_UNMAPPABLE_JOIN keeping the reason. - One shared resolver per concern in projection/report-resolve.ts: dimensionOfField, resolveDimensionViaPath (with viaHopError, moved from extract-report-spec.ts), resolveAggregate and ratioOperand. The view lowering and the Cube build both call them; view messages and SQL are byte-identical and the projection goldens pass unedited. - Identifiers take the literal rule: braces escaped, raw-wrapped when holding {% or {#, endraw refused. - A self-referencing to-one hop joins an alias cube <Cube>_<hop>. A @via that returns to an entity on its path any other way, or crosses one cube twice, is refused naming the path. - A @via reuses any declared dimension (attribute or time) without @via over the field. field.uri and field.inet dimensions are strings. - CubeRollupSpec is a union of its three time forms; a type test pins that two forms at once do not compile. - An alias cube colliding with an entity cube is ERR_CUBE_NAME_COLLISION (tested). MemberNamespace.has removed. The spec header says SQL arrives escaped and free text raw.
…FR-044) crosses() picked the join a @via step crosses by comparing the step's first foreign-key column. Two composite references onto one entity that share a leading column (fkProgram [tenantId, programId] and fkFormerProgram [tenantId, formerProgramId]) both have tenantId first, so a dimension with @via Week.fkFormerProgram was written reading {Week_fkProgram.title}: the wrong alias cube, with nothing failing. A belongs-to step now matches the hop whose identity.reference is the one the step crosses (hopReferenceIdentity, the view walk's own resolution). The one_to_one branch keeps matching by relationship name. Test: two composite references sharing a leading column, each @via reading its own alias cube.
Table F of plan 4. A served object.report (servedReport: concrete, read source @kind view) writes into its @from cube: - a rollup named after the report: its attribute dimensions and measures in listed order (a qualified item such as Week.weeks is the measure, a @via dimension is a member of the @from cube like any other); one time dimension in the documented time_dimension + granularity form, two or more as the time_dimensions list; its @segment, then its scope segment, in segments; - for its @filter, a public segment <report>Scope (lower-camel report name) whose SQL is the filter, rendered by the view lowering's own cond with the Cube renderer, listed after the declared segments. No rollup, only the scope segment, when the report's @filter, its @segment or any listed measure's condition (a ratio's operands included) holds a relative date: a rollup is built at refresh time and would answer with the "now" of its build. The relative date is found on the lowered clauses (RelativeNow operands), never on the authored JSON. A report that is not served (sourceless, abstract, materializedView) contributes nothing. matches selects entities, not reports: a report's rollup and scope segment are members of its @from cube and are written whenever that cube is emitted as the entity's own, so selecting the entity alone keeps them. A selected @from that cannot be a cube (abstract, or no table) is refused, as the view lowering refuses it. Rollup names join the cube's member namespace: Cube 1.7.43's compiler reports a name shared by measures, dimensions, segments and pre-aggregations as "defined more than once", so a rollup named like a member, or two same-named reports on one cube, is ERR_CUBE_MEMBER_COLLISION naming both, and a report named with a Python keyword is ERR_CUBE_INVALID_NAME. The collision message now says so. The canonical model test now pins all of Table H, rollups and the recentProgramsScope segment included.
Alias cubes no longer extend their target. Cube's extends copies every member and every pre-aggregation of the parent, so each alias exposed the target's measures again and rebuilt its rollups under scheduled refresh. An alias cube is now a standalone cube like a join-target cube: the target's sql_table (or a TPH subtype's sql), public: false, the target's primary-key dimensions, the members the dimensions reaching it read, and only the joins of multi-hop @via paths that continue through it, rendered from the alias. The ambiguity check walks those joins only, so a second alias of the same entity no longer counts as a route it does not have. CubeSpec drops the unused extends field. A cube is now written when a selected entity's cube reaches it through joins: an entity reached only through alias cubes gets no plain join-target cube, since nothing joins it. A served report whose @from has no cube (abstract, or no writable table) is ERR_CUBE_UNMAPPABLE_REPORT, naming the report, the entity and the reason. A TPH-subtype @from gets its rollup on the subtype's cube, whose sql applies the discriminator. Tests: alias cubes carry none of the target's measures, segments or rollups; a multi-hop @via through an alias; ambiguity through an alias; two self-references; the alias tests in their new shape; the abstract @from refusal; the TPH-subtype rollup; dimensions in report order.
renderCubeYaml writes one CubeSpec as Table H's bytes: block lists, flow lists for rollup members and meta.grains, single-quoted sql with the quote doubled, free text as a JSON string wrapped in a raw block when it holds a Jinja opener. A name a YAML reader takes for a boolean or null is quoted. Free text holding endraw and a sql value holding a line break are refused with ERR_CUBE_UNESCAPABLE_LITERAL.
…(FR-044) A sql value holding a control character, line break, line or paragraph separator or byte-order mark is legal model data (a string filter value), so sqlScalar now writes it as a JSON double-quoted scalar instead of refusing it; plain values stay single-quoted, so Table H is unchanged. CubeSpec carries exactly one of sqlTable or sql in its type. Tests parse the rendered YAML with Bun's built-in parser and compare it with the spec.
…-044)
cubeModel({ dialect?, filter?, target? }) wires buildCubeModel and
renderCubeYaml into meta gen: one model/cubes/<Cube>.yml per cube under
the target's outDir. Model scope; dialect defaults to the config's;
column naming from the config.
Selection (Ruling 17, amended with the controller): the build covers the
whole loaded model narrowed by the generator's own filter, which is fixed
config, so a cube's bytes never depend on the run. The run's selection
(meta gen <Entity>, scope) only picks which files are written: each
selected entity's cube and every cube it reaches through joins. A reached
entity cube is written too, because a @via adds the member it reads to the
cube it reaches. The filter is the build's universe, so it settles
ERR_CUBE_NAME_COLLISION and steps around a refusal on an excluded entity,
as the error messages advise.
ERR_CUBE_UNSUPPORTED_DIALECT refuses sqlite only when the run would write
a cube; a model or a selection with no reporting vocabulary writes nothing
and raises nothing under any dialect.
fixtures/cube-model/: 38 cases, one per Table B row (measure-default and
report-spine wait for #411), Table K's extras, the controller's extras
(dimension-time-two-grains, join-composite, join-self-reference,
dimension-uri-inet, measure-mysql, free-text-jinja) and one case per
CubeModelError code. Every expected file was written by hand from its
contract row. canonical/expected/ is Table H byte for byte, written by
bun run gen:cube-canonical and drift-checked.
Exports cubeModel, buildCubeModel, hasReportingVocabulary,
renderCubeYaml, CubeModelError, CUBE_ERROR_CODES, the ERR_CUBE_* codes
and the spec types from the package root (Ruling 3). No registry entry,
reference template or eject wiring (Task 6).
…imensions (FR-044) Two mapping-corpus cases, each expected tree written by hand from the plan's Tables C and D before the generator ran: - measure-count-distinct-tuple-condition (postgres): a tuple distinct count with a segment, a filter, both, and an or-group segment keeps ONE filters entry, the not-null terms first and the condition ANDed after them; an or group and a segment-plus-filter arrive parenthesised. - dimension-time-mysql: a timestamp is the backtick-quoted column, and a date, as a time dimension or an attribute, is CAST(... AS DATETIME), since MySQL's CAST has no TIMESTAMP target.
cube-model becomes a first-class catalog citizen. - Registry: layer capability, native, ejectable, no runtime packages or peers, config keys dialect and columnNamingStrategy. meta gen --list shows the row and its use-when/emits facets, read from the template header. - Reference copy src/reference/cube-model.ts (meta eject cube-model). The adopter owns which entities get a cube, the file layout and the YAML call; buildCubeModel and renderCubeYaml stay in the package. - Cross-port manifest: cube-model for typescript only. The other four ports' registry tests select their own slice, so a TS-only entry cannot break them. - Orphan cleanup, built-in and copy together: the generator opts in to the runner's orphanPolicy for model/cubes/*.yml. A stale cube file breaks Cube's whole-model compile. The runner removes only an untouched file that a prior run wrote and this run did not re-emit, refuses a hand-edited one, and skips reconciliation entirely when the run names entities (meta gen <Entity>). - Tests: the copy is byte-identical to the built-in over the canonical model and every corpus case, errors included, and is the same generator (name, target, filter, orphan namespace); orphan cleanup through the real runner for both; the inert test pins the three cubes the with/ model writes and nothing for without/; catalog gates treat cube-model like trace-helper (a model trigger the probe fixture lacks).
A pinned Cube (cubejs/cube:v1.7.43, development mode) over a throwaway postgres:16-alpine on a private docker network loads the cube-model generator's output for the canonical persistence model, and: - the generated tree must equal fixtures/cube-model/canonical/expected/; - /v1/meta must compile it and list every cube and member the files declare, with Cube's type, aggregate and visibility; - each served report's Table F query (UTC) must be answered from the report's rollup when Table F emits one, and its rows must equal SELECT * FROM <view> under Table I's normalization; - every Postgres case of the mapping corpus is swapped into the same Cube (through an empty model, so a shared cube name cannot mask a stale compile) and must compile with the members its files declare. Postgres is reached only through docker exec; Cube binds one ephemeral 127.0.0.1 port. Both containers and the network are force-removed from afterAll, from every failure path in setup, and on exit or SIGINT/SIGTERM. Without docker every test is skipped behind a banner. The file is *.live.ts, run only by its path (bun run test:cube). fixtures/cube-model/canonical/seed.sql is the planning spike's seed. Two checks fail on this commit and need rulings, not edits to the corpus: FitnessTotals is answered from the ProgramMinutes rollup (Cube takes the first matching rollup in declaration order), and Cube rejects the free-text-jinja case (it parses title/description as reference templates, so braces need the SQL escape and a backslash is consumed).
scripts/ci-local.sh gains a `cube` section: accepted by --only and listed in --help, run in the full no-flag run after the 5-port integration suite, kept by --integration-only, dropped by --quick (a SKIP entry) and --no-integration, and never part of `ts`/`ts-slow`. `--only cube` installs and builds the workspace first when no ts-fast or ts-slow step in the selection did. Docker down records a SKIP behind a banner, a FAIL under --strict-toolchains. scripts/integration-test.sh gains a `cube` target (`bun run test:cube`), left out of `all`. integration-tests.yml gains a `cube` matrix entry with the Bun setup and cache steps; it brings its own containers and ignores the job's Postgres sidecar. No local-ci.yml job is added, so the lane map in ci-ports-to-run.sh is unchanged (its test reads only local-ci.yml's jobs).
The live lane found two defects; both are fixed in the generator and the
lane keeps its assertions.
Free text (Ruling 28). Cube 1.7.43 reads `title` and `description` as
templates: `{x}` is a member reference, `${x}` an interpolation, a
backslash an escape. So `a {b} c` failed the whole model and `C:\path`
came back `C:path`. textScalar now doubles every backslash, escapes `{`
and `}`, then raw-wraps when the ORIGINAL text holds `{{`, `{%` or `{#`
(the `endraw` refusal stays), then writes the JSON double-quoted scalar.
free-text-jinja gains a measure description with lone braces, `${` and
a backslash. Its golden is updated by hand. The lane now checks that
every declared title and description comes back verbatim from /v1/meta
(cube title/description, member shortTitle/description).
Rollup order (Ruling 29). Cube serves a query from the first matching
rollup in definition order, and a finer rollup serves a coarser additive
query: FitnessTotals was answered from ProgramMinutes. Each cube's
rollups are now written by dimension count (attribute + time)
ascending, ties in report order. In the canonical golden, Week lists
FitnessTotals before ProgramMinutes and Program lists ProgramsByWeek
before ProgramsByMonth. report-rollup lists ProgramTotals first.
Nothing else moved.
scripts/ci-local.sh --only cube is green: 10 of 10, each report served
from its own rollup (dev_pre_aggregations.<cube>__<rollup>), 28 of 28
corpus cases compiled.
Review minors, ruled in by the controller. - The generated canonical model must equal the golden in beforeAll, before Cube starts. On drift, setup fails, naming each drifted file and the regeneration command. - startCubeStack takes `initSql`: the schema and seed are applied after Postgres is ready and before Cube starts. No rollup can be built over empty tables. - Rollup tie order: dimension count ascending, then the coarser time grain first (listed order), then fewer measures, then report order. The comment now claims only Cube's first-match rule: an exact-match rollup comes before a strictly finer one, and rows are correct either way. Day-vs-month and superset-of-measures ties are unit-tested. No golden moved. - The corpus pass gives each case a deadline drawn from one budget, and its test timeout scales with the case count. A slow Cube names the case it was on and lists the cases already rejected. - load() retries transport errors (refused, reset, aborted, a non-JSON body) with bounded backoff, inside the test timeout. - Rows are compared first. A rollup-key mismatch reports the query, Cube's SQL and whether the rows matched. - stop() warns on stderr, naming the container or network, when a removal fails for any reason but absence. It still never throws. - free-text-jinja carries a backtick in a raw-wrapped and a plain description. Cube 1.7.43 hands both back unchanged, so no encoding change. - Nits: U+2028 written as an escape in a test; the Ruling 29 tests sit under their own banner; a non-numeric MO_CUBE_READY_TIMEOUT_S is refused by name; with docker down, `--only cube` records its SKIP without installing or building.
Document the cube-model reference generator as built, not as planned. - New docs/features/cube-export.md: what it writes, wiring (target, eject, options, dialects), the mapping as prose with a worked example, the refusals, which files a run writes and the orphan cleanup, the Cube query that reproduces a report, where a Cube query and the view differ, how the corpus and the `cube` lane check it, rollups needing Cube Store in production, and known limits (@spine and @default are not mapped yet). - reporting.md: an "Exporting to Cube" section; the "does not exist yet" paragraph no longer calls the list hook missing and names the MetricFlow exporter as on demand. - own-your-codegen.md, ports/typescript.md, docs/README.md: cube-model where reference generators are listed. cli.md lists no generators, so unchanged. - Skills: one paragraph each in metaobjects-codegen (typescript) and metaobjects-authoring (reporting). Agent-context goldens regenerated; the regen also picked up SKILL.md text that already differed on main. - CONFORMANCE.md and AGENTS.md: fixtures/cube-model/ is the 29th corpus (TypeScript only), with its table row and section; the stated total moves from 28 to 29 and counts.test.ts is green again. - CHANGELOG [Unreleased]: the cube-model generator and the `cube` lane; no vocabulary change, metamodelVersion stays 1.1. - spec/roadmap.md FR-044 row brought current. - Code comments from the build ledger: the reference header now carries the built-in's orphan-cleanup caveats (a join no longer reached; a changed scope reconciles on a full run); the corpus README row for error-unmappable-report says it needs an unrelated cube; the alias test comment no longer says Team keeps its own join. - The reference template's dialect message was four concatenated template literals, which failed check-reference-templates-lint. It is one literal now, with the same text (reference-byte-identical stays green). metaobjects/meta.requirements.yaml: the reporting branch (and objectReport) describes declaring the vocabulary and leaves lowering and serving out. No entry is about export, so the ledger is unchanged.
Plan 4 gets "Answers to the open questions" (all six accepted as recommended) and an "As built" section, in Plan 3's style. The contract tables are left as first written; each As built item names the table it changes. Tasks 9 and 10 (@default, @spine) are recorded as not built: the #411 build is not on main. Spec section 5: the relative-filter row is the view's SQL in a segment or measure filter (no rollup), the object.report row is a rollup on the @from cube (coarsest first), and the @spine row is a Cube view with no rollup. Three cells the build contradicted are corrected too: the tuple distinct count is ROW(...) with a not-null filter, @Grains is carried as meta.grains, and a to-one join is one_to_one from the side that does not hold the key. One sentence under the table points at Plan 4.
… built (FR-044) The ledger is not in the repo, so a ruling number told a reader nothing.
- A report with a relative date in its @filter, its @segment's filter or a listed measure's condition gets no rollup; its <report>Scope segment is written only when it has a @filter. Said exactly that in the authoring and codegen skills (goldens regenerated), cube-export.md, reporting.md, the CHANGELOG and the plan's answer 2. - Spec section 5: the note now also excludes measure.derived; the ratio cell gives the Postgres CAST and the MySQL form; the tuple cell names MySQL's JSON_ARRAY. - cube-export.md: an alias cube clash is ERR_CUBE_NAME_COLLISION, not a member collision; --forbid-hand-edits makes a hand edit fail verify. - ports/typescript.md: the config example has outDir; both it and reporting.md say a cube for each concrete, table-backed entity plus the join-target and alias cubes its @via dimensions need. - reporting.md: the filters entry is measure.aggregate only; measure.ratio is a number measure over its operands. - fixtures/cube-model/README.md: no ledger ruling numbers; the error-unmappable-report row states the rule. - Plan answer 5: <m>Raw belongs to the unbuilt @default mapping. - Root README capability matrix, TypeScript reporting cell: the list hook and the cube-model generator (it said "no client hook yet").
…apped (FR-044) The zero-rows/measure-defaults vocabulary (@spine on object.report, an integer @default on measure.aggregate and measure.ratio) is planned and not built. When it lands, cube-model would otherwise ignore both and write wrong Cube output silently: a rollup lacking the spine's zero rows, a measure lacking the COALESCE the view applies. The build now refuses a served report that declares @spine, and a measure.aggregate or measure.ratio written on a cube that declares @default, with the new ERR_CUBE_UNMAPPED_VOCABULARY naming the node and how to proceed. A sourceless report, a report or measure on an entity the generator's filter leaves out, a measure on an entity with no table and a join-target cube's measures stay inert. The attribute names are local constants in cube-pending.ts, read with the resolving hasAttr(); the module goes when the real mapping replaces it. Adds the error-unmapped-vocabulary corpus case (41 cases, 30 trees + 11 errors) and updates the counts and the known-limits entry.
…nal review fixes (FR-044) Cube compiles every `sql` and `sql_table` as a JS template literal after Jinja, so a backslash in a SQL literal or identifier was an escape: 'a\b' reached the database as a backspace, 'A\_%' as 'A_%', a trailing backslash swallowed the closing quote and \u broke the compile. escapeToken now doubles every backslash before it escapes the braces, the order free text already uses. The escaping case gains a trailing backslash and a backslash before a brace, and the cube lane now reads each of its literals (and the braced column) back from Cube's /v1/sql and requires the view's own SQL. A negative control with the old single-backslash golden fails that check on Cube 1.7.43 (Cube's SQL holds a backspace). A declared dimension without @via named after a key field and reading that field is now that key dimension: one dimension, primary_key and public: true, with the declared title, description and grains. Over another field, or with a @via, it stays ERR_CUBE_MEMBER_COLLISION. New corpus case dimension-over-key (hand-written golden). Also from the final review: - The lane's compile pass loads the two MySQL cases too (compiling runs no SQL): 31 of the 42 cases. - Counts corrected everywhere: 42 cases, 31 trees, 11 errors. - Ledger ruling numbers replaced with the rule they stood for. - The filter option and ERR_CUBE_NAME_COLLISION remedy say an excluded entity a @via reaches is still written, as a join-target cube. - CubeViewSpec and its parts are no longer exported from the package root. - A test for a ratio whose operand declares @default; the tautological constant test is gone. - Doc statements on relative dates and the served-report row; test titles and comments that misdescribed the rollup order, the eject install set and the catalog's silent list.
The new fixtures/cube-model corpus raises the corpus count from 28 to 29; the published site payload carries that count.
…les (FR-044) #415 registered @spine and a measure's @default with load rules R8, R9 and M7, so four models in cube-unmapped-vocabulary.test.ts no longer loaded and the build under test was never reached: - the @spine report named a hop Program does not declare (R8) and listed no dimension (R9). It now walks Program.fkOwner, an identity.reference onto a new Owner entity, and lists a dimension read through it. - the abstract-base measure put @default on a count (M7). It is a sum. Each test still pins the same ERR_CUBE_UNMAPPED_VOCABULARY refusal or the same inert result; the guard and its messages are unchanged. Comments that described the attributes as unregistered are corrected.
…iew fixes (FR-044)
Pre-gate review findings CR1 to CR5:
- MySQL tuple distinct count: the view's own COUNT(DISTINCT a, b) as a
`number` measure (a condition on the first component, as the view writes
it). A count_distinct over JSON_ARRAY(a, b) compares JSON bytes, not the
column collation: executed on mysql:8.4 (utf8mb4_0900_ai_ci) over rows with
case and accent variants, nulls and duplicates, it counted 6 tuples where
the view counts 4. Integer tuples agreed (4 and 4).
- One Jinja-opener rule for SQL and free text (cube-template.ts): text holding
{{, {% or {# is raw-wrapped, so a {{x}} literal is now wrapped too.
- endraw is refused only when the text is raw-wrapped; unwrapped text is never
inside a raw block. The error-unescapable-literal case now holds {%.
- Tests: a relationship-backed @via hop crosses its own reference's alias, and
an inherited one crosses the inherited reference's join.
- ERR_CUBE_AMBIGUOUS_PATH says when more routes exist than it lists.
- The join-target remedy text says an entity reached only through alias cubes
gets no join-target cube; a false test comment about String.raw is fixed.
A measure.aggregate with `@default: n` is two Cube members: `<m>Raw`, the
aggregate as before (type, sql and filters), `public: false`, then `<m>`,
`type: number`, `COALESCE({<m>Raw}, n)`, carrying the measure's title and
description. A measure.ratio with `@default: n` wraps its quotient in
COALESCE (Postgres with the cast, MySQL without). An operand that declares
its own default is referenced by member name, so its COALESCE reaches the
ratio (#411 decision 4); a unit test expands the Cube members of such a ratio
and requires the report view's own SQL for it, character for character.
`<m>Raw` is an added member, so a clash is ERR_CUBE_MEMBER_COLLISION. A
rollup lists `<m>`. #415's Tables A and E agree with the plan's Task 9 table,
so the mapping is the table's.
The @default half of the guard is gone; the @spine half stays until @spine
is mapped, and error-unmapped-vocabulary now pins that refusal. Corpus case
measure-default (hand-written golden). The codegen-noop `with/` model now
exports; its defaulted ratio is pinned in the inert test.
The canonical golden still cannot be regenerated: its @spine reports are
refused until the next commit maps them, so the canonical cube tests and the
two canonical reference-byte-identical tests stay red until then.
A served report with @spine is a Cube view, model/views/<Report>.yml, with no rollup and no scope segment (a rollup on the spine cube is built from the fact cube and loses the zero rows). Its first join_path is the spine cube (the cube every listed dimension reaches after the spine's hops), including the member each dimension reads under the dimension's name, with the dimension's title, description and grains; its last is <Report>Facts with the listed measures. <Report>Facts is a standalone public: false cube (no `extends`, which would copy the fact cube's rollups): `SELECT * FROM <@from table> <alias>` with the report's @segment and @filter, ANDed, as its WHERE, so the scope stays inside the join and a spine row whose facts are all filtered out keeps its row. It holds @from's key and @from's own definitions of the listed measures, ratio operands and <m>Raw members. The spine cube gets one one_to_many join onto it; no reverse join is added to an ordinary cube, so ad-hoc answers are unchanged. A multi-hop spine is built with standalone chain cubes <Report>_<hop>, one per entity between @from and the spine entity. Executed on Cube 1.7.43 before building: a view includes a private key and a public: false member under an alias, include-level docs and meta reach /v1/meta, the roster view returns every program (weeks 0, sums null, defaults 0), the scoped facts cube keeps the programs whose weeks are all short, a two-hop chain equals the view lowering's SQL, and {Week.weeks, Program.id} stays rooted at Week. The guard is deleted: cube-pending.ts, ERR_CUBE_UNMAPPED_VOCABULARY and the error-unmapped-vocabulary case. renderCubeViewYaml and the view types are exported; the generator writes a view when every cube it reads is written and cleans up model/views/*.yml. Corpus cases report-spine and report-spine-multi-hop (hand-written goldens); the canonical golden gains the two facts cubes and views. The cube lane compares ProgramRoster and ProgramLongWeeks through their views (7 rows each), FitnessTotalsFilled through its rollup, and reads a defaulted operand's COALESCE inside a ratio from /v1/sql. Docs, spec section 5, roadmap and skills say both attributes are mapped.
- A TPH subtype as the spine entity is exported through its own discriminator-scoped cube, so the view's rows are that subtype's only; pinned by a unit test and documented beside the TPH @from note (the view lowering refuses both reports). - Two listed dimensions over one field and path include one member twice under two aliases. Added to the report-spine corpus case (golden edited by hand); the cube lane's compile pass shows Cube 1.7.43 accepts it. - One exported JOIN_PATH_SEPARATOR, used by the build, both generator copies and the live lane. - One definition of the report scope and of the MySQL tuple count: reportScope and mysqlTupleCount in projection/report-sql.ts, called by the view lowering (extract-report-spec, report-ddl-emit) and the exporter (cube-members, cube-reports, build-cube-model). View SQL, the canonical schema, the report shapes and the projection goldens are unchanged. - ERR_CUBE_NAME_COLLISION says "cube or view" when one side is a view. - A facts or chain cube carries no title or description of its entity.
Collapse the mirrored join arms, share the YAML file framing and the grains tail, derive the unmappable-dimension message from the dispatch sets, and make four helpers with no outside importers module-private. No output byte changes: the corpus, canonical golden and reference byte-identical tests are unchanged.
…ards the gate The branch's first push (the plan document) was squash-merged as fa0a0d7, which this history already contains; this merge keeps every file as is.
…xporter: refresh date and FR-044 feature description in AGENTS.md (authority file).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Build the Cube exporter for FR-044 reporting (Plan 4 of the reporting work) so that 1.1.0 ships reporting to the bar its own design sets: "I don't want a half baked 1.1 and then have to do a 1.2 right away".
The design (spec R6 and section 7) asks for a
cube-modelreference helper, TypeScript first: it emits Cubemodel/cubes/*.yml, one cube per entity that declares dimensions or measures, with joins from to-one relationships, measures, dimensions (time dimensions with their granularities) and segments. It is a reference helper, not core: listed bymeta gen --list, ejectable withmeta eject, its output drift-checked bymeta verify --codegen, and it carries no runtime. The mapping is lossless for the core vocabulary by construction: the spec's section 5 mapping table is a contract, each row backed by a golden fixture. Live check: a real Cube instance loads the exporter's output against the persistence-conformance Postgres database, and a measure query through Cube is compared to the report view's result for the same data; this runs in an integration lane, not the unit gate.The dbt MetricFlow exporter is not part of this: it is built on first adopter demand (decision D5). Other ports' exporters only on adopter demand; the exported files are language-neutral.
What Changed
Added
cube-modelreference generator (FR-044 Plan 4): translates MetaObjects reporting vocabulary (dimension.*,measure.*,segment.filter,object.report) into Cube data-model YAML files (model/cubes/*.ymlper entity,model/views/*.ymlper@spinereport). Generator is ejectable, listed inmeta gen --list, and drift-checked bymeta verify --codegen.Reporting vocabulary mapped to Cube: section 5 mapping (dimensions → attributes/measures, time dimensions with granularities, measures with aggregations, segments with filters, reports with spines as views) is lossless by construction, backed by 45 conformance fixtures covering joins, rollups, relative-date filters, MySQL/Postgres dialects, error conditions, and escaping.
Live integration test: a real Cube instance loads and queries the exporter's output against the persistence-conformance Postgres database in a new
cubeCI lane, comparing measure aggregations to report-view results for correctness.Documentation, conformance, and tooling: added
docs/features/cube-export.md(spec section 5 mapping + build ledger), conformance fixtures underfixtures/cube-model/, integration test (cube-live), and CI lane integration (integration-tests.yml,scripts/integration-test.sh).Core reporting design updated: refined FR-044 Plan 1 spec to clarify
@spinereports as read-only views served by every port's generated API, integrated with the Cube export path.Risk Assessment
✅ Low: All 7 parallel review angles (reuse, efficiency, removed-behavior regression, simplification, CLAUDE.md conventions, cross-file tracing, root-cause/altitude, and finally a deep line-by-line correctness scan) completed; the correctness scan explicitly traced every risky area (self-joins, aliasing, TPH discriminator encoding, relative-date suppression, rollup ordering, YAML/SQL escaping pipeline) and found zero confirmed bugs. Only minor info-level cleanups surfaced across all angles. Ejectability, no-runtime-import, drift-check wiring, and the live-Cube-check's isolation to its own integration lane all match stated intent exactly, verified directly.
Testing
Baseline TS unit/conformance suite passes (exit 0). Core cube-model unit tests: 362 pass (YAML generation, mapping tables, error cases). Integration test with real Cube 1.7.43 + Postgres: 15 tests pass, confirming 9 served reports' Cube queries match report view results exactly, 34 corpus cases compile, SQL escaping validated. Catalog test confirms cube-model registered as ejectable reference helper. Orphan cleanup tests pass. No regressions detected.
Evidence: Cube model validation summary
Evidence: Example: measure-count cube YAML
Evidence: Example: time dimension with granularities
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
server/typescript/packages/codegen-ts/src/cube/build-cube-model.ts:696- Key-field bookkeeping in CubeDraft uses two parallel maps (declaredByField, addedByField) read together only via one ?? fallback (reachedMember); a single map with declared-wins-by-write-order would convey the same precedence with less state to keep in sync.server/typescript/packages/integration-tests/src/cube-container.ts:60- CubeStack's network/pgContainer/cubeContainer fields in the live-test harness are written but never read by any consumer (cube-model.live.ts only uses stop()); unused public surface on a container test helper.server/typescript/packages/codegen-ts/src/cube/cube-errors.ts:17- CUBE_ERROR_CODES/CubeErrorCode are re-exported from the package barrel with no consumer anywhere in the repo; unlike PYTHON_KEYWORDS there is no test asserting this list stays in sync with the ERR_CUBE_* constants actually thrown.server/typescript/packages/integration-tests/src/cube-container.ts:246- The docker-CLI container harness (runDocker/tailLogs/assertRunning-equivalents) is now implemented three times with drifted signatures across cube-container.ts, postgres-container.ts, and mysql-container.ts, with no shared module.server/typescript/packages/codegen-ts/src/cube/build-cube-model.ts:634- The viaCubes self-referencing-hop refusal (crossing the same alias cube twice in one @via path) is covered by a unit test but has no corresponding fixture-corpus directory under fixtures/cube-model/, unlike every other ERR_CUBE_* case.server/typescript/packages/codegen-ts/src/cube/build-cube-model.ts:321- Spine facts/chain cube drafts bypass the normal addJoins/alias flow and are name-collision-checked only by relying on assertCubeNames running after the spine-population loop in build(); no test exercises a facts/chain cube name colliding with an entity or alias cube, so a future reordering of build()'s passes could let a name clash reach the YAML emitter unchecked.server/typescript/packages/codegen-ts/src/cube/cube-errors.ts:17- CubeModelError ships no isCubeModelError type guard, unlike the sibling migrate-ts error convention (isXError) adopted specifically because instanceof is unsafe across two physical copies of a package in one process; a future caller narrowing this error by instanceof would inherit that failure mode.✅ **Test** - passed
✅ No issues found.
scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchainsbun test packages/codegen-ts/test/cube/ (362 tests)bun test packages/cli/test/catalog-listing.test.ts (24 tests)bun test packages/codegen-ts/test/cube/cube-model-orphans.test.ts (21 tests)bun test packages/codegen-ts/test/reference-byte-identical.test.ts (58 tests)bun run test:cube cube-model.live.ts (15 integration tests with real Cube + Postgres)scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains (baseline: pass)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.