From 4f3d97372255ff66c256d748687dbb95111d18a9 Mon Sep 17 00:00:00 2001 From: Matthew Thomas <25988124+Ma11hewThomas@users.noreply.github.com> Date: Sat, 3 Oct 2026 10:22:28 +0100 Subject: [PATCH 1/5] Prepare CTRF specification 0.1.0 --- CHANGELOG.md | 60 ++++++- examples/comprehensive.json | 2 +- examples/minimal.json | 2 +- examples/with-diagnostics.json | 2 +- examples/with-insights.json | 2 +- examples/with-retries.json | 2 +- spec/ctrf.md | 32 ++-- tests/ctrf.test.json | 22 +-- .../normative/additional-properties.test.json | 28 +-- tests/normative/enum-constraints.test.json | 8 +- tests/normative/required-properties.test.json | 56 +++--- tests/normative/string-constraints.test.json | 28 +-- tests/normative/type-validation.test.json | 162 +++++++++--------- tests/normative/valid-documents.test.json | 2 +- 14 files changed, 238 insertions(+), 170 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7233887..2ab9998 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,11 +7,69 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.1.0] - 2026-10-03 + +### Changed + +- Established the first internally consistent CTRF specification release: the + specification header, inline examples, standalone examples, and conformance + fixtures now identify version `0.1.0`. +- Defined the pre-1.0 versioning policy: PATCH releases preserve compatibility, + while MINOR releases may contain additions or breaking contract changes. +- No report-shape or validation changes were introduced relative to `v0.0.4`; + producers should emit `"specVersion": "0.1.0"` for this release. + +## [0.0.4] - 2026-08-15 + +> Retrospectively assigned specification snapshot, tagged on 2026-10-03. Files +> at this revision retain their original version metadata. + ### Changed -- Clarified `retryAttempts` as the ordered history preceding the final attempt, aligned `retries` counting semantics, and updated the schema and examples accordingly ([#62](https://github.com/ctrf-io/ctrf/pull/62)). +- Clarified `retryAttempts` as the ordered history preceding the final attempt, + aligned `retries` counting semantics, and updated the schema and examples + accordingly ([#62](https://github.com/ctrf-io/ctrf/pull/62)). + +## [0.0.3] - 2026-07-26 + +> Retrospectively assigned specification snapshot, tagged on 2026-10-03. Files +> at this revision retain their original version metadata. + +### Added + - Added an optional identity model for CTRF documents, logical runs, test cases, executions, attempts, attachments, and shards ([#57](https://github.com/ctrf-io/ctrf/pull/57)). + +### Changed + - Clarified namespace guidance for `extra` extension keys and examples ([#56](https://github.com/ctrf-io/ctrf/pull/56)). - Clarified immutability guidance for emitted CTRF report artifacts ([#55](https://github.com/ctrf-io/ctrf/pull/55)). - Clarified `tags` as simple keyless classifications and `labels` as structured key-value test metadata ([#54](https://github.com/ctrf-io/ctrf/pull/54)). - Allowed non-empty arrays of strings, numbers, and booleans as label values ([#54](https://github.com/ctrf-io/ctrf/pull/54)). + +## [0.0.2] - 2026-02-07 + +> Retrospectively assigned specification snapshot, tagged on 2026-10-03. Files +> at this revision retain their original version metadata. + +### Added + +- Added structured scalar `labels` metadata to test objects + ([#51](https://github.com/ctrf-io/ctrf/pull/51)). + +## [0.0.1] - 2026-01-24 + +> Retrospectively assigned specification snapshot, tagged on 2026-10-03. Files +> at this revision retain their original version metadata. + +### Added + +- Established the standalone CTRF specification, normative JSON Schema, + examples, and conformance tests + ([#50](https://github.com/ctrf-io/ctrf/pull/50)). + +[Unreleased]: https://github.com/ctrf-io/ctrf/compare/v0.1.0...HEAD +[0.1.0]: https://github.com/ctrf-io/ctrf/compare/v0.0.4...v0.1.0 +[0.0.4]: https://github.com/ctrf-io/ctrf/compare/v0.0.3...v0.0.4 +[0.0.3]: https://github.com/ctrf-io/ctrf/compare/v0.0.2...v0.0.3 +[0.0.2]: https://github.com/ctrf-io/ctrf/compare/v0.0.1...v0.0.2 +[0.0.1]: https://github.com/ctrf-io/ctrf/tree/v0.0.1 diff --git a/examples/comprehensive.json b/examples/comprehensive.json index d9c41d2..70f0431 100644 --- a/examples/comprehensive.json +++ b/examples/comprehensive.json @@ -1,6 +1,6 @@ { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "reportId": "9d2c6a10-3f7a-4e22-9a8f-1a2b3c4d5e6f", "runId": "run-2025-11-24-pr-123", "timestamp": "2025-11-24T12:00:00Z", diff --git a/examples/minimal.json b/examples/minimal.json index 61b4482..0b95527 100644 --- a/examples/minimal.json +++ b/examples/minimal.json @@ -1,6 +1,6 @@ { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "example-runner" diff --git a/examples/with-diagnostics.json b/examples/with-diagnostics.json index 20f6946..c3662f9 100644 --- a/examples/with-diagnostics.json +++ b/examples/with-diagnostics.json @@ -1,6 +1,6 @@ { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "example-runner", diff --git a/examples/with-insights.json b/examples/with-insights.json index 690c7ae..dbb6146 100644 --- a/examples/with-insights.json +++ b/examples/with-insights.json @@ -1,6 +1,6 @@ { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "reportId": "7c4e1c20-7c89-4f30-9b52-1f6f9d6b8f21", "results": { "tool": { diff --git a/examples/with-retries.json b/examples/with-retries.json index 34a205d..3c06ae7 100644 --- a/examples/with-retries.json +++ b/examples/with-retries.json @@ -1,6 +1,6 @@ { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "example-runner", diff --git a/spec/ctrf.md b/spec/ctrf.md index f5a84e3..8d7abed 100644 --- a/spec/ctrf.md +++ b/spec/ctrf.md @@ -2,10 +2,10 @@ ## Common Test Report Format -**Version:** 0.0.0 +**Version:** 0.1.0 (This version corresponds directly to the CTRF `specVersion` field.) -**Date:** 2025-11-24 +**Date:** 2026-10-03 **Status:** Working Draft @@ -1964,9 +1964,19 @@ It is the ONLY permitted extension point within the baseline object. CTRF follows **Semantic Versioning**. -- MAJOR = breaking changes -- MINOR = backward-compatible additions -- PATCH = non-breaking fixes +This document defines CTRF version `0.1.0`. + +Before CTRF `1.0.0`: + +- MINOR versions MAY add capabilities or introduce breaking contract changes. +- PATCH versions MUST preserve compatibility and are reserved for compatible + corrections and clarifications. +- Breaking changes MUST be identified in the changelog and include migration + guidance where action is required from producers or consumers. + +Consumers MUST treat different pre-1.0 MINOR versions as potentially +incompatible. Consumers SHOULD support PATCH releases within a supported +pre-1.0 MINOR version. Consumers MUST reject incompatible MAJOR versions. @@ -2967,7 +2977,7 @@ It includes: ```json title="Minimal CTRF document" { "reportFormat": "CTRF", - "specVersion": "0.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "example-runner" @@ -3008,7 +3018,7 @@ It includes: ```json title="CTRF document with retries" { "reportFormat": "CTRF", - "specVersion": "0.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "example-runner", @@ -3065,7 +3075,7 @@ It includes: ```json title="CTRF document with diagnostics" { "reportFormat": "CTRF", - "specVersion": "0.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "example-runner", @@ -3129,7 +3139,7 @@ It includes: ```json title="CTRF document with insights and baseline" { "reportFormat": "CTRF", - "specVersion": "0.0.0", + "specVersion": "0.1.0", "reportId": "7c4e1c20-7c89-4f30-9b52-1f6f9d6b8f21", "results": { "tool": { @@ -3232,7 +3242,7 @@ It includes: ```json title="Comprehensive CTRF document" { "reportFormat": "CTRF", - "specVersion": "0.0.0", + "specVersion": "0.1.0", "reportId": "9d2c6a10-3f7a-4e22-9a8f-1a2b3c4d5e6f", "runId": "run-20251124-e2e-staging", "timestamp": "2025-11-24T12:00:00Z", @@ -3426,7 +3436,7 @@ The `ctrf.` and `ctrf/` namespace prefixes are reserved for CTRF-defined extensi ```json title="CTRF document with namespaced extra objects" { "reportFormat": "CTRF", - "specVersion": "0.0.0", + "specVersion": "0.1.0", "extra": { "myorg.ci": { "pipeline": { diff --git a/tests/ctrf.test.json b/tests/ctrf.test.json index ae18393..295a9fb 100644 --- a/tests/ctrf.test.json +++ b/tests/ctrf.test.json @@ -31,7 +31,7 @@ "description": "Missing reportFormat - invalid", "valid": false, "data": { - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -56,7 +56,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0" + "specVersion": "0.1.0" } }, { @@ -64,7 +64,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 } @@ -76,7 +76,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -89,7 +89,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "unknownProperty": "should fail", "results": { "tool": { "name": "runner" }, @@ -103,7 +103,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -116,7 +116,7 @@ "valid": false, "data": { "reportFormat": "NOT-CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -142,7 +142,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -155,7 +155,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -168,7 +168,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": {}, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -181,7 +181,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, diff --git a/tests/normative/additional-properties.test.json b/tests/normative/additional-properties.test.json index f9fb13a..a483850 100644 --- a/tests/normative/additional-properties.test.json +++ b/tests/normative/additional-properties.test.json @@ -7,7 +7,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "unknownProperty": "should fail", "results": { "tool": { "name": "runner" }, @@ -21,7 +21,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -34,7 +34,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner", "unknownField": true }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -47,7 +47,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000, "unknownField": true }, @@ -60,7 +60,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -74,7 +74,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -88,7 +88,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -101,7 +101,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -114,7 +114,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -127,7 +127,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -140,7 +140,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -157,7 +157,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -174,7 +174,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -190,7 +190,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, diff --git a/tests/normative/enum-constraints.test.json b/tests/normative/enum-constraints.test.json index 1251fc5..9162738 100644 --- a/tests/normative/enum-constraints.test.json +++ b/tests/normative/enum-constraints.test.json @@ -7,7 +7,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -20,7 +20,7 @@ "valid": false, "data": { "reportFormat": "NOT-CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -33,7 +33,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -46,7 +46,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, diff --git a/tests/normative/required-properties.test.json b/tests/normative/required-properties.test.json index d360984..add52d3 100644 --- a/tests/normative/required-properties.test.json +++ b/tests/normative/required-properties.test.json @@ -6,7 +6,7 @@ "description": "Document missing required reportFormat property", "valid": false, "data": { - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -31,7 +31,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0" + "specVersion": "0.1.0" } }, { @@ -39,7 +39,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, "tests": [{ "name": "test", "status": "passed", "duration": 100 }] @@ -51,7 +51,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "tests": [{ "name": "test", "status": "passed", "duration": 100 }] @@ -63,7 +63,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 } @@ -75,7 +75,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -88,7 +88,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -101,7 +101,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -114,7 +114,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": {}, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -127,7 +127,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -140,7 +140,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -153,7 +153,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -166,7 +166,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -179,7 +179,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -192,7 +192,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -205,7 +205,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "stop": 1609459201000 }, @@ -218,7 +218,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000 }, @@ -231,7 +231,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -244,7 +244,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -257,7 +257,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -270,7 +270,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -283,7 +283,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -296,7 +296,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -309,7 +309,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -322,7 +322,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -338,7 +338,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -360,7 +360,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -382,7 +382,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "skipped": 0, "pending": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, diff --git a/tests/normative/string-constraints.test.json b/tests/normative/string-constraints.test.json index d37e206..c0c4b46 100644 --- a/tests/normative/string-constraints.test.json +++ b/tests/normative/string-constraints.test.json @@ -20,7 +20,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "reportId": "", "results": { "tool": { "name": "runner" }, @@ -34,7 +34,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "runId": "", "results": { "tool": { "name": "runner" }, @@ -48,7 +48,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -61,7 +61,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -74,7 +74,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -87,7 +87,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -100,7 +100,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -113,7 +113,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -126,7 +126,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -139,7 +139,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -152,7 +152,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -165,7 +165,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -178,7 +178,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -192,7 +192,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, diff --git a/tests/normative/type-validation.test.json b/tests/normative/type-validation.test.json index f1c13c6..d49a530 100644 --- a/tests/normative/type-validation.test.json +++ b/tests/normative/type-validation.test.json @@ -7,7 +7,7 @@ "valid": false, "data": { "reportFormat": 123, - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -33,7 +33,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "reportId": 12345, "results": { "tool": { "name": "runner" }, @@ -47,7 +47,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "runId": 12345, "results": { "tool": { "name": "runner" }, @@ -61,7 +61,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "timestamp": 1609459200000, "results": { "tool": { "name": "runner" }, @@ -75,7 +75,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "generatedBy": 12345, "results": { "tool": { "name": "runner" }, @@ -89,7 +89,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "extra": ["invalid"], "results": { "tool": { "name": "runner" }, @@ -103,7 +103,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": [] } }, @@ -112,7 +112,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": [], "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -125,7 +125,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": 12345 }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -138,7 +138,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner", "version": 1.0 }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -151,7 +151,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": [], @@ -164,7 +164,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": "1", "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -177,7 +177,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": "1", "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -190,7 +190,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": "0", "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -203,7 +203,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": "0", "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -216,7 +216,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": "0", "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -229,7 +229,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": "0", "start": 1609459200000, "stop": 1609459201000 }, @@ -242,7 +242,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "flaky": "0", "start": 1609459200000, "stop": 1609459201000 }, @@ -255,7 +255,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "suites": "5", "start": 1609459200000, "stop": 1609459201000 }, @@ -268,7 +268,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": "1609459200000", "stop": 1609459201000 }, @@ -281,7 +281,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": "1609459201000" }, @@ -294,7 +294,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000, "duration": "1000" }, @@ -307,7 +307,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -320,7 +320,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -333,7 +333,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -346,7 +346,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -359,7 +359,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -372,7 +372,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -385,7 +385,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -398,7 +398,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -411,7 +411,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -424,7 +424,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -437,7 +437,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -450,7 +450,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -463,7 +463,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -476,7 +476,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -489,7 +489,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -502,7 +502,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -515,7 +515,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -528,7 +528,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -541,7 +541,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -554,7 +554,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -567,7 +567,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -580,7 +580,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -593,7 +593,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -606,7 +606,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -619,7 +619,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -632,7 +632,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -645,7 +645,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -658,7 +658,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -671,7 +671,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -684,7 +684,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -697,7 +697,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -710,7 +710,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -723,7 +723,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -736,7 +736,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -749,7 +749,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -762,7 +762,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -775,7 +775,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -788,7 +788,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -801,7 +801,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -814,7 +814,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -827,7 +827,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -840,7 +840,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -853,7 +853,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -866,7 +866,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -879,7 +879,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -892,7 +892,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -905,7 +905,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -918,7 +918,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -931,7 +931,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -945,7 +945,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -959,7 +959,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -973,7 +973,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -987,7 +987,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -1001,7 +1001,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -1015,7 +1015,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -1029,7 +1029,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -1043,7 +1043,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -1057,7 +1057,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, @@ -1071,7 +1071,7 @@ "valid": false, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "results": { "tool": { "name": "runner" }, "summary": { "tests": 1, "passed": 1, "failed": 0, "pending": 0, "skipped": 0, "other": 0, "start": 1609459200000, "stop": 1609459201000 }, diff --git a/tests/normative/valid-documents.test.json b/tests/normative/valid-documents.test.json index 0598df8..cc813d0 100644 --- a/tests/normative/valid-documents.test.json +++ b/tests/normative/valid-documents.test.json @@ -32,7 +32,7 @@ "valid": true, "data": { "reportFormat": "CTRF", - "specVersion": "1.0.0", + "specVersion": "0.1.0", "runId": "run-1", "results": { "tool": { "name": "runner" }, From bf9d82761859d9444bfef59e343858d7fc7fd40d Mon Sep 17 00:00:00 2001 From: Matthew Thomas <25988124+Ma11hewThomas@users.noreply.github.com> Date: Sat, 3 Oct 2026 11:17:54 +0100 Subject: [PATCH 2/5] Document pre-1.0 release process --- RELEASE.md | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 137 insertions(+) create mode 100644 RELEASE.md diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 0000000..8adf785 --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,137 @@ +# CTRF Specification Release Process + +This document describes how versions of the CTRF specification are prepared and +published during the pre-1.0 period. + +## Versioning policy + +CTRF release versions and the `specVersion` field use `MAJOR.MINOR.PATCH` +version numbers. + +Before CTRF `1.0.0`: + +- MINOR versions may add capabilities or introduce breaking contract changes. +- PATCH versions must preserve compatibility and are reserved for compatible + corrections and clarifications. +- Breaking changes must be identified in `CHANGELOG.md` and include migration + guidance when producers or consumers need to take action. + +Consumers should treat different pre-1.0 MINOR versions as potentially +incompatible and should support PATCH releases within a supported pre-1.0 MINOR +version. + +The version in a published tag, the specification header, inline examples, +standalone examples, and valid conformance fixtures must agree. Producers should +emit the `specVersion` of the published specification they fully support. + +## Release contents + +A CTRF specification release contains the repository at the tagged commit, +including: + +- the specification in `spec/ctrf.md`; +- the JSON Schema in `schema/ctrf.schema.json`; +- the examples in `examples/`; +- the conformance fixtures in `tests/`; and +- the release history in `CHANGELOG.md`. + +These artifacts are versioned together. If the schema changes, the schema +embedded in `spec/ctrf.md` must be byte-for-byte identical to +`schema/ctrf.schema.json` after extraction. + +## Publication model + +During the current pre-1.0 period, an annotated Git tag named `vMAJOR.MINOR.PATCH` +is the publication record for a CTRF specification version. The tagged repository +contents are the canonical source for that version. + +GitHub Releases are not part of the current publication process. The existing +tag-triggered GitHub Release workflow must remain disabled while this tags-only +policy is in effect. + +Published tags are immutable. Do not move, replace, or delete a published tag to +correct a release. Prepare and publish a new version instead. + +## Preparing a release + +All release preparation must be made on a dedicated branch and reviewed through +a pull request. Never prepare or push a release commit directly on `main`. + +Before opening the pull request: + +1. Choose the release version using the pre-1.0 versioning policy. +2. Update the version and publication date in `spec/ctrf.md`. +3. Update `specVersion` in inline examples, standalone examples, and valid test + fixtures. +4. Move the relevant `Unreleased` changelog entries into a dated release entry. +5. Describe breaking changes and required migrations in the changelog. +6. Confirm that intentionally invalid version fixtures remain invalid rather + than mechanically replacing every version-like value. +7. Run the release verification checks. + +The release pull request should state whether the report shape or validation +rules changed relative to the preceding tag. + +## Verification + +Run the repository's complete schema and conformance checks before merging the +release pull request: + +```bash +jsonschema fmt schema/ctrf.schema.json --check +jsonschema lint schema/ctrf.schema.json --exclude top_level_examples +jsonschema validate schema/ctrf.schema.json examples/*.json +jsonschema test tests/ctrf.test.json +jsonschema test tests/normative +``` + +Also verify: + +- the embedded schema matches `schema/ctrf.schema.json` byte-for-byte; +- released examples and valid fixtures use the intended `specVersion`; +- the changelog comparison links identify the correct tags; and +- `git diff --check` passes. + +## Publishing a release + +After the release pull request is approved and merged: + +1. Update local `main` with a fast-forward pull. +2. Confirm the merged commit contains the reviewed version and release date. +3. Re-run the release verification checks on the merged commit. +4. Confirm the GitHub Release workflow is still disabled. +5. Create an annotated tag on the verified merged commit. +6. Push only the tag. +7. Verify that the remote tag resolves to the intended commit and that no GitHub + Release was created. + +For example: + +```bash +git switch main +git pull --ff-only +git tag -a v0.1.0 -m "CTRF specification v0.1.0" +git push origin v0.1.0 +``` + +Creating and pushing the tag is an explicit publication action. It must not be +performed merely because the preparation pull request was merged. + +## Current pre-1.0 history + +The `v0.0.1` through `v0.0.4` tags were assigned retrospectively to historical +specification snapshots. Files at those commits retain their original version +metadata, so those tags do not have the internal version consistency required of +new releases. Their tag annotations and `CHANGELOG.md` record this limitation. + +Version `0.1.0` is the first release prepared so that its specification header, +inline examples, standalone examples, and valid conformance fixtures all identify +the same specification version. It introduces no report-shape or validation +change relative to `v0.0.4`; producers adopting it should emit +`"specVersion": "0.1.0"`. + +## Authority + +The tagged repository contents define a published CTRF specification version. If +the written specification and JSON Schema disagree, the written specification +takes precedence, and the inconsistency must be corrected in a new release. From 9506fbea90ee26b71db11d15682cec2d052d1d76 Mon Sep 17 00:00:00 2001 From: Matthew Thomas <25988124+Ma11hewThomas@users.noreply.github.com> Date: Sat, 3 Oct 2026 11:19:32 +0100 Subject: [PATCH 3/5] Align pre-1.0 release guidance --- CONTRIBUTING.md | 6 ++++-- README.md | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6efd70e..c39da08 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -88,8 +88,10 @@ If the schema and specification disagree, the **written specification takes prec Backward compatibility is a core CTRF principle. -- PATCH and MINOR releases MUST NOT introduce breaking changes -- Breaking changes MUST be explicit and well-justified +- PATCH releases MUST NOT introduce breaking changes +- Before `1.0.0`, MINOR releases MAY introduce breaking contract changes +- Breaking changes MUST be explicit, well-justified, and documented with + migration guidance when action is required --- diff --git a/README.md b/README.md index 8a8d77c..ea9bce3 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,8 @@ The written specification defines the semantics and rules. CTRF follows Semantic Versioning. -Releases are defined in [Releases](https://github.com/ctrf-io/ctrf/releases). +Published specification versions are defined by annotated Git tags. See the +[release process](RELEASE.md) for the current pre-1.0 policy. ## Reference Implementation From eb038045fba5b9ee158c63c667ab420d942581ca Mon Sep 17 00:00:00 2001 From: Matthew Thomas <25988124+Ma11hewThomas@users.noreply.github.com> Date: Sat, 3 Oct 2026 11:32:26 +0100 Subject: [PATCH 4/5] Condense pre-1.0 release policy --- RELEASE.md | 155 ++++++++++++++--------------------------------------- 1 file changed, 40 insertions(+), 115 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index 8adf785..fbcb008 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,137 +1,62 @@ -# CTRF Specification Release Process +# CTRF Releases -This document describes how versions of the CTRF specification are prepared and -published during the pre-1.0 period. +CTRF is currently in pre-1.0 development. -## Versioning policy +## Versioning -CTRF release versions and the `specVersion` field use `MAJOR.MINOR.PATCH` -version numbers. +CTRF specification versions and the `specVersion` field use +`MAJOR.MINOR.PATCH` version numbers. -Before CTRF `1.0.0`: +Before `1.0.0`: - MINOR versions may add capabilities or introduce breaking contract changes. -- PATCH versions must preserve compatibility and are reserved for compatible - corrections and clarifications. -- Breaking changes must be identified in `CHANGELOG.md` and include migration - guidance when producers or consumers need to take action. +- PATCH versions preserve compatibility and contain compatible corrections or + clarifications. +- Breaking changes are identified in `CHANGELOG.md`, with migration guidance + when producer or consumer action is required. -Consumers should treat different pre-1.0 MINOR versions as potentially -incompatible and should support PATCH releases within a supported pre-1.0 MINOR -version. +Different pre-1.0 MINOR versions may be incompatible. PATCH versions within the +same MINOR version are compatible. -The version in a published tag, the specification header, inline examples, -standalone examples, and valid conformance fixtures must agree. Producers should -emit the `specVersion` of the published specification they fully support. +## Releases -## Release contents - -A CTRF specification release contains the repository at the tagged commit, -including: - -- the specification in `spec/ctrf.md`; -- the JSON Schema in `schema/ctrf.schema.json`; -- the examples in `examples/`; -- the conformance fixtures in `tests/`; and -- the release history in `CHANGELOG.md`. - -These artifacts are versioned together. If the schema changes, the schema -embedded in `spec/ctrf.md` must be byte-for-byte identical to -`schema/ctrf.schema.json` after extraction. - -## Publication model - -During the current pre-1.0 period, an annotated Git tag named `vMAJOR.MINOR.PATCH` -is the publication record for a CTRF specification version. The tagged repository -contents are the canonical source for that version. - -GitHub Releases are not part of the current publication process. The existing -tag-triggered GitHub Release workflow must remain disabled while this tags-only -policy is in effect. - -Published tags are immutable. Do not move, replace, or delete a published tag to -correct a release. Prepare and publish a new version instead. - -## Preparing a release - -All release preparation must be made on a dedicated branch and reviewed through -a pull request. Never prepare or push a release commit directly on `main`. +Before `1.0.0`, CTRF specification releases are published as annotated Git tags +named `vMAJOR.MINOR.PATCH`. -Before opening the pull request: +The tag and its repository contents are the canonical release record. Published +tags are immutable; corrections are published as a new version. -1. Choose the release version using the pre-1.0 versioning policy. -2. Update the version and publication date in `spec/ctrf.md`. -3. Update `specVersion` in inline examples, standalone examples, and valid test - fixtures. -4. Move the relevant `Unreleased` changelog entries into a dated release entry. -5. Describe breaking changes and required migrations in the changelog. -6. Confirm that intentionally invalid version fixtures remain invalid rather - than mechanically replacing every version-like value. -7. Run the release verification checks. +Release changes are reviewed through a pull request. Merging the pull request +does not publish the release; publication occurs when the version tag is pushed. -The release pull request should state whether the report shape or validation -rules changed relative to the preceding tag. - -## Verification - -Run the repository's complete schema and conformance checks before merging the -release pull request: - -```bash -jsonschema fmt schema/ctrf.schema.json --check -jsonschema lint schema/ctrf.schema.json --exclude top_level_examples -jsonschema validate schema/ctrf.schema.json examples/*.json -jsonschema test tests/ctrf.test.json -jsonschema test tests/normative -``` - -Also verify: - -- the embedded schema matches `schema/ctrf.schema.json` byte-for-byte; -- released examples and valid fixtures use the intended `specVersion`; -- the changelog comparison links identify the correct tags; and -- `git diff --check` passes. - -## Publishing a release +## Release contents -After the release pull request is approved and merged: +The specification, JSON Schema, examples, conformance fixtures, and changelog +are versioned together. -1. Update local `main` with a fast-forward pull. -2. Confirm the merged commit contains the reviewed version and release date. -3. Re-run the release verification checks on the merged commit. -4. Confirm the GitHub Release workflow is still disabled. -5. Create an annotated tag on the verified merged commit. -6. Push only the tag. -7. Verify that the remote tag resolves to the intended commit and that no GitHub - Release was created. +For each release: -For example: +- the specification header identifies the release version and date; +- inline examples, standalone examples, and valid conformance fixtures use the + same `specVersion`; +- `CHANGELOG.md` describes the changes from the preceding version; and +- the schema embedded in `spec/ctrf.md` matches + `schema/ctrf.schema.json` byte-for-byte. -```bash -git switch main -git pull --ff-only -git tag -a v0.1.0 -m "CTRF specification v0.1.0" -git push origin v0.1.0 -``` +Schema formatting and linting, example validation, and the reference and +normative conformance suites form the release verification record. -Creating and pushing the tag is an explicit publication action. It must not be -performed merely because the preparation pull request was merged. +If the written specification and JSON Schema disagree, the written +specification takes precedence and the inconsistency is corrected in a new +release. -## Current pre-1.0 history +## Current release history The `v0.0.1` through `v0.0.4` tags were assigned retrospectively to historical specification snapshots. Files at those commits retain their original version -metadata, so those tags do not have the internal version consistency required of -new releases. Their tag annotations and `CHANGELOG.md` record this limitation. - -Version `0.1.0` is the first release prepared so that its specification header, -inline examples, standalone examples, and valid conformance fixtures all identify -the same specification version. It introduces no report-shape or validation -change relative to `v0.0.4`; producers adopting it should emit -`"specVersion": "0.1.0"`. - -## Authority +metadata and may not identify the version shown by the tag. -The tagged repository contents define a published CTRF specification version. If -the written specification and JSON Schema disagree, the written specification -takes precedence, and the inconsistency must be corrected in a new release. +Version `0.1.0` is the first release in which the specification header, inline +examples, standalone examples, and valid conformance fixtures identify the same +specification version. It introduces no report-shape or validation change from +`v0.0.4`; producers adopting it use `"specVersion": "0.1.0"`. From 0c0e45a1ad575fd2a525c2bcffd96c0dee573488 Mon Sep 17 00:00:00 2001 From: Matthew Thomas <25988124+Ma11hewThomas@users.noreply.github.com> Date: Sat, 3 Oct 2026 12:20:56 +0100 Subject: [PATCH 5/5] Clarify pre-v1 and v1 versioning --- CHANGELOG.md | 5 +++-- CONTRIBUTING.md | 5 ++++- RELEASE.md | 8 ++++++++ spec/ctrf.md | 19 ++++++++++++++++--- 4 files changed, 31 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ab9998..b8a1a68 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,8 +14,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Established the first internally consistent CTRF specification release: the specification header, inline examples, standalone examples, and conformance fixtures now identify version `0.1.0`. -- Defined the pre-1.0 versioning policy: PATCH releases preserve compatibility, - while MINOR releases may contain additions or breaking contract changes. +- Defined the versioning policy: before `1.0.0`, PATCH releases preserve + compatibility while MINOR releases may contain additions or breaking contract + changes; from `1.0.0`, breaking changes require a MAJOR release. - No report-shape or validation changes were introduced relative to `v0.0.4`; producers should emit `"specVersion": "0.1.0"` for this release. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c39da08..2821b57 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -77,7 +77,8 @@ The normative JSON Schema lives at: Schema changes MUST: - match the written specification -- avoid breaking existing valid documents +- avoid breaking existing valid documents except for an explicitly approved + pre-1.0 MINOR release - use consistent validation constraints If the schema and specification disagree, the **written specification takes precedence**. @@ -90,6 +91,8 @@ Backward compatibility is a core CTRF principle. - PATCH releases MUST NOT introduce breaking changes - Before `1.0.0`, MINOR releases MAY introduce breaking contract changes +- Beginning with `1.0.0`, breaking changes require a MAJOR release and MINOR + releases contain only backward-compatible additions - Breaking changes MUST be explicit, well-justified, and documented with migration guidance when action is required diff --git a/RELEASE.md b/RELEASE.md index fbcb008..0c740d9 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -18,6 +18,12 @@ Before `1.0.0`: Different pre-1.0 MINOR versions may be incompatible. PATCH versions within the same MINOR version are compatible. +Beginning with `1.0.0`: + +- MAJOR versions contain breaking changes. +- MINOR versions contain backward-compatible additions. +- PATCH versions contain backward-compatible corrections and clarifications. + ## Releases Before `1.0.0`, CTRF specification releases are published as annotated Git tags @@ -26,6 +32,8 @@ named `vMAJOR.MINOR.PATCH`. The tag and its repository contents are the canonical release record. Published tags are immutable; corrections are published as a new version. +GitHub Releases are not used during the pre-1.0 tags-only period. + Release changes are reviewed through a pull request. Merging the pull request does not publish the release; publication occurs when the version tag is pushed. diff --git a/spec/ctrf.md b/spec/ctrf.md index 8d7abed..000d6d5 100644 --- a/spec/ctrf.md +++ b/spec/ctrf.md @@ -206,14 +206,21 @@ This reduces ambiguity and ensures consistent interpretation across consumers. CTRF follows Semantic Versioning and is designed to evolve without breaking existing consumers. -Backward-compatible additions are introduced through: +Before CTRF `1.0.0`, MINOR versions may introduce additions or breaking contract +changes, while PATCH versions preserve compatibility. + +Beginning with CTRF `1.0.0`: + +- MAJOR versions contain breaking changes. +- MINOR versions contain backward-compatible additions. +- PATCH versions contain backward-compatible corrections and clarifications. + +Backward-compatible additions may be introduced through: - optional fields - new insight metrics - use of `extra` objects -Breaking changes are reserved for major version increments. - --- ### 2.9. Interoperability Over Optimization @@ -1978,6 +1985,12 @@ Consumers MUST treat different pre-1.0 MINOR versions as potentially incompatible. Consumers SHOULD support PATCH releases within a supported pre-1.0 MINOR version. +Beginning with CTRF `1.0.0`: + +- MAJOR versions contain breaking changes. +- MINOR versions contain backward-compatible additions. +- PATCH versions contain backward-compatible corrections and clarifications. + Consumers MUST reject incompatible MAJOR versions. Producers MUST set `specVersion` to the highest CTRF version they fully support.