Skip to content

test: Promote the center and isdigit function-library fixtures - #165

Merged
leongdl merged 3 commits into
OpenJobDescription:mainlinefrom
leongdl:conformance-func-lib-expected-failures
Sep 15, 2026
Merged

leongdl merged 3 commits into
OpenJobDescription:mainlinefrom
leongdl:conformance-func-lib-expected-failures

Conversation

@leongdl

@leongdl leongdl commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Promotes the two remaining function-library fixtures out of EXPR/jobs/proposed/ and into jobs/, where the runner discovers them. Both spec decisions they were waiting on have landed, and both fixtures pass.

expr2.2.4--center-odd-padding

Section 2.2.4 does not say which side of an odd pad takes the extra space. Per @mwiebe's review: #177 settles ambiguities of this kind by matching Python. CPython's 'hi'.center(7) is ' hi ' (extra space left), which is what the fixture already asserted as a strawman and is now the rule.

expr2.2.4--isdigit-unicode

The classification functions carry Python str semantics, and openjd-rs#341 implemented that from CPython-generated tables. Per @mwiebe's review, the Unicode expectation now holds, so the fixture goes live asserting isdigit is true for U+0663 and U+0969.

Coverage note: #177 adds expr2.2.4--string-classification-unicode, which is a superset for U+0663 (same ISDIGIT_ARABIC marker). This fixture also covers U+0969, so it is not a pure duplicate, but it is reasonable to fold it into #177's fixture instead if you would rather have one file. Say the word and I will drop it.

Fixture changes made while promoting

  • Rewrote both headers: dropped the PARKED framing and the 2026-08 divergence tables, and stated the spec rule, the inputs, and what a wrong implementation prints.
  • Wrote the digits as \u0663 and \u0969 escapes so the file stays ASCII. The runner reads fixtures with open(path) at the locale encoding, which mojibakes a UTF-8 literal on Windows. Same reason fix: Specify Python str semantics for character classification functions #177 uses escapes.
  • Bracketed the isdigit assertions, matching the sibling expr2.2.4--string-classification fixture. The runner matches output as a substring, so an unbracketed value is a weaker assertion.
  • Deleted proposed/README-func-lib.md. Nothing is left parked in it: the repr_py fixture was promoted on test: repr_py control-character conformance tests #162.

The third fixture from the original description, expr2.2.6--repr-py-newline-roundtrip, was already promoted on #162 and dropped from this branch.

Comment thread conformance-tests/2023-09/EXPR/proposed/expr2.2.4--center-odd-padding.test.yaml Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quorum verdict: GOOD-WITH-NITS — spec: EL §2.2.4 leaves 'digit' undefined while §2.2.5 gives \d explicit Unicode semantics — correctly classified as a spec ambiguity, and the README's dual-run (both implementations ASCII-only, agreeing) matches the sweep. Nits: (1) the fixture header claims 'live implementation divergence' — there is none; both agree, the divergence is vs host-language str.isdigit — align the header with the README; (2) the strawman asserts Unicode-true, the OPPOSITE of the de facto agreement — if the spec ratifies ASCII (likely), the expectation flips; two reviewers suggest either flipping now or leaving this out of the inventory entirely since it prejudges an open spec decision rather than pinning a bug.

@leongdl

leongdl commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Quorum review (5 independent agents). 3 fixtures + README. Net: 1 GOOD, 1 GOOD-WITH-NITS, 1 NOT-GOOD requiring a README rewrite before merge. The center-odd-padding README is internally contradictory and factually wrong about CPython (verified by execution: 'hi'.center(7) == ' hi ', extra space LEFT); per the 2026-08-12 sweep, openjd-rs now matches CPython and the PYTHON CLI is the outlier — the fixture's committed expectation actually matches the de facto answer, and the README is the defect. Cross-PR blocker: this branch and PR #164 both create EXPR/proposed/README.md (verified add/add conflict).

@leongdl
leongdl force-pushed the conformance-func-lib-expected-failures branch from 4fd8d2b to 6fc5ab9 Compare August 15, 2026 19:26
@leongdl

leongdl commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Quorum-review fixes applied and pushed (rebased onto mainline 3069673):

  • expr2.2.4--center-odd-padding README/header rewritten on the measured facts (the NOT-GOOD): CPython's 'hi'.center(7) is ' hi ' — extra space LEFT (verified by execution, three independent runs this review); current openjd-rs matches CPython (re-verified against an upstream/main build: emits ' hi '); the Python openjd CLI is now the right-heavy outlier. The fixture's committed expectation was correct all along — the old README's "both implementations are right-heavy / the expectation was wrong" claims are retracted in the new text.
  • expr2.2.4--isdigit-unicode: header reframed as spec ambiguity with no divergence between the openjd implementations (both ASCII-only; the divergence is vs host CPython), and the strawman's flip-on-ASCII-ratification path is documented.
  • expr2.2.6--repr-py-newline-roundtrip: stale "bug in openjd-rs" comment corrected to BOTH implementations.
  • Restructured to kind-level parking (EXPR/jobs/proposed/) with a family-named README (README-func-lib.md) — removes the add/add merge conflict with test: [expected failures] EXPR fixtures needing implementation fixes #164.

Comment thread conformance-tests/2023-09/EXPR/jobs/proposed/expr2.2.4--isdigit-unicode.test.yaml Outdated
leongdl added a commit to leongdl/openjd-specifications that referenced this pull request Sep 10, 2026
Both were parked as expected failures. openjd-rs#374 landed the fix, so
they pass and become the regression tests the proposed/README promised.

Measured against a binary built from openjd-rs main at 5b04959, and
against a pre-fix binary as the control:

  fixture                                        pre-fix   merged
  EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip  FAIL      PASS
  WRAP_ACTIONS/.../wrap-repr-py-escapes-newline   FAIL      PASS

Both pre-fix failures are "SyntaxError: unterminated string literal" with
the expected output missing, which is the defect they were written for:
repr_py quoted its output without escaping, so a value carrying a newline
emitted it raw and CPython would not parse the result.

Discovered by the runner's directory scan from their new locations, not
just single-file mode, which is what leaving proposed/ was for. No
regression in either category: WRAP_ACTIONS 73 passed 0 failed, EXPR 352
passed 0 failed.

The EXPR fixture comes from the func-lib expected-failures branch (OpenJobDescription#165),
which should drop its copy so the two do not collide on merge.

Headers rewritten, since both described the parked state and claimed the
implementations fail. Now spec, test, expectation in three paragraphs each,
naming the paired fixture, within the comment-line budget.

proposed/README.md and the proposed/ directory are removed: the fixture it
described was its only entry.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
leongdl added a commit to leongdl/openjd-specifications that referenced this pull request Sep 10, 2026
Both were parked as expected failures. openjd-rs#374 landed the fix, so
they pass and become the regression tests the proposed/README promised.

Measured against a binary built from openjd-rs main at 5b04959, and
against a pre-fix binary as the control:

  fixture                                        pre-fix   merged
  EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip  FAIL      PASS
  WRAP_ACTIONS/.../wrap-repr-py-escapes-newline   FAIL      PASS

Both pre-fix failures are "SyntaxError: unterminated string literal" with
the expected output missing, which is the defect they were written for:
repr_py quoted its output without escaping, so a value carrying a newline
emitted it raw and CPython would not parse the result.

Discovered by the runner's directory scan from their new locations, not
just single-file mode, which is what leaving proposed/ was for. No
regression in either category: WRAP_ACTIONS 73 passed 0 failed, EXPR 352
passed 0 failed.

The EXPR fixture comes from the func-lib expected-failures branch (OpenJobDescription#165),
which should drop its copy so the two do not collide on merge.

Headers rewritten, since both described the parked state and claimed the
implementations fail. Now spec, test, expectation in three paragraphs each,
naming the paired fixture, within the comment-line budget.

proposed/README.md and the proposed/ directory are removed: the fixture it
described was its only entry.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
leongdl added a commit that referenced this pull request Sep 11, 2026
* test: Add expected-failure WRAP_ACTIONS fixtures (parked in proposed/)

Spec-correct fixtures that FAIL against current reference implementations, parked in a proposed/ directory the conformance runner does not discover, so merging keeps the suite green. Each entry in proposed/README.md records the observed output, spec citation, and classification. Promote each fixture up one directory unchanged when its implementation fix lands. Companion to conformance-wrap-actions-gaps.

Review: quorum-review fixes — kind-level proposed/ placement
(jobs/proposed/); README corrected: the defect fails BOTH implementations
(2026-08-12 sweep), the §5.2 ArgString conflict is now the stated
promotion gate rather than a footnote (the fixture's own args are
arguably spec-invalid until that is resolved), and the unit-level repr_py
twin in the func-lib expected-failures PR is cross-referenced.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>

* test: Promote the repr_py fixtures out of proposed/

Both were parked as expected failures. openjd-rs#374 landed the fix, so
they pass and become the regression tests the proposed/README promised.

Measured against a binary built from openjd-rs main at 5b04959, and
against a pre-fix binary as the control:

  fixture                                        pre-fix   merged
  EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip  FAIL      PASS
  WRAP_ACTIONS/.../wrap-repr-py-escapes-newline   FAIL      PASS

Both pre-fix failures are "SyntaxError: unterminated string literal" with
the expected output missing, which is the defect they were written for:
repr_py quoted its output without escaping, so a value carrying a newline
emitted it raw and CPython would not parse the result.

Discovered by the runner's directory scan from their new locations, not
just single-file mode, which is what leaving proposed/ was for. No
regression in either category: WRAP_ACTIONS 73 passed 0 failed, EXPR 352
passed 0 failed.

The EXPR fixture comes from the func-lib expected-failures branch (#165),
which should drop its copy so the two do not collide on merge.

Headers rewritten, since both described the parked state and claimed the
implementations fail. Now spec, test, expectation in three paragraphs each,
naming the paired fixture, within the comment-line budget.

proposed/README.md and the proposed/ directory are removed: the fixture it
described was its only entry.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>

* test: Assemble the repr_py round-trip FAIL marker at run time

windows-latest failed the Python lane on this fixture and ubuntu-latest
passed the same installed CLI. Not an implementation difference: the
fixture matched its own forbidden marker.

The harness scans a job's whole output for forbidden substrings, and on
Windows openjd-sessions-for-python echoes the full child command line at
INFO (_subprocess.py logs list2cmdline(self._args)); the POSIX path logs
only the temp .sh path. This fixture's script source contained the literal
FAIL, and FAIL is its forbidden entry, so the echo matched even though
every assertion passed.

The run log discriminates the two cases. It reports "Found forbidden
output: FAIL" with no "Missing expected output" line, so NEWLINE:PASS was
printed and the escaping worked on Windows too. A genuine failure of this
fixture looks different: the pre-fix control run reported the missing
NEWLINE:PASS line alongside a SyntaxError.

Split the marker across two adjacent literals, the convention this repo
already uses for the same hazard (20c1a84, and the timeout fixture's
'SHOULD_NOT' + '_PRINT'). Python concatenates them at compile time, so the
run-time value is unchanged at 'FAIL got=' while the source no longer
carries the literal. Verified both halves: the value still compares equal,
and the script body no longer contains FAIL.

Verified the marker still catches a real failure: comparing against a
wrong value makes the runner report 0 passed, 1 failed, and restoring the
fixture byte-identically returns it to 1 passed, 0 failed.

The Rust lane was never affected, because it prints only COMMAND_OUTPUT
records while the command echo is tagged FILE_PATH|PROCESS_CONTROL. The
WRAP_ACTIONS fixture is unaffected too: it declares no forbidden entries.

Windows itself is unverified from here.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>

---------

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
@leongdl leongdl changed the title test: [expected failures] function-library fixtures pending spec decisions test: Promote the center and isdigit function-library fixtures Sep 14, 2026
@leongdl

leongdl commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

CI on fb2898c: both promoted fixtures are now discovered by the runner and pass on every lane that has completed.

Lane OS Result Suite total
Conformance Tests (Rust) ubuntu ✓ expr2.2.4--center-odd-padding, ✓ expr2.2.4--isdigit-unicode 1184 passed, 0 failed
Conformance Tests (Rust) windows both 1139 passed, 0 failed
Conformance Tests (Python) ubuntu both 1184 passed, 0 failed
Conformance Tests (Python) windows both 1139 passed, 0 failed

macOS (Rust lane) has been queued for 45 minutes without starting, which is a hosted-runner backlog rather than a result. Both fixtures are platform-independent string operations and pass on the other four lanes.

Worth recording because it retracts an earlier measurement: the old center-odd-padding header claimed the Python openjd CLI was the right-heavy outlier and would fail this fixture. It passes. Whatever the 2026-08 sweep measured, the current Python CLI agrees with CPython and openjd-rs on 'hi'.center(7) == ' hi ', so there is no live divergence left on either fixture.

…sed/)

Three parked fixtures with dual-implementation observations: center() odd-padding and ASCII-only isdigit() are spec gaps where BOTH implementations agree (no divergence - the spec should document the de facto behaviour), and repr_py raw-newline emission is a bug in BOTH implementations per the explicit spec text. Details in proposed/README.md. Companion to conformance-func-lib-gaps.

Review: quorum-review fixes — corrected the center-odd-padding entry on
the measured facts: CPython's str.center('hi',7) is '   hi  ' (extra
space LEFT, verified by execution), current openjd-rs matches CPython
(re-verified against an upstream/main build this session), and the
Python CLI is now the right-heavy outlier — the fixture's committed
expectation is the de facto answer and the earlier README/header claims
were wrong. isdigit reframed as spec ambiguity with NO divergence between
the openjd implementations (both ASCII-only; the divergence is vs host
CPython). repr_py attribution corrected to BOTH implementations.
Kind-level proposed/ placement with a family-named README
(README-func-lib.md) to avoid add/add conflicts with co-located
expected-failures PRs.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
openjd-rs#374 landed the repr_py escaping fix, so
expr2.2.6--repr-py-newline-roundtrip passes and no longer belongs in
proposed/. It is promoted to EXPR/jobs/ on OpenJobDescription#162 alongside its
WRAP_ACTIONS twin, where the two land together and the control run
against a pre-fix binary is recorded.

Removing it here so the two copies do not collide when both merge. Its
README section goes with it.

The other two fixtures on this branch are untouched and still parked:
both are spec decisions, not implementation bugs.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
Both fixtures were parked in EXPR/jobs/proposed/ pending spec decisions. Both
decisions have landed, so they move up into jobs/ where the runner discovers
them.

center("hi", 7): section 2.2.4 does not say which side of an odd pad takes the
extra space. OpenJobDescription#177 settles ambiguities of this kind by matching Python, so the
fixture's CPython expectation ("   hi  ", extra space left) is now the rule
rather than a strawman.

isdigit over non-ASCII decimal digits: the classification functions carry
Python str semantics, and openjd-rs#341 implemented that from CPython-generated
tables, so the Unicode expectation now holds.

Fixture changes made while promoting:

- Rewrite both headers: drop the PARKED framing and the 2026-08 divergence
  tables, and state the spec rule, the inputs, and what a wrong implementation
  prints.
- Write the digits as \u0663 and \u0969 escapes so the file stays ASCII. The
  runner reads fixtures with the locale encoding, which mojibakes a UTF-8
  literal on Windows. Same reason OpenJobDescription#177 uses escapes.
- Bracket the isdigit assertions, matching the sibling
  expr2.2.4--string-classification fixture. The runner matches output as a
  substring, so an unbracketed value is a weaker assertion.
- Delete proposed/README-func-lib.md. Nothing is left parked in it: the
  repr_py fixture was promoted on OpenJobDescription#162.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
@leongdl
leongdl force-pushed the conformance-func-lib-expected-failures branch from fb2898c to 85c5215 Compare September 14, 2026 23:32
@leongdl
leongdl marked this pull request as ready for review September 15, 2026 00:08
@leongdl
leongdl requested a review from a team as a code owner September 15, 2026 00:08
@leongdl
leongdl merged commit 24e9062 into OpenJobDescription:mainline Sep 15, 2026
7 checks passed
@leongdl
leongdl deleted the conformance-func-lib-expected-failures branch September 15, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants