Skip to content

feat(test-specs): check transaction fixtures against the reference implementation - #3916

Open
Mannan-Goyal wants to merge 4 commits into
ethereum:forks/bogotafrom
Mannan-Goyal:feature/fill-check-transaction-fixtures
Open

Mannan-Goyal wants to merge 4 commits into
ethereum:forks/bogotafrom
Mannan-Goyal:feature/fill-check-transaction-fixtures

Conversation

@Mannan-Goyal

@Mannan-Goyal Mannan-Goyal commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

Before this change, TransactionTest.generate discarded the transition tool (del t8n). A transaction fixture therefore recorded tx.error without any check against EELS. Block-based fills can't catch this either: they send the tool the transaction as JSON, and the t8n's JSON loader (TransactionLoad) rebuilds the transaction from its fields, ignoring rlpOverride.

With this PR, when filling with the in-process EELS t8n, every transaction fixture's exact bytes are decoded by the fork's EELS code:

  • Bytes EELS can't decode, including empty or truncated bytes, must have an expected error.
  • Once a fork defines FrameTransaction (EIP-8141), decoded frame txs also go through validate_frame_transaction, so a frame tx it accepts must have no expected error. On forks without it, this step does nothing.
  • Known limit: validate_frame_transaction applies every transaction-only rule except the chain ID, which EELS checks against the block's chain ID when it processes the transaction. A frame transaction_test whose only fault is its chain ID would therefore be reported as an unexpected success. None of the EIP-8141 transaction_test cases does this today: test_wrong_chain_id is a state_test, and the raw-RLP chain ID of 2**64 already fails decoding.

Only the accept/reject verdict is compared, not the exception class. Other t8n tools are unaffected.

The frame step is exercised for real on the EIP-8141 branch (#3878). No fork on this branch ships FrameTransaction, so the success-direction error is unit-tested with a fake fork whose decoder returns a stand-in frame transaction (test_reference_check_rejects_accepted_frame_transaction).

Documented under docs/writing_tests/types_of_tests.md → Transaction Tests.

Verified:

  • every transaction_test directory fills clean --until Bogota (11265 passed);
  • on the EIP-8141 branch, all 162 EIP-8141 transaction_test cases fill clean, and an EELS FrameTransaction.sender width mutation is caught by sender_19_bytes, sender_21_bytes and sender_empty;
  • breaking the legacy to decoding in EELS is caught by test_valid_reencoded_transaction;
  • just test-tests: 4013 passed, 0 failed; the new unit test fails (DID NOT RAISE UnexpectedExecutionSuccessError) when the success-direction raise is removed.

Related Issues or PRs

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <description>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

A dog licking its own nose with a side-eye

@Mannan-Goyal Mannan-Goyal changed the title feat(test-fill): check transaction fixtures against the reference implementation feat(test-specs): check transaction fixtures against the reference implementation Oct 27, 2026
Note in types_of_tests.md that filling with the in-process t8n decodes a
transaction test's bytes with the fork's own code, so undecodable bytes
without error= fail the fill. Use imperative one-line summaries for the
new unit tests and check_reference_verdict.
Add a unit test where a frame transaction decodes and validates but the
test sets error=, so the reference check must raise. No fork on this
base ships FrameTransaction, so Cancun's type-2 class stands in for it.
…test

Patch resolve_fork in the transaction spec with a SimpleNamespace fork
instead of adding attributes to Cancun's transactions module and putting
a Mock in sys.modules. No real spec module is changed by the test.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant