Skip to content

feat(tests): add missing EIP-8141 frame transaction coverage - #3917

Closed
Mannan-Goyal wants to merge 15 commits into
ethereum:eips/bogota/eip-8141from
Mannan-Goyal:feature/8141-missing-tests
Closed

Mannan-Goyal wants to merge 15 commits into
ethereum:eips/bogota/eip-8141from
Mannan-Goyal:feature/8141-missing-tests

Conversation

@Mannan-Goyal

@Mannan-Goyal Mannan-Goyal commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Split into smaller PRs

Closed in favour of smaller PRs, opened one at a time (see the last comment). I'll keep this table updated as they go in.

# Theme PR Status
1 VERIFY frames run as static calls; transient storage cleared between frames #3939 merged
2 APPROVE edge cases; sender and payer balance against max_cost #3947 open for review

Description

Adds tests for EIP-8141 rules that the suite didn't cover yet, found by going through the EIP text rule by rule and checking each against the existing tests (including the open draft PRs, to avoid duplicates). Every new test was checked by temporarily breaking the spec rule it targets and confirming it fails. The whole eip8141_frame_transactions directory fills on Bogota: 2,455 passed, 0 failed. Gas expectations are read off the bytecode under test and the fork's calculators (.state_cost/.refund/.execution_cost, frame_transaction_intrinsic_cost_calculator, call_value_stipend, GasConsumer); the only fork.gas_costs() reads left are single constants for charges no opcode makes (frame-entry and APPROVE account creation) and precompile prices the framework does not model.

It's large (+12.5k lines; 16 new test files, 9 extended). Happy to split it if that's easier to review: it divides by the areas below, with shared helpers first. Several of the files marked * also hold tests grouped into them by subject, so a split would go by file, not strictly by area.

Area Files What it covers (EIP section)
Helpers helpers.py, signature_helpers.py Shared builders used by the new tests; a pure-Python P-256 signer (RFC 6979) so tests don't depend on OpenSSL. Existing fixtures are byte-identical except 20 in test_static_validity.py: test_gas_limit_cap_from_calldata_floor*[below_cap, above_cap] (the frames now call a target and the floor lands exactly on the cap, so below_cap is one byte under it, with a new at_cap case between), and test_signature_constraints*[secp256k1_v_27, secp256k1_s_zero, p256_signer_mismatch] (each now carries a signature that otherwise verifies, so only the rule under test rejects it).
Transaction validity test_static_validity.py, test_admission_validity.py, test_raw_rlp_rejections.py, test_default_code_signatures.py, test_approve_balance.py Malformed type-6 RLP (truncation, trailing bytes, arity, non-canonical integers, address widths, blob-hash length); signature s/v/r bounds for secp256k1 and P256; default code rejecting a missing entry, wrong scheme, wrong signer or explicit message; APPROVE balance at max_cost and one wei below, read when APPROVE runs; wrong chain id; frame-gas sum and calldata-floor caps; reservations for a second transaction in a block
Execution test_transient_storage.py, test_verify_frame_static.py, test_caller_origin.py, test_frame_opcodes_outside_frames.py, test_approval_edges.py, test_delegated_sender.py, test_batch_status.py, test_frame_environment.py, test_introspection.py, test_frame_transactions.py, test_target_resolution.py* Transient storage cleared between frames; VERIFY frames as static context (direct and nested); CALLER/ORIGIN per mode and depth; frame opcodes halting outside type 6; APPROVE edges (double payment approval, split approval, approval through CALLCODE, halt discarding the approval, default-code refusals); EIP-7702-delegated senders; atomic batch status; out-of-range introspection operands; GASPRICE/CALLVALUE/logs per frame
Gas test_frame_create.py, test_frame_selfdestruct.py, test_frame_opcode_gas.py, test_intrinsic_cost.py, test_payer_settlement.py, test_fee_budget.py, test_gas_settlement.py, test_state_gas.py, test_warmth.py CREATE/CREATE2 and SELFDESTRUCT state gas inside frames (including batch unroll); introspection opcode gas and memory-expansion boundaries; state gas in sub-calls; refund cap and refunds across frames; intrinsic cost; payer settlement; fee products above 2**256; warm/cold access across frames

* existing file, tests appended or tables extended (no existing test id renamed or removed).

Where no test was written. The EIP is silent, or EELS differs from the text. Raising these rather than pinning either reading:

  1. The EIP allows nonce < 2**64, while EELS rejects 2**64 - 1, and two existing tests pin EELS's bound.
  2. The EIP allows chain_id < 2**256, while EELS decodes it as 64-bit (invalid either way; only the error differs).
  3. No order is given for which error wins when two checks fail.
  4. Approvals are discarded when a frame reverts. EELS also discards them on an exceptional halt, and the text doesn't say so.
  5. Fee products (max_priority_fee * max_gas) can exceed 2256, and nothing says they must not wrap. A test with a 2255 tip pins EELS's arbitrary-precision result.
  6. Rationale l.1294 says creation charges are undone "only" when the whole frame rolls back. l.629 and EELS also undo them when the CREATE's own call fails.
  7. A sponsor can pay away its whole balance and become an empty account. EELS deletes it; the EIP doesn't say.
  8. The sender's nonce can rise by 2 in one transaction (a CREATE before APPROVE), and the payer's balance is read when APPROVE runs. These tests follow the literal text; happy to adjust if that isn't the intent.

Also checked against a client: all fixtures at 1d798f927 were run through Nethermind's nethtest on its bogota-devnet-2 branch, with the open devnet-2 PRs (#15045, #14791, #15033) merged locally: blockchain 775/775, state 743/743, engine 766/775, transaction 144/162. None of the 27 failures changes block validity, and all were present before the first push: 2 are the one real Nethermind difference the new tests found — frame gas sums just below 2**64, which the EIP allows, are rejected because the intrinsic gas is added in 64 bits (block validity is not affected; fix: NethermindEth/nethermind#15270); 22 are the test runner lacking a mapping from Nethermind's rejection message to the fixture's exception label (5 engine fixtures for INSUFFICIENT_MAX_FEE_PER_GAS, 16 raw-RLP and 1 nonce transaction fixtures; NethermindEth/nethermind#15274 adds the decode mappings); 3 engine fixtures report the block access list error before the transaction's own error, rejecting the block either way.

Related: #3916 makes fill also check the raw-RLP cases here against EELS (today transaction fixtures aren't run through EELS). The two PRs don't depend on each other.

Related Issues or PRs

cc @spencer-tb @gurukamath

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <description>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

A mastiff in glasses and a tie

Read every expected gas amount off the bytecode under test or the
fork's calculators instead of rebuilding it from schedule constants:
state gas from `.state_cost(fork)`, refunds from `.refund(fork)`,
stipends from `fork.call_value_stipend()`, the refund cap from
`fork.max_refund_quotient()`, intrinsic terms as calculator deltas and
burns from `GasConsumer`. Drop the self-check asserts and the
`storage_set_cost`/`account_creation_cost`/`clearing_refund`/
`restoring_refund` helpers, and restore the framework-priced lines in
`test_state_gas.py` that the helpers had replaced.

Also, in the same modules: cover the call and create opcode families
with `with_all_call_opcodes`/`with_all_create_opcodes`, handle every
parametrized value with an explicit branch, add the passing side of
the one-sided batch boundaries, keep gas numbers out of docstrings, and
stop restating the gas limit as the expected gas used.
Cover the call and create opcode families with the covariant markers,
handle every parametrized value with an explicit branch, say why each
listed exception of a multi-error case is valid, drop the module-level
`inclusion_test` mark from the raw RLP rejections, add the passing side
of the one-sided introspection boundaries and keep gas numbers out of
docstrings.
Move each new module whose subject an existing module already owns into
that module, keeping every test name and case:

- test_cross_frame_access.py -> test_warmth.py
- test_tx_validity_bounds.py -> test_static_validity.py (transaction
  tests) and test_admission_validity.py (in-block test)
- test_block_limits.py -> test_admission_validity.py
- test_introspection_bounds.py -> test_introspection.py
- test_refund_edges.py -> test_gas_settlement.py
- test_state_gas_subcalls.py -> test_state_gas.py
- test_default_code_approval_refusals -> test_approval_edges.py

Duplicate constants and helpers now use the destination's version, and
helpers shared by two modules live in helpers.py.
- Deploy non-zero runtime code from the frame-create init code again,
  so the `code=` post checks fail for an init code that copies nothing.
- Split the pre-existing victim out of the create-opcode-marked
  self-destruct refill test; it creates nothing, so its CREATE2 case
  duplicated the CREATE one.
- Restore the premise that the calldata floor sits exactly
  `cap_excess` byte prices from the gas cap.
@Mannan-Goyal
Mannan-Goyal marked this pull request as ready for review October 29, 2026 10:01
@Mannan-Goyal

Copy link
Copy Markdown
Contributor Author

Taking this out of draft. I've kept it as one PR, but happy to split it by the areas in the description if that's easier to review. It merges cleanly with the current eips/bogota/eip-8141, and the 8141 tests still fill with 0 failures with that base merged in.

@gurukamath

Copy link
Copy Markdown
Contributor

Thanks @Mannan-Goyal for the offer to split, let's do that. At 12.5k lines, this can't get a careful review as one PR. This should ideally be 6-7 different PRs (at least), split based on theme (or even which section of the EIP text the PR tackles). I'll leave that up to you but a couple of guidelines.

  1. Make sure the PRs are small enough that they tackle 1 theme (or at most 2 related themes)
  2. PLEASE DO NOT open multiple open PRs at once. Would appreciate if you open 1 PR and then wait for it to be reviewed and actioned on, before opening the next one.

This will save the maintainers a lot of review time while also making sure we get a chance to go through the proposed changes thoroughly.

@Mannan-Goyal

Copy link
Copy Markdown
Contributor Author

Thanks @gurukamath, that makes sense. I'll split this into smaller PRs, one or two related themes each, and open them one at a time, waiting for each to be reviewed and dealt with before opening the next.

I've closed this one so only one part is open at a time (GitHub also wouldn't let me open a third PR here while this and #3916 were open). The table below is also at the top of this PR's description; I'll keep it updated there as the parts go in. Part 1 is #3939.

# Theme PR Status
1 VERIFY frames run as static calls; transient storage cleared between frames #3939 open for review

#3916 (the test-specs framework change, against forks/bogota) is separate from this series and stays as it is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants