Repository navigation
[Bug Report] _BLOCK_LIST_ATTRS hardcoded name list silently drops Raven's blocks from composition-score / head-label analysis #1791
Description
Describe the bug
_enumerate_blocks() (transformer_bridge.py) walks a hardcoded tuple, _BLOCK_LIST_ATTRS = ("blocks", "encoder_blocks", "decoder_blocks", "L_blocks", "H_blocks") (bridge_core.py), checking self._modules.get(name) for each. Raven/Huginn's three block lists are registered under prelude, core_block, and coda -- none of which are in that tuple -- so _enumerate_blocks() silently returns [] for a Raven bridge, with no error.
This breaks every method built on it, silently:
blocks_with(submodule)-> always[]composition_layer_indices()-> always[]attn_head_labels-> always[](should report labels for every attention head across prelude/core_block/coda)
Inconsistently, some other methods built on the same broken enumeration fail loudly instead of silently, because they separately guard against an empty result: all_composition_scores() raises ValueError("No attention layers found"), and stack_params_for() (used by QK_for_attn_layers()/OV_for_attn_layers()) raises ValueError("No blocks have submodule ..."). Both messages are misleading -- Raven's blocks clearly do have attention -- but at least they don't silently hand back wrong data the way attn_head_labels does.
This is the same class of fragility as #1769 (stop_at_layer's reject-list), which jlarson4 is already having fixed by moving to an allowlist/_modules-based check -- except this one is on the enumeration side rather than the rejection side, and its failure mode is silent wrongness rather than a silent no-op.
Reproduction (no model download needed)
import torch.nn as nn
from transformer_lens.model_bridge.transformer_bridge import TransformerBridge
# Mirrors Raven's actual registered block-list shape (prelude/core_block/coda).
fake = nn.Module()
fake.add_module("prelude", nn.ModuleList([nn.Linear(2, 2), nn.Linear(2, 2)]))
fake.add_module("core_block", nn.ModuleList([nn.Linear(2, 2) for _ in range(4)]))
fake.add_module("coda", nn.ModuleList([nn.Linear(2, 2), nn.Linear(2, 2)]))
result = TransformerBridge._enumerate_blocks(fake)
print(result) # -> [] -- silently drops all 8 blocks, no errorSystem Info
- Installed from source,
dev-4.x - Pure Python logic, no weights involved; verified without downloading Raven's ~14GB checkpoint
Expected behaviour & fix pointers
Same direction as the fix in progress for #1769: replace the hardcoded name tuple with structural discovery. Every block-list component set via component_setup.py's list-item path carries hook_out_is_single_residual_stream = True on its items (already used for exactly this purpose in all_composition_scores()'s prerequisite check) -- _enumerate_blocks() could instead walk self._modules.items() and treat any nn.ModuleList whose first element is a GeneralizedComponent with that marker set as a block list, picking up prelude/core_block/coda (and any future non-standard adapter) automatically, with no name list to maintain.
Acceptance:
-
_enumerate_blocks()(and everything built on it:blocks_with,composition_layer_indices,attn_head_labels,stack_params_for,all_composition_scores) correctly discovers Raven'sprelude/core_block/codablocks, or raises a clear error consistently -- never silently returns an empty/wrong result - Existing
blocks/encoder_blocks/decoder_blocks/L_blocks/H_blocksbehavior is unchanged - A fast regression test (matching the reproduction above, no checkpoint download) pins the fix
-
make unit-testanduv run mypy .pass
Checklist
- I have checked that there is no similar issue in the repo (required)
Activity
LightWork666 commentedon Sep 18, 2026
I'd like to take this on -- happy to be assigned.
jlarson4 commentedon Sep 22, 2026
Sorry I don't know how I missed this before now, you are now assigned! Thank you for reporting
fix(bridge): discover non-standard block lists in _enumerate_blocks (#…