Addressable Attention Heads / V96
Every one of the 264 attention heads in the reader’s frozen encoder now has an address and can be switched off exactly, without changing a weight. Masked one at a time, 93.6% of heads cost at most one point of field accuracy; masking the cheapest 20% together cost 14.3 points, so the joint pruning gate failed.
SCOPE + LIMITATIONS
- The infrastructure works and a full ablation map exists; pruning for speed is optimisation and would need iterative pruning that re-measures after each removal. One task, one encoder, 100 configurations.
- This public edition reviews saved evidence; experiments were not rerun for publication.
CONTROL COMPARISON
Give each head an address of the form layers.i.head.j across 22 layers of 12 heads and switch any set off through a forward hook. Measure the learned reader’s field accuracy on 100 configurations only the learned reader can read: 99.3% with nothing masked. Mask each head alone, then mask the 52 individually cheapest heads together.
Identity with nothing masked, exact zeroing of each head’s slice, and the share of individually cheap heads (93.6% against 25%) passed. The joint gate failed at 14.3 points against 3. Heads that are each redundant cover for one another, which matches published pruning results beyond a pruning fraction.
- Nothing masked
- 99.3% of fields right; bit-identical to the unmodified encoder
- Single heads costing at most one point
- 93.6% of 264
- Most important single heads
- At most 2.0 points each
- Cheapest 20% masked together
- 14.3 points lost; gate at most 3
Question and method
Give each head an address of the form layers.i.head.j across 22 layers of 12 heads and switch any set off through a forward hook. Measure the learned reader’s field accuracy on 100 configurations only the learned reader can read: 99.3% with nothing masked. Mask each head alone, then mask the 52 individually cheapest heads together.
Recorded results
| Condition | Recorded result |
|---|---|
| Nothing masked | 99.3% of fields right; bit-identical to the unmodified encoder |
| Single heads costing at most one point | 93.6% of 264 |
| Most important single heads | At most 2.0 points each |
| Cheapest 20% masked together | 14.3 points lost; gate at most 3 |
Comparative standing and locked gates
Identity with nothing masked, exact zeroing of each head’s slice, and the share of individually cheap heads (93.6% against 25%) passed. The joint gate failed at 14.3 points against 3. Heads that are each redundant cover for one another, which matches published pruning results beyond a pruning fraction.
Interpretation and limitations
The infrastructure works and a full ablation map exists; pruning for speed is optimisation and would need iterative pruning that re-measures after each removal. One task, one encoder, 100 configurations.
SOURCE PROVENANCE
V96: addressable attention heads, with an ablation of every head of the reader's encoder
LABORATORY REPORT / 2026-10-21SOURCE CHECKSUM / SHA-256
26976b65ac5d45d66816b3a3aabed53eacfd3ae7d3fb42cc15c859609dde5f0fPublic journal edition reviewed 2026-10-22. Source documents and saved evidence were inspected; experiments were not rerun for this edition. Proprietary implementation code, model binaries, private infrastructure, and detailed machine records are not published here. Journal identifiers are editorial references. Catalog inclusion does not imply qualification or runtime promotion.