Parallel Composition V3
Raw cross-lane bank and gate candidates fail exact development behavior; a modular path-reader reproduction supplies a separate positive control.
Objective
Test source-preserving, all-at-once access to C1-C4 and Global Context G, with tasks requiring G plus two edges stored in different local lanes. This continues the original parallel-Context hypothesis: every lane remains available during the same Foundation reasoning pass; there is no winner-take- all Context source router in this experiment.
The run compares two modular access mechanisms after shared lane encoding and cross-lane interaction:
cross_lane_bank: tag LOCAL/GLOBAL lane roles, then let Foundation attend to one joint key/value bank.cross_lane_gated: attend to every lane separately and combine lane deltas with independent sigmoid contribution gates.
The qualified Foundation and active Context-native checkpoint are frozen controls. The fusion candidate, including its copied Context encoder/access weights, is the only trainable component.
Protected starting state and data
- Active Context-native checkpoint SHA-256: [checksum retained in the private evidence record]
- Qualified Foundation parent SHA-256: [checksum retained in the private evidence record]
- Direct Context Transfer baseline: 124/128 (96.875%).
- The protected file hashes and frozen Foundation tensor hashes matched after both initial arms. Final retention is rerun after the corrected arm.
- Repository HEAD at run start:
7298687(Add V3.1 Context routing experiment); relevant run code was in the working tree, so the report also retains candidate/config/data hashes.
Fresh sets were locked before training: 4,096 train, 512 development, and 1,024 sealed examples. The sealed manifest remains sealed_opened: false; there has been no sealed scoring or tuning. Identity values are disjoint lowercase six-character alphanumeric strings. Each task requires G.START to select one of three FRIEND→CITY paths whose edges appear in distinct C1-C4 lanes. The target string itself is absent from inference fields.
- Manifest content SHA-256: [checksum retained in the private evidence record]
- Train SHA-256: [checksum retained in the private evidence record]
- Development SHA-256: [checksum retained in the private evidence record]
- Sealed input SHA-256: [checksum retained in the private evidence record]
- Sealed label SHA-256: [checksum retained in the private evidence record]
Integrity checks streamed the locked files to recompute hashes only. The sealed examples were not parsed, scored, or used to select a candidate.
Matched setup
Both initial arms used the same random initialization (3e1b266a8b6b8acea04c cd13811dde5a91cd5b1083f742f01bead5b7892af7ca), seed 2026092554, train examples, batch size 8, AdamW, learning rate 3e-4, and 4,096 updates. Each candidate has 4,648,806 parameters. The matching makes the access mechanism the intended experimental variable.
Both preflights passed: five lanes were supplied together, access occurred at all four placements, losses/logits were finite, and the cross-lane and access parameters received gradients. Passing preflight establishes executable plumbing only, not useful behavior.
Results: initial matched arms
| Variant | Full dev exact (512) | Counterfactual query accuracy | Complete 3-query groups | 128-row permutation exact | Sealed gate |
|---|---|---|---|---|---|
cross_lane_bank | 84/512 (16.41%) | 29/192 (15.10%) | 0/64 | 14/128 (10.94%) | Failed |
cross_lane_gated | 12/512 (2.34%) | 3/192 (1.56%) | 0/64 | 3/128 (2.34%) | Failed |
The bank arm's all-context 128-row diagnostic was 17/128 (13.28%); G-only was 0/128, and the oracle-selected evidence diagnostic was 0/64. Thus simplifying the context to the correct two edges did not rescue this frozen Foundation generation path. The gated arm's one-source diagnostics were at or below 0.78%, and its correct-evidence diagnostic was also 0/64. Neither model learned the target behavior at a usable rate.
The bank arm's candidate SHA-256 is [checksum retained in the private evidence record]. The gated arm's candidate SHA-256 is [checksum retained in the private evidence record]. Both are retained as negative evidence; neither is qualified.
Failure and implementation diagnosis
The initial joint-bank implementation flattened all source tokens and passed one monotonically increasing rotary position sequence to Context Access. That made the bank encode physical lane offsets even though C1-C4 are meant to be interchangeable. The bank's 14/128 permuted accuracy and only 55.47% identical generated strings confirmed serious permutation sensitivity. The implementation was corrected to reset key rotary positions to zero within each lane.
The independent-gating arm does not use a flattened rotary bank, yet its permuted free-generation accuracy was also poor. Because the architecture is designed to be equivariant to local-lane permutation and small logit changes can cascade through greedy generation, free-generation exact accuracy alone cannot establish an order shortcut. A separate teacher-forced logit/argmax consistency diagnostic was added for the corrected candidate.
An earlier runner attempt reached the final diagnostics and exposed a harness bug: the evaluation wrapper did not forward source-ablation arguments. The candidate was saved before diagnostics after the fix, and the complete matched arms above were rerun. This was a runner failure, not a model score.
Corrected bank candidate
The bank architecture now declares independent-source-encode-cross-lane- interaction-v2, records its source-local rotary-position policy, and has regression tests for permutation invariance with unequal lane lengths. The corrected matched-seed bank arm completed 4,096 updates under cross-lane-interaction-rope-reset-v1:
- Full development exact: 86/512 (16.80%).
- Counterfactual query accuracy: 30/192 (15.63%); complete groups: 0/64.
- 128-row full-context diagnostic: 14/128 (10.94%).
- Oracle-selected two-edge evidence diagnostic: 1/64 (1.56%).
- Best single lane: C4 at 16.41%; G-only: 0%.
- Removing G gave 14.84% exact versus 10.94% full; removing a required lane did not reliably degrade performance. The candidate did not demonstrate causal all-lane integration.
- Free-generation permutation: 14/128 exact both before and after local-lane permutation, with all 128 generated strings identical.
- Teacher-forced permutation: token argmax agreement 128/128; mean logit absolute difference
1.06e-6, maximum8.58e-6. - Candidate SHA-256: [checksum retained in the private evidence record].
The rotary reset fixed the bank's physical lane-order leak. It did not fix relation composition or frozen-Foundation reading. The candidate failed the predeclared development gate; the V3 sealed set remains unopened.
Positive structural-reader diagnostic on the held-out V3 identities
To isolate the missing structural bias, the existing EMMA ContextPathReader was retrained on V3 train/development data only. The reader deterministically segments the declared edge records, then learns character representations, G.START-to-FRIEND matching, FRIEND-object-to-CITY-subject matching, and a joint path score. It receives all five lane contents together; it does not select a lane before reading. It copies the selected CITY object from its source edge with provenance.
This was an 800-update AdamW run, batch size 32, learning rate 1e-3, seed 20260927. It trained only the reader's model; Foundation, Context Encoder, and Context Access were not updated.
- Development exact payload and complete path: 512/512 (100%).
- Counterfactual queries: 1,536/1,536 (100%); complete three-query groups: 512/512 (100%).
- Source permutation: 100% same selected payload.
- Best single local lane plus G: 16.41%; G-only: 0%.
- Removing G or the selected CITY lane reduced exact output to 0%.
- Removing the selected FRIEND lane reduced structural path validity to 0%; exact copying remained 8.20% because the correct value can still be present among unlinked CITY candidates.
- Candidate SHA-256: [checksum retained in the private evidence record].
Run artifacts are under [retained internal evidence]. This is strong evidence that a learned relation-aware reader can use complementary information from multiple Context lanes on this structured task. It is not evidence that Foundation's raw cross-attention fused the lanes: the graph reader is a distinct specialist MicroModel, and no sealed qualification was performed for this reproduction.
The candidate was then attached to the Modular Agent V3 reader registry and run across all five source lanes on the same 512 development cases. The reader packet, deterministic RETURN_EXACT, selected-edge provenance, and runtime availability/access record were each 512/512. This is an integration smoke test only: the runtime used PlumbingFoundationPort, whose context gate is not a neural Foundation access path. Report: [retained internal evidence].
Tests
- Fusion V3 and native Context tests: 10 passed after the position-reset fix.
- Python compilation of the runner and fusion/access modules passed.
- Full repository suite: 216 passed, 2 upstream deprecation warnings.
- Final Direct Context Transfer: 124/128 (96.875%); active candidate and qualified parent hashes matched; frozen Foundation tensors were unchanged.
Interpretation and follow-up
The failure is specific to these raw token-level bank/gate implementations and their frozen-Foundation generation path. It is not evidence that all-at- once independent Context is impossible. The path-reader reproduction is a positive, separately modular mechanism for the structured task, while the Foundation-fusion hypothesis remains unresolved.
The next candidate should add the missing structural inductive bias without collapsing the Drones into one source or exposing the answer procedurally:
- Preserve the successful relation-aware
ContextPathReaderas one typed reader MicroModel behindContextPacket; do not promote it as proof of Foundation fusion. - Keep all C1-C4/G present and source-provenant in one inference pass.
- Compare per-lane evidence encoding plus learned relation-aware message passing against the current pairwise path scorer. Keep source contribution decisions separate from source acquisition/routing, and allow several lanes to contribute at once.
- Use a learned pointer over candidate evidence for
RETURN_EXACT, then pass the provenance-bearing evidence representation to the frozen Foundation for a distinct reasoning/reading evaluation. - If the frozen Foundation still cannot consume correct evidence, test a bounded late Context adapter as a separate candidate; keep the qualified parent immutable.
- Lock fresh train/development/sealed sets for that next candidate before tuning. The V3 sealed set remains unopened.
Research references are architectural references only; no external project code was copied:
- Fusion-in-Decoder paper and official implementation: independently encode passages and let a decoder attend over their joined representations.
- KG-FiD paper: use graph structure and GNN updates over intermediate passage representations to model relations and reduce irrelevant evidence.
- Relational Graph Attention Networks: incorporate relation types into learned graph attention; the paper also reports that results depend on configuration, so this is a candidate mechanism, not a guaranteed fix.
The sealed set remains closed, neither fusion variant is qualified, and no later architecture track (K=2 source routing, Memory, movement, or adaptive weights) is started by this run.
SOURCE PROVENANCE
Context Parallel Composition V3 Experiment Report
LABORATORY REPORT / 2026-09-26SOURCE CHECKSUM / SHA-256
0e7a9c804ff40155f5e4c0c79f0072f651d7d5395ebad5b31294661370e913f6Public journal edition reviewed 2026-10-01. Source documents and saved evidence were inspected; experiments were not rerun for this edition. Proprietary implementation code, model binaries, private infrastructure, and detailed machine records are not published here. Journal identifiers are editorial references. Catalog inclusion does not imply qualification or runtime promotion.