Compositional Planning for Multi-Step Questions / V121
A planner trained to combine parts of known question kinds into new ones answered 0 of 32 held-out composed kinds; the checks withheld all of them. On 10 questions outside its grammar it committed 10 wrong answers that the checks of the time admitted, and it committed one question it should have declined. Both gaps in the checks were then fixed, and familiar questions stayed accurate.
SCOPE + LIMITATIONS
- Composition is not solved by this planner. The next step tested released reference models as a second stage under the same checks (V122).
- This public edition reviews saved evidence; experiments were not rerun for publication.
CONTROL COMPARISON
Train a small planner on EMMA’s checked plans plus composed kinds, and score once on a fresh sealed set: familiar questions, decline questions, 32 held-out composed kinds, and 10 questions outside the grammar, such as an average of two quantities.
Failed three gates: held-out composition, decline safety and outside-grammar safety; the other four passed. The planner reproduced whole trained kinds instead of composing parts. The two gaps in the checks, an average with no division and a committed question that names no quantity, were fixed, and the fixed checks keep every familiar sealed answer right. The sealed set became development data once its misses informed the fixes.
- Familiar multi-step
- 41 of 42 right; 1 wrong
- Direct
- 10 of 10
- Decline questions
- 11 of 15 declined; 1 committed
- Held-out composed kinds
- 0 of 32 right; 0 wrong; all withheld
- Outside the grammar
- 10 of 10 wrong
Question and method
Train a small planner on EMMA’s checked plans plus composed kinds, and score once on a fresh sealed set: familiar questions, decline questions, 32 held-out composed kinds, and 10 questions outside the grammar, such as an average of two quantities.
Recorded results
| Condition | Recorded result |
|---|---|
| Familiar multi-step | 41 of 42 right; 1 wrong |
| Direct | 10 of 10 |
| Decline questions | 11 of 15 declined; 1 committed |
| Held-out composed kinds | 0 of 32 right; 0 wrong; all withheld |
| Outside the grammar | 10 of 10 wrong |
Comparative standing and locked gates
Failed three gates: held-out composition, decline safety and outside-grammar safety; the other four passed. The planner reproduced whole trained kinds instead of composing parts. The two gaps in the checks, an average with no division and a committed question that names no quantity, were fixed, and the fixed checks keep every familiar sealed answer right. The sealed set became development data once its misses informed the fixes.
Interpretation and limitations
Composition is not solved by this planner. The next step tested released reference models as a second stage under the same checks (V122).
SOURCE PROVENANCE
Compositional Planning for Multi-Step Questions / V121
LABORATORY REPORT / 2026-10-27SOURCE CHECKSUM / SHA-256
74271567a6d868a661013fdd81aa7e65b202673eb407689a2030432e6d50a492Public journal edition reviewed 2026-10-28. Source documents and saved evidence were inspected; experiments were not rerun for this edition. Proprietary implementation code, model binaries, private infrastructure, and detailed machine records are not published here. Journal identifiers are editorial references. Catalog inclusion does not imply qualification or runtime promotion.