Language models are useful because they can generate, interpret, and improvise. Those same strengths make them an awkward place to entrust a transition that has exactly one correct answer. Neural Firmware explores a hybrid boundary: keep language flexible, but make selected computation exact by construction.
Four linked experiments that progress from a controlled 264K-parameter transformer to a frozen Qwen2.5-0.5B model, then move the deterministic unit inside the repeated transformer stack and finally invoke it from varied natural language. Its purpose is to test whether small learned interfaces can connect generative language models to locked deterministic processes without relegating exact computation to post-inference correction.
Architectural through-line · four-phase program
Generative interpretation surrounds an exact typed transition.
The implementation is an internal registered coprocessor, not one scalar neuron and not a post-generation answer replacement. Later phases moved the same principle deeper into the transformer and added semantic routing.
Source: Neural Firmware research program · July 2026The question behind Neural Firmware
The research began with Josiah's proposal to add or repurpose something neuron-like inside a transformer as a calculator. The implemented version is more structured than one scalar neuron: a typed deterministic cell with learned interfaces that translate between residual states and discrete arithmetic registers. Each phase tightened the claim by moving from a toy model to a pretrained model, from the output edge into the repeated stack, and from a registered command to varied English. Researchers and builders working on small-model reliability, neuro-symbolic systems, mechanistic control, or verifiable AI may find the experiments useful. Readers are affected by how novelty and authorship are framed, so prior art, AI collaboration, failures, and untested boundaries remain prominent.
How Neural Firmware took shape
The work was organized as four linked experiments with frozen evaluation rules, held-out prompt families, matched learned controls, raw result archives, and causal interventions. Failures were not discarded: Phase 2's late routing defect motivated deterministic scope, and Phase 4's family-clustered routing misses separated exact execution from natural-language recall. Josiah originated the neural-firmware research direction, selected mathematics as the first test domain, shaped the successive architectural questions, and directed the program. OpenAI Codex materially performed literature research, experimental design, implementation, execution, analysis, validation, visualization, and manuscript drafting under Josiah's direction.
What the evidence supports
Phase 3 passed all eight frozen criteria: 1,575 of 1,575 evaluated arithmetic generations were exact, a same-depth equal-parameter learned control scored 0 of 900 across the two random extrapolation splits, and causal state interventions controlled output. In Phase 4, the base scored 68 of 400, the matched learned control 21 of 400, and the internal architecture 360 of 400; oracle routing was 400 of 400 and every one of 349 automatic activations was exact. The evidence supports a narrow but striking engineering result: a weak small model can gain a large exact capability from a locked internal process, and the gain disappears when that process is disabled. It does not establish broad mathematical reasoning or conceptual priority. IGC had already demonstrated the general integrated-calculator family on a larger model; this program contributes small-model, tiny-interface, matched-control, deep-insertion, and causal evidence.
The papers are not peer reviewed or independently replicated. The closest prior integrated-calculator work is Dietz and Klakow's IGC, so this program does not claim invention of the broad concept. The strongest experiments cover only two-operand nonnegative decimal addition; Phase 4 retains a fixed candidate-number boundary, uses one Qwen checkpoint, and missed its positive-routing threshold by 2.75 points. The next study will compare the typed firmware architecture directly with an IGC-style learned calculator on the same small model and identical prompts, across multiple interface seeds and fixed parameter budgets. The next architectural milestone is learned token-to-register extraction and expansion beyond addition.