Language models are useful because they can generate, interpret, and improvise. Those same strengths make them an awkward place to entrust a transition that has exactly one correct answer. Neural Firmware explores a hybrid boundary: keep language flexible, but make selected computation exact by construction.
Seven linked stages progressing from a controlled 264K-parameter transformer to a frozen Qwen2.5-0.5B model, through natural-language and learned-register interfaces, and finally to a calculator circuit occupying 28 ordinary MLP coordinates. Its purpose is to test whether small learned interfaces can connect generative language models to locked deterministic processes without relegating exact computation to post-inference correction.
Architectural through-line · seven-stage program
Learned representations surround an exact in-place transition.
The practical implant is a small neuron bank, not one scalar and not a post-generation answer replacement. Phase 7 places its deterministic outputs into selected coordinates of Qwen's existing layer-16 MLP; repeated open-ended calls remain future work.
Source: Neural Firmware research program · July 2026The question behind Neural Firmware
The research began with my proposal to add or repurpose something neuron-like inside a transformer as a calculator. The practical unit is a compact bank rather than one scalar: typed deterministic computation with learned interfaces that translate between residual states and arithmetic registers. Seven stages moved from a toy model to a pretrained model, from the output edge into the repeated stack, from a fixed parser to learned registers, and finally into 28 existing coordinates of Qwen's own MLP. Researchers and builders working on small-model reliability, neuro-symbolic systems, mechanistic control, or verifiable AI may find the experiments useful. Readers are affected by how novelty and authorship are framed, so prior art, AI collaboration, failures, and untested boundaries remain prominent.
How Neural Firmware took shape
The work used frozen evaluation rules, held-out prompt families, matched learned controls, raw result archives, multiple training seeds, and causal interventions. Failures were retained: unsafe late routing, family-clustered misses, learned-parser limits, compound-prompt failures, operand drift, and a downstream decode error each narrowed the next architectural question rather than disappearing from the record. I originated the neural-firmware research direction, selected mathematics as the first test domain, shaped the successive architectural questions, and directed the program. OpenAI Codex materially performed literature research, experimental design, implementation, execution, analysis, validation, visualization, and manuscript drafting under my direction.
What the evidence supports
Phase 5's typed condition scored 1,200/1,200 while a 24.68-times-larger native IGC-style interface scored 1,084/1,200 on identical prompts. In Phase 7, three independently initialized in-place implants produced 173/180 exact additions and 90/90 exact word problems; all 174 correct calculator trajectories were stable, while result-channel ablation left only 9/180 exact answers. Targeted route repair produced 0/360 false routes and 360/360 token-exact negative-prompt preservation. The evidence now supports a narrower version of my literal-neuron idea: a frozen arithmetic circuit can occupy ordinary activation coordinates inside a small model, and the learned network can depend causally on the results returned through them. It does not yet establish arbitrary calculator use during reasoning, broad mathematics, cross-model generality, or conceptual priority. IGC remains direct prior art for integrated calculators.
The reports are not peer reviewed or independently replicated. IGC is direct prior work, so the program does not claim invention of integrated calculators. The current implant covers one nonnegative addition per response in one Qwen checkpoint. Six final-audit prompts framed the wrong operands, one exact circuit trajectory was misdecoded downstream, and the route latch, operand register, and answer-position counter remain runtime microcircuit state rather than arbitrary recurrent neural use. Replicate the in-place implant on a Llama-family small model with three seeds and the same frozen prompts, controls, and causal tests. Then replace the single-response runtime controller with a bounded recurrent calculator register trained on multi-step problems, including explicit stop supervision and a hard call budget to prevent loops.