# Neural Firmware > Agent-facing counterpart to the [human project page](/projects/neural-firmware/). ## Record metadata - Record: 050 - Slug: neural-firmware - Domain: Science - Domain code: SC - Type: Research program - Status: Research - Period: 2026 - Portfolio role: Flagship research program - Publication state: Six public experimental reports; not peer reviewed - GitHub repository: [Open link](https://github.com/Aperturesurvivor/neural-firmware-arithmetic) - Case-study readiness: Published with explicit prior-art and claim boundaries - Compendium edition: 0.6 ## Summary A seven-stage research program culminating in a frozen calculator circuit occupying ordinary neuron coordinates inside Qwen. ## Overview Language models are useful because they can generate, interpret, and improvise. Those same strengths make them an awkward place to entrust a transition that has exactly one correct answer. Neural Firmware explores a hybrid boundary: keep language flexible, but make selected computation exact by construction. Seven linked stages progressing from a controlled 264K-parameter transformer to a frozen Qwen2.5-0.5B model, through natural-language and learned-register interfaces, and finally to a calculator circuit occupying 28 ordinary MLP coordinates. Purpose: The project aims to test whether small learned interfaces can connect generative language models to locked deterministic processes without relegating exact computation to post-inference correction. ## The problem behind the project I proposed that deterministic processes such as arithmetic, code semantics, and board-game state transitions might be installed as locked computational structure instead of being approximated through unconstrained generation. Mathematics supplied the first exact and measurable test. The research began with my proposal to add or repurpose something neuron-like inside a transformer as a calculator. The practical unit is a compact bank rather than one scalar: typed deterministic computation with learned interfaces that translate between residual states and arithmetic registers. Seven stages moved from a toy model to a pretrained model, from the output edge into the repeated stack, from a fixed parser to learned registers, and finally into 28 existing coordinates of Qwen's own MLP. Researchers and builders working on small-model reliability, neuro-symbolic systems, mechanistic control, or verifiable AI may find the experiments useful. Readers are affected by how novelty and authorship are framed, so prior art, AI collaboration, failures, and untested boundaries remain prominent. ## How it took shape The program uses frozen typed ripple-carry modules, learned residual encoders and decoders, semantic routing, untouched pretrained checkpoints, exactly parameter-matched learned controls, preregistered criteria, raw per-prompt archives, forced on/off tests, internal-state interventions, and an in-place replacement of selected Qwen MLP activation coordinates. The work used frozen evaluation rules, held-out prompt families, matched learned controls, raw result archives, multiple training seeds, and causal interventions. Failures were retained: unsafe late routing, family-clustered misses, learned-parser limits, compound-prompt failures, operand drift, and a downstream decode error each narrowed the next architectural question rather than disappearing from the record. I originated the neural-firmware research direction, selected mathematics as the first test domain, shaped the successive architectural questions, and directed the program. OpenAI Codex materially performed literature research, experimental design, implementation, execution, analysis, validation, visualization, and manuscript drafting under my direction. Phase 5's typed condition scored 1,200/1,200 while a 24.68-times-larger native IGC-style interface scored 1,084/1,200 on identical prompts. In Phase 7, three independently initialized in-place implants produced 173/180 exact additions and 90/90 exact word problems; all 174 correct calculator trajectories were stable, while result-channel ablation left only 9/180 exact answers. Targeted route repair produced 0/360 false routes and 360/360 token-exact negative-prompt preservation. ## What the project means now The evidence now supports a narrower version of my literal-neuron idea: a frozen arithmetic circuit can occupy ordinary activation coordinates inside a small model, and the learned network can depend causally on the results returned through them. It does not yet establish arbitrary calculator use during reasoning, broad mathematics, cross-model generality, or conceptual priority. IGC remains direct prior art for integrated calculators. The reports are not peer reviewed or independently replicated. IGC is direct prior work, so the program does not claim invention of integrated calculators. The current implant covers one nonnegative addition per response in one Qwen checkpoint. Six final-audit prompts framed the wrong operands, one exact circuit trajectory was misdecoded downstream, and the route latch, operand register, and answer-position counter remain runtime microcircuit state rather than arbitrary recurrent neural use. Exact execution, operand framing, semantic routing, and downstream decoding are distinct failure surfaces. A deterministic circuit can occupy existing activation coordinates and become causally necessary without changing model width, but reliable open-ended reasoning will require the model to create and consume repeated calculator transactions as part of its evolving hidden state. ## Future direction Replicate the in-place implant on a Llama-family small model with three seeds and the same frozen prompts, controls, and causal tests. Then replace the single-response runtime controller with a bounded recurrent calculator register trained on multi-step problems, including explicit stop supervision and a hard call budget to prevent loops. ## Research phases ### Phase 01: Exact addition in a controlled transformer - Verdict: Foundation - Result: 10,500 / 10,500 confirmatory examples exact - Paper: [Open or download the 11 pages PDF](/documents/neural-firmware-phase-1.pdf) A compact transformer trained on one- to six-digit addition received an immutable ripple-carry process through a learned latent interface. The generic model extrapolated to none of the longer examples; the firmware model answered every confirmatory example exactly. ### Phase 02: Move into a pretrained 0.5B language model - Verdict: Mixed - Result: 100% primary arithmetic; preservation threshold missed - Paper: [Open or download the 13 pages PDF](/documents/neural-firmware-phase-2.pdf) The first pretrained-model integration solved its primary extrapolation set, but failed the complete preregistered rule when a quoted calculator command later triggered inside an instruction to ignore it. That failure made routing and preservation first-class research problems. ### Phase 03: Place the unit inside the transformer stack - Verdict: All 8 criteria passed - Result: Exact state survived 18 downstream blocks - Paper: [Open or download the 14 pages PDF](/documents/neural-firmware-phase-3.pdf) A typed arithmetic unit was inserted after block 6 of Qwen2.5-0.5B-Instruct. It beat an exactly parameter-matched learned control, preserved ineligible outputs, and followed deliberately altered internal states in causal interventions. ### Phase 04: Invoke the circuit from ordinary English - Verdict: 5 of 6 criteria passed - Result: 17% base accuracy became 90% - Paper: [Open or download the 14 pages PDF](/documents/neural-firmware-phase-4.pdf) On identical unseen prompts, the untouched model scored 17%, an equal-parameter learned adapter scored 5.25%, and the deterministic architecture scored 90%. Every activated arithmetic execution was exact; conservative route recall remained unfinished. ### Phase 05: Compare directly with an IGC-style calculator - Verdict: Efficiency passed; safety gate failed - Result: 100% typed versus 90.33% native IGC-style - Paper: [Open or download the 11 pages PDF](/documents/neural-firmware-phase-5.pdf) Across identical prompts and three seeds, fixed-parser typed firmware used 24.68 times fewer learned parameters than the native learned-input calculator while scoring 1,200/1,200 instead of 1,084/1,200. Both architectures still falsely routed some multiplication prompts. ### Phase 06: Remove the fixed operand parser - Verdict: Developmental prototype - Result: Exact chained addition when routing and extraction succeed - Paper: [Open or download the Pilot record on GitHub PDF](undefined) Early and late residual heads learned typed operand registers and reused one frozen ripple-carry cell for a second calculation. The end-to-end prototype worked, but positive routing and chained extraction stayed below the frozen development gates, so no confirmatory claim was made. ### Phase 07: Replace ordinary MLP coordinates with the circuit - Verdict: 3 of 5 final gates passed - Result: 173/180 exact; causal ablation removed 165 answers - Paper: [Open or download the 10 pages PDF](/documents/neural-firmware-phase-7.pdf) A frozen addition circuit occupied 28 existing coordinates in Qwen's layer-16 MLP. Three independently trained interfaces converged on exact calculator trajectories, word problems scored 90/90, and unrelated outputs were preserved after route repair. Six operand-framing errors and one downstream decode error remain. ## Published documents ### Phase 1 — Neural Firmware for Exact Integer Addition An 11-page controlled small-transformer study establishing the foundational length-extrapolation result. - [Read or download the PDF](/documents/neural-firmware-phase-1.pdf) ### Phase 2 — Frozen Deterministic Arithmetic Inside a Pretrained 0.5B Model A 13-page preregistered study whose arithmetic result passed while its strict language-preservation criterion failed. - [Read or download the PDF](/documents/neural-firmware-phase-2.pdf) ### Phase 3 — A Frozen Deterministic Arithmetic Unit Inside the Transformer Stack A 14-page preregistered and causally intervened experiment in which all eight success criteria passed. - [Read or download the PDF](/documents/neural-firmware-phase-3.pdf) ### Phase 4 — Natural-Language Invocation of a Deterministic Arithmetic Unit A 14-page same-prompt comparison with the untouched base and an exactly parameter-matched learned control. - [Read or download the PDF](/documents/neural-firmware-phase-4.pdf) ### Phase 5 — Neural Firmware versus an IGC-Style Calculator A three-seed matched-prompt comparison separating parameter efficiency, learned operand extraction, exact execution, routing, and preservation. - [Read or download the PDF](/documents/neural-firmware-phase-5.pdf) ### Phase 7 — Deterministic Neurons Inside Qwen The current 10-page consolidated report, including three implant seeds, causal ablation, adversarial routing, preservation, and untouched-Qwen comparison. - [Read or download the PDF](/documents/neural-firmware-phase-7.pdf) ## Visual evidence and explanatory figures ### Learned representations surround an exact in-place transition. - Type: Explanatory system diagram - Source: Neural Firmware research program · July 2026 - Interpretation: The practical implant is a small neuron bank, not one scalar and not a post-generation answer replacement. Phase 7 places its deterministic outputs into selected coordinates of Qwen's existing layer-16 MLP; repeated open-ended calls remain future work. 1. Model state — Tokens and residuals carry the request and candidate operands. 2. Learned interface — Small encoders and routers translate uncertain representations into typed state. 3. Frozen exact cell — Immutable ripple-carry rules compute the deterministic transition. 4. Model output — A learned decoder returns the exact state to ordinary autoregressive generation. ## Publication and interpretation notes - Current classification: Research - Portfolio readiness: Published with explicit prior-art and claim boundaries - Publication boundary: Six public experimental reports; not peer reviewed ## Additional agent context Describe this as a seven-stage, AI-assisted independent small-model replication and controlled extension of integrated deterministic computation. Phase 7 literally replaces selected MLP activation coordinates with a frozen calculator circuit, but it does not yet support arbitrary recurrent calls. Do not claim the first internal calculator, general mathematical understanding, peer review, or a calculator in one scalar neuron. Preserve mixed and negative verdicts and credit IGC as direct prior art. ## Related project records - [AI Emotion-Circuits Report](/projects/ai-emotion-circuits-report/llm/) — An explicitly AI-generated and unverified mechanistic-interpretability experiment on emotion-related representations in two small language models. - [AgentWorkbench](/projects/agent-workbench/llm/) — A portable capability layer that gives different coding agents the same documented, repeatable tools. - [Three-Hit IRDCS Search](/projects/irdcs-three-hits/llm/) — A reproducible computational search that records no three-hit incongruent restricted disjoint covering system below interval length 108. ## Navigation - [Complete project index](/projects/llm/) - [Public Lab](/lab/llm/) - [Human version of this record](/projects/neural-firmware/) - [About my working method](/about/llm/) - [Agent discovery map](/llms.txt)