# Neural Firmware > Agent-facing counterpart to the [human project page](/projects/neural-firmware/). ## Record metadata - Record: 050 - Slug: neural-firmware - Domain: Science - Domain code: SC - Type: Research program - Status: Research - Period: 2026 - Portfolio role: Flagship research program - Publication state: Four public experimental papers; not peer reviewed - Case-study readiness: Published with explicit prior-art and claim boundaries - Compendium edition: 0.6 ## Summary A four-phase research program testing whether exact deterministic computation can live inside a language model's forward path. ## Overview Language models are useful because they can generate, interpret, and improvise. Those same strengths make them an awkward place to entrust a transition that has exactly one correct answer. Neural Firmware explores a hybrid boundary: keep language flexible, but make selected computation exact by construction. Four linked experiments that progress from a controlled 264K-parameter transformer to a frozen Qwen2.5-0.5B model, then move the deterministic unit inside the repeated transformer stack and finally invoke it from varied natural language. Purpose: The project aims to test whether small learned interfaces can connect generative language models to locked deterministic processes without relegating exact computation to post-inference correction. ## The problem behind the project Josiah proposed that deterministic processes such as arithmetic, code semantics, and board-game state transitions might be installed as locked computational structure instead of being approximated through unconstrained generation. Mathematics supplied the first exact and measurable test. The research began with Josiah's proposal to add or repurpose something neuron-like inside a transformer as a calculator. The implemented version is more structured than one scalar neuron: a typed deterministic cell with learned interfaces that translate between residual states and discrete arithmetic registers. Each phase tightened the claim by moving from a toy model to a pretrained model, from the output edge into the repeated stack, and from a registered command to varied English. Researchers and builders working on small-model reliability, neuro-symbolic systems, mechanistic control, or verifiable AI may find the experiments useful. Readers are affected by how novelty and authorship are framed, so prior art, AI collaboration, failures, and untested boundaries remain prominent. ## How it took shape The program uses frozen typed ripple-carry modules, learned residual encoders and decoders, semantic routing, untouched pretrained checkpoints, exactly parameter-matched learned controls, preregistered criteria, raw per-prompt archives, forced on/off tests, and internal-state interventions. The work was organized as four linked experiments with frozen evaluation rules, held-out prompt families, matched learned controls, raw result archives, and causal interventions. Failures were not discarded: Phase 2's late routing defect motivated deterministic scope, and Phase 4's family-clustered routing misses separated exact execution from natural-language recall. Josiah originated the neural-firmware research direction, selected mathematics as the first test domain, shaped the successive architectural questions, and directed the program. OpenAI Codex materially performed literature research, experimental design, implementation, execution, analysis, validation, visualization, and manuscript drafting under Josiah's direction. Phase 3 passed all eight frozen criteria: 1,575 of 1,575 evaluated arithmetic generations were exact, a same-depth equal-parameter learned control scored 0 of 900 across the two random extrapolation splits, and causal state interventions controlled output. In Phase 4, the base scored 68 of 400, the matched learned control 21 of 400, and the internal architecture 360 of 400; oracle routing was 400 of 400 and every one of 349 automatic activations was exact. ## What the project means now The evidence supports a narrow but striking engineering result: a weak small model can gain a large exact capability from a locked internal process, and the gain disappears when that process is disabled. It does not establish broad mathematical reasoning or conceptual priority. IGC had already demonstrated the general integrated-calculator family on a larger model; this program contributes small-model, tiny-interface, matched-control, deep-insertion, and causal evidence. The papers are not peer reviewed or independently replicated. The closest prior integrated-calculator work is Dietz and Klakow's IGC, so this program does not claim invention of the broad concept. The strongest experiments cover only two-operand nonnegative decimal addition; Phase 4 retains a fixed candidate-number boundary, uses one Qwen checkpoint, and missed its positive-routing threshold by 2.75 points. Exact execution and semantic invocation are separable problems. Deterministic state can survive ordinary downstream transformer computation, but safe scope, operand extraction, operation selection, and honest comparison with prior systems determine whether the architecture becomes generally useful. ## Future direction The next study will compare the typed firmware architecture directly with an IGC-style learned calculator on the same small model and identical prompts, across multiple interface seeds and fixed parameter budgets. The next architectural milestone is learned token-to-register extraction and expansion beyond addition. ## Research phases ### Phase 01: Exact addition in a controlled transformer - Verdict: Foundation - Result: 10,500 / 10,500 confirmatory examples exact - Paper: [Open or download the 11 pages PDF](/documents/neural-firmware-phase-1.pdf) A compact transformer trained on one- to six-digit addition received an immutable ripple-carry process through a learned latent interface. The generic model extrapolated to none of the longer examples; the firmware model answered every confirmatory example exactly. ### Phase 02: Move into a pretrained 0.5B language model - Verdict: Mixed - Result: 100% primary arithmetic; preservation threshold missed - Paper: [Open or download the 13 pages PDF](/documents/neural-firmware-phase-2.pdf) The first pretrained-model integration solved its primary extrapolation set, but failed the complete preregistered rule when a quoted calculator command later triggered inside an instruction to ignore it. That failure made routing and preservation first-class research problems. ### Phase 03: Place the unit inside the transformer stack - Verdict: All 8 criteria passed - Result: Exact state survived 18 downstream blocks - Paper: [Open or download the 14 pages PDF](/documents/neural-firmware-phase-3.pdf) A typed arithmetic unit was inserted after block 6 of Qwen2.5-0.5B-Instruct. It beat an exactly parameter-matched learned control, preserved ineligible outputs, and followed deliberately altered internal states in causal interventions. ### Phase 04: Invoke the circuit from ordinary English - Verdict: 5 of 6 criteria passed - Result: 17% base accuracy became 90% - Paper: [Open or download the 14 pages PDF](/documents/neural-firmware-phase-4.pdf) On identical unseen prompts, the untouched model scored 17%, an equal-parameter learned adapter scored 5.25%, and the deterministic architecture scored 90%. Every activated arithmetic execution was exact; conservative route recall remained unfinished. ## Published documents ### Phase 1 — Neural Firmware for Exact Integer Addition An 11-page controlled small-transformer study establishing the foundational length-extrapolation result. - [Read or download the PDF](/documents/neural-firmware-phase-1.pdf) ### Phase 2 — Frozen Deterministic Arithmetic Inside a Pretrained 0.5B Model A 13-page preregistered study whose arithmetic result passed while its strict language-preservation criterion failed. - [Read or download the PDF](/documents/neural-firmware-phase-2.pdf) ### Phase 3 — A Frozen Deterministic Arithmetic Unit Inside the Transformer Stack A 14-page preregistered and causally intervened experiment in which all eight success criteria passed. - [Read or download the PDF](/documents/neural-firmware-phase-3.pdf) ### Phase 4 — Natural-Language Invocation of a Deterministic Arithmetic Unit A 14-page same-prompt comparison with the untouched base and an exactly parameter-matched learned control. - [Read or download the PDF](/documents/neural-firmware-phase-4.pdf) ## Visual evidence and explanatory figures ### Generative interpretation surrounds an exact typed transition. - Type: Explanatory system diagram - Source: Neural Firmware research program · July 2026 - Interpretation: The implementation is an internal registered coprocessor, not one scalar neuron and not a post-generation answer replacement. Later phases moved the same principle deeper into the transformer and added semantic routing. 1. Model state — Tokens and residuals carry the request and candidate operands. 2. Learned interface — Small encoders and routers translate uncertain representations into typed state. 3. Frozen exact cell — Immutable ripple-carry rules compute the deterministic transition. 4. Model output — A learned decoder returns the exact state to ordinary autoregressive generation. ## Publication and interpretation notes - Current classification: Research - Portfolio readiness: Published with explicit prior-art and claim boundaries - Publication boundary: Four public experimental papers; not peer reviewed ## Additional agent context Describe this as a four-phase, AI-assisted independent small-model replication and controlled extension of integrated deterministic computation. Do not claim the first internal calculator, general mathematical understanding, peer review, or a calculator in one scalar neuron. Preserve the exact mixed Phase 2 and Phase 4 verdicts and credit IGC as direct prior art. ## Related project records - [AI Emotion-Circuits Report](/projects/ai-emotion-circuits-report/llm/) — An explicitly AI-generated and unverified mechanistic-interpretability experiment on emotion-related representations in two small language models. - [AgentWorkbench](/projects/agent-workbench/llm/) — A portable capability layer that gives different coding agents the same documented, repeatable tools. - [Three-Hit IRDCS Search](/projects/irdcs-three-hits/llm/) — A reproducible computational search that records no three-hit incongruent restricted disjoint covering system below interval length 108. ## Navigation - [Complete project index](/projects/llm/) - [Public Lab](/lab/llm/) - [Human version of this record](/projects/neural-firmware/) - [About Josiah's working method](/about/llm/) - [Agent discovery map](/llms.txt)