Tiny Recursive Model (TRM): A Tiny 7M Model that Surpass DeepSeek-R1, Gemini 2.5 pro, and o3-mini at Reasoning on both ARG-AGI 1 and ARC-AGI 2

Tiny Recursive Model (TRM): A Tiny 7M Model that Surpass DeepSeek-R1, Gemini 2.5 pro, and o3-mini at Reasoning on both ARG-AGI 1 and ARC-AGI 2

Estimated reading time: 6 minutes

Key Takeaways

• TRM is a groundbreaking ~7M-parameter, 2-layer recursive solver designed for efficient reasoning tasks, challenging the notion that bigger models are always better.
• It employs an iterative “think” then “act” refinement mechanism, unrolled up to 16 steps with deep supervision, and critically, full backpropagation through the recursive loop for effective learning.
• TRM achieves impressive performance on ARC-AGI-1 (~44.6–45%) and ARC-AGI-2 (~7.8–8%), significantly outperforming much larger LLMs like DeepSeek-R1, o3-mini-high, and Gemini 2.5 Pro.
• Its success highlights that strategic compute allocation and recursive refinement, particularly during test-time inference, can surpass brute-force parameter scaling for symbolic-geometric problems.
• The model offers a compact, from-scratch solution with publicly available code, demonstrating a paradigm shift towards more sustainable and deployable AI architectural design.

Table of Contents

• Tiny Recursive Model (TRM): A Tiny 7M Model that Surpass DeepSeek-R1, Gemini 2.5 pro, and o3-mini at Reasoning on both ARG-AGI 1 and ARC-AGI 2
• The Dawn of Recursive Reasoning: What Makes TRM Unique?
• Unpacking TRM’s Remarkable Performance
• The Secret Sauce: Why TRM Outperforms Giants
• Actionable Steps for Innovators
• Real-World Impact: Enhancing Automated Design
• Conclusion
• FAQ

Tiny Recursive Model (TRM): A Tiny 7M Model that Surpass DeepSeek-R1, Gemini 2.5 pro, and o3-mini at Reasoning on both ARG-AGI 1 and ARC-AGI 2

In the rapidly evolving landscape of artificial intelligence, the conventional wisdom often dictates that bigger models yield better results. Larger parameter counts, vast datasets, and expansive architectures are typically seen as prerequisites for achieving state-of-the-art performance, especially in complex reasoning tasks. However, a recent development challenges this paradigm, demonstrating that strategic architectural choices and efficient compute allocation can empower smaller models to punch far above their weight.

Enter the Tiny Recursive Model (TRM), a groundbreaking innovation that redefines what’s possible with compact neural networks. This lean, 7-million-parameter model has not only matched but surpassed the reasoning capabilities of vastly larger counterparts, including industry giants like DeepSeek-R1, Gemini 2.5 Pro, and o3-mini, on critical benchmarks like ARC-AGI 1 and ARC-AGI 2.

“Can an iterative draft–revise solver that repeatedly updates a latent scratchpad outperform far larger autoregressive LLMs on ARC-AGI? Samsung SAIT (Montreal) has released Tiny Recursive Model (TRM)—a two-layer, ~7M-parameter recursive reasoner that reports 44.6–45% test accuracy on ARC-AGI-1 and 7.8–8% on ARC-AGI-2, surpassing results reported for substantially larger language models such as DeepSeek-R1, o3-mini-high, and Gemini 2.5 Pro on the same public evaluations. TRM also improves puzzle benchmarks Sudoku-Extreme (87.4%) and Maze-Hard (85.3%) over the prior Hierarchical Reasoning Model (HRM, 27M params), while using far fewer parameters and a simpler training recipe.”

The Dawn of Recursive Reasoning: What Makes TRM Unique?

TRM’s success isn’t just about reducing model size; it’s about a fundamental shift in how reasoning is approached. Unlike traditional autoregressive models that generate output token by token, TRM employs a recursive, iterative draft-and-revise mechanism. This novel approach allows the model to refine its solutions through repeated internal consistency checks, a stark contrast to the fixed, sequential generation of larger LLMs.

The core innovations that set TRM apart are:

• Single Tiny Recurrent Core: TRM discards the two-module hierarchy found in its predecessor, HRM, in favor of a singular, two-layer network. This compact core jointly manages a latent “scratchpad” (z) and a current solution embedding (y). The model alternates between a “think” phase, where z is updated via f(x,y,z) for several inner steps, and an “act” phase, where y is refined using g(y,z). This elegant loop allows for dynamic, internal deliberation.
• Deeply Supervised Recursion: The think→act block is not a static process. It’s unrolled up to 16 times during training, benefiting from deep supervision. A learned halting head is employed during this phase, guiding the model to optimal stopping points. At test time, the full unroll is utilized, maximizing the model’s iterative refinement capabilities. Signals are consistently carried across these recursive steps via the (y,z) embeddings.
• Full Backprop Through the Loop: A critical differentiator is TRM’s ability to backpropagate gradients through all recursive steps. This contrasts sharply with HRM’s one-step implicit (fixed-point) gradient approximation. The research team found this full backpropagation “essential for generalization,” allowing the model to learn much more effectively from its iterative refinements.

Architecturally, TRM is also adaptable. For ARC and Maze tasks, it retains self-attention. However, for Sudoku’s constrained grid structures, the team ingeniously swaps self-attention for an MLP-Mixer-style token mixer, demonstrating a tailored approach to problem types. Effective depth in TRM isn’t built by stacking layers, but rather through recursion (e.g., T = 3, n = 6), a method shown to yield better generalization at equivalent computational cost.

Unpacking TRM’s Remarkable Performance

The numbers speak for themselves, clearly illustrating TRM’s superior performance in specific reasoning domains, especially when compared against much larger, general-purpose LLMs:

• ARC-AGI-1 / ARC-AGI-2 (two tries): TRM-Attn (7M) achieves 44.6% on ARC-AGI-1 and 7.8% on ARC-AGI-2. This significantly outperforms HRM (27M) which scored 40.3% / 5.0%. Even more strikingly, it surpasses leading LLM baselines: DeepSeek-R1 (671B) at 15.8% / 1.3%, o3-mini-high at 34.5% / 3.0%, and Gemini 2.5 Pro at 37.0% / 4.9%. While larger bespoke Grok-4 entries are higher (66.7–79.6% / 16–29.4%), TRM’s performance from such a small footprint is unprecedented.
• Sudoku-Extreme (9×9, 1K train / 423K test): TRM, utilizing its attention-free mixer, achieves an impressive 87.4% accuracy, a substantial leap over HRM’s 55.0%.
• Maze-Hard (30×30): TRM records 85.3% accuracy, besting HRM’s 74.5%.

It’s crucial to understand that these results are not from few-shot prompting but from direct-prediction models trained from scratch on relatively small, heavily augmented datasets. This highlights TRM’s inherent learning efficiency and architectural strengths. The ARC-AGI challenges, with a grand-prize threshold at 85% on the private set of ARC-AGI-2, remain a formidable target, and TRM marks a significant step forward in the pursuit of general artificial intelligence for such tasks.

The Secret Sauce: Why TRM Outperforms Giants

The ability of a mere 7M parameter model to eclipse models orders of magnitude larger stems from several ingenious design choices that optimize for reasoning over sheer scale:

• Decision-then-Revision Instead of Token-by-Token: Unlike autoregressive decoding, which can suffer from “exposure bias” when generating structured outputs, TRM drafts a complete candidate solution. It then iteratively refines this draft through latent consistency checks against the input. This ‘think-and-then-act’ cycle allows for robust self-correction and reduces the compounding errors often seen in sequential generation.
• Compute Spent on Test-Time Reasoning, Not Parameter Count: TRM’s effective depth isn’t a function of its physical layers but of its recursive unrolling. By dedicating computational resources to iterative refinement during inference (emulated depth ≈ T·(n+1)·layers), the model achieves better generalization. The research shows this approach yields superior performance at a constant compute budget compared to simply adding more layers. This is a paradigm shift from focusing on large model sizes to optimizing the inference process itself.
• Tighter Inductive Bias to Grid Reasoning: For specialized tasks like Sudoku, which involve small, fixed grids, TRM’s attention-free mixing strategy reduces overcapacity. This design choice leads to improved bias/variance trade-offs, preventing the model from becoming overly complex for the problem at hand. For larger, more complex grid tasks like Maze-Hard, the model intelligently reintroduces self-attention, demonstrating architectural flexibility tailored to the problem’s needs.

Actionable Steps for Innovators

The success of TRM offers valuable lessons for researchers and developers:

1. Explore Recursive and Iterative Architectures: Don’t limit design to feed-forward or purely autoregressive structures. Investigate models that can iteratively refine solutions, especially for symbolic, geometric, or constraint-satisfaction problems. The think then act paradigm could unlock new levels of efficiency.
2. Optimize for Test-Time Compute Allocation: Shift focus from merely scaling parameter counts to strategically allocating computational effort during inference. Maximizing effective depth through unrolling or recursive steps can offer better generalization and performance with fewer parameters, leading to more sustainable and deployable AI.
3. Tailor Inductive Biases to Problem Domains: General-purpose models are powerful, but specialized architectures with tighter inductive biases can excel in specific areas. Experiment with different token mixers, attention mechanisms, or even attention-free designs that align perfectly with the structure of your data and task.

Real-World Impact: Enhancing Automated Design

Consider the field of automated engineering design or verification. Instead of generating a design component by component, a TRM-like approach could first draft a complete preliminary design. Then, through iterative “think” cycles, it could simulate and analyze the design’s structural integrity, material compatibility, or energy efficiency. The “act” phase would then refine the design based on these internal consistency checks, leading to optimized and verified blueprints, significantly reducing human error and development time for complex systems like microchip layouts or robotic assembly sequences.

Key Takeaways: TRM stands as a testament to intelligent architectural design. It’s a ~7M-parameter, 2-layer recursive solver that alternates latent “think” updates and an “act” refinement, unrolled up to 16 steps with deep supervision. Its ability to backpropagate through the full recursion, rather than relying on approximations, is critical. The model reports ~44.6–45% on ARC-AGI-1 and ~7.8–8% on ARC-AGI-2, outperforming several much larger LLMs. This demonstrates that allocating test-time compute to recursive refinement can outperform brute-force parameter scaling on symbolic-geometric tasks, offering a compact, from-scratch recipe with publicly released code.

Conclusion

The Tiny Recursive Model (TRM) represents a pivotal moment in AI research, proving that efficiency and elegant design can often triumph over sheer scale. While the pursuit of ever-larger language models continues, TRM shines a spotlight on the untapped potential of recursive reasoning, efficient compute allocation, and domain-specific architectural choices. It showcases that for tasks requiring structured, iterative problem-solving, a small, cleverly designed model can indeed outperform much larger, more general-purpose counterparts.

As the editorial comments aptly put it, this research demonstrates a ~7M-parameter, two-layer recursive solver that unrolls up to 16 draft-revise cycles with ~6 latent updates per cycle and reports ~45% on ARC-AGI-1 and ~8% (two-try) on ARC-AGI-2. The research team released code on GitHub. ARC-AGI remains unsolved at scale (target 85% on ARC-AGI-2), so the contribution is an architectural efficiency result rather than a general reasoning breakthrough. Nevertheless, TRM is a powerful reminder that innovation often lies in rethinking fundamental approaches, not just in expanding existing ones.

FAQ