Meet ZAYA1-8B, a super efficient, open reasoning model trained on AMD Instinct MI300 GPUs

Even as leading AI providers like OpenAI and Anthropic battle over the compute to train and release ever larger, more powerful models, other labs are going in a different direction — pursuing the development of smaller, more efficient models and often open sourcing them.

The latest worth paying attention to comes from the lesser-known Palo Alto startup Zyphra, which this week released its new reasoning, mixture-of-experts (MoE) language model, ZAYA1-8B, with just over 8 billion parameters and only 760 million active — far fewer than the trillions estimated for the likes of the big labs. Yet, ZAYA1-8B retains competitive performance on third-party benchmarks against GPT-5-High and DeepSeek-V3.2.

It can be downloaded from Hugging Face now free of charge under a permissive, standard, enterprise-friendly Apache 2.0 license — and enterprises and indie developers can begin using and customizing it immediately to suit their needs. Individual users can also test it themselves here free at Zyphra Cloud, the startup’s inference solution.

But the real headline is what ZAYA1-8B was trained on: a full stack of AMD Instinct MI300 graphics processing units (GPUs), the rival to Nvidia GPUs released by AMD nearly three years ago, and which shows that this platform is capable of producing useful models and is a viable alternative to the preferential position Nvidia has maintained in recent years among AI model developers.

How ZAYA1-8B was trained

The «intelligence density» touted by Zyphra is the result of what they describe as a «full-stack innovation» approach, spanning architecture, pretraining, and reinforcement learning (RL).

ZAYA1-8B is built on Zyphra’s proprietary MoE++ architecture, described in a technical report released by the lab. This architecture introduces three fundamental changes to the standard Transformer architecture that gave rise to large language models (LLMs) and the entire generative AI era:

1. Compressed Convolutional Attention (CCA): Unlike standard attention mechanisms that struggle with memory as context windows grow, CCA performs sequence mixing in a compressed latent space. This results in an 8x reduction in KV-cache size compared to full multi-head attention, enabling more efficient long-context reasoning.

2. The ZAYA1 MLP Router: Most MoE models use a linear router to decide which «experts» handle a specific token. Zyphra replaced this with a more expressive multi-layer MLP-based design. To maintain stability during training—a common hurdle for MoEs—they implemented a bias-balancing scheme inspired by PID controllers from classical control theory.

3. Learned Residual Scaling: This controls the growth of the «residual norm» as data flows deeper into the model’s 40 layers, preventing gradient vanishing or explosion with negligible computational overhead.

Reasoning-First Pretraining

A critical differentiator for ZAYA1-8B is that reasoning was integrated from the start of pretraining, rather than being «bolted on» during post-training.

To handle long chain-of-thought (CoT) traces that would otherwise exceed the initial 4K pretraining context, Zyphra developed Answer-Preserving (AP) Trimming.

Think of AP-trimming like a film editor cutting a long scene: instead of cutting the ending (the solution) or dropping the scene entirely, the editor removes the «middle» of the character’s monologue while keeping the beginning (the problem setup) and the final reveal (the answer).

This ensures the model learns the relationship between complex problems and their solutions even when the full internal logic doesn’t yet fit into memory.

It seemed to work well on my test query about countertop stain removal to ZAYA1-8B running on Zyphra Cloud.

Markovian RSA: redefining test-time compute

The model’s most significant performance leap comes from Markovian RSA, a novel test-time compute (TTC) methodology.

Traditionally, if you want a model to «think harder,» you let it generate a longer chain of thought. However, this often leads to «context bloat,» where the model loses focus as the history grows too long.

Markovian RSA solves this by decoupling «thinking depth» from «context size». It functions like a recursive scientific peer-review process:

• The model generates multiple parallel reasoning traces (candidates).

• It then extracts only the «tails» (the last few thousand tokens) of these traces.

• These tails are subsampled and presented to the model in a new «aggregation prompt,» asking it to reconcile the different approaches into a better solution.

By carrying forward only the tails (typically a 4K-token budget), the model can reason indefinitely without the context window ever overflowing. In practice, this allows the 700M active parameter ZAYA1-8B to achieve a 91.9% score on AIME ’25, closing the gap with models that have 30 to 50 times its active parameter count.

Because ZAYA1-8B maintains a small total parameter footprint (8.4B), it is uniquely positioned for on-device deployment and local LLM applications. For enterprises, this enables the deployment of high-tier reasoning capabilities—traditionally reserved for massive cloud-based models—directly onto local hardware or edge devices. This «local-first» reasoning approach addresses common enterprise hurdles regarding data residency, latency, and the high cost of persistent API dependencies.

Benchmarks show a remarkably performant small model that punches above its weight class

Zyphra is positioning ZAYA1-8B as a «punch above its weight» model for developers who need high-tier reasoning without the latency or cost of massive frontier models. After all, its active parameter count is much lower than other similarly-sized models, making it much cheaper and less compute-intensive to run in inference.

• Instruction Following: ZAYA1-8B scores 85.58 on IFEval, remaining competitive with much larger models like Intellect-3 (106B).

• Agentic Capabilities: On the τ² benchmark, the model reaches 43.12, and 39.22 on BFCL-v4, providing a baseline for its ability to handle tool-calling and multi-turn tasks.

In single-rollout evaluations (without the extra «thinking» time), ZAYA1-8B already outperforms its weight class. It beats Qwen3.5-4B and Gemma-4-E4B on math and code benchmarks.

When Markovian RSA is enabled, the results are startling:

• HMMT ’25 (Math): ZAYA1-8B hits 89.6%, surpassing Claude 4.5 Sonnet (79.2%) and GPT-5-High (88.3%).

• LiveCodeBench (Coding): The model achieves 69.2%, outperforming DeepSeek-R1-0528.

Zyphra notes that while the model is a specialist in algorithmic reasoning, it lags slightly behind larger models on «knowledge-heavy» tasks like broad factual retrieval (MMLU-Pro), which suggests that while reasoning can be compressed into smaller cores, factual memory still benefits from raw parameter count.

Apache 2.0 open licensed for research and commercial usage

Zyphra has released ZAYA1-8B under the Apache-2.0 license. This is a critical choice for the developer community. Unlike «copyleft» licenses like the GPL, which require any derived work to also be open-source, Apache-2.0 is highly permissive.

For developers and enterprises, this means they can use, modify, and distribute ZAYA1-8B—even within proprietary, commercial applications—without being forced to open-source their own codebases.

It also includes an explicit grant of patent rights from contributors, providing a layer of legal safety for startups building on top of Zyphra’s architecture. By opting for Apache-2.0 over more restrictive «research-only» licenses often seen from frontier labs, Zyphra is signaling a commitment to the open-weight ecosystem.

To deploy ZAYA1-8B, developers must use specific branches from Zyphra’s forks of core libraries, as the architecture requires specialized handling:

• Custom Forks: Users should install the zaya1 branch from Zyphra’s versions of the vllm and transformers libraries.

• Deployment Flags: When starting a vLLM server, specific flags are required to handle the reasoning parser and tool-calling (e.g., --reasoning-parser qwen3 and --tool-call-parser zaya_xml).

• Parallelism Strategy: For multi-GPU environments, Zyphra recommends using Data Parallelism (DP) combined with Expert Parallelism (EP). Notably, Tensor Parallelism (TP) for the model’s CCA mechanism is not currently supported, making DP+EP the optimal path for scaling inference throughput.

Background on Zyphra

Zyphra: A New Paradigm for Intelligence Density

Founded in 2021 and headquartered in Palo Alto, California, Zyphra Technologies is a full-stack artificial intelligence laboratory dedicated to building human-aligned artificial general intelligence (AGI) — that which outperforms people at most tasks — through a decentralized, open-source framework.

According to the company’s official mission statement, Zyphra seeks to challenge the «centralized» dominance of monolithic cloud models by focusing on «intelligence density»—a core guiding principle that aims to maximize the reasoning and logic extracted per parameter and per FLOP.

Zyphra CEO and Co-Founder Krithik Puthalath explained previously to VentureBeat that this strategy is essential for enabling high-performance AI to run locally on hardware such as tablets, wearable glasses, and enterprise servers, thereby ensuring user privacy and reducing reliance on third-party cloud infrastructure.

The company’s technical identity is deeply informed by computational neuroscience, led by Co-Founder and Chief Scientist Beren Millidge.

According to Millidge’s personal website, he currently serves as a Postdoctoral Researcher at the University of Oxford’s Nuffield Department of Clinical Neurosciences, where his research focuses on deep credit assignment and mathematical models of the brain.

Millidge, who earned his PhD from the University of Edinburgh, has pioneered research into active inference and the «free-energy principle,» concepts that directly influence Zyphra’s pursuit of multimodal architectures capable of long-term memory and continual learning.

This neuroscientific influence was central to the design of Zyphra’s prior Zamba model, released in 2024, which mimics the cortex-hippocampus interaction to share information across sequential layers. A recent TED Talk video provides insight into Millidge’s perspective on the intersection of biological neuroscience and AI, which serves as the theoretical foundation for Zyphra’s model architectures.

Zyphra has achieved significant technical milestones through a deep integration with the AMD hardware ecosystem, as detailed in the company’s research documentation.

Financial data from PitchBook indicates that Zyphra is currently a venture-backed company that attained «Unicorn» status in June 2025 following a $110 million Series A funding round. According to PitchBook and company press releases, Zyphra is supported by a group of strategic investors including Advanced Micro Devices (AMD), IBM, Bison Ventures, and BC VC. With a team of approximately 31 employees as of 2026, the company continues to expand its footprint through the Zyphra Inference Cloud and Maia, an intelligent assistant platform designed to bring advanced search and productivity tools to enterprise teams.

Community reactions and industry context

The announcement has resonated strongly within the AI community, garnering nearly 1 million views on X/Twitter within 24 hours. The excitement largely centers on two factors: the viability of the AMD stack and the efficiency of the reasoning «cascade.»

Technologists have noted that Zyphra’s post-training process—a 4-stage RL cascade—is unusually disciplined. Most labs use a single round of RL, but Zyphra’s pipeline includes a «reasoning warmup» followed by a curriculum of 400 adaptive puzzle-like environments (RLVE-Gym) before finally moving to behavioral polishing.

One of the most praised «under-the-hood» details is Router Replay. In MoE models, training can become unstable if the «trainer» engine and the «inference» engine make slightly different decisions about which expert to use for a token due to floating-point noise. Zyphra’s system records the exact expert choices made during generation and forces the trainer to use them, effectively «pinning» the computation path and ensuring higher learning stability.

As the industry faces a potential plateau in the benefits of simply adding more parameters, ZAYA1-8B provides a compelling counter-narrative: that the next frontier of AI isn’t just about bigger clusters, but about smarter «thinking» algorithms that can do more with less.