Devstral 2 Review: Mistral's Open-Weight Coding Model Hits 72.2% SWE-Bench (2026)
Released December 9, 2025 · Reviewed June 14, 2026
At a Glance
On December 9, 2025, Mistral AI released Devstral 2 — a two-model open-weight coding family that immediately set the bar for open-source software engineering agents. Devstral 2 (123B) achieves 72.2% on SWE-bench Verified under a modified MIT license. Devstral Small 2 (24B) reaches 68.0% under Apache 2.0, runs on consumer hardware, and is available as open weights on Hugging Face.
Both models are currently free to use via the Mistral API. Mistral also shipped Mistral Vibe CLI, an open-source terminal agent built specifically for Devstral — bringing end-to-end code automation to the command line without needing an IDE or GUI.
What's New vs. Prior Devstral
72.2% on SWE-bench Verified — SOTA open-weight at launch
SWE-bench Verified is the standard benchmark for coding agents: models are given real GitHub issues and must produce a patch that passes the project's own test suite. Devstral 2 reached 72.2% — placing it among the best coding models available at launch, open or closed. Devstral Small 2 (24B) reached 68.0%, which puts it firmly above models five times its size including DeepSeek V3.2 and Kimi K2 when adjusted for parameter count. Both models achieve this while being 5× and 28× smaller than competing open models, respectively, making them far more practical to deploy.
Up to 7× cost efficiency vs. Claude Sonnet
Mistral ran independent human evaluations of Devstral 2 against DeepSeek V3.2 and Claude Sonnet 4.5 using an independent annotation provider, with tasks scaffolded through Cline. Devstral 2 beat DeepSeek V3.2 with a 42.8% win rate vs. 28.6% loss rate. Against Claude Sonnet 4.5, Claude still wins — but Mistral claims up to 7× cost efficiency at real-world tasks, meaning teams that can tolerate a modest quality trade-off can slash coding-agent API spend dramatically. At scale — millions of tokens per day — that differential is significant.
Mistral Vibe CLI: terminal-native code automation
Alongside Devstral 2, Mistral released Mistral Vibe — an open-source terminal CLI built specifically for Devstral. Unlike IDE-embedded agents (Copilot, Cursor, Cline), Vibe runs natively in your shell, enabling autonomous end-to-end code automation without needing a GUI. It's designed to scaffold, edit, and iterate on codebases directly from the command line. For CI/CD automation, server-side code generation, and headless coding agents, a terminal-native interface is a meaningful architectural advantage.
Devstral Small 2 runs on consumer hardware
The 24B variant ships under Apache 2.0 and is designed to run locally — on a single H100, high-end consumer GPU, or equivalent. At 68.0% SWE-bench Verified, it's a credible local coding agent for most software engineering tasks. Apache 2.0 means no commercial restrictions, no attribution requirements beyond normal open-source practice, and no data leaving your infrastructure. For teams with HIPAA/GDPR requirements or proprietary codebases they can't send to a cloud API, Devstral Small 2 is one of the strongest options at this parameter count.
Built for production-grade agentic workflows
Devstral 2 was designed for multi-step agentic coding — the kind where a model needs to explore a large codebase, understand framework dependencies, make changes across multiple files, detect when its first attempt failed, and retry with a correction. The 256K context window lets the model hold entire repository structures in context. Mistral's tooling integrations with Cline received positive feedback ("tool-calling success rate on par with the best closed models" per Cline's team), indicating reliability in real agent scaffolding — not just benchmark conditions.
Benchmark Comparison
SWE-bench Verified scores from official announcements. Pricing as of December 2025.
| Model | SWE-bench | License | Price | Local |
|---|---|---|---|---|
| Devstral 2 (123B) | 72.2% | Modified MIT | Free (API) | Large GPU |
| Devstral Small 2 (24B) | 68.0% | Apache 2.0 | Free (API) | Consumer GPU |
| Claude Sonnet 4.5 | ~70%+ | Proprietary | $3/$15 per M tokens | No |
| DeepSeek V3.2 (~671B) | ~65% | MIT | API pricing | Datacenter |
| Kimi K2 | ~65% | Modified MIT | API pricing | Datacenter |
Source: Mistral AI announcement, December 2025. Claude Sonnet SWE-bench score is approximate from Anthropic's published results. Scores reflect announced benchmarks at release; current rankings may differ.
Pricing
Devstral 2 (123B)
Free APIFree
Available via Mistral API at console.mistral.ai. Modified MIT license. 256K context. Best for production agentic coding workloads where cost efficiency matters.
Devstral Small 2 (24B)
Open WeightsFree + Self-host
Free via Mistral API. Open weights under Apache 2.0 on Hugging Face. Runs on consumer-grade GPUs. Best for local/on-prem deployments with data-privacy requirements.
Enterprise / On-prem
EnterpriseContact Mistral
Both models support custom fine-tuning and enterprise on-prem deployment. Contact Mistral AI directly for SLAs, support, and volume pricing.
Use Cases
- →High-volume agentic coding pipelines: For teams running thousands of automated coding tasks daily, Devstral 2's free API and 7× cost efficiency vs. Claude Sonnet changes the unit economics dramatically.
- →Local / on-prem code agents: Devstral Small 2's Apache 2.0 weights on consumer hardware make it the default choice for coding workloads that can't use a cloud API — proprietary codebases, air-gapped environments, HIPAA-sensitive data.
- →CI/CD and DevOps automation: Mistral Vibe CLI enables terminal-native coding agents that fit naturally into shell scripts, GitHub Actions, and other headless automation environments.
- →Fine-tuned enterprise coding assistants: Both models support custom fine-tuning, making them suitable for organizations with specialized languages, internal frameworks, or large legacy codebases that generic models struggle with.
- →Bug fixing and modernization: Devstral 2's multi-file context and failure-retry loop make it particularly well-suited for the kind of large-scale refactoring and bug-fixing tasks where a single-turn model falls short.
Frequently Asked Questions
What is Devstral 2?
Devstral 2 is Mistral AI's next-generation open-weight coding model, released December 9, 2025. It comes in two sizes: Devstral 2 (123B parameters, modified MIT license) and Devstral Small 2 (24B parameters, Apache 2.0). Both achieve state-of-the-art results on SWE-bench Verified — 72.2% and 68.0% respectively — making them the strongest open-weight coding models available at launch. Both are currently free to use via the Mistral API.
How does Devstral 2 compare to Claude Sonnet?
Claude Sonnet 4.5 is still significantly preferred over Devstral 2 in Mistral's own human evaluations. However, Mistral claims Devstral 2 is up to 7× more cost-efficient at real-world coding tasks. For teams where quality is paramount and budget is not a constraint, Claude Sonnet 4.5 remains the stronger choice. For teams running high-volume agentic pipelines where Claude's per-token cost is prohibitive, Devstral 2's combination of strong benchmark performance and free/low-cost API access makes it a compelling alternative.
Can I run Devstral locally?
Yes. Devstral Small 2 (24B) is available as open weights under Apache 2.0 on Hugging Face and is designed to run locally on consumer hardware — a single H100 or equivalent GPU. The 123B model requires more substantial hardware but can also be self-hosted. Local deployment means no data leaves your infrastructure, which is essential for HIPAA-compliant environments, proprietary codebases, or air-gapped systems.
What is Mistral Vibe CLI?
Mistral Vibe is an open-source terminal CLI released alongside Devstral 2. It's built specifically for Devstral and enables end-to-end code automation directly from the command line — without a GUI or IDE. It's designed for autonomous software engineering tasks: scaffolding, editing, debugging, and iterating on codebases. Vibe is particularly useful for CI/CD integration, server-side automation, and headless coding pipelines.
What license does Devstral 2 use?
Devstral 2 (123B) ships under a modified MIT license; Devstral Small 2 (24B) ships under Apache 2.0. Both licenses are permissive and allow commercial use without royalties. The 'modified' MIT for the 123B model may include usage terms — check the Mistral GitHub repository or license file for the current text before deploying at scale.
Is Devstral 2 good for SWE-bench?
Yes. Devstral 2 scores 72.2% on SWE-bench Verified, which placed it as the best open-weight coding model at launch (December 2025). Devstral Small 2 scores 68.0%. SWE-bench Verified tests models on real GitHub issues — patches must pass the project's existing test suite, making it a harder and more realistic evaluation than many synthetic coding benchmarks.
Verdict
Devstral 2 is the strongest open-weight coding model at its December 2025 launch, and the 7× cost efficiency claim vs. Claude Sonnet is credible enough to evaluate seriously. If you're running high-volume coding pipelines and the quality ceiling of Claude Sonnet is above what your tasks actually require, Devstral 2's free API access dramatically changes the math.
Devstral Small 2 is the standout for on-prem and local deployments: Apache 2.0, 24B parameters, consumer hardware, 68% SWE-bench. That combination doesn't exist elsewhere at this parameter count.
The honest caveat: Claude Sonnet 4.5 is still preferred in Mistral's own human evals. If your tasks are complex, ambiguous, or require frontier reasoning alongside coding ability, the quality gap is real. But for well-defined engineering tasks at scale, Devstral 2 deserves a head-to-head eval.