Together AI has published a benchmark comparison suggesting that the smartest way to use two very different AI coding models isn’t to pick one it’s to use both, in sequence. The company ran DeepSeek V4 Pro 0813, an open-weight model from DeepSeek, against GPT-5.6 Sol, OpenAI’s flagship model, on a coding benchmark called DeepSWE. The individual results were unsurprising: the expensive flagship model was more accurate on a single attempt. What stood out was what happened when Together AI combined the two models into a routing strategy — a “cascade” that tries the cheap model first and only calls in the expensive one when the first attempt fails a test suite.
That cascade solved 83.0% of coding tasks at a cost of $3.35 per task. GPT-5.6 Sol alone, tested by itself, solved 72.7% at $8.37 per task. In other words, according to this test, combining the models produced better results for less than half the price of the flagship model working alone.
Table of Contents
What Together AI Tested
Together AI ran 904 total rollouts 452 attempts each for DeepSeek V4 Pro 0813 and GPT-5.6 Sol across all 113 tasks in the DeepSWE benchmark, with each task attempted four times per model. The results were published in a Together AI blog post on August 18, 2026. This is a single company’s benchmark run, using its own testing harness and settings, not an independently replicated study, and the figures throughout should be read with that in mind.
Why This Comparison Matters Now
Both models being compared are live, commercially available products, not early previews. GPT-5.6 Sol went from a limited government-reviewed preview to full general availability across ChatGPT, Codex, and the OpenAI API on July 9, 2026. DeepSeek V4 Pro 0813 is the official, non-preview release of DeepSeek’s V4 Pro model, made available across multiple inference platforms in mid-August 2026. That timing matters: this is a comparison of two models that businesses can actually deploy today, not a hypothetical matchup.
It’s also worth noting that pricing is a moving target. OpenAI cut GPT-5.6 Sol’s API pricing by more than 20% on August 21, 2026 — after Together AI’s test was run. The roughly 35-times cost gap reported between the two models reflects pricing at the time of testing and may look different now.
| Date | Event |
|---|---|
| June 26, 2026 | GPT-5.6 Sol enters a limited, government-reviewed preview |
| July 9, 2026 | GPT-5.6 family reaches full general availability |
| ~August 13, 2026 | DeepSeek V4 Pro 0813 official non-preview release becomes available |
| August 18, 2026 | Together AI publishes the DeepSWE benchmark comparison |
| August 21, 2026 | OpenAI cuts GPT-5.6 Sol API pricing by over 20% |
Timeline based on company announcements and Together AI’s benchmark publication.
The Models
DeepSeek V4 Pro 0813 is a large open-weight model 1.65 trillion total parameters, with 49 billion active at any given time — built specifically for coding and long-running agentic tasks. It includes a speculative-decoding feature called DSpark, designed to speed up inference. Because it’s open-weight, it can be run through multiple providers rather than a single company’s API.
GPT-5.6 Sol is the top tier of OpenAI’s three-model GPT-5.6 family, which also includes Terra (a mid-tier, lower-cost option) and Luna (the fastest and cheapest tier). Sol is positioned as OpenAI’s strongest model for complex reasoning, coding, and long-horizon agentic work.
The DeepSWE Benchmark
The two models were tested on DeepSWE, a benchmark built by the AI data company Datacurve and introduced in July 2026. It consists of 113 original software engineering tasks drawn from 91 real, actively maintained open-source repositories, spanning five programming languages. A key design choice is that every task’s solution is written from scratch and never merged back into the source repositories — the goal is to prevent models from having effectively “seen the answer” during training, a known weakness of older coding benchmarks built from real pull requests. Task success is verified with hand-written tests that check whether the code actually works, rather than whether it matches a specific reference implementation.
DeepSWE is a legitimate, technically documented benchmark with an open dataset and an independent tracking page on Epoch AI. It is not, however, an established industry-wide standard the way older benchmarks like SWE-bench have become — it’s a newer addition to the field, and how widely it will be adopted for evaluating coding models remains to be seen.
The Benchmark Results
On a single attempt (pass@1), GPT-5.6 Sol was clearly more accurate: 72.7% compared to DeepSeek V4 Pro 0813’s 62.8%. But that gap closed, and then reversed, as both models were given more tries. At two attempts, Sol still led slightly (81.0% to 78.5%). At four attempts, Pro pulled ahead, solving 88.5% of tasks compared to Sol’s 85.8%.
Cost per attempt told a starker story. Sol cost $8.37 per rollout in this test; DeepSeek V4 Pro 0813 cost $0.24 — a roughly 35-fold difference. Measured as value, Together AI calculated that $100 spent on Pro solved about 261 tasks, versus roughly 9 tasks for the same spend on Sol.
| Metric | DeepSeek V4 Pro 0813 | GPT-5.6 Sol |
|---|---|---|
| Pass@1 (single attempt) | 62.8% | 72.7% |
| Pass@4 (four attempts) | 88.5% | 85.8% |
| Cost per rollout | $0.24 | $8.37 |
| Solves per $100 spent | ~261 | ~9 |
| Test-regression rate on failures | 11% | 20% |
Source: Together AI benchmark, published August 18, 2026.
It’s worth flagging one small wrinkle: DeepSeek’s own model card lists a slightly different DeepSWE score for Pro (62.7% versus Together’s reported 62.8%). The difference is trivial and likely reflects separate test runs rather than any real discrepancy, but it’s a reminder that these numbers come from independently run evaluations rather than a shared, official scorecard.
The Cascade Routing Strategy
The more interesting result, according to Together AI, wasn’t either model on its own — it was combining them. The company tested a “cascade”: run DeepSeek V4 Pro 0813 first, and only escalate to GPT-5.6 Sol if the cheaper model’s output fails an automated test suite. That approach solved 83.0% of DeepSWE tasks at an average cost of $3.35 per task.
That’s about 10 percentage points more accurate than using Sol alone, at roughly 60% less than Sol’s per-task cost. Together AI also compared the cascade to a theoretical “perfect” router one that always knew in advance which model would get a given task right — and found the real cascade actually outperformed that oracle (83.0% versus 80.8%), because sending a task through two independent attempts catches more edge cases than a single “best guess” pick ever could.
The strategy depends entirely on having a way to check the cheap model’s work automatically. Without an existing test suite or another reliable verification method, there’s no way to know when to escalate to the more expensive model, and the whole approach breaks down.
Failure Modes and Reliability
Accuracy percentages don’t tell the whole story of how a model fails. Together AI found that when GPT-5.6 Sol got a task wrong, its failures were more likely to break code that was previously working a regression in 20% of cases. DeepSeek V4 Pro 0813’s failures broke existing tests less often, in 11% of cases, tending instead to simply fall short of a full solution.
That distinction matters operationally: it suggests that output from either model, but especially from Sol, benefits from being checked against a full regression suite before being accepted, rather than being reviewed only for whether it solved the immediate task.
Practical Implications for Teams
For organizations already running automated tests against their codebases, this benchmark suggests a concrete way to cut coding-agent costs without sacrificing and potentially improving accuracy: default to a cheaper model, and only pay for the expensive one when the cheap model’s work doesn’t pass verification. For teams without that kind of test infrastructure in place, the routing benefit doesn’t apply in the same way, since there’s no automated signal to trigger escalation.
It’s also important to keep the results scoped to what was actually tested. These figures come from one benchmark, one testing harness, and one specific pair of model versions. They say nothing definitive about how either model performs on tasks outside DeepSWE’s scope, how other model pairings might route, or how the economics would look on a different benchmark or in production use with real, messier codebases.
Limitations and Open Questions
Several things about this comparison remain unresolved. No independent party has yet replicated Together AI’s specific 904-rollout test, so the exact numbers should be treated as one company’s reported findings rather than an established fact. How widely DeepSWE will be adopted as a benchmark going forward is unclear it’s well-documented and technically credible, but still new. Together AI’s blog post didn’t address its own infrastructure capacity in the context of this test, so that remains outside what can be verified from the source. And because pricing on both models has already shifted since the test was conducted, the exact cost multiples reported here are a snapshot in time rather than a permanent figure.
What to Watch Next
The core evidence here is solid: both models are real, currently available products, the benchmark is credible, and the arithmetic behind the “10 points better, 60% cheaper” claim checks out against Together AI’s own data. What’s still open is whether this specific routing advantage holds up under independent testing, on different benchmarks, and as pricing for both models continues to change. For now, the clearest takeaway is narrower than “routing always wins” — it’s that a test-gated cascade between a cheap and an expensive coding model, in this one benchmark, beat using the expensive model by itself.
FAQ
Is DeepSeek V4 Pro 0813 cheaper than GPT-5.6 Sol? Yes, substantially. In Together AI’s test, DeepSeek V4 Pro 0813 cost $0.24 per rollout versus $8.37 for GPT-5.6 Sol — about 35 times cheaper at the time of testing. OpenAI has since cut Sol’s pricing by more than 20%, so the current gap may be somewhat smaller.
What is the DeepSWE benchmark and why does it matter? DeepSWE is a coding benchmark built by Datacurve, consisting of 113 original software engineering tasks across 91 real repositories and five programming languages. Its tasks are written from scratch and never merged upstream, which is intended to prevent AI models from having encountered the solutions during training.
How does the Pro-to-Sol cascade routing strategy work? DeepSeek V4 Pro 0813 attempts a task first. If its output fails an automated test suite, the task escalates to GPT-5.6 Sol for a second attempt. This reached 83.0% accuracy at $3.35 per task in Together AI’s test.
Which model performs better on a single attempt (pass@1)? GPT-5.6 Sol, with a 72.7% pass@1 rate compared to DeepSeek V4 Pro 0813’s 62.8%. Pro closes the gap and eventually overtakes Sol when given multiple attempts.
Does routing between models actually save money in practice? In this specific benchmark, yes — the cascade cost about 60% less than using GPT-5.6 Sol alone while also improving accuracy by roughly 10 percentage points. That result depends on having automated tests available to gate when the more expensive model gets called in.
Are DeepSeek V4 Pro 0813 and GPT-5.6 Sol both generally available? Yes. GPT-5.6 Sol has been generally available since July 9, 2026, and DeepSeek V4 Pro 0813’s official, non-preview release became available in mid-August 2026.
What are the risks of relying on this benchmark’s results? The results come from a single company’s benchmark run that hasn’t been independently replicated, are specific to the DeepSWE benchmark and Together AI’s testing setup, and reflect pricing that has already changed since the test was conducted.