Local Models Caught Up. Here's What Actually Runs on Your Desk.

Someone on a client call last month asked me why I still pay for a Claude subscription when "the open ones are just as good now."

I gave him a hedge. Something about benchmarks not being the whole story. Then I went back to my desk, opened the numbers properly, and realised he was more right than I was.

Open-weight models caught up. Not "almost." Not "for simple tasks." The measured distance between the best open model you can download for free and the best closed model you can rent is now four months. Four months is one billing cycle and a bit. And a chunk of that capability now fits on a machine sitting under your desk.

Four Months, Measured, Not Vibes

Epoch AI tracks this on an actual index instead of an argument. Since January 2026, the most capable open-weight models have trailed frontier closed models by an average of four months, or 8 points on their Epoch Capabilities Index. Eight ECI points is roughly the distance between GPT-5 and GPT-5.5. Half a generation.

The individual scores as of late May 2026 tell it better than the average does. Kimi K2.6 at 151.60, released April 20. GLM-5.1 at 149.94, April 7. Qwen 3.6 Plus at 149.08, March 31. Against that: Gemini 3.5 Flash at 156.31, Claude Opus 4.7 at 156.18, GPT-5.5 Pro at 159.35.

I want to be honest about one thing, because it cuts against my own headline. That four-month gap is slightly wider than the three months Epoch measured across January 2023 to October 2025. The catching up did not happen this year. It happened over the previous three, and 2026 has been roughly a holding pattern.

But look at where the holding pattern is holding. Four months behind Claude Opus 4.7 is a position that would have been science fiction in 2024. The story is not that the gap is closing this quarter. The story is that it already closed, to a distance small enough that for most of my actual work it stopped mattering.

Apache 2.0, 27 Billion Parameters, On My Desk

Alibaba shipped Qwen3.8-27B on August 14, 2026. Dense, vision-capable, 262,144-token native context, and licensed Apache 2.0. Not a bespoke community licence with a revenue clause. Not "open weights, terms and conditions apply." Apache 2.0. I can run it for a paying client and nobody sends me an email about it.

The vendor benchmark card claims 61.7 on SWE-bench Pro and 90.3 on LiveCodeBench v6, which would put it at or above Claude Opus 4.6 on agentic coding. I do not believe that number, and neither should you, because it is Qwen benchmarking Qwen. Independent verification is still pending as I write this.

What is independently verified is smaller and more convincing. In its first two weeks the model took the top open-weight spot on Harvey's Legal Agent benchmark, ranked #9 overall on Code Arena's WebDev leaderboard, and took the top open-weight spot on Arena.ai's Image-to-WebDev board at #7 overall. Ninth overall, on a leaderboard whose top eight are models I rent, from a file I downloaded.

Worth separating two things that get confused constantly: Qwen3.8-27B is the Apache 2.0 model you can actually run. Qwen3.8-Max is a 2.4-trillion-parameter flagship on its own bespoke terms, served from someone else's cluster. Only one of those two is a thing you own.

Six to Twelve Months Until It Runs at Home

The number I care about more than the frontier gap is Epoch's other one: models small enough to fit on a single consumer GPU, meaning roughly 28B parameters for an RTX 4090 or 40B for a 5090, reach frontier-level scores after a lag of six to twelve months across GPQA Diamond, MMLU-Pro, and LMArena.

So the pattern is: whatever the expensive model does today, something on your desk does by roughly next summer. That is not a promise about the future. That is a measured pattern that has already repeated several times, and it is why I stopped treating my local setup as a toy.

What 32 GB and 64 GB Actually Get You

Here is where the marketing and my Activity Monitor part ways.

Qwen3.8-27B at BF16 wants about 56 GB of weights, plus roughly 16 GB of KV cache at full context. That is a rented H100, not a laptop. At FP8 it drops to about 28 GB. At 4-bit it lands around 14 to 19 GB depending on the build, and that is the number that matters, because 4-bit is where a Mac becomes a real option.

On a 32 GB Mac you get the 4-bit build with room for a 128K context window and nothing to spare. On 64 GB you get the 6-bit build at 21 to 22 GB and you stop making compromises. Under 24 GB, don't bother.

Speed is where the sources genuinely contradict each other and I'd rather show you the contradiction than pick a flattering number. One writeup running the MLX 4-bit build on an M4 Mac mini with 32 GB reports 5 to 6 tokens per second. A measured benchmark repo on a Mac Studio M4 Max reports 48 to 65 tokens per second on an optimised MLX recipe with native multi-token prediction. That is a tenfold spread, and it is not one of them lying. It is memory bandwidth plus how much work went into the inference recipe. A base M4 and an M4 Max are not the same machine wearing different badges.

My honest read: if you are on base Apple Silicon, expect single-digit tokens per second and plan around it. If you are on a Max or Ultra with real bandwidth, expect something you can hold a conversation with. I went through the whole setup of pointing Claude Code at a local model in Ollama, MLX and a local coding agent, and the throughput question was the entire difference between "neat demo" and "I use this daily."

It Isn't One Model Anymore, It's a Field

The thing that changed most is that there is no longer a single open model to root for.

On Artificial Analysis Intelligence Index v4.0, Kimi K2.6 leads open weights at 54, DeepSeek V4 Pro at 52, GLM-5.1 at 51. DeepSeek V4 Pro runs 1.6 trillion total parameters with 49 billion active and MIT-licensed weights, with a V4 Flash variant at 284B total and 13B active that is far more plausible to self-host. OpenAI's gpt-oss-120b and gpt-oss-20b are Apache 2.0 and were built with a chat schema designed around tool calls, which shows. Google's Gemma 4 arrived in April 2026, also Apache 2.0, in a dense 31B and a 26B mixture-of-experts.

Five serious labs shipping downloadable weights on a rolling cadence. When one of them is behind for a quarter, another one isn't. That is a much more durable situation than any single release being good.

Where They Still Lose, Because They Do

I am not going to pretend the local model handles everything, because it doesn't, and I know exactly where it stops.

Single question, everything the model needs in the prompt, one answer back: local wins on convenience and loses nothing I can feel. Renaming things across a file, writing a Zod schema from a Directus collection, explaining a regex, converting a JSON blob into a TypeScript interface, summarising a client email into three things I have to do. All of it lands.

Thirty tool calls deep into a real codebase is a different animal. I asked it to find why a form submission was failing silently across four files in a Next.js app. It read the first two fine. Somewhere around the fifth or sixth tool call it started confidently editing a file that was not in the project, with a plausible-looking diff referencing a function that has never existed.

That is not a mystery, it is the known failure shape. The plumbing is where open models are behind, more than the intelligence: malformed tool arguments, reaching for the wrong tool, getting an error back and retrying the identical call forever. I watched one run burn eleven minutes in that loop. I wrote about the general version of this in the mistakes AI coding agents keep making, and local models make them harder and earlier.

I also have to admit I have not ruled out my own configuration. Tighter tool schemas and a smaller context per step might fix half of it. I have not done that work properly yet.

The Split I Run Now, and Why It Isn't About Money

Everyone frames local models as a cost play. I already ran that math and the break-even on buying hardware is further out than most people want it to be, especially if your usage is spiky.

The reason I keep the local setup is duller and more valuable. I am a German agency with US clients, and some of those contracts make me answerable for where their data goes, in a "list your subprocessors" way. When a client sends me a CSV of customer records to debug an import, the local model is the only answer that does not need a DPA and a paragraph of explanation. It never leaves the machine.

So: one-shot work goes local. Anything touching client-confidential data goes local regardless of difficulty, and if it is too hard for the local model I do it by hand. Long agentic runs across a real codebase go hosted. That is about 60 percent local for me now, against zero a year ago.

I picked the same way when I stayed on Opus 4.6 while everyone chased newer releases, and it is the same lesson: choose per job, not per hype cycle. If you want the wider picture of what runs my one-person setup, most of the AI tools that actually run my agency have nothing to do with which model is winning this quarter.

Download Qwen3.8-27B, or whatever the equivalent is when you read this. Run it for a week on your real work. The surprise is not that it fails somewhere. The surprise is how far you get before it does.

I can't tell you which model to run on your hardware. What I can do is look at your actual workflow and work out which parts genuinely need a frontier API and which parts you are paying for out of habit. Let's talk if that's useful.