What VRAM Buys You When the Coding Agent Runs Locally

My day job is C++/Qt on embedded displays. On that kind of target the hardware is fixed before I write a line. The memory is what it is, and the software gets fitted to it.

Local coding agents turn out to be the same problem. Tech With Tim has a long setup guide arguing that local models are now good enough to run an agentic coding workflow with no internet connection, no subscription and no per-token bill. I think that is true in a narrow sense, and the useful part of the video is the budget.

I have not built this setup. What follows is how I read the constraint and what I would check before trusting it.

One number sets the ceiling

Tim's claim is that the model size you can run is dictated almost entirely by VRAM. On a Windows machine that means the dedicated memory on the graphics card. On an M-series Mac it means unified memory, which is shared between CPU and GPU, and he puts the usable share at roughly 75 to 80 percent once the operating system has taken its cut.

What makes this a hard limit is the behavior past it. A model bigger than your VRAM still loads. It spills into system RAM or onto disk and, by his estimate, runs about 100 times slower. He states that figure, and my notes do not show him measuring it. Whatever the real multiplier, performance falls off a cliff at the boundary. On a fixed target, software either fits the budget or it does not ship.

His approximate cheat sheet for what fits:

These numbers move with quantization.

Capacity and speed are separate axes

The point I found most useful is that how much memory you have and how fast it moves are different properties. Tim cites memory bandwidth of about 1008 GB/s for an RTX 4090 against about 546 GB/s for an M4 Max. A Mac with a lot of unified memory can load a larger model than a 24 GB card can. On a model that fits both, the card will usually produce tokens faster.

So a hardware choice is two questions: how large a model the work needs, and how long each response may take. I would weight the second more heavily for an agent than for chat, because an agent makes many model calls to finish one task.

Two ways to make a model fit

The first is quantization. A quantized model is a compressed copy of the full one, labeled Q6, Q4 and so on, with a much smaller footprint. Tim's advice is to pick the lowest quantization you can run, because smaller is faster and the quality loss is, in his word, tiny.

He does not show a comparison. The check I would want is narrow: whether a Q4 model still emits well-formed tool calls on the twentieth step of a task. A general benchmark would not tell me that.

The second is active parameters. A model name with "A3B" in it means three billion active parameters: a mixture-of-experts design where only part of the network runs for a given token. Tim presents this as a way to run larger models on weaker hardware. My reading is that it helps speed more than fit, since the full set of weights still has to live somewhere. He does not separate the two, so I would verify it.

The agent part has its own requirements

Tim recommends two models. One is a very small autocomplete model (he names Qwen 2.5 Coder 1.5B and says it runs on almost any machine). The other is a larger model from the Qwen family for chat and edits.

The larger model must support tool use. Without tool calling it can answer questions about code, but it cannot create or modify a file. Tool support is a pass/fail requirement that comes before size.

The second requirement is easier to miss. In his setup he adjusts context length while watching the memory estimate against his VRAM. Context comes out of the same budget as the weights, and an agent is the workload that wants long context: file contents, tool results and the running plan all accumulate. I read the table above as optimistic for agent work.

The stack itself is LM Studio serving models over a local API, a custom endpoint in VS Code, and the Continue extension for autocomplete. I expect that wiring to change. The selection rules should last longer.

What you give up

Tim is direct that Opus-class models do not run locally. He puts realistic local quality at roughly Sonnet or Haiku level depending on hardware. His closing verdict is that local works for offline sessions, for the moment your credits run out, and for small inline edits and single functions. He excludes complicated prompts.

That verdict arrives about 33 minutes into a video titled as a complete guide. The real scope is much smaller than the title.

The larger problem is the cheat sheet. It maps memory to parameter count. It does not map memory to capability. Nothing in my notes says what a 14B model can do as an agent that a 7B model cannot, or at which tier multi-file edits start to hold up. The Sonnet-or-Haiku comparison is his characterization, with no benchmark behind it. That mapping is what a buyer needs, and it is missing.

My main harness at home is Claude Code, and this does not change that. Local is a fallback for offline work and for code that should not leave the machine. If I set one up, I would decide in this order: confirm the chat model supports tool use, pick a size that fits in VRAM with room for context, then take the lowest quantization that still calls tools reliably.

Key Takeaways

• VRAM (or unified memory on a Mac) sets the largest model you can run at usable speed.
• Capacity and bandwidth are different axes. One loads a bigger model. The other answers faster.
• Context length shares the budget with the weights, so agent work needs more headroom than a size chart suggests.
• Tool-use support in the chat model is pass/fail. Check it first.
• The source reports what fits at each tier, not what each tier can do.

Sources

• Tech With Tim, "The Best LOCAL Agentic Coding Workflow (Complete Guide)"

What I want to find next is a tool-calling reliability comparison across quantization levels, because the whole fallback depends on that one claim holding.