LLM VRAM Requirements: The 2026 Master Table (Every Model, Every Quant)

LLM VRAM Requirements: The 2026 Master Table (Every Model, Every Quant)

Here's the number that catches everyone off guard: DeepSeek-V3.2 has 671 billion parameters, but only 37 billion fire on any given token. So how much VRAM does it actually need? All 671 billion worth, roughly 382 GB at Q4. LLM VRAM requirements rarely follow intuition, and the gap between "active parameters" and "what you have to load" is exactly where hardware budgets blow up. This guide hands you the master table (every major open model, every quantization level, the GB figure and the GPU that runs it) plus the formula to size any model yourself in about ten seconds.

Key Takeaways

• VRAM for the weights ≈ parameters × bytes-per-param: FP16 = 2.0, Q8 = 1.0, Q5_K_M ≈ 0.68, Q4_K_M ≈ 0.57. Add KV cache and ~15-20% overhead on top.
• Mixture-of-Experts models (DeepSeek, GLM-5.2, Qwen3-235B) must load every expert into VRAM. "Active parameters" buy you speed, not memory.
• KV cache is the hidden cost. Llama 3.3 70B needs about 2.6 GB of cache at 8K context and roughly 41 GB at 128K, on top of the weights.
• Q4_K_M is the sensible default: near-full quality at roughly a quarter of the FP16 footprint.
• A 12B model like Gemma 4 fits an 8 GB card at Q4. A 70B dense model needs about 40 GB. A 671B frontier MoE needs a small server.

LLM VRAM Requirements by Model: The Master Table

The short answer: at Q4_K_M, small models (under 14B) fit consumer 8-12 GB cards, mid-size models (24-32B) want 16-24 GB, a 70B dense model needs about 40 GB, and the frontier MoE models jump into hundreds of gigabytes because every expert has to be resident. Here is the full picture in one place. All figures are the memory for the weights alone, computed from each model's parameter count and cross-checked against the official model cards from Meta AI, Qwen, and Hugging Face.

Two things to read off this table. First, quantization is the biggest lever you have: dropping from FP16 to Q4 cuts the footprint by roughly 4x with a barely perceptible quality hit. Second, the MoE rows look brutal because they are. Qwen3-30B-A3B activates only 3B parameters per token, so it runs at the speed of a tiny model, but you still need to hold all 30B in memory to have every expert ready. Want the model-by-model details behind these numbers? Our Gemma 4 12B deep dive and the best open-source LLMs of 2026 roundup cover the benchmarks and licenses.

The VRAM Formula: Calculate Any Model Yourself

To size any model, multiply its parameter count by the bytes-per-parameter for your quantization, then add a bit for the KV cache and runtime overhead. That's it. Weights are the dominant term, and the arithmetic is simple enough to do on the back of a napkin.

The core equation for the weights:

VRAM_weights (GB) = parameters (billions) × bits_per_weight ÷ 8

The bits-per-weight values you need (these are the effective rates for GGUF k-quant files, which carry a little block metadata on top of the nominal bit depth):

Worked example, Gemma 4 12B at Q4_K_M: 11.95 × 4.5 ÷ 8 = about 6.7 GB for the weights. That lines up with the roughly 6.6 GB the official model card quotes and explains why it fits an 8 GB card with room for a modest context. Run the same math for a 70B model at Q4 and you get 70 × 4.5 ÷ 8 = 39.4 GB, which is why "you need two 24 GB cards or one 48 GB card for a 70B" is the rule of thumb everyone repeats.

The full picture adds two more terms: total VRAM ≈ weights + KV cache + ~15-20% overhead. The overhead covers activation buffers, the CUDA context, and memory fragmentation, and your GPU also reserves half a gigabyte or so for the driver, so never plan to use 100% of the sticker VRAM.

Why KV Cache Is the Number That Bites You

The KV cache stores the attention keys and values for every token already in the context, and it grows linearly with context length. On short prompts it's a rounding error. Push toward a long context and it can rival or exceed the weights themselves. This is the single most common reason a model that "should fit" throws an out-of-memory error mid-generation.

The formula, per token:

KV_cache_per_token (bytes) = num_layers × 2 × kv_dim × precision_bytes
kv_dim = num_kv_heads × head_dim   (grouped-query attention shrinks this)

Take Llama 3.3 70B: 80 layers, 8 KV heads, head dimension 128, so kv_dim is 1024. At FP16 that's 80 × 2 × 1024 × 2 = 327,680 bytes per token, about 0.31 MB. Multiply by context length and the story writes itself: at 8K tokens the cache is roughly 2.6 GB, at 32K it's about 10 GB, and at 128K it balloons to around 41 GB. That last figure is on top of the 40 GB of weights, so a "40 GB model" quietly becomes an 80 GB problem the moment you fill the window.

Two practical escapes. Grouped-query attention (which every recent model uses) already slashes kv_dim versus the old multi-head design, so modern models are far kinder here than Llama 2 was. And most inference engines can quantize the KV cache to 8-bit or 4-bit, halving or quartering its size for a small quality cost. Google's newer TurboQuant memory-compression approach targets the same bottleneck at roughly 3 bits per value. If you're serving long contexts in production, the vLLM vs SGLang comparison covers which backend manages this memory most efficiently with paged attention.

MoE Models: Why "Active Parameters" Don't Save VRAM

This is the trap that costs people the most money. A Mixture-of-Experts model like DeepSeek-V3.2 (671B total, 37B active, sharing the V3 architecture) or GLM-5.2 (744B total, 40B active) routes each token through a small subset of its experts. The marketing leans on the active number because it describes speed: you only pay 37B parameters' worth of compute per token, so inference is fast for the model's size. But every expert must sit in memory, ready to be picked, which means your VRAM budget is set by the total parameter count, not the active one.

So the honest reading of the table above: GLM-5.2 runs at the speed of a 40B model but occupies the memory of a 744B one. That's why these frontier open models need an 8-GPU server or a large unified-memory machine, even though a single forward pass is cheap. Qwen3-235B-A22B is the same shape at smaller scale, quick per token, heavy to host.

The upside of MoE shows up on unified-memory hardware. A Mac Studio with 512 GB of unified memory can hold a 671B model at Q4 and still run it at usable speeds precisely because only 37B activate, so the memory bandwidth demand per token stays reasonable. If you're new to running these locally, start with our local LLM setup guide before you spend on hardware.

Which Quantization Should You Pick?

For nearly everyone, Q4_K_M is the right default: it holds near-full quality while cutting the FP16 footprint by about 4x. Step up to Q5_K_M or Q8 only if you have spare VRAM and a quality-sensitive task, and reach for FP16 only when you're fine-tuning or benchmarking against a reference. Below Q4, quality degradation becomes noticeable fast, so Q3 and lower are a last resort for squeezing a model onto a card that's genuinely too small.

One caveat: quantization quality is not identical across models. Very small models (under 4B) feel Q4 more than large ones, because they have less redundancy to spare. On a 70B model, Q4 versus Q8 is hard to tell apart on most tasks. On a 1.7B model, the gap is real.

What GPU Do You Actually Need?

Match the Q4 column of the master table to a card with a little headroom for KV cache. Here's the practical mapping from budget consumer hardware up to the data center, with the model tier each class comfortably runs at Q4.

Apple Silicon deserves a special mention because unified memory changes the calculus. A Mac doesn't split VRAM from system RAM, so a 128 GB M-series machine can load models that would need multiple discrete GPUs, trading peak throughput for the ability to fit huge weights on one desktop. For the backends that squeeze the most out of any of these cards, our roundup of the best tools to run LLMs locally benchmarks the real-world speed differences.

How We Size VRAM for Client Deployments

At Techsy we deploy open models for clients often enough that VRAM sizing is the first conversation, before model choice, before prompts, before anything. Our method is boring on purpose, because the failure mode (an OOM in production under real context load) is expensive. Here's the process we actually run.

We start from the table math, then measure. After loading a model we check the real resident footprint rather than trusting the estimate:

# What the GPU is actually holding
nvidia-smi --query-gpu=memory.used,memory.total --format=csv

# For an Ollama-served model, its real memory + how much sits on GPU vs CPU
ollama ps

# llama.cpp: control the split explicitly and cap context to bound KV cache
llama-server -m model-Q4_K_M.gguf --n-gpu-layers 999 --ctx-size 8192

The lesson that keeps repeating: teams size for the weights and forget the KV cache, then wonder why a model that loaded fine dies three long requests into a demo. We size for the weights plus the KV cache at the maximum context the app will really use, plus headroom, and we cap --ctx-size so a runaway request can't OOM the box. For anything customer-facing we'd rather run a quantized 32B that never falls over than an FP16 70B that OOMs under load.

If you're weighing whether to self-host an open model or stay on a hosted API, that trade (hardware cost and ops burden versus per-token pricing and control) is exactly what our team scopes during an AI integration engagement. If it would help to have someone run the numbers against your actual workload, get a free consultation and we'll size it with you.

Frequently Asked Questions

How much VRAM do I need to run a 70B model?

A 70B dense model like Llama 3.3 70B needs about 40 GB of VRAM for the weights at Q4_K_M, so plan for a 48 GB card (RTX 6000 Ada) or two 24 GB cards. Add several more gigabytes for KV cache if you use a long context, which pushes practical requirements toward 48 GB or more.

How much VRAM does Llama, Qwen, or DeepSeek need?

It depends entirely on the variant. Llama 4 Scout needs about 62 GB at Q4, Qwen3-32B about 18 GB, and Qwen3-8B under 5 GB. DeepSeek-V3.2, a 671B MoE, needs roughly 382 GB because every expert must load. Always check the total parameter count, not the active one, for MoE models.

Can I run an LLM on an 8GB GPU?

Yes, comfortably. An 8 GB card such as an RTX 4060 runs models up to about 12B parameters at Q4_K_M. Gemma 4 12B fits in roughly 6.8 GB, leaving room for a modest context. For anything larger you either quantize harder, keep the context short, or move to a bigger card.

What can a 24GB GPU like the RTX 4090 run?

A 24 GB card handles dense models up to about 32B at Q4_K_M with headroom for a reasonable context, so Qwen3-32B and Mistral Small 3.2 24B are comfortable. It also runs the Qwen3-30B-A3B MoE, which loads 30B of weights but generates at the speed of a 3B model thanks to sparse activation.

Does quantization hurt model quality?

At Q4_K_M and above, the quality loss is small and often imperceptible on real tasks, especially for models over 13B. The gap widens as you go lower and as models get smaller, so Q4 on a 70B is nearly free while Q4 on a 1.7B is noticeable. Q8 is effectively lossless if you have the memory.

Do MoE models need less VRAM than dense models?

No, and this is the most common misconception. A Mixture-of-Experts model must hold every expert in VRAM, so its memory is set by the total parameter count. The active-parameter figure only describes inference speed. GLM-5.2 runs at the speed of a 40B model but needs the memory of a 744B one.

Is unified memory the same as VRAM?

Functionally, for loading models, yes. Apple Silicon and some other systems share one memory pool between CPU and GPU, so a 128 GB Mac can load models that would otherwise need multiple discrete GPUs. The trade is bandwidth: unified memory usually delivers lower peak throughput than a high-end data-center GPU, so tokens-per-second is lower.

Can I offload part of a model to system RAM or CPU?

Yes. Engines like llama.cpp and Ollama let you keep some layers on the GPU and the rest in system RAM with a flag like --n-gpu-layers. It lets you run a model too big for your VRAM, but every layer on the CPU slows generation sharply, so use it to make a model possible, not fast.

How do I calculate VRAM for a model that isn't in the table?

Multiply the parameter count in billions by the bits-per-weight for your quantization, then divide by 8. For Q4_K_M use about 4.5 bits, so a 40B model needs 40 × 4.5 ÷ 8 = about 22.5 GB for the weights. Add roughly 15-20% overhead plus your KV cache for the real requirement.