LLM VRAM CALCULATOR
[ LLM VRAM calculator ]
How much VRAM does your language model need?
Pick a model, a context length and how many sequences run at once. You get a memory estimate, a fit check for your own GPU, and the Xesktop card that runs it.
- Weights and cache
- GPU fit check
- 96 GB Xesktop card
MethodHow the estimate works
Weights = parameters × effective bits ÷ 8Standard KV = 2 × KV heads × head dimension × sequences × cache bytes × (full layers × tokens + local layers × min(tokens, window))MLA = layers × sequences × tokens × (KV latent rank + RoPE dimension) × cache bytesQwen hybrid = full-attention KV + fixed Gated DeltaNet recurrent state + convolution stateDSA = MLA + sparse-index key cache + index scalesTotal = weights × (1 + weight overhead / 100) + inference cache + compute buffer + runtime reserve
Gemma 3n presets count only unique cache-producing layers because later layers share KV values. Qwen3.5 and Qwen3-Next recurrent state is modeled in FP32 from the published Gated DeltaNet dimensions. The full-attention KV portion follows the selected cache precision. DeepSeek V3.2 and GLM-5 DSA presets add an FP8 index-key cache with FP32 per-block scales, matching the published DeepSeek reference implementation. Byte totals are converted to GiB (2³⁰ bytes). Decimal GB (10⁹ bytes) is also shown. A tight fit means under 10% of VRAM remains.
The page address stays clean while you change settings. Open a shared link to restore a saved setup, or use Copy settings link to make one. Resetting defaults clears a shared link’s settings from the address.
MoE weight-memory rule: active parameters per token are not the model-size input. The weight estimate uses the total stored parameter count, including inactive experts, because those weights still occupy memory. Quantization is applied to the stored weights, then the inference cache, buffer, runtime reserve and selected headroom are added.
Recommended VRAM capacity is the total estimate × (1 + extra headroom / 100), rounded up to the next tier: 8, 12, 16, 24, 32, 48, 64, 80 or 96 GiB. Larger targets round up to a whole GiB without claiming a matching product exists. These are capacity targets, not minimum requirements or benchmark results.
Legacy alternatives are first filtered by the unrounded base workload. GTX 1080 Ti appears only at or below 11 GiB. Tesla V100 16GB appears only at or below 16 GiB. Tesla V100 32GB appears only at or below 32 GiB. Passing that memory check does not make them Recommended. GTX 1080 Ti and V100 are marked Compatible with limitations because software, precision and kernel support depend on the architecture. The selected headroom target is shown separately and can mark a tight fit even when the base workload fits. CPU or GPU offloading does not qualify a legacy card.
One RTX PRO 6000 Blackwell GPU has 96 GB. The dual server has 192 GB across two 96 GB GPUs. A supported inference engine can distribute a model across both devices. Dual-server suggestions compare combined capacity only. They do not simulate the split, duplicated buffers or extra per-device overhead. A 150 GiB estimate plus 20% headroom is a 180 GiB target and a candidate for the dual server, subject to validation. An estimate of 192 GiB leaves no spare capacity, even with headroom set to zero.
For listed GPUs, reference VRAM bandwidth is a rough proxy for memory-bandwidth-bound decode throughput. A single RTX PRO 6000 Blackwell is modeled at 1,792 GB/s. The dual server shows 3,584 GB/s of aggregate theoretical bandwidth and does not assume perfect scaling. A direct “faster” comparison is used only when the selected local GPU can hold the same workload. Otherwise the calculator reports the theoretical bandwidth advantage and explains that there is no matching tokens-per-second comparison. The time example divides a hypothetical 10-second local step by the bandwidth-derived multiplier. It does not predict a real prompt. AMD figures use raw external-memory bandwidth rather than vendor “effective bandwidth” numbers that include Infinity Cache.
VRAM fit and GPU suitability are separate. A GPU can have enough memory and still be a limited choice because of architecture, precision, kernels, drivers or framework support. These figures are not measured tokens-per-second benchmarks. Training and general CPU offloading are outside this estimate. Multimodal encoder memory, speculative-decoding state, backend workspaces and multi-GPU sharding can add memory beyond the estimate.
Architecture inputs: num_hidden_layers, num_key_value_heads, and head_dim (often hidden_size / num_attention_heads).
References: llama.cpp multi-GPU documentation · Hugging Face: KV cache memory and quantization.
How to use the LLM VRAM calculator
Start by selecting an example model or entering the size of your own LLM. Then choose the weight format or quantization you plan to use, such as Q4_K_M, Q8_0, FP16 or BF16.
Set the context length you expect to use and the number of concurrent sequences the GPU needs to keep in memory. These settings are important because the model weights are only one part of total VRAM usage. Longer contexts and additional concurrent sequences increase the KV cache and can substantially change the amount of GPU memory required.
You can also enter the VRAM available on your GPU. The calculator compares it with the estimated requirement and shows whether the configuration fits, is likely to be tight, or needs more VRAM.
For more precise estimates, the advanced settings let you adjust model architecture, KV cache precision, runtime reserve, compute buffer and additional headroom.
How is LLM VRAM calculated?
An LLM’s memory requirement is not determined by parameter count alone. The calculator considers three main parts of the GPU memory budget: model weights, KV cache and runtime overhead.
Model weights
The first part is the memory required to load the model weights.
A rough calculation is:
Model weights = parameters × effective bits per weight ÷ 8
Quantization can make a large difference here. A model using a 4-bit weight format requires much less memory for its weights than the same model using FP16, although real quantized files can include tensors stored at different precisions.
That is why two models with the same number of parameters do not necessarily have exactly the same VRAM requirement.
KV cache
The KV cache stores attention information for tokens that have already been processed. It allows the model to continue generating tokens without repeatedly recomputing the entire previous sequence.
Its memory use depends on the model architecture as well as your workload, including:
context length, number of KV heads, head dimension, number of attention layers, KV cache precision and the number of concurrent sequences.
This is why increasing a model from an 8K context to a much longer context can increase VRAM usage even when the model weights stay exactly the same.
The calculator includes the KV cache in the total rather than estimating VRAM from model size alone, so it can also be used as an LLM KV cache calculator when planning longer-context or multi-user inference workloads.
Runtime memory and headroom
Inference engines also require memory for temporary buffers, the GPU runtime and other allocations.
The calculator adds adjustable allowances for these requirements and lets you add extra headroom before choosing a recommended VRAM capacity.
The result should therefore be treated as a planning estimate rather than a guarantee of exact peak memory usage. Actual VRAM use can differ depending on the model, inference engine, GPU drivers and configuration.
Can I run this LLM on my GPU?
Enter your GPU’s available VRAM into the calculator to check whether your selected model and inference workload will fit. The estimate includes model weights, KV cache, and runtime overhead.
If your GPU doesn’t have enough memory, try stronger quantization, a shorter context length, or fewer concurrent sequences. If those adjustments aren’t suitable, you’ll need access to a GPU with more VRAM.
How much VRAM do 14B, 30B, 32B, and 70B LLMs need?
| Model size | 4-bit weights | FP16 weights |
|---|---|---|
| 14B | 7 GB | 28 GB |
| 30B | 15 GB | 60 GB |
| 32B | 16 GB | 64 GB |
| 70B | 35 GB | 140 GB |
Architecture matters too. Models using grouped-query attention can require a different amount of KV-cache memory from models with a different attention configuration, even if their parameter counts are similar.
For a more useful estimate, select a matching model preset when available and set the quantization, context length and concurrency you plan to use.
Why two 32B models can have different VRAM requirements
Two LLMs with similar parameter counts don’t necessarily need the same amount of VRAM. Their attention architecture, context length, and number of concurrent sequences can affect memory usage.
Take Qwen2.5-32B-Instruct and Qwen3-32B. Despite having different attention-head counts (40 vs. 64), both use 8 KV heads and 64 layers. As a result, their FP16 KV-cache requirements are identical under the same settings.
| Configuration | Qwen2.5-32B | Qwen3-32B |
|---|---|---|
| 8K context, 1 sequence | 2 GiB | 2 GiB |
| 32K context, 1 sequence | 8 GiB | 8 GiB |
| 32K context, 4 sequences | 32 GiB | 32 GiB |
Calculated KV-cache memory only, assuming FP16 precision and fully populated contexts. Model weights and runtime overhead are excluded.
Increasing context length from 8K to 32K quadruples the KV-cache requirement. Running four concurrent sequences at 32K increases it further to 32 GiB.
The takeaway: Parameter count and attention-head count alone don’t determine VRAM usage. Use the LLM VRAM Calculator to estimate memory based on the model’s architecture and your intended workload.
GGUF VRAM calculator
GGUF models are commonly available in several quantization levels, which means the same model can have very different GPU memory requirements depending on the file you choose.
The calculator includes common effective weight formats such as Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K and Q8_0.
If you know the actual size of your GGUF weight files, you can also use a custom effective bit value to get closer to the memory footprint of that specific model file.
Remember that GGUF file size represents the model weights. The complete inference workload still needs additional VRAM for the KV cache, runtime and temporary buffers.
Using the calculator for Ollama GPU requirements
Ollama GPU requirements depend heavily on the model and quantization being loaded.
To estimate an Ollama workload, match the model size and weight format as closely as possible, then enter the context length and concurrency you expect to use.
The result provides a GPU-memory planning estimate rather than an Ollama compatibility guarantee. Actual allocation can depend on the Ollama version, underlying inference backend, model architecture and how much of the model is placed on the GPU.
If you are comparing several Ollama models, you can use the calculator to test each configuration before deciding how much VRAM you need.
Using Hugging Face model information
If your model is hosted on Hugging Face, its configuration can provide several of the architecture values needed for a more accurate estimate.
Useful values commonly include the total parameter count, number of hidden layers, number of key/value heads and attention head dimension.
The included model presets use configuration information for supported models, while the custom mode lets you enter architecture values for other models.
This makes the calculator useful when researching Hugging Face models even though it does not automatically load every model from the Hugging Face Hub.
Local LLM hardware requirements beyond VRAM
VRAM is one of the most important constraints for GPU inference, but it is not the only hardware requirement for running an LLM locally.
System RAM is still required by the operating system and inference software and becomes more important when part of the model is offloaded from the GPU. CPU performance can affect loading, preprocessing and CPU-based inference. Storage capacity and speed also matter when keeping multiple large model files locally.
GPU compatibility is equally important. Having enough VRAM does not guarantee that a particular GPU, driver or inference engine supports the model and weight format you want to use.
Use the VRAM estimate as one part of your hardware planning rather than treating it as a complete PC or server specification.
Choosing the best GPU for LLM inference
The best GPU for an LLM is not necessarily the GPU with the highest raw compute performance. First, it needs enough usable VRAM for the model, KV cache and runtime requirements.
Once the workload fits, other factors such as memory bandwidth, inference speed, software support, power consumption and cost become more important.
That is why it is useful to calculate the memory requirement before comparing GPUs. A faster card with insufficient VRAM may be less useful for a large model than a higher-memory GPU that can keep the entire workload on the GPU.
Running larger LLMs on Xesktop
For workloads that exceed local GPU memory, Xesktop provides dedicated RTX PRO 6000 Blackwell GPU servers with 96GB of VRAM on a single GPU.
For supported multi-GPU inference workloads, a dual RTX PRO 6000 Blackwell server provides 192GB of total VRAM across two 96GB GPUs.
Two GPUs do not automatically behave like one larger GPU. The inference engine and model must support splitting the workload between devices, and each GPU can still require its own runtime memory and buffers.
Use the calculator’s recommended VRAM capacity as a starting point, then confirm that your inference framework supports the GPU configuration you plan to use.
Where Xesktop fits into LLM VRAM planning
The calculator tells you how much GPU memory your LLM workload is likely to require. What you do with that number depends on the hardware you already have and how often you need to run the model.
If your local GPU comfortably meets the estimated VRAM requirement, running the model locally may be the simplest option. There is little reason to move the workload to a larger GPU simply because more VRAM is available.
The situation changes when the model, quantization, context length or concurrency pushes the workload beyond your available GPU memory. At that point, you can reduce the workload, upgrade your hardware or use a higher-memory remote GPU.
This is where Xesktop fits.
Xesktop provides dedicated remote GPU servers that let you access higher-memory GPUs without purchasing and maintaining the hardware yourself. This can be useful when you want to test a larger LLM, occasionally need more VRAM than your local workstation provides, or want to evaluate a model before investing in new hardware.
When 96GB is not enough
Some larger models, long-context configurations or higher-concurrency workloads can require more than 96GB of GPU memory.
For these workloads, Xesktop also offers a dual RTX PRO 6000 Blackwell configuration with 192GB of combined VRAM across 2 × 96GB GPUs. Supported multi-GPU LLM workloads can use the memory capacity of both GPUs, making it possible to run configurations that require more than a single 96GB GPU can provide.
This makes the dual-GPU option particularly relevant when the calculator shows a VRAM requirement above 96GB but within the available combined capacity.
Local GPU, buying a GPU or using Xesktop?
Which option makes sense depends on how often you need the additional GPU capacity.
If your existing GPU already fits the workload, use it.
If you regularly run workloads that exceed your current GPU, purchasing higher-memory hardware may make sense.
But if you only occasionally need to run a larger model, longer context window or higher-memory configuration, using a remote GPU can give you access to that capacity without sizing your entire local workstation around your peak workload.
A useful way to approach the decision is:
Model → quantization → context → concurrency → estimated VRAM → available GPU capacity
If you need more VRAM than your local GPU provides, compare Xesktop GPU server options and pricing for 96GB and 192GB configurations.
Frequently asked questions
What is an LLM VRAM calculator?
An LLM VRAM calculator estimates how much GPU memory a large language model may require during inference. A useful estimate includes the model weights, KV cache and additional runtime memory rather than looking at parameter count alone.
How much VRAM does an LLM need?
It depends on the model size, quantization, architecture, context length, concurrency, KV cache precision and runtime overhead. Enter those values into the calculator to estimate the requirement for your specific configuration.
Can my GPU run this LLM?
Enter your GPU’s available VRAM and the model configuration into the calculator. It will compare the estimated memory requirement with your GPU capacity and show whether the workload fits or needs additional VRAM.
What increases LLM VRAM usage?
Larger model weights, higher weight precision, longer context windows, additional concurrent sequences and higher-precision KV caches can all increase memory use.
Does quantization reduce VRAM requirements?
Yes. Lower-bit weight formats reduce the amount of memory required to store model weights. The total VRAM requirement still includes the KV cache, runtime memory and other allocations.
Can I use this as a KV cache calculator?
Yes. The calculator estimates KV-cache memory using the model’s attention architecture, context length, number of concurrent sequences and selected KV cache precision, then includes it in the overall VRAM estimate.
Can I use this calculator for Ollama?
You can use it to estimate the GPU memory needed for an Ollama model by matching the model size, quantization, context and concurrency. The result is a planning estimate and does not replace testing with the actual Ollama configuration.
Can I use it for Hugging Face models?
Yes. You can enter model information from a Hugging Face model configuration, including parameter count and attention architecture. Several model presets are already included for convenience.
Can I use this for LLM training?
No. This calculator is designed for LLM inference. Training and fine-tuning require additional memory for gradients, optimizer states, activations and other training-specific allocations.
Can two GPUs combine their VRAM for an LLM?
They can be used together when the inference engine supports model splitting or another compatible multi-GPU strategy. The VRAM is still physically attached to separate GPUs, so a dual-GPU system should not automatically be treated as one unified memory pool.