Every QLoRA VRAM Table Leaves Out Sequence Length

Every QLoRA guide gives you a table like this: 8B needs 10 GB, 14B needs 16 GB, 70B fits on a 48 GB card. You buy or rent accordingly, start the run, and it dies with an out-of-memory error on the first step.

The table was not lying. It just quoted one number from a bill with four lines on it, and left out the line that grows fastest.

I ran the numbers on five models using the same architecture-aware formulas our AI VRAM calculator uses, with parameter counts read from each model's Hugging Face config. For Llama 3.1 8B at rank 16 and a 2,048-token sequence, QLoRA needs 8.59 GiB. Push the sequence to 8,192 tokens and the same job needs 18.49 GiB. Nothing about the model changed. The model was never the variable.

Table of contents

• The four lines on a QLoRA bill
• Your adapters are not the problem
• Activations scale with tokens, and they win
• What each card actually holds
• The 70B on 48 GB claim
• Why full fine-tuning is 16x worse
• Two switches worth more than a bigger card
• FAQ
• Conclusion
• Sources

The four lines on a QLoRA bill

QLoRA charges you for four separate things, and they scale on different axes.

Base weights are fixed. Llama 3.1 8B in NF4 is 4.19 GiB and stays 4.19 GiB no matter what else you do, because it is frozen.

Adapter state is tiny, and I will come back to how tiny.

Activations are the intermediate tensors the backward pass needs, and they scale with batch size times sequence length times layers times hidden size. Not with parameter count.

Runtime overhead is the CUDA context, allocator fragmentation and kernel scratch. I budget 10% here; go lower and you are gambling.

For the 8B at 2,048 tokens the split is 48.7% base weights, 7.27% adapters, 34.9% activations, and the rest overhead. Published tables usually quote roughly the first two lines and a short sequence, then readers apply the number to a 4k or 8k training set.

Your adapters are not the problem

Rank is the knob everyone reaches for first, and it is close to the least important one.

At rank 16 targeting all seven projection modules, Llama 3.1 8B gets 41.94 million adapter parameters. That is 0.522% of the base model. Those adapters do carry the full mixed-precision training cost of 16 bytes per parameter (a bf16 weight, a bf16 gradient, an FP32 master copy and two FP32 AdamW moments), but 16 bytes on 42 million parameters is 640 MiB.

Going from rank 8 to rank 32, a 4x increase in trainable capacity, costs about 1 GiB. Meanwhile doubling your sequence length from 2,048 to 4,096 costs 3.3 GiB at any rank.

Cutting rank to save memory is treating the smallest line on the bill. If you are out of memory at rank 16, rank 8 will probably not save you.

Activations scale with tokens, and they win

Here is the same 8B model, rank 16, batch size 1, gradient checkpointing on, with only the sequence length changing.

Past 4,096 tokens, activations are the majority of your VRAM. At 32k they are 82.6% of it, and the 4-bit weights everyone optimizes are a footnote.

The rate is per model, and it is worth knowing yours. With gradient checkpointing on, bf16, batch 1:

Multiply by your sequence length in thousands of tokens and by your batch size. That is your activation budget, and batch size behaves exactly like sequence length here: batch 4 at 2,048 tokens costs the same 12 GiB as batch 1 at 8,192.

What each card actually holds

Flip the question around. Instead of asking how much VRAM a model needs, ask what sequence length your card can actually train at. Llama 3.1 8B, QLoRA, rank 16, batch 1, gradient checkpointing on, 10% runtime overhead:

An 8 GB card cannot train an 8B model at 2,048 tokens. It tops out at 1,664. Most "8B on 8 GB" claims are quietly assuming 512 tokens, which is a real configuration but not one that suits instruction data with long answers.

A 12 GB card handles 4,096 tokens, which covers most instruction tuning. A 16 GB card gets you to 6,528, and the jump to 24 GB roughly doubles that again.

If your dataset has a few long outliers, they set your memory ceiling, not your average. Check the token-length distribution of your training set before you pick a card, and if you need to inspect the JSONL you are about to train on, our JSON formatter will pretty-print it in the browser without uploading it anywhere.

The 70B on 48 GB claim

This one deserves its own section, because it is repeated everywhere and it is technically true in the narrowest possible sense.

Llama 3.3 70B, QLoRA, rank 16: the NF4 base weights are 36.80 GiB. Add 3.09 GiB of adapter state and 10% overhead and you are at 48.00 GiB before a single activation, which happens to land on a 512-token sequence.

So a 48 GB card fits 70B QLoRA at 512 tokens with no headroom at all. At 1,024 tokens you are over the card. An 80 GB H100 tops out at 4,480 tokens.

The claim is not a lie, it is a specification with the sequence length removed. Remove the sequence length from any QLoRA memory figure and you can make almost anything fit almost anything.

Why full fine-tuning is 16x worse

Full fine-tuning is where the parameter count finally matters, because every parameter now carries baggage.

Mixed-precision AdamW holds a bf16 weight, a bf16 gradient, an FP32 master copy and two FP32 optimizer moments: 16 bytes per parameter. On 8.03 billion parameters that is 119.66 GiB of parameter state before any activations at all.

Note the asymmetry in where the memory goes. For full fine-tuning of the 8B, parameter state is 88.7% of the bill and activations are a rounding error. For QLoRA on the same model at the same sequence length, parameter state is under 9% and activations are a third. They are different problems wearing the same name, which is why advice that works for one fails for the other.

Switching AdamW to 8-bit states drops the per-parameter cost from 16 bytes to 10, taking the 8B parameter state from 119.66 GiB to 74.79 GiB. Still multiple 80 GB cards for a model you can QLoRA on a laptop GPU.

Two switches worth more than a bigger card

Gradient checkpointing is the one to check first. It recomputes activations during the backward pass instead of storing them, roughly halving the activation term for a small time penalty. On the 8B at 8,192 tokens it takes activations from 24.00 GiB to 12.00 GiB and the total from 31.69 GiB to 18.49 GiB. A 24 GB card that cannot run the job runs it with one flag flipped.

Sequence packing is the second. If your examples average 600 tokens and you have set a 4,096-token maximum, you are paying for 4,096 on every step and using a seventh of it. Either set the maximum near your real distribution, or pack short examples together so the tokens you pay for carry actual data.

What does not help as much as people expect: dropping rank, as the table above shows, and dropping the base model to int4 from NF4. NF4's extra scaling metadata costs 0.56 bytes per parameter against int4's 0.5, which on 8B is 4.19 GiB versus 3.74 GiB. You save 450 MiB and give up the quantization quality NF4 was designed for.

FAQ

How much VRAM does QLoRA fine-tuning of an 8B model need?

Between 6.12 GiB and 18.49 GiB for Llama 3.1 8B at rank 16 and batch 1, depending entirely on sequence length: 6.12 GiB at 512 tokens, 8.59 GiB at 2,048, 11.89 GiB at 4,096 and 18.49 GiB at 8,192. Any single number quoted without a sequence length is incomplete.

Why does my fine-tune run out of memory when the guide said it would fit?

Almost always sequence length or batch size. Guides usually quote short sequences, and activation memory scales linearly with both. For Llama 3.1 8B each 1,024 tokens costs 1.5 GiB with gradient checkpointing on, so a jump from 512 to 4,096 tokens adds 5.25 GiB to a job whose weights never changed.

Does lowering LoRA rank save much VRAM?

Not much. On Llama 3.1 8B, rank 8 to rank 32 spans 8.25 GiB to 9.28 GiB, about 1 GiB for a 4x change in trainable parameters. Halving your sequence length saves far more, and costs you less capacity.

Can you really QLoRA a 70B model on a single 48 GB GPU?

Only at about 512 tokens. NF4 base weights for Llama 3.3 70B are 36.80 GiB, and with adapter state and 10% runtime overhead you reach 48.00 GiB before activations. At 1,024 tokens the job needs 52.12 GiB and no longer fits.

How much VRAM does full fine-tuning need compared to QLoRA?

Roughly 15 to 19 times more. Mixed-precision AdamW costs 16 bytes per parameter, so Llama 3.1 8B needs 119.66 GiB of parameter state alone, against 8.59 GiB total for QLoRA at 2,048 tokens.

Is gradient checkpointing worth the slowdown?

Usually yes, because it halves the term that dominates your bill. On the 8B at 8,192 tokens it takes the total from 31.69 GiB to 18.49 GiB, which is the difference between needing an 80 GB card and running on a 24 GB one.

Why is NF4 listed at 0.56 bytes per parameter instead of 0.5?

NF4 stores quantization scaling factors alongside the 4-bit weights, so the effective cost is slightly above four bits per parameter. For an 8B model that is 4.19 GiB rather than the 3.74 GiB a pure 4-bit encoding would give.

Conclusion

The number worth memorizing is not a VRAM figure for your model. It is your model's activation rate: 1.5 GiB per 1,024 tokens for an 8B, 2.3 for a 14B, 7.5 for a 70B. Multiply by sequence length and batch size, add the 4-bit weights, add 10%, and you have an estimate that survives contact with a real training run.

Do that and the published tables become readable. "8B on 8 GB" means 1,664 tokens. "70B on 48 GB" means 512. Neither is wrong, both are missing the variable that moves.

If you are sizing a rental or deciding between a 16 GB and a 24 GB card, put your real sequence length and batch size into the AI VRAM calculator and read the component breakdown rather than the total. The line that is largest is the line to attack, and for QLoRA past about 3,000 tokens that line is never the model.

Sources

• Every VRAM figure in this post was computed locally from the architecture-aware formulas in ToolMintX's AI VRAM calculator, using parameter counts, layer counts, hidden sizes, KV head counts and intermediate sizes taken from each model's Hugging Face config: Llama 3.1 8B Instruct (8,030,261,248 params, 32 layers, hidden 4096), Qwen3 8B (8,190,735,360, 36 layers), Qwen3 14B (14,768,307,200, 40 layers), Qwen3 32B (32,762,123,264, 64 layers) and Llama 3.3 70B Instruct (70,553,706,496, 80 layers).
• Adapter parameter counts follow the standard LoRA placement of rank-r matrices on all seven projections (q, k, v, o, gate, up, down), with the KV projections sized by KV heads rather than attention heads to account for grouped-query attention.
• Mixed-precision AdamW memory accounting of 16 bytes per trainable parameter (bf16 weight, bf16 gradient, FP32 master copy, two FP32 moments) and the 8-bit optimizer variant at 10 bytes per parameter.
• Dettmers et al., "QLoRA: Efficient Finetuning of Quantized LLMs" (2023), for the NF4 data type and the double-quantization scaling factors that put NF4's effective cost slightly above four bits per parameter.
• Unsloth documentation, Unsloth requirements and the LoRA hyperparameters guide, for gradient checkpointing behaviour and the recommended rank and learning-rate ranges for QLoRA fine-tuning.