Qwen3.8 Explained: 27B Multimodal, 2.4T MoE, and the VRAM Math

Alibaba shipped the Qwen3.8 family across two weeks in August 2026, and the pair it released is unusual: a 2.4-trillion-parameter flagship whose weights you can actually download, and a 27B dense multimodal model small enough to run on one high-end consumer GPU. Both went out as open weights. Neither is API-only.

The more interesting story for anyone running models locally is not the benchmark table. It is the architecture. Qwen3.8 uses a hybrid attention stack that changes how much VRAM long context actually costs — and it changes it by roughly 4x in your favour.

What Alibaba actually released

Two checkpoints, two very different deployment stories, two different licences.

The 27B is the one most people will actually run. It is Apache 2.0, it is a native vision-language model rather than a text model with an adapter bolted on, and at 27B parameters it lands in the size class that fits a single 24–48 GB card once quantized.

Qwen3.8-Max is the headline act — Alibaba's first Max-class model released as open weights rather than kept behind an API. But "open weights" and "runnable" are different things at 2.4 trillion parameters, which we will get to.

The architecture detail that matters for VRAM

Both models use the same trick, and it is the single most practical thing to understand about this family.

Instead of making every layer a full attention layer, Qwen3.8 interleaves them in a 3:1 ratio. The 27B's pattern is 16 repetitions of:

3 x (Gated DeltaNet -> FFN)  ->  1 x (Gated Attention -> FFN)

That is 64 layers total, but only 16 of them are real attention layers. The other 48 are Gated DeltaNet — a linear-attention variant that maintains a fixed-size recurrent state per head instead of a per-token key-value cache.

Why that matters: a KV cache grows linearly with the number of tokens you feed the model. A recurrent state does not. It is the same size whether you send 1,000 tokens or 1,000,000.

So when you compute VRAM for Qwen3.8-27B, only 16 layers contribute to the cache that scales with context. The published attention config is 24 query heads, 4 KV heads (grouped-query attention), head dim 256. Working it through at BF16:

• Per token, per attention layer: 2 (K and V) x 4 KV heads x 256 head dim x 2 bytes = 4 KiB
• Across 16 attention layers: 64 KiB per token
• At the native 262,144 context: 16 GiB

If all 64 layers used that same config, you would be looking at roughly 64 GiB of KV cache instead of 16 GiB — and about 256 GiB at the 1M ceiling rather than 64 GiB. The DeltaNet layers add a constant overhead measured in tens of megabytes, and it does not move as context grows.

This is why a 27B model can credibly advertise a 262k context for local use. On a conventional full-attention stack of the same depth, the cache alone would exceed the weights several times over.

One caveat worth stating plainly: the model card documents the layer pattern and the attention head config, but it does not spell out DeltaNet's memory behaviour. That the DeltaNet layers hold a fixed state rather than a growing cache follows from the layer type, not from an explicit claim on the card. Treat the 4x figure as a well-grounded inference, not a vendor-published number.

What Qwen3.8-27B actually needs in VRAM

Here is the practical ladder. Weights at BF16 are about 51.75 GiB, which is more than any single consumer card, so quantization is not optional — it is the whole point.

The community GGUF quants give real, measured file sizes rather than estimates:

Pair that with the KV cache figures above and the picture gets concrete:

• 24 GB card (RTX 4090, 3090): Q4_K_M weights at ~17.8 GB leave roughly 5–6 GB for cache and overhead. That is about 32k–64k of context in practice, not 262k. Comfortable for coding and document work, not for feeding it a whole repository.
• 32 GB: Q4_K_M or Q5_K_M with room for meaningfully longer context.
• 48 GB (RTX 6000-class, or 2x24 GB): Q6_K plus six-figure context, or Q4 with the full 262k.
• 80 GB (A100/H100): BF16 weights plus the full native context, no compromises.

Two details that catch people out. First, the vision encoder is separate in GGUF form — image and video input needs an mmproj projector file alongside whatever quant you download. It is not baked into the quant. Second, the quants bundle MTP (multi-token prediction) layers for speculative decoding, which adds a little to the file size but can buy you throughput.

If you want to run these numbers against your own card, context length, and concurrent-user count, our AI VRAM Calculator now includes both Qwen3.8 presets. It accounts for the hybrid stack correctly — the preset uses 16 attention layers rather than the full 64 depth, which is why its KV figures match the arithmetic above instead of overstating them by 4x.

Qwen3.8-Max: open weights you probably cannot run

The flagship deserves an honest framing. At 2.4T total parameters, BF16 weights alone are roughly 4.4 TB. Even at 4-bit that is over a terabyte of memory.

MoE does not rescue you here in the way people often assume. Only ~95B parameters are active per token, so compute per token is modest for a model this size. But every expert has to be resident somewhere the GPU can reach, so memory scales with the full 2.4T, not the active 95B. Sparse activation buys you speed, not footprint.

That puts Qwen3.8-Max firmly in multi-node datacentre territory — clusters of H100/H200 or MI300X-class accelerators with tensor and expert parallelism, served through SGLang or vLLM. It is genuinely open in that you can download, inspect, fine-tune, and self-host it if you have the hardware. It is not a local model.

It also carries a custom qwen3.8-max licence rather than Apache 2.0, unlike the 27B. If licensing matters for your use case, check the terms before building on it — the two models in this family are not interchangeable on that front.

Also worth knowing: thinking mode on Max cannot be turned off. Every response opens with a <think> block. You control depth via reasoning_effort (low, medium, xhigh, with xhigh the default), but you cannot get a bare answer. Budget output tokens accordingly — the card suggests up to 262,144 tokens for reasoning and 131,072 for the final response inside a 1M context.

Benchmarks, with the caveat attached

Alibaba's published numbers are strong. On the 27B: 61.7 on SWE-bench Pro, 73.0 on Terminal Bench 2.1, 89.2 on GPQA Diamond, 90.3 on LiveCodeBench v6. On vision: 84.3 on OSWorld-Verified, 90.0 on MathVision, 91.1 on OmniDocBench 1.5.

Qwen3.8-Max posts 67.7 on SWE-bench Pro, 86.6 on Terminal Bench 2.1, 92.6 on GPQA Diamond, and 93.0 on PaperBench — the last of which is above the Claude and GPT-5.6 comparison figures in Alibaba's own table.

Here is the caveat that belongs next to all of it: several of these are in-house benchmarks (QwenSWEBench, QwenQoderBench, CoWorkBench, RecreationBench), and the comparison scores for competing models were re-run by Alibaba rather than lifted from independent leaderboards. The Max card even notes that some competitor results "may involve fallbacks." That does not make the numbers fake — Qwen has a solid track record — but cross-model deltas from a vendor's own harness deserve more skepticism than the same deltas from a neutral evaluation.

The genuinely notable claim is not a benchmark at all. Alibaba says Max sustained work for over 125 hours reproducing and improving research papers, and continued a software project for 16 days without human intervention. Long-horizon autonomy is the thing this family is built around, and it is much harder to fake than a leaderboard row.

How to actually run the 27B

For a single-GPU local setup:

1. Pick your quant against your VRAM. Q4_K_M at 17.77 GB is the sensible default on a 24 GB card. Step up to Q6_K if you have 32 GB or more.
2. Grab the mmproj file too if you want image or video input. With llama.cpp's -hf flag it is pulled automatically; loading files manually means passing --mmproj yourself.
3. Set context deliberately. Do not default to 262k because the model supports it — at 64 KiB per token, 262k costs 16 GiB of cache on its own. Start at 32k and raise it only when a task needs it.
4. Decide on thinking mode. Unlike Max, the 27B lets you disable it with enable_thinking: False, or dial it with reasoning_effort. Thinking mode uses temperature 1.0 / top_p 0.95 / top_k 20; non-thinking uses temperature 0.7 / top_p 0.80 / top_k 20 with presence_penalty 1.5.
5. Use a real serving engine for throughput. Ollama and LM Studio are fine for single-user work. For concurrency, the card points to SGLang, vLLM, or TokenSpeed.

AMD users get a head start here: AMD announced day-0 support for the family on Instinct MI300X, MI325X, and MI355X via ROCm with SGLang and vLLM.

One quantization gotcha, if you are making your own quants: the 27B sets attn_output_gate: true, which fuses an output gate into q_proj so that tensor is [12288, 5120] rather than [6144, 5120]. Half of it is a multiplicative per-head gate, and quantization error behaves worse on a multiplicative gate than on a plain additive projection. Community FP8 and NVFP4 builds account for this; a naive uniform quant may not.

Where this family fits

If you run models locally, Qwen3.8-27B is the release worth your attention. Apache 2.0, native multimodal, a genuinely usable long context thanks to the hybrid stack, and it fits a single card at Q4. That combination is rare — most models that handle images well are either closed or too large to self-host, and most models advertising 262k context make that context financially painful to actually use.

Qwen3.8-Max matters for a different reason. A Max-class frontier model released with downloadable weights is a meaningful shift in what "open" means at the top of the market, even if almost nobody can run it on their own hardware. Its practical route today is a hosted provider rather than your own GPUs.

And the architectural lesson generalises beyond Qwen. Hybrid linear-plus-full attention stacks make long context affordable in a way that pure full-attention models cannot match at the same depth. If more labs adopt the 3:1 pattern, the "how much VRAM does 200k context cost" question gets a very different answer next year than it had last year.

Before you download 18 GB of weights, it is worth checking the numbers against your actual hardware. Plug your card, context length, and quantization into the AI VRAM Calculator — both Qwen3.8 models are in the preset list, with the hybrid-attention accounting already handled.

FAQ

Can I run Qwen3.8-27B on a 24 GB GPU?

Yes, quantized. Q4_K_M weights are 17.77 GB, which leaves roughly 5–6 GB for KV cache and runtime overhead on a 24 GB card — enough for about 32k–64k of context. You cannot run it at BF16 (51.75 GiB) or use the full 262k context on 24 GB.

Is Qwen3.8-27B really multimodal?

Yes, natively. It is described as a "Causal Language Model with Vision Encoder" and accepts both images and video, with video sampled at 2 fps by default. In GGUF form the vision projector ships as a separate mmproj file that you load alongside your chosen quant.

Why does the VRAM calculator show 16 layers for a 64-layer model?

Because only 16 of the 64 layers use Gated Attention and therefore keep a growing KV cache. The other 48 are Gated DeltaNet linear-attention layers with a fixed-size recurrent state. Entering 64 layers would overstate the KV cache by roughly 4x.

Can I run Qwen3.8-Max locally?

Realistically, no. At 2.4T total parameters, BF16 weights are around 4.4 TB and even aggressive 4-bit quantization exceeds a terabyte. MoE sparsity reduces compute per token, not memory footprint — all experts must stay resident. It needs multi-node datacentre GPUs.

Which Qwen3.8 model has the more permissive licence?

The 27B is Apache 2.0. Qwen3.8-Max uses a custom qwen3.8-max licence. If you need permissive commercial terms, the 27B is the straightforward choice.

Can I turn off thinking mode?

On the 27B, yes — pass enable_thinking: False, or use reasoning_effort set to low or medium to reduce depth. On Qwen3.8-Max, no: the card states thinking cannot be disabled, and every response begins with a <think> block.

Sources

• Qwen/Qwen3.8-27B model card — architecture, layer pattern, context, benchmarks, sampling settings
• Qwen/Qwen3.8-2.4T-A95B model card — MoE config, licence, thinking-mode behaviour, benchmark table
• bartowski/Qwen3.8-27B-GGUF — measured quant file sizes and mmproj notes
• AMD: Day-0 support for Qwen 3.8 on Instinct GPUs
• DataNorth: Alibaba releases Qwen3.8-27B open weights — release dates, licence confirmation, hosted pricing
• huginnfork/Qwen3.8-27B-FP8 — attn_output_gate quantization note