Arete-27B VRAM: Same Qwen Memory, No MTP Head

Nous Research published Arete-27B on Hugging Face on 6 November 2026. It's Qwen3.8-27B after one round of supervised fine-tuning, aimed at how the model answers politically sensitive questions about China. The same evening bartowski posted GGUF builds from 8.64 GB to 28.67 GB, and by 9 November the original repo showed 1,747 downloads.

If you already run Qwen3.8-27B, most of your setup carries over. Arete keeps the same 64-layer hybrid stack, the same 262,144-token context and the same vision encoder. One thing doesn't carry over. Arete ships without the multi-token prediction (MTP) head the base model has, so any speed you were getting from MTP drafting is gone. Below is the memory you need at each quant and context, what the missing head means, and a llama-server setup that matches how Nous tested it.

Table of contents

• What Nous changed and what it kept
• The missing MTP head
• Memory by quant and context
• Which GPU to buy or use
• Running it with llama-server
• Reading the evaluation numbers
• FAQ
• Sources

What Nous changed and what it kept

Nous says it saw refusals, deflections and answers framed in the Chinese government's terms on politically sensitive topics in Qwen3.8-27B, and trained to change that. The method is plain SFT. Nous lists no RL, no preference optimization and no change to the architecture.

The config.json matches the base model line for line on the parts that drive memory: 64 layers, 48 of them Gated DeltaNet and 16 full attention, 4 KV heads, head dimension 256, hidden size 5,120. That means the KV cache math from our Qwen3.8 family guide still applies.

Because the vision encoder was frozen and the training data was text only, image input still works through the same mmproj file. It just wasn't part of what Nous changed.

The missing MTP head

Here is the line in Arete's config.json that the model card doesn't call out: "mtp_num_hidden_layers": 0. The base Qwen3.8-27B config has 1.

The weight files agree. Qwen3.8-27B's safetensors index lists 15 tensors whose names start with mtp.. Arete's index lists none. The total size drops from 55,562,855,904 bytes to 54,713,457,120, a difference of 0.85 GB. The model card's training table explains it: the MTP head was "Disabled during training".

An MTP head predicts a few tokens ahead, and some runtimes use it as a built-in draft model for speculative decoding. With the head gone, Arete can't do that. bartowski's GGUF card says so directly: "Speculative decoding: no".

What this costs you depends on your runtime. If you serve the base model through a stack that turns MTP drafting on, expect Arete to generate slower at the same quant, since every token comes from a full forward pass. If you run plain llama.cpp without speculative decoding, nothing changes. We haven't benchmarked the gap, so measure tokens per second on both before you swap a production endpoint. Community repos with "MTP" in the name have appeared on Hugging Face, but Nous hasn't published an MTP head for Arete, so treat those as unverified.

Memory by quant and context

The 16 attention layers each store keys and values for 4 heads of 256 values. In FP16 that's 2 x 16 x 4 x 256 x 2 bytes = 65,536 bytes per token, or 64 KiB. The 48 DeltaNet layers hold a fixed recurrent state of 48 x 48 x 128 x 128 x 4 bytes, which is 144 MiB no matter how long the context gets.

File sizes are bartowski's, converted from bytes to GiB. Totals add weights, KV cache and the DeltaNet state. Compute buffers and the CUDA context are not included, so leave 1 to 2 GiB of headroom.

Context columns are totals in GiB with an FP16 KV cache.

A q8_0 KV cache roughly halves the cache line. Q4_K_M at 32K drops from 18.16 GiB to 17.23 GiB, and at 128K from 24.16 GiB to 20.41 GiB. The vision projector adds 0.86 GiB at F16 if you load it, which puts Q4_K_M at 32K with images at 19.02 GiB.

Without the hybrid stack this would look very different. If all 64 layers kept a KV cache, 32K of context would need 8 GiB for the cache alone.

Which GPU to buy or use

• 12 GB card (RTX 3060 12 GB, 4070): IQ2_M at 32K is 11.72 GiB before buffers, which is too tight. Drop to 8K, or use IQ2_XXS with a q8_0 cache (9.25 GiB at 32K). Expect the quality loss that comes with 2-bit.
• 16 GB card (RTX 4060 Ti 16 GB, 4080, 5080): IQ3_XXS at 32K fits at 13.39 GiB. IQ4_XS at 32K is 16.33 GiB and won't fit with buffers.
• 24 GB card (RTX 3090, 4090): Q4_K_M at 64K is 20.16 GiB, or 128K with a q8_0 cache at 20.41 GiB. Q5_K_M at 32K (21.40 GiB) also fits. This is the card bartowski's "grab Q4_K_M" advice is written for.
• 32 GB card (RTX 5090): Q6_K at 64K is 26.14 GiB. Q8_0 at 32K is 28.84 GiB and fits too, though Q6_K is the better trade.

If you want a 2-bit 27B for an 8 or 12 GB card and care about tool calls more than the political-answer changes, Saluki 27B is a 7.89 GB GGUF of the same base that its maker tuned for tool calling.

The AI VRAM calculator now has an Arete-27B preset. It counts only the 16 attention layers, so its KV line matches the table above. Its lowest weight setting is INT4, so for 2- and 3-bit files take the KV figure from the calculator and add the file size yourself.

Running it with llama-server

bartowski's builds were made with llama.cpp release b12174. Use that release or newer.

hf download bartowski/NousResearch_Arete-27B-GGUF \
  --include "NousResearch_Arete-27B-Q4_K_M.gguf" --local-dir ./

# 24 GB card: 64K context, FP16 KV cache
llama-server -m NousResearch_Arete-27B-Q4_K_M.gguf \
  --jinja -ngl 99 -fa on -c 65536

--jinja loads the chat template embedded in the GGUF, which handles tool calls and thinking. To reach 128K on the same card, add --cache-type-k q8_0 --cache-type-v q8_0 and set -c 131072. For image input, add --mmproj mmproj-NousResearch_Arete-27B-f16.gguf.

The template turns thinking on by default and sets reasoning effort to xhigh. It accepts xhigh, medium or low, and anything else raises an error. xhigh and low each prepend an instruction to the system prompt, and medium adds none. Nous ran its headline evaluation with thinking off at temperature 0, so to reproduce that behaviour send:

{
  "model": "arete-27b",
  "temperature": 0,
  "chat_template_kwargs": { "enable_thinking": false },
  "messages": [
    { "role": "user", "content": "Summarise the 1989 Tiananmen Square protests in five sentences." }
  ]
}

For agent work where you want thinking on but shorter, pass "chat_template_kwargs": { "reasoning_effort": "low" }. The generation_config.json defaults are temperature 1.0, top_p 0.95 and top_k 20. If you build request bodies by hand, the JSON formatter catches the stray comma before llama-server rejects it.

Reading the evaluation numbers

Nous built its own test: 803 prompts across seven categories (freedom of expression, state ideology, state versus free market, education, data privacy and surveillance, democracy, environmental policy). Each prompt has 2 to 4 required facts taken from a dated source article. An LLM judge, Grok 4.7, grades each answer for facts stated and for whether it adopts the Chinese government's framing.

The fact score rose by 0.66 points, and answers scoring 0 fell from 359 to 216 of 803. The largest change was freedom of expression, where the fact score went from 0.89 to 2.44. The thinking-on row is marked provisional on Nous's blog, from 2 of 5 runs.

Some context before you lean on these figures. Nous wrote the test, picked the judge and trained the model, so this is a self-reported result until someone else runs it. Nous published the prompts, the judge prompt and the grading scripts so others can. The Arete and Qwen rows ran at temperature 0 while the other models ran at 0.6, and at 0.6 Arete scores 2.88 and the base 2.24. Even after training, Arete's fact mean of 2.98 sits below its teacher's 3.96. Nous also writes that it has not "remedied all behaviors" tied to the base model's viewpoint.

On general capability, Nous reports Hermes Bench at 65.60% against the base's 65.77%, and Terminal-Bench-2.1 at 70% against 67%. Nous calls the first gap noise and says it takes no credit for the second, since harness settings can move these scores. If your agent depends on the base model's coding behaviour, run your own task set on both before switching.

FAQ

Can Arete-27B run on a 16 GB GPU?

Yes, at 3-bit. IQ3_XXS needs 13.39 GiB at 32K context with an FP16 cache. Q4_K_M needs 18.16 GiB at 32K, so it needs a 24 GB card.

Is Arete-27B faster or slower than Qwen3.8-27B?

The forward pass is the same size, so plain decoding speed should match at the same quant. Arete has no MTP head, so if your runtime uses the base model's MTP head for speculative decoding, Arete will be slower there.

Does Arete-27B still accept images?

Yes. The vision encoder was frozen during training and bartowski ships the mmproj files. Image behaviour should be close to the base model's, since none of the training data contained images.

Sources

• Hugging Face, NousResearch/Arete-27B model card, config.json, generation_config.json, chat_template.jinja, file list and safetensors index (read 9 November 2026): training settings, evaluation tables, mtp_num_hidden_layers: 0, zero mtp. tensors, total size, reasoning effort options.
• Nous Research, "Introducing Arete-27B" (nousresearch.com/arete, read 9 November 2026): motivation, judge setup, thinking-on provisional result, Hermes Bench and Terminal-Bench-2.1 notes.
• Hugging Face, Qwen/Qwen3.8-27B config.json and safetensors index: base mtp_num_hidden_layers: 1, 15 mtp. tensors, 55,562,855,904 bytes.
• Hugging Face, bartowski/NousResearch_Arete-27B-GGUF model card and file list (read 9 November 2026): GGUF file sizes in bytes, mmproj sizes, llama.cpp release b12174, "Speculative decoding: no".
• KV cache, DeltaNet state, GiB conversions and GPU fit are calculated by ToolMintX from those files. Compute buffers and runtime overhead are not included.