Unlimited OCR 2 VRAM: 300-Page PDFs, 1.77 GB KV

Baidu released Unlimited OCR 2 on 27 October 2026, a 3B document parser that reads a whole PDF in one request instead of page by page. The headline claim is a 300-page document on a fixed memory budget. The model card shows how: peak KV cache for a hundred-page document drops from 47.00 GB to 1.77 GB once the document runs in page windows. The weights take another 6.69 GB.

That makes a long-PDF parser something you could run on one workstation card. Baidu only tested it on an NVIDIA A800 80 GB, though, and the serving path depends on a custom SGLang build. I read the config, the file list and the technical report, then worked out what each part costs in memory.

Table of contents

• What Baidu released
• Memory for a long PDF
• How the page windows work
• Speed from the paper
• Benchmarks and what they leave out
• Running it with SGLang
• FAQ
• Sources

What Baidu released

Unlimited OCR 2 is built on the first Unlimited OCR (June 2026), which itself extends DeepSeek-OCR. It keeps the DeepEncoder vision front end and a mixture-of-experts decoder with 3B total and about 0.5B active parameters per token. The repo is MIT licensed.

From config.json:

The new part is the attention. Every decoder layer now uses Sparse Reference Sliding Window Attention (SR-SWA). A "page gate" limits each new token to blocks from the page it is transcribing. A "token sieve" then keeps only the top blocks within a fixed budget. Each token attends to a set of visual tokens that does not grow with document length, so the cost of one decoding step stays flat whether you are on page 5 or page 250.

Memory for a long PDF

The KV cache is where long documents usually fail. With 12 layers, 10 KV heads and a head size of 128, each token stores 2 x 12 x 10 x 128 x 2 bytes in BF16. That is 60 KiB per token, large for a 3B model, because none of the heads share keys.

In base mode a page is 273 image tokens. If the model writes about 1,024 tokens per page (the length the paper uses for its speed test), one page holds 1,297 tokens of cache:

My numbers are a floor. Single pages default to "gundam" mode, which tiles each page into extra crops and adds image tokens. The paper measured 47.00 GB of peak KV with standard decoding across MarathonDocBench, whose documents run from 103 to 300 pages. With the window scheme turned on, the same run peaked at 1.77 GB.

So a single document needs about 6.69 GB of weights plus 1.77 GB of KV, plus activations and the vision encoder's working memory. The README warns that one SAM attention-bias allocation can reach about 462 MiB at its default chunk size. The parts with published sizes add up to about 8.9 GB at batch 1, before activations. Nobody has published a run on a consumer card yet, so treat a 12 GB or 16 GB card as plausible rather than proven.

The serving setup reserves 70% of GPU memory for weights and KV (--mem-fraction-static 0.7). Applied to smaller cards, that leaves:

Batch 16, the paper's throughput setting, runs sixteen documents at once and multiplies the KV need, so that's an 80 GB-class job. For a preset-based estimate on other models, the AI VRAM calculator does the same weights-plus-KV sum.

How the page windows work

The memory drop comes from the "Unlimited-Horizon Decoding" scheme. SGLang splits the document into overlapping page windows and processes them in order. Neighbouring windows share their boundary page, and that page's cache is reused so a paragraph or table that runs across the break keeps its context. Caches outside the active window are freed.

Four environment variables control it:

export SGLANG_UOCR_EANABLE_DOC_LIMIT=1        # 1 = page windows, 0 = whole document at once
export SGLANG_UOCR_DOC_GUNDAM_PAGE_WINDOW=10  # pages per window, gundam mode
export SGLANG_UOCR_DOC_BASE_PAGE_WINDOW=10    # pages per window, base mode
export SGLANG_UOCR_MAX_PAGE_TOKENS=20000      # output cap per page, must be above 0

EANABLE is misspelled in Baidu's own code. Copy it as written, or the variable is silently ignored and the whole document loads at once. A smaller window lowers peak memory, and the README notes the cost: blocks that continue across a window boundary lose context. On a tight card, I'd drop the window to 5 pages before lowering anything else and check a few cross-page tables by hand.

Speed from the paper

The paper reports two speed figures:

• At a hundred pages, one decoding step is 7.0x faster than full attention and 5.6x faster than R-SWA, the windowed attention the first Unlimited OCR used. That test was run in gundam mode at batch 3 with 1,024 output tokens per page.
• End to end on MarathonDocBench at batch 16, throughput rises from 360 to 1,315 tokens per second with page windows on. That's 3.65x.

The paper doesn't name the GPU behind the throughput table. Training ran on H800 nodes and the README's tests on an A800 80 GB, so assume a data-centre card.

A worked example at that rate: sixteen 100-page reports at 1,024 tokens per page is 1,638,400 output tokens. At 1,315 tokens per second, that's about 20.8 minutes for the batch. Standard decoding at 360 tokens per second would take about 75.9 minutes, if it fit in memory at all. For a single document on a smaller card, expect fewer tokens per second, since batch 1 leaves most of the GPU idle.

Benchmarks and what they leave out

Baidu reports three benchmarks. Higher Overall is better.

The gap on hundred-page documents is 43.32 points over the next model. On the shorter MPDocBench it's 6.29 points over Qwen3.8-Max. The first Unlimited OCR is left out of MarathonDocBench entirely because it stops at 40 pages.

Some caveats the launch post doesn't make:

• MarathonDocBench is Baidu's own benchmark, released the same day. Its labels were first produced by "a strong page-by-page parser" and then corrected by people and a VLM pipeline.
• The training labels came from MinerU2.5-Pro and PaddleOCR-VL-v1.6. Neither is in the comparison tables.
• On single pages it isn't the leader. On OmniDocBench v1.6 it scores 94.02, behind TeleOCR (96.91, 1.2B) and OvisOCR2 (96.47, 0.8B), and barely ahead of the first Unlimited OCR (93.92). For scans of one page at a time, a smaller model does better.
• Heading structure is the weak spot. HeadingTEDS is 48.20 on MarathonDocBench, and the limitations section says the sparse attention "does not preserve global document information". Expect to fix heading levels in the Markdown output.

Running it with SGLang

The quick test path is plain Transformers with trust_remote_code=True, pinned to transformers==4.52.1 and torch==2.13.0 on Python 3.10. Use model.infer_multi(...) with a list of page images. For real documents, serve it with the customised SGLang wheel that ships inside the model repo:

uv venv --python 3.10 && source .venv/bin/activate
uv pip install wheel/sglang-0.5.20.dev10+g8d62f5fb9c-py3-none-any.whl
uv pip install pymupdf==1.27.2.2 fastokens==0.3.2

export SGLANG_UOCR_EANABLE_DOC_LIMIT=1
export SGLANG_OPT_UOCR_SAM_REL_POS_BIAS_MB=64
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

python -m sglang.launch_server --model-path baidu/Unlimited-OCR-2 \
  --served-model-name Unlimited-OCR-2 --chat-template unlimited-ocr \
  --tokenizer-backend fastokens --mem-fraction-static 0.7 \
  --max-prefill-tokens 2048 --enable-custom-logit-processor \
  --enable-streaming-session --disable-radix-cache --port 10000

The full command in the README adds CUDA-graph flags tuned for 64 concurrent requests. Leave those out on a small card. The largest --cuda-graph-bs-prefill bucket has to equal --max-prefill-tokens, and halving --max-prefill-tokens from 4096 to 2048 halves one temporary tensor.

Requests go to the OpenAI-style /v1/chat/completions endpoint. Each page is a base64 image, and the prompt is "<PAGE><image>" repeated once per page followed by Multi page parsing. Baidu's client renders PDFs at 300 dpi with PyMuPDF and sets max_tokens to 8,000 per page. It also passes an n-gram logit processor to stop repetition loops, with page_token_id 128827 and eos_token_id 1. Skip the processor and long documents can loop on a repeated line.

If you only need text from a screenshot or a single scanned page, the browser-based image to text tool runs Tesseract locally and needs no GPU. To try Unlimited OCR 2 on part of a large file first, cut a 20-page sample with Split PDF.

FAQ

Does it run on Windows or Mac? The custom SGLang wheel and the Triton kernels in the repo target Linux with NVIDIA CUDA. Baidu lists no Mac or AMD path. The Transformers route calls .cuda(), so it needs an NVIDIA card too.

Can I use it for Hindi or other Indian scripts? The model card tags it as multilingual but publishes no per-language scores, and none of the three benchmarks reports Indian scripts. Test on your own pages before you trust it.

Is there a hosted version? Baidu says it's on Baidu Cloud, and there's a free Hugging Face Space demo. I didn't check Baidu Cloud's pricing, so this post only costs the local route.

Sources

• Hugging Face, baidu/Unlimited-OCR-2 model card, config.json and file listing (read 29 October 2026): release dates, layer and head counts, 273 image tokens per page, BF16 parameter total, file sizes, MIT license, SGLang and Transformers instructions, environment variables, memory notes.
• Baidu ERNIE Team, "Unlimited OCR Works 2" technical report (Unlimited-OCR-2.pdf in the model repo): architecture, SR-SWA, MarathonDocBench design, Tables 1 to 4, the 7.0x and 5.6x speedups, training data and hardware, limitations.
• GitHub, baidu/Unlimited-OCR README and LICENSE.
• KV cache per token, window memory, card budgets and batch times are calculated by ToolMintX from the figures above.