If you have ever built a document-parsing pipeline, you know the ritual. Split the PDF into pages. Run each page through the OCR model. Stitch the outputs back together. Then write a pile of glue code to fix the tables that got cut in half at a page boundary, and the heading that lost its section, and the footnote that ended up orphaned three pages away from its reference.
Baidu's you can drop a file into.
Path 1: Transformers (start here)
This is the fastest way to a working result. Set up the environment:
python -m venv .venv
source .venv/bin/activate
pip install torch==2.10.0 torchvision==0.25.0 transformers==4.57.1
pip install Pillow==12.1.1 matplotlib==3.10.8 einops==0.8.2
pip install addict==2.4.0 easydict==1.13 pymupdf==1.27.2.2 psutil==7.2.2
Pin these versions. The model ships custom modeling code via trust_remote_code, and that code is written against these specific releases. Version drift here produces confusing import errors rather than clean failures.
Now a single image:
import torch
from transformers import AutoModel, AutoTokenizer
model_name = 'baidu/Unlimited-OCR'
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(
model_name,
trust_remote_code=True,
use_safetensors=True,
torch_dtype=torch.bfloat16,
)
model = model.eval().cuda()
model.infer(
tokenizer,
prompt='<image>document parsing.',
image_file='your_image.jpg',
output_path='your/output/dir',
base_size=1024, image_size=640, crop_mode=True, # gundam mode
max_length=32768,
no_repeat_ngram_size=35, ngram_window=128,
save_results=True,
)
Two details in there matter more than they look:
The prompt must literally start with <image>. It is not decoration, it is the placeholder token the visual features get spliced into. Drop it and you will get output that looks like the model hallucinating a document it never saw.
no_repeat_ngram_size and ngram_window are load-bearing. Long-horizon generation on repetitive layouts, think a table of contents or a price list, can send the model into a loop where it happily emits the same row forever. These parameters block any 35-gram from repeating inside a sliding window. Do not remove them because they look like tuning noise.
gundam vs base
Single-image inference gives you two configurations, and the naming is not self-explanatory:
| Mode | Settings | Use for |
|---|---|---|
| gundam | base_size=1024, image_size=640, crop_mode=True | Single images, especially dense ones. Crops the image into tiles and processes each, so small text survives. |
| base | base_size=1024, image_size=1024, crop_mode=False | Whole image at once. Required for multi-page. |
Multi-page and PDF paths only support base mode. That is a hard constraint, not a default.
Multiple pages
model.infer_multi(
tokenizer,
prompt='<image>Multi page parsing.',
image_files=['page1.png', 'page2.png', 'page3.png'],
output_path='your/output/dir',
image_size=1024,
max_length=32768,
no_repeat_ngram_size=35, ngram_window=1024,
save_results=True,
)
Note that ngram_window jumps from 128 to 1024 here. Across many pages there is legitimately more repeated structure, so the repeat-detection window has to widen to keep up.
PDFs
There is no direct PDF entry point. You rasterize first, then feed the images to infer_multi:
import os, tempfile
import fitz # PyMuPDF
def pdf_to_images(pdf_path, dpi=300):
doc = fitz.open(pdf_path)
tmp_dir = tempfile.mkdtemp(prefix='pdf_ocr_')
mat = fitz.Matrix(dpi / 72, dpi / 72)
paths = []
for i, page in enumerate(doc):
out = os.path.join(tmp_dir, f'page_{i+1:04d}.png')
page.get_pixmap(matrix=mat).save(out)
paths.append(out)
doc.close()
return paths
model.infer_multi(
tokenizer,
prompt='<image>Multi page parsing.',
image_files=pdf_to_images('your_doc.pdf', dpi=300),
output_path='your/output/dir',
image_size=1024,
max_length=32768,
no_repeat_ngram_size=35, ngram_window=1024,
save_results=True,
)
300 DPI is the recommended default. Going lower to save memory costs you small text and table rules, which is usually the exact content you cared about.
Path 2: SGLang server (for anything real)
Transformers is fine for evaluating the model. For a service handling concurrent requests, run SGLang and talk to it over an OpenAI-compatible API.
Set up the environment. Note that SGLang ships as a local wheel in the repo rather than from PyPI:
uv venv --python 3.12
source .venv/bin/activate
uv pip install wheel/sglang-0.0.0.dev11416+g92e8bb79e-py3-none-any.whl
uv pip install kernels==0.11.7
uv pip install pymupdf==1.27.2.2
Small heads-up: the README prose mentions pinning kernels==0.9.0 while the command block right beneath it installs 0.11.7. Follow the command block. If you hit kernel-related errors, that mismatch is the first thing to check against the current repo state.
Launch the server:
python -m sglang.launch_server \
--model baidu/Unlimited-OCR \
--served-model-name Unlimited-OCR \
--attention-backend fa3 \
--page-size 1 \
--mem-fraction-static 0.8 \
--context-length 32768 \
--enable-custom-logit-processor \
--disable-overlap-schedule \
--skip-server-warmup \
--host 0.0.0.0 \
--port 10000
The flags that are not optional:
--enable-custom-logit-processor— the no-repeat-ngram processor runs as a custom logit processor. Without this flag, requests referencing it fail.
--attention-backend fa3— FlashAttention 3, which requires Hopper-class hardware. On older GPUs you will need a different backend.
--page-size 1and--disable-overlap-schedule— R-SWA's cache eviction does not play nicely with the usual paged-attention and overlapped-scheduling optimizations.
Then a client. The image goes in as a base64 data URL, standard OpenAI vision format, with a couple of model-specific extras:
import base64, json, os
import requests
from sglang.srt.sampling.custom_logit_processor import (
DeepseekOCRNoRepeatNGramLogitProcessor,
)
server_url = "http://127.0.0.1:10000"
session = requests.Session()
session.trust_env = False
def encode_image(image_path):
ext = os.path.splitext(image_path)[1].lower()
mime = "image/jpeg" if ext in (".jpg", ".jpeg") else f"image/{ext.lstrip('.')}"
with open(image_path, "rb") as f:
data = base64.b64encode(f.read()).decode("utf-8")
return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{data}"}}
def generate(prompt, image_paths, image_mode, ngram_window):
content = [{"type": "text", "text": prompt}] + [
encode_image(p) for p in image_paths
]
payload = {
"model": "Unlimited-OCR",
"messages": [{"role": "user", "content": content}],
"temperature": 0,
"skip_special_tokens": False,
"images_config": {"image_mode": image_mode},
"custom_logit_processor": DeepseekOCRNoRepeatNGramLogitProcessor.to_str(),
"custom_params": {"ngram_size": 35, "window_size": ngram_window},
"stream": True,
}
response = session.post(
f"{server_url}/v1/chat/completions",
headers={"Content-Type": "application/json"},
data=json.dumps(payload),
timeout=1200,
stream=True,
)
response.raise_for_status()
chunks = []
for line in response.iter_lines(chunk_size=1, decode_unicode=True):
if not line or not line.startswith("data: "):
continue
data = line[len("data: "):]
if data == "[DONE]":
break
delta = json.loads(data)["choices"][0].get("delta", {}).get("content", "")
if delta:
print(delta, end="", flush=True)
chunks.append(delta)
return "".join(chunks)
generate("document parsing.", ["your_image.jpg"],
image_mode="gundam", ngram_window=128)
temperature: 0 and skip_special_tokens: False are both deliberate. This is transcription, not generation, so any sampling randomness is pure downside. And the special tokens carry layout structure you want in the output.
The 1200 second timeout is also not paranoia. Long documents take a while, which is precisely why you want streaming.
Batch processing
The repo ships infer.py, which starts the SGLang server for you and fires concurrent requests at it:
# A directory of images
python infer.py \
--image_dir ./examples/images \
--output_dir ./outputs \
--concurrency 8 \
--image_mode gundam
# A PDF
python infer.py \
--pdf ./examples/document.pdf \
--output_dir ./outputs \
--concurrency 8 \
--image_mode gundam
Useful extras: --model_dir accepts a local path or a Hugging Face ID, --gpu sets CUDA_VISIBLE_DEVICES, and --server_log puts server output somewhere you can read it.
Start concurrency low. Eight is the documented example, but the right number depends on your VRAM and how long your documents are.
Path 3: vLLM via Docker
If you want to skip environment management entirely:
# Default, CUDA 13.0
docker pull vllm/vllm-openai:unlimited-ocr
# Hopper GPUs, CUDA 12.9
docker pull vllm/vllm-openai:unlimited-ocr-cu129
docker run --rm --gpus all --network host --ipc host \
vllm/vllm-openai:unlimited-ocr \
baidu/Unlimited-OCR \
--trust-remote-code \
--logits_processors vllm.model_executor.models.unlimited_ocr:NGramPerReqLogitsProcessor \
--no-enable-prefix-caching \
--mm-processor-cache-gb 0
Same story as SGLang regarding the ngram processor, it has to be registered at server start. Prefix caching and the multimodal processor cache are both disabled because R-SWA's fixed-window cache invalidates the assumptions those optimizations make.
Full details are in the
SOCIAL SHARE CARD GENERATOR