Adding 70-language translation to an image API without paying per word

I run with int8
quantization. This is the key piece. It shrinks the model to ~1.3 GB and runs
CPU inference fast. Do not run raw transformers on CPU for this.

A multi-stage Docker build does the conversion once, so the runtime image

carries no torch:

# stage 1: convert + quantize (needs torch, thrown away)

FROM python:3.11-slim AS converter

RUN pip install ctranslate2 "transformers[sentencepiece]" torch \

    --extra-index-url https://download.pytorch.org/whl/cpu

RUN ct2-transformers-converter \

      --model facebook/nllb-200-distilled-1.3B \

      --quantization int8 --output_dir /models/nllb-ct2



# stage 2: runtime (ctranslate2 + tokenizer only)

FROM python:3.11-slim

RUN pip install ctranslate2 "transformers[sentencepiece]" fastapi "uvicorn[standard]"

COPY --from=converter /models /models

The service itself is a tiny FastAPI app with /translate and

/translate_batch. NLLB uses FLORES-200 codes (spa_Latn), so I map ISO codes

(es) and normalize typographic punctuation (em dashes, curly quotes) that the

tokenizer would otherwise drop as <unk>.

Wiring it into rendering

The render service is the only place that needs translation, and it already has

a Redis/Kvrocks cache. The flow:

1. A text field can carry a language: { "headline": { "text": "Spring Sale", "lang": "es" } }.


2. The render cache key already includes the full payload (so es and fr are
   different cache entries). On a cache miss only, translate.


3. 
   translatePayload() collects every text value with a lang, checks the
   cache (tr:<lang>:<sha(text)>), batches the misses to the translation
   service, writes results back to the cache, and swaps the text in.


4. The harness draws the translated text. The same cache key is used by the
   editor, so a translation done in the editor is reused at render time and
   vice versa.

Two operational details that mattered:

• The translation container is CPU-capped (a couple of cores). Even a burst
  of cache misses can't starve the renderer.


• It's best-effort: if the service is unavailable, it falls back to the
  original text. Translation never breaks a render.

The result

POST /v1/render

{

  "templateId": "...",

  "payload": {

    "headline": { "text": "Welcome to our spring sale", "lang": "es" }

  }

}

renders "Bienvenido a nuestra venta de primavera" onto the image. In the

editor there's a one-click "translate the whole template" button with a

searchable 70-language picker, and the bulk-CSV flow supports a per-row lang

column so a single upload can render a batch across markets.

Because every (text, language) pair is cached forever, the model only runs on

genuinely new strings, which for marketing copy is rare after warmup. A small

CPU model plus aggressive caching turned out to be a better fit than a cloud API

for this particular shape of problem.

If you want to see it in action: pixeldrive.pro.

Happy to answer questions about the setup in the comments.