A 12-megapixel phone photo is around 4MB and contains far more detail than any current vision model uses. Resizing it in the browser before it leaves the device makes the upload faster, the request cheaper and the answer no worse. The question is what to resize to, and that has an arithmetic answer.
How much resolution is worth paying for
Vision models do not see pixels. They cut the image into fixed-size patches or tiles, encode each one, and feed the resulting vectors into the model as tokens. The consequence is that image cost is roughly proportional to area, and doubling each side quadruples what you pay.
Tiles for a tile size of 512×512, which is a common choice:
512 × 512 → 1 × 1 = 1 tile
1024 × 1024 → 2 × 2 = 4 tiles
1536 × 1536 → 3 × 3 = 9 tiles
2048 × 2048 → 4 × 4 = 16 tiles
4032 × 3024 → 8 × 6 = 48 tiles (a typical phone photo)
So the phone photo costs about 48× what a single-tile thumbnail costs,
and about 12× what a 1024×1024 version costs.
If a tile is T tokens and there is a fixed base cost B:
total tokens ≈ B + T × ceil(w / 512) × ceil(h / 512)
The tile size, the tokens per tile and the base cost are all provider-specific and model-specific, and they change. The shape of the formula — fixed cost plus a per-tile cost multiplied by a tile count that grows with area — is what holds across providers. Put your provider’s current numbers into the expression above rather than trusting a figure printed anywhere; if you are processing several images.
// resize.ts — runs in the browser, no dependencies
export type Resized = { blob: Blob; width: number; height: number; bytes: number };
export async function resizeImage(
file: File,
maxEdge = 1024,
quality = 0.82,
): Promise<Resized> {
// Decoding a 12MP JPEG is the expensive step; createImageBitmap does it
// off the main thread and can downscale during decode.
const probe = await createImageBitmap(file);
const scale = Math.min(1, maxEdge / Math.max(probe.width, probe.height));
const width = Math.round(probe.width * scale);
const height = Math.round(probe.height * scale);
probe.close();
const bitmap = await createImageBitmap(file, {
resizeWidth: width,
resizeHeight: height,
resizeQuality: "high",
});
const canvas = new OffscreenCanvas(width, height);
const ctx = canvas.getContext("2d");
if (!ctx) throw new Error("2d context unavailable");
ctx.drawImage(bitmap, 0, 0);
bitmap.close(); // release the decoded pixels promptly
const blob = await canvas.convertToBlob({ type: "image/jpeg", quality });
return { blob, width, height, bytes: blob.size };
}
Three decisions in there are worth stating. JPEG rather than PNG for photographs, because a PNG of a photograph is often five times larger for no visible difference — but PNG for screenshots and diagrams, where JPEG artefacts around text are exactly what destroys the thing you wanted the model to read. Quality 0.82 rather than 0.95, because the difference is invisible after a downscale and the file is roughly half the size. And bitmap.close(), because decoded bitmaps hold real memory and a loop over twenty images without it will make a phone browser reload the tab.
A 4032×3024 phone photo, in practice:
original JPEG ~4,000 KB
resized to 1024×768, quality 0.82 ~120 KB ~33× smaller
resized to 768×576, quality 0.82 ~70 KB ~57× smaller
Upload time on a 5 Mbit/s uplink, which is a realistic mobile figure:
4,000 KB × 8 / 5,000 kbit/s ≈ 6.4 s
120 KB × 8 / 5,000 kbit/s ≈ 0.19 s
That six seconds is time the user spends staring at a progress bar before the model has even started, and it is entirely removable.
Data URL or signed upload
| Approach | Description |
|---|---|
| Base64 data URL in the request | Simplest: read the blob as a data URL and put it in the message content. Base64 inflates by about 33%, so a 120KB image becomes a 160KB JSON field. Fine for one small image; poor for several, and it makes your request body large enough to hit body-size limits on some platforms. |
| Public URL | The provider fetches the image itself. Requires the image to be publicly reachable, which for user uploads is usually unacceptable, and adds a fetch you cannot see failing. |
| Signed upload, then a signed read URL | The browser PUTs straight to object storage with a short-lived signature; your server never handles the bytes. This is the one that scales, and it is the version below. |
The reason to prefer the third is not elegance. Proxying image bytes through a serverless function means paying for the bandwidth twice, holding the whole file in the function’s memory, and running into request body size limits that are far smaller than people expect. A signed upload removes all three at the cost of one extra round trip.
The signed upload, end to end
- The browser resizes and asks your server for an upload URL, sending only the content type and byte length.
- The server authenticates the user, checks quota, validates the declared size and type, and returns a short-lived signed PUT URL plus the object key.
- The browser PUTs the blob directly to storage. Your server sees none of the bytes.
- The browser calls your analyse endpoint with the key. The server generates a short-lived signed read URL and sends that to the model.
- The signed read URL expires in minutes, so the image is never publicly reachable for longer than the request needs.
// app/api/uploads/route.ts — issue the signed URL
import { auth } from "@/lib/auth";
const MAX_BYTES = 8 * 1024 * 1024;
const ALLOWED = new Set(["image/jpeg", "image/png", "image/webp"]);
export async function POST(request: Request) {
const session = await auth();
if (!session) return Response.json({ error: "sign in" }, { status: 401 });
const { contentType, bytes } = await request.json();
if (!ALLOWED.has(contentType)) {
return Response.json({ error: "unsupported type" }, { status: 400 });
}
// The client's declared size is a hint, not a guarantee. Enforce the real
// ceiling in the storage policy as well; see the note below.
if (typeof bytes !== "number" || bytes <= 0 || bytes > MAX_BYTES) {
return Response.json({ error: "too large" }, { status: 400 });
}
const key = "uploads/" + session.accountId + "/" + crypto.randomUUID();
const url = await signPutUrl(key, contentType, { expiresInSeconds: 120 });
return Response.json({ url, key });
}
// The browser side, all three steps.
export async function analyseImage(file: File, question: string) {
const { blob } = await resizeImage(file, 1024);
// 1. Ask for somewhere to put it.
const presign = await fetch("/api/uploads", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ contentType: "image/jpeg", bytes: blob.size }),
});
if (!presign.ok) throw new Error("upload refused");
const { url, key } = await presign.json();
// 2. Send the bytes straight to storage.
const put = await fetch(url, {
method: "PUT",
headers: { "Content-Type": "image/jpeg" },
body: blob,
});
if (!put.ok) throw new Error("upload failed: " + put.status);
// 3. Ask the server to run the model against the stored object.
const res = await fetch("/api/analyse", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ key, question }),
});
return res.json();
}
The declared byte length in step one is attacker-controlled. Enforce the real ceiling where it cannot be lied about: most object stores let you bind a content-length range into the signature itself, and that is the check that actually holds. Validating only the JSON field means a client can declare 100KB and upload 500MB.
Capping the bill
Images make the per-request cost variable in a way text does not, so the caps belong on the server and not in the component that submits the form.
- Cap the pixels server-side, not just client-side. The resize runs in the browser, which means an attacker skips it. Read the stored object’s dimensions before calling the model and reject or downscale anything beyond your ceiling. This is the single most important control on the page.
- Cap images per request and per user per hour. The same limiter as everything else — and
-
SOCIAL SHARE CARD GENERATOR