The scale wall
A computer vision pipeline that works on one image at one resolution isn't a pipeline. It's a prototype. The moment you move beyond controlled inputs, you hit the reality of production images: a 4K video frame, a satellite capture, a whole-slide pathology image, a high-resolution document scan. These images don't fit in a single model call. They're too large, too detailed, and too information-dense for one inference pass to handle well.
So you tile it. You divide the image into a grid of regions and run inference on each region independently. A 3×3 grid means 9 inference calls. An 8×8 grid means 64. A whole-slide pathology image at diagnostic resolution? Tens of thousands of tiles.
The orchestration problem scales directly with the image.
And as that tile count grows, so do the failure modes. Nine concurrent inference calls might all succeed. Sixty-four concurrent calls will occasionally hit a throttle limit or a timeout. At hundreds of tiles, partial failures aren't edge cases. They're expected. You need orchestration for your CV pipeline. The real requirement is that your orchestration scales with your image.
The pattern you already use
Tiled inference isn't a niche technique. It's the industry standard for any image that exceeds a model's input constraints. introduce an operation called context.map() that maps directly onto this pattern. It fans out an array of items as independent concurrent invocations, each independently checkpointed, with a configurable concurrency cap. One failed tile retries only that tile, not the entire image. The same line of code handles 9 tiles or 900.
What I built
In this post, I walk through an image analysis pipeline I built using durable functions to demonstrate this pattern concretely. The application accepts an image and divides it into an N×N grid of regions. It runs concurrent .
Trigger: The browser calls the analyze endpoint. An API Lambda fires the durable pipeline asynchronously and returns :
export async function invokeNova(
prompt: string,
imageBase64: string,
imageFormat: ImageFormat
): Promise<string> {
const response = await client.send(new ConverseCommand({
modelId: MODEL_ID,
messages: [{
role: 'user',
content: [
{ image: { format: imageFormat, source: { bytes: new Uint8Array(Buffer.from(imageBase64, 'base64')) } } },
{ text: prompt }
]
}],
inferenceConfig: { maxTokens: 512 }
}));
return response.output?.message?.content?.[0]?.text;
}
I'm using endpoint for a custom-trained detection model, or use different models for different steps entirely.
The orchestration pattern doesn't change. Only the inference call changes.
Step 3: Synthesize
After the map operation completes, all successful region findings are available as an array. The synthesize step aggregates them into a coherent scene description with overall object detection results and computer vision insights.
const successfulFindings = mapResults.succeeded()
.map(item => item.result as RegionFinding);
const synthesis = await context.step('synthesize', () =>
synthesizeFindings(successfulFindings)
);
Model selection becomes a scaling lever at this step. The tiled inference step runs N times concurrently, so you want it fast and cheap. The synthesis step runs once and needs to reason across all findings. You might want a more capable model here. Same orchestration code, different model routing per step based on the complexity of the task.
Step 4: Store
The final step persists the analysis result to . With S3 Files, the Lambda function reads the image directly from the local filesystem. No GetObject calls, no SDK overhead, no presigning. The image is a file path. At 9 tiles the difference is negligible. At 400 concurrent tiles each making a GetObject call, filesystem access becomes a meaningful optimization.
Partial failure at scale
At 9 tiles, one failure is an annoyance. You might tolerate restarting all 9. At 64 tiles, restarting all 64 because tile 47 hit a timeout is a waste of compute, time, and money. At 400 tiles, it's unacceptable. The mapResults object gives you fine-grained failure handling:
const successfulFindings = mapResults.succeeded()
.map(item => item.result as RegionFinding);
if (mapResults.failureCount > 0) {
mapResults.failed().forEach(item =>
context.logger.error('Region failed', { index: item.index, error: String(item.error) })
);
}
Successful tiles keep their checkpointed results. Failed tiles can be logged, retried independently, or excluded from the synthesis. The pipeline degrades gracefully rather than failing catastrophically.
Model selection as a scaling lever
As tile count grows, cost per inference call matters more. With 9 tiles, using a capable (expensive) model for each tile is reasonable. With 400 tiles, you want the cheapest model that produces acceptable results for the per-tile work, and reserve the capable model for the single synthesis step. The orchestration code stays identical. You change a model ID parameter, not the pipeline structure.
Real-time observability at scale
Every tile publishes its completion status through .
To experiment with scale, change the gridSize parameter when triggering the pipeline. Start with 3 (9 tiles). Try 5 (25 tiles). Push to 8 (64 tiles) and watch how the same code handles increased concurrency with checkpointed resilience.
Tiled inference is already your pattern. If you're working with images that don't fit in one model call (and at production resolution, most interesting images don't), you're already tiling, processing in parallel, and aggregating results. With durable functions, you get checkpointed, resilient orchestration for that pattern without building separate infrastructure. The context.map() call that handles 9 tiles handles 900. Your orchestration scales with your image.
This isn't a toy demo. It's the skeleton of production batch inference.
SOCIAL SHARE CARD GENERATOR