How to Build a Scalable File-Processing Pipeline: Validation, Metadata Extraction, Queues, and Workers

Here's a failure I've seen more than once on upload-heavy products: the endpoint returns 200, the user closes the tab, and twenty minutes later there's no order in the system because a worker choked on a vendor's proprietary container format. Nobody told the user. Nobody told the worker's author either, because the retry policy quietly redelivered...