Sending a Million Rows from the Backend: Streaming, Batching, Compression & Protocol Buffers


Your database has a million rows. Your client needs them. What could go wrong?

Everything, actually. If you try to serialize a million rows into a single JSON response, you'll blow up your server's memory, timeout the request, make the client wait...