When retries, timeouts and fallbacks start interacting

You call an LLM API and it times out. So you retry. The retry gets a 429. You wait and try again. This time the provider responds, but the output fails validation. Maybe you have a fallback provider, so you call that instead. Now the interesting question is no longer whether you know how to retry an HTTP request. It is how all of those decisions... Weiterlesen: When retries, timeouts and fallbacks start interacting