OpenAI API 429 and Timeout Errors in Production: Fix Guide

An OpenAI API 429 can mean two different things: a rate limit you can retry after waiting, or insufficient_quota which no retry will fix; read the error code in the response body, retry only the first with exponential backoff and jitter, and add timeouts and a fallback for slow responses.

AI features fail differently from normal APIs: calls are slow, usage is bursty, and quota is a billing issue as much as an engineering one.

Tell the Two 429s Apart

Both return HTTP 429. The body differs:

Treating insufficient_quota as retryable creates a retry storm that fails every time and delays the real fix. Log the code on every error.

Fix 1: Retry With Backoff and Jitter

The official SDKs already retry some errors, with a default of 2 retries. Tune it and set a timeout:

import OpenAI from 'openai'

const client = new OpenAI({
  maxRetries: 3,
  timeout: 30_000, // ms. The SDK default is much longer, which can outlive your function limit
})

For custom logic, honor the retry-after header when present:

async function callWithBackoff<T>(fn: () => Promise<T>, tries = 4): Promise<T> {
  for (let i = 0; i < tries; i++) {
    try {
      return await fn()
    } catch (e: any) {
      const retryable = e?.status === 429 && e?.code !== 'insufficient_quota'
      if (!retryable || i === tries - 1) throw e
      const wait = Number(e?.headers?.['retry-after']) * 1000 || 500 * 2 ** i
      await new Promise(r => setTimeout(r, wait + Math.random() * 250))
    }
  }
  throw new Error('unreachable')
}

More on retry design, budgets, and what is safe to retry in API Timeout Errors in Node.js.

Fix 2: Respect Your Own Function Timeout

LLM calls can take many seconds. If your serverless function limit is shorter than the SDK timeout, the platform kills you first with a 504. Set the client timeout below your function's maxDuration and stream the response so the user sees output sooner. See Vercel function timeouts.

Fix 3: Smooth Out Bursts With a Queue

Rate limits are per minute. A burst of 200 users hitting at once exceeds it even if average load is fine. Put non-interactive work (summaries, batch jobs) in a queue with a concurrency limit and let it drain. Monitor the worker as described in BullMQ failed jobs monitoring.

Fix 4: Cache and Reduce Tokens

• Cache identical prompts (hash the prompt and parameters) in Redis for deterministic tasks; see Redis Upstash on serverless.
• Trim system prompts and context, since tokens per minute is often the real limit.
• Use a smaller model for easy tasks.

Fix 5: Degrade Gracefully

Decide what the feature does when the model is unavailable: show a cached answer, a simpler rule-based fallback, or a clear message with a retry button. A spinner that never ends is the worst outcome.

Log What You Need to Debug

Record the model, token counts, latency, status, error.code, and the x-request-id response header. That ID lets OpenAI support trace a specific failed call.

Validate the Output Too

Model output is untrusted input. If you expect JSON, parse and validate it with a schema; see Zod validation errors. Failed parses are production errors, not edge cases.

Let Relia Watch the AI Path

Relia watches your AI-agent runs and the production paths around your AI calls. When failures pile up, it writes the incident up in plain words: what happened, why it thinks so (with a confidence level), and one thing to do next. On request it writes a fix plan, such as adding backoff, timeouts, or fallback handling, with a "Copy for your AI assistant" button; Relia doesn't change code itself.

FAQ

Why do I get 429 when I am under my rate limit?

Limits apply to both requests and tokens per minute, and bursts count. A short spike can exceed the limit even when your average does not.

What does insufficient_quota mean? Your account has no remaining credit or hit a spend limit. Retries will not help; fix billing.

What timeout should I set for LLM calls?

Shorter than your platform's function limit, typically 20-60 seconds, with streaming for long outputs.