What your AI feature does when the credit runs out

A feature that worked yesterday starts answering everyone with the same apology. The logs are full of one line repeated ten thousand times, the service itself is healthy, and the cause is a number in somebody else's dashboard that reached zero overnight. A model provider stopped accepting calls, and the code had exactly one plan for that.

The four ways a provider says no

The failures arrive over the same channel and look similar in a stack trace, but they need different treatment:

1. Billing. The account has no credit, the card was declined, or the plan expired. This is terminal. It will fail identically in one second and in one hour.
2. Quota. A rate limit, a token per minute cap, or a daily allowance. This is temporary and usually carries a hint about when to come back.
3. Outage. The provider is degraded or a region is down. Temporary, unpredictable, often partial, so one model answers while another does not.
4. Our own mistake. A malformed request, a context that grew past the limit, a parameter the endpoint does not accept. Retrying this is pointless until the request changes.

Most codebases collapse all four into one branch, because the first version of the integration had a try block around the call and a toast that said something went wrong. That branch is correct for exactly zero of the four cases. Rate limits in particular deserve their own path, for the reasons in treating a rate limit as a permanent failure.

The second half of the problem is that nothing was watching the money. Uptime monitoring watches processes, ports and response codes. A prepaid balance dropping toward zero produces no alert at all until it produces every alert at once.

How to see which one you have

Stop swallowing the response body. The status code alone is ambiguous, because providers overload 429 for both a per minute limit and an exhausted plan, and some report a billing problem as 403. Classify on status plus body, and log the class:

type ProviderFailure =
  | { kind: 'billing' }                        // terminal until someone pays
  | { kind: 'quota'; retryAfterMs: number }    // come back later
  | { kind: 'transient' }                      // retry with backoff
  | { kind: 'request' };                       // our bug, fix the call

function classify(status: number, body: string): ProviderFailure {
  const text = body.toLowerCase();
  if (status === 402 || /credit|balance|billing|payment/.test(text)) return { kind: 'billing' };
  if (status === 429) return { kind: 'quota', retryAfterMs: parseRetryAfter(body) ?? 20_000 };
  if (status >= 500 || status === 408) return { kind: 'transient' };
  return { kind: 'request' };
}

With that class written on every failed call, one query answers the question that matters during an incident:

SELECT failure_kind, count(*), max(created_at) AS last_seen
FROM model_calls
WHERE created_at >= now() - interval '30 minutes' AND failure_kind IS NOT NULL
GROUP BY failure_kind
ORDER BY 2 DESC;

Ten thousand rows of billing and zero of anything else is a different incident from a mix of transient and quota, and you want to know which one you are in before you start editing code.

The fix: a ladder, not a branch

Write the degradation as an ordered list of attempts, each one cheaper and less capable than the last, with a rule that says when to stop climbing down:

const ladder = [
  { name: 'primary_large',   run: () => call(primary,   'large', messages) },
  { name: 'secondary_large', run: () => call(secondary, 'large', messages) },
  { name: 'primary_small',   run: () => call(primary,   'small', messages) },
  { name: 'cached',          run: () => cachedAnswer(cacheKey) },
  { name: 'static',          run: () => templateAnswer(intent) }
];

for (const step of ladder) {
  const result = await attempt(step);
  if (result.ok) return { ...result, servedBy: step.name };
  if (result.failure.kind === 'request') break;   // our bug, the next step fails too
}
return honestMessage();

Four things make that list work rather than just look tidy.

The order follows the failure, not the price list. A billing failure on the primary provider means every model there is gone, so the next step must be a different provider, not a smaller model on the same one. A quota failure on one model often leaves a smaller one available, so that ordering flips. Pass the failure class into the chooser instead of hard coding one sequence.

A request level mistake stops the ladder. If the context is too long or the schema is wrong, every step below will fail for the same reason, five times slower and five times more expensively.

The answer carries its origin. Return servedBy with the response, store it on the record, and show it where it matters. A draft written by the small model and a draft written from a template are not interchangeable to the person who is about to send it.

The feature has a state, not just a result. Keep a small circuit breaker per route so that after a handful of billing failures the primary is skipped entirely for the next few minutes, and expose that state:

curl -s http://localhost:3000/health/ai | jq .
# {
#   "primary":   { "state": "open",   "reason": "billing", "since": "2026-04-06T02:14:11Z" },
#   "secondary": { "state": "closed", "reason": null },
#   "serving":   "secondary_large"
# }

Then alert on the balance instead of the crash. Read the remaining credit on a schedule, divide it by the last seven days of spend, and alert on days of runway:

# runway in days, from the cost table built for the cost work
days=$(psql -At -c "SELECT round(:balance / (sum(cost_micros)/1e6/7.0), 1)
                    FROM model_calls WHERE created_at >= now() - interval '7 days'")
[ "${days%.*}" -lt 7 ] && notify "AI credit runway is ${days} days"

Seven days is a threshold a person can act on. Zero is a threshold that pages somebody at two in the morning. The per feature cost table from keeping an AI feature affordable already has the numbers for this.

How to check it worked

Do not wait for a real outage. Inject the failure in staging and assert on the answer, not on the absence of an exception:

AI_FAULT=billing:primary npm run test:integration

# assertion output
# primary returns 402 -> ladder falls to secondary_large     ok
# secondary also 402  -> ladder falls to cached              ok
# no cache entry      -> honest message, HTTP 200            ok
# retries against primary during the window: 0               ok

That last line is the one people forget. A fallback that still sends a hundred doomed requests per minute to a dead provider is a fallback on paper only.

What to watch out for

• The secondary provider needs its own credit, its own quota and its own monitoring. A backup nobody has called in three months is an untested backup, so send a small share of live traffic through it every day.
• Cached answers go stale. Label them, cap their age, and never serve a cached answer for a request whose inputs differ from the cached one in any way that matters.
• Streaming needs its own fallback. A call that fails halfway through a stream has already sent tokens to the browser, and the client has to be able to replace them. That path has the same hazards as a streamed response that never closes.
• Prompts drift apart. When the prompt changes for the primary route and nobody runs the evaluation set against the secondary, the fallback silently gets worse until the day you need it.

Every integration with something you do not run has a state you cannot see, and money is one of the most common versions of it. The useful habit is to treat the provider as a dependency with a health of its own: classify its failures, keep a route that does not share its failure modes, and alert on the thing that predicts the outage rather than on the outage. A feature that answers with a smaller model and says so is still a working feature. A feature that answers with a generic error is a support ticket for every user who tried.