# AI feature failing or costing too much

> Find why the AI feature in your product fails (often a provider error like a rate limit), catch a cost spike before the bill does, and see which of your MCP tools fail for AI assistants.

Section: Find and fix real problems · Page: https://tryrelia.com/docs/ai-feature-failing · Last checked against the product: 2026-11-01
Video walkthrough: https://youtu.be/OCDIWdXLIQs

AI features fail in their own ways: the model provider refuses a call, a loop sends far more tokens than it should, or a tool the model calls breaks. Relia records every model call your product makes as a **run**, with its steps, tokens, estimated cost and whether it failed, and raises the problems.

The example is Relia's demo shop: its support assistant sometimes gives no answer, has started costing far more than usual, and one of its MCP tools fails.

## Before you start

- Relia's server SDK in the app that calls the model: `relia-sdk` 0.8 or later for Node.js (`npm i relia-sdk@latest`), 0.6 or later for Python (`pip install -U relia-sdk`). Calls to OpenAI, Anthropic, Gemini, OpenRouter and other common providers are recorded with no code change. See [Monitor AI calls in your app](https://tryrelia.com/docs/ai-calls-setup.md).
- Costs are estimates from public list prices. A model Relia doesn't know is counted at zero cost rather than a guess.

## What you'll see

Relia raises three kinds of AI problems, all under the **AI in your product** part on Home:

| Problem | Example headline | Level |
|---|---|---|
| Runs fail | *Your AI agent support-bot is failing* | **Needs you** once at least 3 runs fail |
| Spend jumps | *Your AI agent support-bot is costing far more than usual* | **Watch** |
| An MCP tool fails | *Your MCP tool search_products fails for AI assistants (claude-code)* | **Needs you** once at least 3 calls fail |

A cost spike means at least $1 and at least 3 times the usual spend for the same length of time (or at least $5 when there is no history yet). A failing MCP tool means at least 3 failed calls that are either at least 10% of its calls and twice its usual share, or half or more of all its calls. Relia also flags runs and tools that get much slower.

## Find the cause

### Failing runs

1. **Open the incident.** It gives the numbers, for example *failed 11 of 184 runs (6%)*. **Do this next** says to open the failing runs and check the model provider error (API key, quota, timeout).
2. **Open AI in your product, then Runs**, and choose **Errors Only**.
3. **Click a failed run.** Its steps show each model call and tool call in order. The failed step shows the error, for example *rate limited by provider, 429*.

Common provider errors: `429` (rate limit or quota used up), `401` (a wrong or revoked API key), `400` (a request the model can't take, often a prompt that is too long), timeouts and `5xx` (the provider's side).

### A cost spike

1. **Open the incident.** It compares spend with usual, for example *$8.70 in 190 runs, usually about $0.46*.
2. **Open Runs** and look at the **Cost (USD)** column: compare recent runs with older ones.
3. **Open Agent issues.** **Token Bloat** marks runs that used 50,000 tokens or more, or at least 3 times the agent's usual.

### A failing MCP tool

If your product offers tools to AI assistants through an MCP server, open **AI in your product**, then **Tool calls**. **Your MCP server** lists each tool with its calls, failures, p95 and **Called by** (which assistants called it). Click a tool to see its latest calls, and open one to see the error in its request. What the assistant asked and what the tool answered are never recorded.

> **Tip:** watch [Monitor AI agents, LLM runs, tool calls and your MCP server](https://youtu.be/szMNhbJQauU) (2 min).

## Fix it

- **Rate limit or quota:** check your quota with the provider, raise it, or add retries with backoff. Spread heavy work out instead of sending it all at once.
- **Key errors:** replace the key in your host's environment variables.
- **Cost spike:** look for a loop that repeats model calls, retries that pile up, a switch to a bigger model, or prompts that grew (for example a growing chat history). Cap the steps or tokens per run.
- **Failing tool:** fix the error inside the tool. Assistants get it as an error answer and may give up.

On Growth and Pro, **Show how to fix** on the incident writes steps from the evidence.

## Check the fix worked

Deploy your change. Relia picks up a new version of the service by itself, or click **I fixed it** on the incident. Relia watches real runs and calls and tells you *Your fix worked* or *Still happening after your fix*.

## Next

- [Monitor AI in your product](https://tryrelia.com/docs/ai-in-your-product.md)
- [Monitor AI calls with no code](https://tryrelia.com/docs/ai-calls-setup.md)
- [Payment provider failing?](https://tryrelia.com/docs/payment-provider-failing.md): the same steps help when any outside API fails.

---
All Relia docs for AI assistants: https://tryrelia.com/llms.txt (index) · https://tryrelia.com/llms-full.txt (everything in one file)
