Docs Find and fix real problems

AI feature failing or costing too much

Find why the AI feature in your product fails (often a provider error like a rate limit), catch a cost spike before the bill does, and see which of your MCP tools fail for AI assistants.

AI features fail in their own ways: the model provider refuses a call, a loop sends far more tokens than it should, or a tool the model calls breaks. Relia records every model call your product makes as a run, with its steps, tokens, estimated cost and whether it failed, and raises the problems.

The example is Relia's demo shop: its support assistant sometimes gives no answer, has started costing far more than usual, and one of its MCP tools fails.

Before you start

  • Relia's server SDK in the app that calls the model: relia-sdk 0.8 or later for Node.js (npm i relia-sdk@latest), 0.6 or later for Python (pip install -U relia-sdk). Calls to OpenAI, Anthropic, Gemini, OpenRouter and other common providers are recorded with no code change. See Monitor AI calls in your app.
  • Costs are estimates from public list prices. A model Relia doesn't know is counted at zero cost rather than a guess.

What you'll see

Relia raises three kinds of AI problems, all under the AI in your product part on Home:

Problem Example headline Level
Runs fail Your AI agent support-bot is failing Needs you once at least 3 runs fail
Spend jumps Your AI agent support-bot is costing far more than usual Watch
An MCP tool fails Your MCP tool search_products fails for AI assistants (claude-code) Needs you once at least 3 calls fail

A cost spike means at least $1 and at least 3 times the usual spend for the same length of time (or at least $5 when there is no history yet). A failing MCP tool means at least 3 failed calls that are either at least 10% of its calls and twice its usual share, or half or more of all its calls. Relia also flags runs and tools that get much slower.

Find the cause

Failing runs

  1. Open the incident. It gives the numbers, for example failed 11 of 184 runs (6%). Do this next says to open the failing runs and check the model provider error (API key, quota, timeout).
  2. Open AI in your product, then Runs, and choose Errors Only.
  3. Click a failed run. Its steps show each model call and tool call in order. The failed step shows the error, for example rate limited by provider, 429.

Common provider errors: 429 (rate limit or quota used up), 401 (a wrong or revoked API key), 400 (a request the model can't take, often a prompt that is too long), timeouts and 5xx (the provider's side).

A cost spike

  1. Open the incident. It compares spend with usual, for example $8.70 in 190 runs, usually about $0.46.
  2. Open Runs and look at the Cost (USD) column: compare recent runs with older ones.
  3. Open Agent issues. Token Bloat marks runs that used 50,000 tokens or more, or at least 3 times the agent's usual.

A failing MCP tool

If your product offers tools to AI assistants through an MCP server, open AI in your product, then Tool calls. Your MCP server lists each tool with its calls, failures, p95 and Called by (which assistants called it). Click a tool to see its latest calls, and open one to see the error in its request. What the assistant asked and what the tool answered are never recorded.

Tip: watch Monitor AI agents, LLM runs, tool calls and your MCP server (2 min).

Fix it

  • Rate limit or quota: check your quota with the provider, raise it, or add retries with backoff. Spread heavy work out instead of sending it all at once.
  • Key errors: replace the key in your host's environment variables.
  • Cost spike: look for a loop that repeats model calls, retries that pile up, a switch to a bigger model, or prompts that grew (for example a growing chat history). Cap the steps or tokens per run.
  • Failing tool: fix the error inside the tool. Assistants get it as an error answer and may give up.

On Growth and Pro, Show how to fix on the incident writes steps from the evidence.

Check the fix worked

Deploy your change. Relia picks up a new version of the service by itself, or click I fixed it on the incident. Relia watches real runs and calls and tells you Your fix worked or Still happening after your fix.

Next

Last checked against the product: 1 Nov 2026