Note: the stories in this post are hypothetical scenarios assembled from common failure patterns in agent deployments — not accounts of any real company.
It was 8:14 on a Monday morning when the head of finance posted in the company Slack: "Did we get hacked?" Attached was a screenshot of the weekend's AI API bill. The support-triage agent, the quiet, well-behaved bot that sorted incoming tickets, had spent $1,900 between Saturday night and Sunday afternoon. Nobody had deployed anything new. Nobody had even been online.
The company was a 38-person SaaS shop selling marketing analytics. Their agent was unremarkable: read new support tickets, look up the customer in the CRM, draft a reply for the human queue. Average cost per ticket was about nine cents. Saturday at 11:47 PM, the CRM's API started returning 500 errors. The agent did what it was built to do: retry. Then retry again. Each retry pulled in the full ticket thread, the customer's entire history, and a fresh reasoning pass. Roughly 4,200 API calls later, each one a little more expensive than the last as context grew, the CRM came back online Sunday afternoon. The agent calmly marked its retries "successful" and moved on. The bill arrived Monday.
Why nobody noticed for two days
This is the part that stings. They had monitoring. They had a dashboard with a big green monthly-spend number. The problem: the dashboard tracked the whole account, and the number moved slowly enough that $1,900 blended into a normal month's usage until someone did the math. Alerts were set on the monthly total at 90% of budget. An agent can burn a weekend's worth of money at 3 AM on a Saturday and never trip a monthly alert until it's far too late.
No human was watching because the whole point of the agent was that no human needed to watch. That's the trap. Autonomy without per-agent limits means the first time you learn about a runaway is the invoice.
The setup that stopped it
They didn't kill the agent. They didn't even rewrite it much. They put a budget on it, per agent rather than per account, and wired three tripwires around it:
- A daily cap per agent. The triage agent got $25/day. That number came from real math: its busiest day on record had cost $11.40, so $25 was headroom, not a guess. Hit the cap and the agent stops calling paid APIs until midnight.
- A monthly cap per agent. $400/month for the triage bot. Monthly caps catch the slow bleed that daily caps miss — the agent that's slightly too chatty every single day.
- Alerts at 70% and a hard stop at 100%. The 70% alert goes to the agent's owner, not to a general channel. Owners ignore general channels. They read pings with their agent's name on them.
Then they added the one thing most teams skip: a per-action cost estimate. Every tool call the agent makes declares an estimated cost before it runs. The gateway checks the estimate against the remaining budget before the call goes out, not after the invoice arrives. A retry loop against a failing API trips the cap in minutes instead of burning all weekend.
What happened next
Two months later, the CRM went down again. Same agent, same retry behavior. This time it hit the daily cap at 1:12 AM and went quiet. The owner got the 70% ping at 12:40 AM, saw the pattern, and added a five-minute backoff to the retry logic the next morning. Total cost of the incident: $17.50. The fix that mattered wasn't the backoff. It was the cap that bought them the night to write the backoff.
They also found something they didn't expect. Once every agent had a budget, the team started comparing agents. The triage bot cost $180/month. The internal research agent cost $640/month and nobody could explain why. Budgets turned out to be a debugging tool, not just a safety one. Cost per agent is a number you can argue about in a planning meeting. A single account-wide total is just weather.
How to size the caps
The most common mistake is setting caps from anxiety instead of data. Run the agent for two weeks with monitoring only (no caps, just logging) and take the worst day's spend. Multiply by 2.5. That's your daily cap. The 2.5x covers real spikes without giving a runaway room to breathe. Monthly is roughly the daily cap times 22 working days, rounded up.
Set it too tight and the agent dies mid-task at 2 AM, which is its own kind of incident. The triage team learned this the hard way: their first daily cap was $12, and the agent kept going dark during Monday-morning ticket surges. Caps that cry wolf get raised by annoyed engineers until they're meaningless. Start from measured peaks and you'll only touch them when something genuinely changes. You can watch caps enforced in real time in our policy playground — paste an action log and see allow/deny verdicts with no signup.
The honest caveats
A budget doesn't stop a bad decision. It stops an expensive one from becoming a catastrophic one. An agent with a $25 daily cap can still send a terrible email to a customer. It'll just do it cheaply. Caps bound the blast radius; they don't fix judgment. You still need action policies and approval queues for the things that matter. That's the part nobody wants to hear, because caps feel like a complete solution and they're only the floor.
Also: caps need an owner per agent. A cap that alerts a shared channel is a cap nobody owns. When we built the gateway, we learned this from watching teams set beautiful per-agent budgets and then route every alert to #engineering-general, where they died unread. Every agent gets a named human. No name, no agent in production.
And log every verdict. A cap without a record of what it stopped is just a number. Keep a tamper-evident trail of every allow and deny so the next incident review has a timeline ready — that's the job our sister platform Agent Flight Recorder was built for.
The short version
Per-agent daily caps sized from measured peaks. Per-agent monthly caps for the slow bleed. Alerts at 70% to a named owner, hard stop at 100%. Cost estimates checked before each call, not after the invoice. It's boring, it's mechanical, and it would have turned a $1,900 weekend into a $17.50 footnote.
The check has to happen before the call
There's a quiet detail in that setup that does most of the work, and it's easy to miss: the budget is checked before the API call goes out, not when the invoice arrives. Every tool call declares an estimated cost. The gateway looks at the agent, the action, the estimate, and the remaining budget, then returns a verdict. The call only goes out if the verdict is allow.
In practice it looks like this. The triage agent tries a CRM lookup at 2 AM:
{
"agent": "triage-bot",
"action": "crm.lookup_customer",
"estimated_cost": 0.012,
"daily_remaining": 24.83,
"verdict": "allow"
}And here's the same call forty minutes into a retry loop:
{
"agent": "triage-bot",
"action": "crm.lookup_customer",
"estimated_cost": 0.012,
"daily_remaining": 0.00,
"verdict": "deny",
"reason": "daily_cap_exhausted",
"resets_at": "00:00 UTC"
}The agent gets a clean, machine-readable no. It can log it, tell the ticket queue it's offline, or wake someone up. What it can't do is keep spending.
Invoice-time checks are archaeology. They tell you what happened. Pre-call checks are control. The difference sounds academic until it's the difference between a $17.50 footnote and a $1,900 Monday.
One honest wrinkle: estimates drift. A model call that usually costs nine cents can cost fourteen when the ticket thread is long. We recalibrate estimates monthly from actuals, and the gateway keeps a 10% buffer under the cap so a slightly wrong estimate can't overshoot into real money. If your estimates are guesses, your caps are decorations.
Make the alert land where someone actually reads it
The 70% alert almost failed in this story. The first version went to #engineering-general, where it sat between a deploy notification and a lunch poll for six hours. The fix was embarrassingly simple: the alert goes to the agent's named owner as a direct ping, and if it's unacknowledged after 30 minutes, it escalates to whoever's on call.
People read pings with their agent's name on them. They do not read channel noise. A budget alert nobody reads is a dashboard, and we already covered how well dashboards worked here.
FAQ: per-agent budgets
- How do I pick a starting daily cap for an AI agent?
- Run the agent for two weeks with monitoring only (no caps, just logging), take the worst day's spend, and multiply by 2.5. That covers real spikes without giving a runaway room to breathe. Monthly caps run about the daily cap times 22 working days, rounded up. Start from measured peaks, not anxiety.
- What happens when an agent hits its budget cap?
- With a proxy gateway in place, the next paid tool call is denied with a machine-readable reason (for example,
daily_cap_exhaustedwith a reset time) before any money is spent. The agent can log the denial, pause work, or alert its owner — it simply can't keep burning budget. - Are per-agent caps better than one account-wide cap?
- Yes, because account-wide caps are blind to which agent is burning. Per-agent caps isolate runaways: one looping agent hits its own ceiling while the rest of the fleet keeps working. They also turn spend into a debugging metric — cost per agent is a number you can argue about in planning; a single account total is just weather.