A Batch Task Cost an Extra $200 in a Month: The Bill Exploded Before the Errors Did—Track Token Costs for Every Task

During last month’s reconciliation, we noticed that the daily cost of our batch export pipeline had risen from $0.30 to nearly $1.00. No one received any alerts because we hadn’t set up any cost-related monitoring. The system didn’t crash, time out, or degrade in quality; it just quietly became a bit more expensive each day. This kind of "failure" is much harder to detect than a crash: crashes scream for attention, while cost drift silently settles the bill at the end of the month.

Let’s clarify how that extra $200 slipped through. There were three causes, none of which were particularly complex:

First, one task stuffed the entire context into the prompt. Instructions that could have been clearly conveyed with 8,000 tokens ended up including 40,000 tokens of related documentation. At the time, no one thought twice about it since the cost per single run was only a few dollars.

Second, the retry logic after hitting 429 rate limits was unintelligent. Each retry resent the full conversation history, tripling the cost for three retries, while failed calls were still billed.

Third, an extra "just summarize it again" call was added to a certain task type. No one remembered why it was added initially, and no one remembered to remove it.

Individually, each issue seemed "manageable," but combined, they tripled the unit cost.

Therefore, the fix we implemented focused on just one thing: **track costs for every task, and take action when the numbers don’t add up.**

1. Track Calls at the Invocation Layer, Settle Accounts at the Task Level

We attached a hook to the router’s invocation wrapper, logging a record for every model call:

ledger.insert({
    "task_id": task_id,          # Unique per run
    "task_type": "batch_export", # Budgeting is tied to this granularity
    "model": model_name,         # Pin the version; do not use 'latest'
    "prompt_tokens": usage.prompt_tokens,
    "completion_tokens": usage.completion_tokens,
    "cost_usd": cost,
    "ts": now(),
})

A SQLite table with fewer than twenty fields is sufficient. The key is `task_id`: calls are attributed to specific tasks. When a task completes, all its invocation records are aggregated by `task_id` into a single summary line stating, "This task cost X dollars," along with a `status` column indicating success or failure.

2. The Denominator Should Be "Cost per Success," Not "Cost per Call"

This step is where things often go wrong. If you calculate cost based on invocations, a task that makes 10 calls costing $5 might seem cheap. However, if 8 of those were retries and only 2 successful outputs were produced, the actual cost per output is $2.50, not $0.50.

Therefore, our reports focus on a single metric: **`cost_usd` divided by the number of successful outputs**. Failed calls are not considered "waste" but rather part of the cost—because it is the retries themselves that make the output more expensive.

3. Budgets Are Gates, Not Just Reports

Tracking costs without taking action is pointless. We set budget caps for each `task_type`, such as $0.50 per run for `batch_export`. The execution logic is straightforward:

- During task execution, check the cumulative cost after each call. If it exceeds 80%, log a warning; if it exceeds 100%, **terminate the task immediately** and mark it with `status=over_budget`.

- Tasks that exceed the budget are not retried. Retries are justified when failures have external causes, but exceeding the budget indicates a flaw in the task design itself; retrying would only incur additional costs.

- Every morning, send a summary report listing yesterday’s `over_budget` tasks, including their `task_id` and final costs.

Terminating over-budget tasks comes with a minor trade-off: occasionally, a task that might have succeeded with just one more step gets killed prematurely. However, the cost of such a false positive is far lower than letting a task run wild, and the complete ledger left behind makes post-mortem analysis quick and easy.

4. Weekly Review of the "Most Expensive Tasks"

After two weeks of tracking, we streamlined our reporting into a single query:

SELECT task_type,
       COUNT(*)            AS runs,
       SUM(cost_usd) / SUM(CASE WHEN status='success' THEN success_count ELSE 0 END) AS cost_per_success
FROM task_summary
GROUP BY task_type
ORDER BY cost_per_success DESC;

The top row identifies the target for optimization this week. After improvements, its position in next week’s report should drop. If it doesn’t, it means the changes didn’t hit the right spot.

The bill exploding before errors appear is unavoidable when using pay-per-use models. What we can avoid is seeing these numbers for the first time only when the bill arrives. The value of ledger tracking isn’t just about saving money; it transforms "where is the money burning?" from a monthly shock into a daily glance at a single line of data.