On the simulated day the underwriting agent handles 48 runs. The SLO dashboard of the SLOs and alerts page is green outside the provider outage. The error rate is low. The underwriters have no complaints about the recommendations. The model recommends; a person decides, and every run ended in a controlled state.
At the end of the month the platform engineer reads the model bill, and it is far above the estimate. Nobody can say which runs spent it. The dashboards count requests and errors; none of them counts what a run cost. The answer was in the records all along: every model span carries the number of tokens it read and wrote. Nobody added them up.
This page treats cost as one more signal. It starts from the token counts on a span, turns them into the cost of a step and of a run, attributes the cost to the broker who caused it, finds the run that spent far more than it should, and shows what routing some steps to a smaller model changes. One note before any number appears: the agent here is a scripted stand-in, so these records show how the mechanism works, not how good any model is. And one more, binding on this page: every cost here is in invented units, called cost units (cu). They are not a price and say nothing about any vendor's price.
1. From a token count to a cost#
Start with the smallest piece. A token is the unit a model reads and writes: a word or part of a word. A model call reads some tokens (the prompt, the retrieved passages, the conversation so far) and writes some tokens (the answer). The model span of GenAI telemetry records both numbers. For the last model step of case UW-006, the worker's log line says it:
2026-09-14 14:07:19 INFO [agent-worker] write_recommendation finished: 723 tokens in, 111 out (large-model)
A rate turns the counts into cost: so many cost units per thousand tokens read, and a higher number per thousand tokens written. The example invents two model tiers with generic names and two rates each.
| Tier | Read (cu per 1,000 tokens) | Written (cu per 1,000 tokens) |
|---|---|---|
small-model | 1.0 | 4.0 |
large-model | 10.0 | 40.0 |
These rates are invented for teaching. The shape is common (writing costs more than reading; a larger model costs more than a smaller one), and the numbers are not anyone's.
The cost of one step is one line of arithmetic. This is the function in cost.py:
def step_cost(tokens_in: int, tokens_out: int, model: str, rates: dict[str, dict[str, float]]) -> float:
return round(tokens_in / 1000 * rates[model]["in"] + tokens_out / 1000 * rates[model]["out"], 6)
For write_recommendation in UW-006: 723 tokens read at 10.0 per thousand is 7.23 cu, and 111 tokens written at 40.0 per thousand is 4.44 cu, so the step costs 11.67 cu. The run had one other model step, plan_steps on the small tier, at 0.826 cu. The per-run cost of run-7f3a is the sum: 12.496 cu.
Plain English. A run's cost is its token counts multiplied by the rates, step by step, then added. If your spans already record tokens and the model tier, you already have everything you need. What is missing is the sum.
Common confusion. The two failed calls toget_roof_assessmentinUW-006cost no tokens: they are tool calls, not model calls. They cost time, not cost units. A run can be slow and cheap, or fast and expensive; the two signals answer different questions.
2. The day, run by run#
Now apply the same arithmetic to every run of the simulated day. The command prints the median run, the threshold for an anomaly, the outlier, the total under different routing rules, and the cost per broker.
cd "$TMPDIR/uw-observe-cost" && /opt/homebrew/bin/python3 run_observe.py --show cost
== cost
rates (invented cost units per 1,000 tokens): {'small-model': {'in': 1.0, 'out': 4.0}, 'large-model': {'in': 10.0, 'out': 40.0}}
median run 12.4335 cu · anomaly threshold 37.3005 cu (3.0 x median)
anomaly run-48f5 UW-502: 265.6070 cu = 21.4 x median · 26485 tokens in, 205 out, 10 model calls
day total by routing rule: as-run 851.1230, all-large 1210.0700, all-small 121.0070, small-except-write 612.8210
by tenant: broker-01 60.3460, broker-02 123.4220, broker-03 74.5920, broker-04 63.0130, broker-05 113.3480, broker-06 416.4020
Read the lines one at a time. The median run costs about 12.4 cu, close to UW-006's 12.496. One run, run-48f5, costs 265.6 cu: 21.4 times the median. Its case is UW-502, this site's own case of an agent that loops. And one broker, broker-06, accounts for 416 cu of the day's 851. All of these numbers are synthetic and illustrative.
Diagram: From a model span to a cost per broker, in invented units
Read left to right. A model span records tokens in and out and the tier. A rate turns that into the cost of one step. The steps of one run add up to the run's cost. Runs carry the broker as tenant, so they add up per broker. The same chain finds the anomaly: run-48f5 sits far above the median. All costs are invented units.
- 1 · Model span
- Tokens on the span: The span of write_recommendation in UW-006 records how many tokens the model read and wrote.
- 2 · Step cost
- Times the rate: Tokens per thousand times the tier's invented rate, read and written separately.
- 3 · Run cost
- Sum of the steps: All model steps of one run added up; tool calls add time, not tokens.
- The looping run: Nine next-step decisions with a growing context: 21.4 times the median run.
- 4 · Attribution
- Per broker: Runs grouped by the tenant that sent them; drill down before acting on a total.
3. The run that loops#
UW-502 is a submission on which the agent keeps searching. It calls search_guidance, asks the model what to do next, searches again, asks again: nine times, until it reaches the example's step limit and ends with an escalate recommendation for the underwriter. That end is a controlled state. Nothing failed. Every span has status ok. On an error dashboard the run is invisible.
The log of the run shows why its cost grows faster than its step count:
2026-09-14 10:41:04 INFO [agent-worker] decide_next_step finished: 1062 tokens in, 20 out (large-model)
2026-09-14 10:41:05 INFO [agent-worker] decide_next_step finished: 1512 tokens in, 20 out (large-model)
2026-09-14 10:41:05 INFO [agent-worker] decide_next_step finished: 1962 tokens in, 20 out (large-model)
2026-09-14 10:41:06 INFO [agent-worker] decide_next_step finished: 2412 tokens in, 20 out (large-model)
Each call reads everything the run has gathered so far, and each search adds about 450 tokens of passages. So the input of each call is larger than the one before: 1,062 tokens, then 1,512, then 1,962, up to 4,662 on the ninth. The output stays at 20 tokens. The cost of the run is dominated by reading, and reading grows with every loop. Nine loops cost far more than nine times one loop.
This is a cost anomaly: a run whose cost is far outside what runs of its kind usually cost. The example's rule is the simplest one that works on a day: a run costing more than three times the median run is flagged.
def anomalies(costs: list[dict[str, Any]], factor: float = ANOMALY_FACTOR) -> dict[str, Any]:
"""Runs whose cost is above `factor` x the median cost of the day. `costs` rows have run_id and cost."""
m = median([c["cost"] for c in costs])
flagged = [{"run_id": c["run_id"], "cost": c["cost"], "multiple": round(c["cost"] / m, 4) if m else None}
for c in costs if m and c["cost"] > factor * m]
return {"median": round(m, 6), "factor": factor, "threshold": round(factor * m, 6), "anomalies": flagged}
The median, not the mean, is the anchor on purpose: one looping run would pull the mean up and hide itself. The threshold is 37.3 cu; run-48f5 is the only run above it.
In our insurance agent. The anomaly is not a billing curiosity. A run that loops nine times on retrieval is also a run whose recommendation rests on a search that never converged. The cost signal finds it first; the trace then shows what the agent kept asking, and the underwriter sees an escalation with a reason. Cost and quality point at the same run from two sides.
What this is not. Stopping the loop is not observability. The step limit that ended UW-502 belongs to the harness (see harness.sammuthu.com), and budgets that stop a run belong to reliability (see reliable.sammuthu.com). This page measures, attributes and alerts.
4. Attribution: who spent it#
The month's bill is one number. Cost attribution means splitting it by something a person can act on: per run, per step, per tenant, per feature. In the example every run carries the broker that sent the submission as its tenant, so the per-run costs add up per broker:
| Broker (tenant) | Cost on the day (cu) |
|---|---|
| broker-01 | 60.346 |
| broker-02 | 123.422 |
| broker-03 | 74.592 |
| broker-04 | 63.013 |
| broker-05 | 113.348 |
| broker-06 | 416.402 |
broker-06 stands out, and attribution alone would suggest a heavy customer. Drill one level down and the picture changes: run-48f5 alone is 265.6 of that broker's 416.4 cu. One looping run, not a busy broker. Attribute first, then look at the runs inside the number, or the wrong conversation happens with the wrong customer.
Attribution needs the identifiers of Identifiers on the cost record: run_id, submission_id, the step name and the tenant. Tenant is a bounded label and may sit on a cost metric; run_id belongs on the span or the cost row, never on a metric label (see Advanced).
5. Routing: what a smaller model changes#
Model routing means choosing a model tier per step instead of one model for everything. The example has three routing rules besides the one the day really ran (as-run, in which the plan step uses the small tier and the other steps the large tier).
| Routing rule | Plan steps | Next-step decisions | Write the recommendation | Day total (cu) |
|---|---|---|---|---|
| as-run (what the day used) | small | large | large | 851.123 |
| all-large | large | large | large | 1210.070 |
| all-small | small | small | small | 121.007 |
| small-except-write | small | small | large | 612.821 |
The re-pricing keeps each step's token counts and changes only the rate. That is a simplification the example states in its code: a different model would write different text, and so different token counts, and possibly a different recommendation. So the table answers one question honestly ("what would these same tokens cost under another rule?") and leaves the real question open: is the recommendation still good when a smaller model writes the decision steps? That is a measurement for eval.sammuthu.com, not a conclusion this page can draw from a cost table.
| Control in the lab | What it changes | What to watch |
|---|---|---|
| Rate of each tier (read and written) | The cost of every step | The day total and the median run |
| Routing rule | Which tier each step uses | The total, and whether run-48f5 is still an anomaly |
| Anomaly factor | The threshold above the median | Which runs are flagged |
Try one thing in the lab: switch to small-except-write. The day total falls, and the looping run falls the most, because its cost was nine large-tier decision calls. Routing reduced the damage of the loop. It did not remove the loop.
Before AI. A Java service measured CPU time and database calls per request, and a slow query that ran in a loop was found the same way: a per-request number far above the median, then the trace. The new part is that the unit of cost is tokens, it is visible on every model span, and it grows with the context the agent carries, not with the size of the request.
6. Budgets and alerts#
A budget is an amount of cost units set aside for a period, a tenant or a run. It turns cost from a report into an alert. Two kinds matter, and they answer different questions.
- A per-run budget answers "is this run behaving?" With a median of 12.4 cu, a run budget of three times the median catches
UW-502while it runs: its cost so far passes 37.3 cu after the third decision call (0.827 + 11.42 + 15.92 + 20.42 = 48.587 cu), not at the end of the month. Whether the run is then stopped is a reliability decision; the observability part is the signal and the alert. - A period budget answers "are we spending as planned?" It burns down over a month like an error budget on SLOs and alerts, and the same idea of burn rate applies: alert when the spend rate would exhaust the month early.
Diagram: Two budgets, two questions
Read each lane on its own. A per-run budget watches one run while it happens and asks whether it is behaving. A period budget watches the month and asks whether spending follows the plan. Both raise alerts; stopping a run is a reliability decision, not an observability one.
- Per run
- Cost so far: The sum of step costs up to now, checked after each model step.
- Run alert: Flag the run, keep its trace by a tail rule, and put it in the review queue.
- Per period
- Spend rate: How fast the period budget burns compared with an even spend.
- Burn alert: Alert when the rate would exhaust the budget early, with a link to attribution.
The same idea for a coding agent. A coding-agent session such asCA-001(read the instructions, edit, run the tests, fail, edit, pass, open a pull request) is priced the same way: tokens per model step, summed per session. Claude Code, for one, documents an OpenTelemetry export with aclaude_code.token.usagemetric (with types for input, output and cache tokens) and aclaude_code.cost.usagemetric, and describes the cost on its request event as "Estimated cost". A session that retries a failing test many times is the coding agent'sUW-502. How such a session is run belongs to harness.sammuthu.com; counting it belongs here.
Why loop cost grows faster than loop count
If each loop iteration appends about a passages of tokens to the context, and the first decision call reads c tokens, the k-th call reads about c + (k − 1)a tokens. Over n calls the tokens read sum to about n·c + a·n(n − 1)/2: linear in n from the base context, quadratic from the growth. For UW-502 (c = 1,062, a = 450, n = 9) that is 9 × 1,062 + 450 × 36 = 25,758 tokens read across the nine decision calls, which matches the run's log exactly; with the plan step's 727 tokens the run read 26,485.
Two consequences for design. A step limit bounds n, and so bounds the quadratic term. Summarizing or trimming the context between iterations lowers a, which changes the growth from quadratic to nearly linear. Both are decisions for the harness and context sites; the formula is how you see which one a cost curve calls for.
Prompt caching, where a provider offers it, changes the rate of the repeated prefix rather than the token count. Record cache tokens as their own count (Claude Code's metric has separate cache types), or a cost model built on two counts will overstate the cost of a cached run.
Better anomaly rules than three times the median
The example's rule works on one day of one workload. Production needs more.
- Per class of run. A run that writes a long recommendation is legitimately more expensive than one that requests information. Compare each run with the median of its own class (case type, route, tenant tier).
- Robust spread. Use the median absolute deviation, or a high percentile of the last N days, instead of a fixed multiple, so that the threshold follows real variation.
- Early signals. Cost so far plus steps so far, checked during the run, catches a loop before it finishes. Model-call count is a good proxy and is known earlier than the bill.
- Flag, then sample. A flagged run is a reason to keep its trace (a tail-sampling rule) and to put it in the review queue of Quality.
Checklist#
- Every model span records tokens in, tokens out and the model tier, as numbers.
- A cost function with your rates (or invented units in examples) turns spans into a cost per step and per run.
- Cost rows carry
run_id,submission_id, step and tenant, so that a total can be split down to one run. - An anomaly rule compares each run with the median of runs like it, and flags far outliers.
- A per-run budget alerts during the run; a period budget alerts on burn rate.
- Re-pricing under a routing rule is labelled as re-pricing; quality under the new rule is measured separately.
- Cache tokens, where a provider reports them, are recorded as their own count.
In one breath#
Cost is token counts times rates, and the token counts are already on every model span, so the cost of a step, a run and a tenant is a sum away. The median run is the anchor: a run far above it, like the looping UW-502 at 21 times the median, is an anomaly even when every span says ok. Attribute cost before acting on it, measure what routing changes, and leave the question of whether a cheaper model is still good to the evals.
Where this leads#
You can now price a run, find the one that spends ten times too much, and say what routing would change. Every page so far built its own records in its own code. The next page looks at the libraries, standards and products that do this work for you, each read in its own documentation: Tracing tools.