LLM Cost Monitoring: Track Spend Before It Spikes (2026)

LLM Cost Monitoring: Track Spend Before It Spikes (2026)

LLM cost monitoring is the difference between a $412 surprise invoice and a Slack ping at 80% of budget. Claude Sonnet 5 bills at $3.00 per million input tokens this month, and a single runaway agent loop can burn through that in an afternoon. Most teams wire up the tracking in a day; the alert is the part they skip.

Key Takeaways:

• LLM cost monitoring attaches a token count and dollar estimate to every request, then aggregates by model and team.
• Track five metrics: tokens per request, cost per feature/team/model, cache hit ratio, cost per conversation, spike rate.
• LiteLLM budgets, Langfuse traces, or Datadog LLM Observability each wire up spend visibility in under a day.
• Dashboards show estimates; cached-input pricing and batch discounts mean the invoice usually lands below them.

What Does LLM Cost Monitoring Actually Track?

LLM cost monitoring is the practice of attaching a token count and a dollar estimate to every LLM request, then aggregating those estimates by model, feature, and team so spend can be alerted on before the invoice arrives. The estimate is computed per request from published token prices, rolled up by whatever tags you stamp on the call, and compared against a budget set in advance.

The math underneath is vendor-neutral. Every provider's usage response breaks tokens into fields, and Datadog's cost docs document the relationships explicitly. The OpenTelemetry GenAI semantic conventions standardize those same fields across vendors, so a dashboard built on them is not locked to one provider.

Three relationships do most of the work: total tokens = input + output; input = non-cached + cache_read + cache_write; and reasoning tokens sit inside output, at the output rate. Get those right and the per-request estimate stays close; ignore them and the dashboard drifts from the bill every month. Cost monitoring is one pillar of AI observability; tracing and evals are the other two, and they share this same field vocabulary.

Your dashboard shows an estimate; the invoice is the only cost metric that is never cached.

How much is 1 million tokens in LLM?

It depends on the model and the direction: 1M tokens costs $0.14 as DeepSeek-V4 input and $15.00 as GPT-5.6 Terra output, a 100x spread on the same unit. Here are four models from our July 14, 2026 pricing research, verified against the official pages:

Sources: OpenAI pricing and Anthropic pricing. One footnote: Claude Sonnet 5 runs introductory pricing of $2.00 input / $10.00 output through August 31, 2026, then reverts to the numbers above.

The 5 Metrics That Actually Matter

Five metrics cover LLM cost tracking, and most teams only ever alert on two of them. Start with the first two rows: tokens per request catches prompt bloat the day it appears, and cost per team is the number finance eventually asks for. The other three refine the picture once those are wired.

One granularity note: Datadog stores request-level cost in nanodollars (billionths of a dollar) per its cost docs. At $0.14 per million tokens, a single DeepSeek-V4 request can cost less than a thousandth of a cent, so aggregate before you round, or small-model traffic disappears from the report.

If you track nothing else, track rows one and two. Tokens per request is the earliest warning you get; cost per team is the one that survives contact with an accounting department.

Three Ways to Wire Up LLM Cost Monitoring

Zero of the top-ranking pages for this query ship a single runnable line of code, so here are three working setups. Each goes live in under a day, and they compose: we run the first two together.

What we actually run in production: in our setup, every client agent talks to the LiteLLM proxy through its own virtual key, with a monthly budget on each key and Langfuse tracing every request behind the proxy. When we deployed the two together, the split of labor was the point: the proxy enforces the caps, and the traces explain what consumed them. Each of our client keys also carries a tpm_limit of 100,000 tokens per minute as a second fuse; based on LiteLLM's docs, the proxy rejects requests once a limit is reached, and that documented behavior is what we rely on, not a benchmark we measured ourselves. Our config skips LiteLLM 1.82.7 and 1.82.8 entirely, the two versions caught in the March 2026 supply chain incident, and pins the image tag instead of floating on latest.

LiteLLM proxy: virtual keys + budgets

Techsy runs a LiteLLM proxy setup in production with one virtual key per client and a monthly budget on each key. The request below is the documented shape from LiteLLM's virtual_keys docs: based on those docs, once cumulative spend on this key crosses $50 in a month, the proxy rejects further requests with a budget-exceeded error rather than logging a warning after the fact.

curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key_alias": "client-acme-search",
    "max_budget": 50.0,
    "budget_duration": "monthly",
    "tpm_limit": 100000,
    "models": ["anthropic/claude-sonnet-4-5"]
  }'

Do the arithmetic against published prices: at Claude Sonnet 5's $3.00 / $15.00 per million tokens, $50 buys roughly 16.7M input tokens or 3.3M output tokens, about a week of traffic for one of our lighter client agents. That's exactly the blast radius we want. The tpm_limit is the second fuse: a runaway loop trips 100K tokens-per-minute long before it trips the monthly budget. Per-user attribution works the same way through the users endpoint, and our guide to the best LLM gateway tools covers when a proxy earns its place versus when it is just another hop.

Langfuse: @observe cost tracing

Google autocomplete pairs "langfuse monitoring" with this query, and for good reason: Langfuse is the open-source tracing default. Its Python SDK wraps your provider client so every call becomes a trace carrying token counts and a computed cost, per the Langfuse tracing docs:

# pip install langfuse openai
from langfuse import observe
from langfuse.openai import openai  # drop-in wrapper, auto-traces

@observe()
def answer(question: str):
    return openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": question}],
    )

answer("what is llm cost monitoring")
# the trace now carries usage.total_tokens and total_cost,
# priced from Langfuse's model price cards

Cost lands on the trace with no token math on your side; group traces by session or user id for per-feature rollups, and self-host if the data can't leave your network.

Datadog LLM Observability: if you're already in it

If your team already ships Datadog agents, its LLM cost docs are the deepest single reference on this page: request-level cost in nanodollars, custom cost_tags for team and feature breakdowns, and a real troubleshooting section for partial-cost gaps. The honest limit: it's Datadog-only. The metrics don't leave the platform, and there's no open-source fallback if the pricing stops fitting. Pick it for consolidation, not for flexibility.

How Do You Wire Budgets, Alerts, and Chargeback?

You wire it in three layers: a budget cap that rejects requests at the limit, a webhook alert that fires at a percentage threshold before the cap, and per-team virtual keys that turn showback and chargeback into a query instead of an argument. LiteLLM ships all three natively; the patterns transfer to any gateway with key-level budgets.

A budget that only counts is a report; a budget that rejects requests at the cap is a control.

Alerts that fire before the spike

Set the alert at 80% of the cap, not at 100%. LiteLLM ships Slack alerting built in: set alerting: ["slack"] and alerting_threshold: 80 in general_settings, point the SLACK_WEBHOOK_URL environment variable at a channel, and a message like this lands when a key crosses the threshold:

{
  "text": "LLM budget alert: client-acme-search spent $40.18 of its $50.00 monthly cap (80.4%). Top model: anthropic/claude-sonnet-4-5."
}

At 80%, a human still has days to act: downgrade the model, tighten the prompt, or raise the cap with a name on the approval. An alert at 100% is an autopsy.

From showback to chargeback

Showback means each team sees its own spend; chargeback means it comes out of their budget. Both run on one ingredient: a virtual key per team, tagged at creation. The monthly report is then a group-by over the spend table:

Example format, not client data. The evals-batch row is the pattern working as intended: batch work ran to its cap and stopped, instead of bleeding quietly into the invoice. Enforcement behavior is documented in LiteLLM's virtual_keys docs, including how budget duration resets.

Why Does Your Dashboard Disagree With the Bill?

Because dashboards bill at list price while invoices apply discounts your monitoring never sees. Cached input lands at 0.1x to 0.25x of base price, batch runs at half, and reasoning tokens bill at the output rate from inside the output count. When the two numbers diverge, the invoice is usually the lower one, and the gap is almost always one of four modifiers.

The open pydantic/genai-prices catalog shows why these multipliers differ per model: each provider sets its own cache and batch factors, so a single hardcoded price table drifts the day a provider revises its cards. Datadog's cost docs document the other half of the problem, partial-cost gaps where a missing token field returns COST UNAVAILABLE for a request. Check there when the dashboard reads lower than the invoice; check the discount table when it reads higher. The mechanism behind the biggest multiplier is LLM prompt caching, and a high cache hit ratio is the most common reason a tracked estimate overshoots the real bill.

A cost estimate without cache-hit and batch discounts is a ceiling, not a forecast.

Which Tools Actually Do LLM Cost Tracking?

Six tools cover most production setups: Langfuse, LiteLLM, Helicone, and Portkey on the open-source and gateway side, Datadog and Braintrust on the commercial side. The honest split is enforcement versus observability: a proxy can reject requests at a budget, while a tracing tool measures spend after the fact. Most mature stacks end up with one of each.

One caveat on vendor content in this space: Braintrust's own 2026 comparison of cost-tracking tools ranks itself first and omits every open-source option in the table above. Read vendor listicles for their data, not their rankings.

For a team starting from zero, we'd run LiteLLM in front and Langfuse behind it: the proxy enforces budgets, the traces explain them, and the pairing costs nothing until you cross into hosted plans. If you're weighing the two tracing defaults, our Langfuse vs Langsmith breakdown goes deep on that choice, and the best AI observability platforms roundup covers the full field.

From Monitoring to Cutting Costs

Monitoring shows where the money goes; the next step is deciding how much goes there. The three levers, in order of payoff: cache repeated input, route easy requests to cheaper models, and downsize the model where quality still holds. Our reduce LLM API costs guide covers each lever, and the LLM API pricing comparison is the input to all the math above.

Our perspective: when a client's LLM spend surprises them, we monitor first and cut second, never the reverse. Teams that cache before they measure usually end up caching the wrong prompts. Monitoring tells you where the money goes; caching and routing decide how much goes there. If your invoice already stings, get a free consultation and we'll look at the numbers with you.

Frequently Asked Questions

How much is 1 million tokens in LLM?

At list price, anywhere from $0.14 to $15 per million depending on the model and direction: DeepSeek-V4 input costs $0.14 per million, while GPT-5.6 Terra output costs $15. Output tokens run 3 to 6 times the input rate on every major provider. The table in the first section has four models, and our LLM API pricing comparison prices all seventeen, verified against OpenAI and Anthropic pages in July 2026.

What is LLM cost optimization?

The practice of lowering cost per request without dropping quality: prompt caching for repeated input, routing easy tasks to cheaper models, batch APIs for async work, and right-sizing the model per feature. LLM cost monitoring is the prerequisite, since you can't optimize what you haven't attributed. Cache hit ratio and cost per feature are the two metrics that point at the biggest levers first.

Why are LLMs so expensive?

Inference is compute-bound: every generated token runs a full forward pass of the model on GPUs, and reasoning models spend extra tokens thinking before they answer, billed at output rates. Long system prompts multiply that cost across every single request. The bill grows with verbosity on both sides of the call, which is why tokens per request is the first metric worth watching.

How much does an LLM cost in the US?

API pricing is USD-denominated and global: OpenAI, Anthropic, and Google charge the same list price per million tokens whether the request originates in Ohio or Osaka. Regional differences show up in self-hosting, where GPU hours vary by cloud region, and in hosted platforms that add markup. For API work, location changes latency, not price.

How do I track LLM costs per team?

Issue one virtual key per team through a proxy like LiteLLM, tag each key with the team name at creation, and aggregate spend by that tag. Every request then carries attribution from the moment it's made, with no log parsing. The showback table in the budgets section above is the end product: team, monthly budget, spend to date, status.

Langfuse vs Datadog for LLM cost tracking: which should I pick?

Pick Langfuse if you want open source, self-hosting, and data you control; it's free to run and traces cost per request out of the box. Pick Datadog only if your team already runs it for infrastructure, since its LLM cost features don't leave the platform. For most teams starting fresh, Langfuse plus a LiteLLM proxy beats either option alone.

Do estimated LLM costs match the actual invoice?

No, and usually the invoice is lower. Dashboards bill at list price while cached input lands at 0.1x to 0.25x and batch jobs at 0.5x, so a high-cache workload sees the real bill come in well under the tracked estimate. Missing token fields push the error the other way. The drift table above lists each multiplier and where to look.

How do I alert on LLM spend spikes?

Set a threshold alert at 80% of each team's monthly budget, delivered to Slack by webhook, plus a day-over-day anomaly alert at +50% for loops that burn fast. LiteLLM ships the threshold pattern natively through its alerting_threshold setting. An alert at 80% leaves days to react; an alert at 100% only confirms the cap did its job.

Is there a free LLM cost tracker?

Yes, three credible ones. Langfuse self-hosted is free and open source, with a free hobby tier on cloud per the Langfuse pricing page. LiteLLM's built-in key budgets cost nothing on the open-source proxy. Helicone's free tier covers proxy-level cost logs with rate limits. Free tiers cover monitoring; enforcement and alerting at scale is where paid plans start.

Conclusion

Pick a setup path today, because the alert you will want in six weeks takes an afternoon to wire now. The short version:

• Track tokens per request and cost per team first; add the other three metrics once those work.
• A budget that rejects requests is a control; one that only counts is a report.
• Set the alert at 80%, in Slack, before the invoice can surprise anyone.
• Your dashboard is an estimate; cached-input and batch pricing mean the bill usually lands below it.

If you'd rather have someone look at your numbers with you, get a free consultation.