Two questions about one LLM call

A call to a language model is one line of code and two very different questions.

Why was it slow, and did it fail? — the model took four seconds, retried twice on a 429, and the whole request the user was waiting on took six. That is a trace.

What did it cost? — this model, this month, for this customer, in tokens and in dollars. That is a metric.

They are questions about the same call, and no single primitive answers both well. Sentrinel answers them with two, and the point of this post is which one to reach for.

The trace: priced GenAI spans

Wrap the call in a span and put the standard GenAI attributes on it — the same names OpenTelemetry's semantic conventions use, so an OTLP exporter produces the identical span:

await traceSpan("llm.chat", async (span) => {
  const res = await client.chat.completions.create({ model: "deepseek-chat", messages });
  span.setAttributes({
    "gen_ai.system": "deepseek",
    "gen_ai.request.model": "deepseek-chat",
    "gen_ai.usage.input_tokens": res.usage.prompt_tokens,
    "gen_ai.usage.output_tokens": res.usage.completion_tokens,
  });
  return res;
});

Sentrinel recognises those attributes and prices the span: the model's rate times the tokens, stored as gen_ai.usage.cost on the span itself. The LLM page lists calls by cost, by model, by operation, with the trace each one belongs to — so "the most expensive call today" opens to the request that made it, the user who made that request, and the four seconds it took.

A span with a model but no token counts is not treated as a usage record. It is a call that happened; nothing is priced from a guess.

The metric: counters

A priced span answers "what did this call cost". It does not answer "what did we spend on this model this month, per customer" — because that is a sum over a month of spans, sliced by a label, and a span is not a number you asked to be added up.

That is what counters are for, and the same usage object feeds them:

import { count, histogram } from "@sentrinel/plugin";

count("llm.tokens", res.usage.prompt_tokens,     { model, direction: "input" });
count("llm.tokens", res.usage.completion_tokens, { model, direction: "output" });
count("llm.cost_usd", priceOf(res.usage),        { model, plan: user.plan });
histogram("llm.cost_per_call_usd", priceOf(res.usage), { model });

Then on the Metrics page: llm.cost_usd, split by plan, read the total. The histogram is there because the average cost per call hides the one prompt that cost fifty times the rest.

Counters fold in memory and leave the process as one row per series per flush, which is why recording a metric per token is a normal thing to do rather than a network write per completion.

Which one, when

The reviewer who first pushed us to build counters put it exactly right: tracing and metrics are complementary, not a choice. The model call that took four seconds and the forty-one dollars spent on that model today are two questions about the same event.

A label is not an id

{ model, plan } is a good label set. { userId } is a series per user, and the classic way to turn a metrics system into an incident. Per-user spend is a question for the trace side — the priced span already knows the user — not for a counter label. The SDK caps live series and drops new ones past the cap, reporting the drop, because a monitoring library that grows without bound inside your process has failed at the one thing it was for.

Both halves are covered in the plugin reference; the pricing table is what the LLM page uses.