Skip to content

The usage record

genai’s usage struct carries prompt, cached, output, reasoning and total token counts. That covers Gemini, nearly covers Claude, and falls short for everything else:

  • It has no cache-write bucket. Claude and OpenAI bill writing to a prompt cache at a premium, so a turn that warms a cache is underpriced.
  • It can’t say “not reported”. A zero might mean the provider reported zero, or that it said nothing. Self-hosted servers often say nothing about cached tokens.

So every adapter also attaches a usage.Detail to the response’s CustomMetadata, under usage.MetadataKey (core_models.usage).

FieldMeaning
CacheReadTokensPrompt tokens served from a provider-side cache
CacheWriteTokensPrompt tokens written to a cache, every TTL
CacheWrite1hTokensThe one-hour share of the above, where it is billed differently
ReasoningTokensReasoning tokens the provider broke out
ToolUseTokensTool-use prompt tokens the provider broke out

A nil count means the provider did not report it. A pointer to zero means it reported zero. In JSON, an unreported count is omitted and a reported zero is written as 0.

FieldMeaning
ServedModelWhat the backend said it ran. May not be something a price table understands, such as a resource path or a deployment name. The response’s ModelVersion is the priceable id
ProviderRequestIDThe id a support ticket needs
BackendThe provider profile that served the call: the key a price is looked up under
RegionThe serving region, where the backend has one

usage.FromMetadata reads the record in both forms it can take:

  • live, as the value an adapter attached;
  • replayed, after a JSON round trip through an event log, as a generic map.

A product’s meter can therefore use the same call on live and resumed sessions.

The shape starts from the record mast shipped first (mast #352). mast and core-agent each read it through their own meters, which keep their own rules for clamping inconsistent counts.