Skip to content

Retries and per-call context

A rate-limited provider tells you how long to wait in a response header: retry-after-ms, retry-after, or x-should-retry. A retry wrapper around the model interface usually can’t see those headers. The Gemini SDK discards the response before the error reaches it, and each vendor SDK retries by its own rules: the Anthropic SDK retries twice, the Gemini SDK never. The same agent then got different resilience depending on which model it named.

So core-models adapters turn their SDK’s own retries off and install retry.Policy.Transport on the HTTP client. One policy and one header rule apply to every provider. A product’s own retry wrapper still runs, and now sees only what this layer gave up on.

RetriedStatus 408, 429, 500, 502, 503 or 504; any failure before a response arrived (refused or dropped connection, timeout, DNS) except a certificate error; any status the server marks x-should-retry: true
Not retriedAny status marked x-should-retry: false; other 4xx; a canceled context; an error that marks itself permanent (Permanent() bool), such as an OAuth2 token endpoint refusing the client secret; a request body that can’t be sent again; a stream that breaks after it started, since replaying it would deliver tokens twice
Waitretry-after-ms if present, otherwise retry-after as seconds or an HTTP date, otherwise exponential backoff with jitter
Too longIf the server asks for more than MaxHeaderDelay (default one minute), the response is handed back without waiting. Retrying sooner than asked just earns another 429
No answerAn attempt that gets no response headers in time is abandoned and retried like a dropped connection, with ErrNoResponse (which is not context.Canceled: the attempt was abandoned, not the call). A streaming request waits HeaderTimeout (default five minutes); the limit never applies to the body, so a long stream is unaffected. A non-streaming request gets its headers only once the whole answer exists, so adapters mark it and it waits UnaryHeaderTimeout (default ten minutes) instead. retry.WithHeaderTimeout(ctx, d) overrides both for one call, and zero removes the limit, for a call the host knows is long, such as a large compaction summary
Default budgetTwo retries, starting at one second
After a successPolicy.AfterSuccess retries one more kind of response, once, and only for a session that has already been served (WithPriorSuccess) and is not a side call. The Gemini adapter uses it for Vertex AI’s bare 400 INVALID_ARGUMENT (“Request contains an invalid argument.” with no details): on a first call that is as likely a malformed request as a fault, but every time it was recorded on a served session it went away on retry

What happened is recorded on a per-call retry.Record. An adapter stamps it onto the response, so a retry shows up in the transcript and not only in a log.

One model serves an agent’s main loop and its side calls alike, such as an approver, a session title or a summary. Anything that differs per call therefore travels on the call’s context, in package callctx:

MarkerEffect
AsSideCall(ctx, name)Names the call as a side call. Retry and logging use the name, and a side call never counts as evidence from its session’s earlier successes
WithoutBuiltins(ctx)The request carries exactly the caller’s tools: no provider-injected search, URL context or code execution
WithoutPromptCache(ctx)No prompt-cache markers. A side call whose prefix won’t be sent again would pay the cache-write premium for nothing
WithPriorSuccess(ctx, rec)Records that this session has already been served once. Some ambiguous rejections are worth one retry only if so

Every marker can only take something away from a request. A marker an adapter doesn’t honor is a missed optimization, never a wrong request.