Retries and per-call context
Retries happen at the HTTP layer
Section titled “Retries happen at the HTTP layer”A rate-limited provider tells you how long to wait in a response header:
retry-after-ms, retry-after, or x-should-retry. A retry wrapper
around the model interface usually can’t see those headers. The Gemini SDK
discards the response before the error reaches it, and each vendor SDK
retries by its own rules: the Anthropic SDK retries twice, the Gemini SDK
never. The same agent then got different resilience depending on which
model it named.
So core-models adapters turn their SDK’s own retries off and install
retry.Policy.Transport on the HTTP client. One policy and one header rule
apply to every provider. A product’s own retry wrapper still runs, and now
sees only what this layer gave up on.
What is retried, and for how long
Section titled “What is retried, and for how long”| Retried | Status 408, 429, 500, 502, 503 or 504; any failure before a response arrived (refused or dropped connection, timeout, DNS) except a certificate error; any status the server marks x-should-retry: true |
| Not retried | Any status marked x-should-retry: false; other 4xx; a canceled context; an error that marks itself permanent (Permanent() bool), such as an OAuth2 token endpoint refusing the client secret; a request body that can’t be sent again; a stream that breaks after it started, since replaying it would deliver tokens twice |
| Wait | retry-after-ms if present, otherwise retry-after as seconds or an HTTP date, otherwise exponential backoff with jitter |
| Too long | If the server asks for more than MaxHeaderDelay (default one minute), the response is handed back without waiting. Retrying sooner than asked just earns another 429 |
| No answer | An attempt that gets no response headers in time is abandoned and retried like a dropped connection, with ErrNoResponse (which is not context.Canceled: the attempt was abandoned, not the call). A streaming request waits HeaderTimeout (default five minutes); the limit never applies to the body, so a long stream is unaffected. A non-streaming request gets its headers only once the whole answer exists, so adapters mark it and it waits UnaryHeaderTimeout (default ten minutes) instead. retry.WithHeaderTimeout(ctx, d) overrides both for one call, and zero removes the limit, for a call the host knows is long, such as a large compaction summary |
| Default budget | Two retries, starting at one second |
| After a success | Policy.AfterSuccess retries one more kind of response, once, and only for a session that has already been served (WithPriorSuccess) and is not a side call. The Gemini adapter uses it for Vertex AI’s bare 400 INVALID_ARGUMENT (“Request contains an invalid argument.” with no details): on a first call that is as likely a malformed request as a fault, but every time it was recorded on a served session it went away on retry |
What happened is recorded on a per-call retry.Record. An adapter stamps
it onto the response, so a retry shows up in the transcript and not only in
a log.
Per-call context markers
Section titled “Per-call context markers”One model serves an agent’s main loop and its side calls alike, such as an
approver, a session title or a summary. Anything that differs per call
therefore travels on the call’s context, in package callctx:
| Marker | Effect |
|---|---|
AsSideCall(ctx, name) | Names the call as a side call. Retry and logging use the name, and a side call never counts as evidence from its session’s earlier successes |
WithoutBuiltins(ctx) | The request carries exactly the caller’s tools: no provider-injected search, URL context or code execution |
WithoutPromptCache(ctx) | No prompt-cache markers. A side call whose prefix won’t be sent again would pay the cache-write premium for nothing |
WithPriorSuccess(ctx, rec) | Records that this session has already been served once. Some ambiguous rejections are worth one retry only if so |
Every marker can only take something away from a request. A marker an adapter doesn’t honor is a missed optimization, never a wrong request.