Providers and models
Model quality moves month to month. A platform team that hard-codes one vendor’s client into its agent layer is carrying that curve as a risk it cannot cheaply unwind — which is why multi-provider is a pillar here rather than a compatibility shim.
Gemini and Claude reach that table through mast’s own adapters, and
everything OpenAI-compatible reaches it through provider profiles. The
pillar is the substitutability rather than the number: the tier
indirection, a meter that prices the (backend, model) pair, and a judged
corpus that runs nightly against both. There is no pkg/provider
interface to implement — the extension point is mast.Config.Model, a
field typed on ADK’s model.LLM, and the root package it lives on is one
of the six paths the v1.0 promise covers.
What you can point mast at
Section titled “What you can point mast at”| Provider | Selected by | Notes |
|---|---|---|
| Gemini | a model id like gemini-3.5-flash, or --provider=gemini | Google Search and URL-context builtins are available through the provider |
| Gemini on Vertex | --provider=vertex, or GOOGLE_GENAI_USE_VERTEXAI=true | same models, ADC instead of a key |
| Gemini on Vertex, express mode | GOOGLE_GENAI_USE_VERTEXAI=true with an API key and no project or location | Vertex AI with an API key instead of ADC; see Vertex AI express mode |
| Claude, first-party | claude-* with ANTHROPIC_API_KEY set | |
| Claude on Vertex | claude-* with a Vertex project, or --provider=anthropic-vertex | context caching supported |
| A provider profile | --provider=<profile> with --model, or the profile’s tier | Vertex AI partner models (vertex-maas), Ollama (ollama), vLLM, SGLang, or any OpenAI-compatible endpoint. See Provider profiles |
echo | --model=echo (the default) | offline fake, no credentials, never emits tool calls |
scripted | --model=scripted | replays recorded turns from a JSONL file (MAST_SCRIPT) |
toolactor | --model=toolactor | offline fake that does drive tool calls |
Switching is a config change, not a code change. --model picks the model;
--provider is an alias that validates it and chooses the backend within a
family. Each family has two aliases for the same model ids — gemini /
vertex and anthropic / anthropic-vertex — because a backend is not a
different model line. Without an alias the backend comes from the
environment: GOOGLE_GENAI_USE_VERTEXAI (or GOOGLE_GENAI_USE_ENTERPRISE,
which wins when set) for gemini-*, and for claude-*
ANTHROPIC_API_KEY selecting the first-party API with a Vertex project
selecting Vertex.
Provider profiles
Section titled “Provider profiles”A model outside the Gemini and Claude families is named by a profile plus
a model id, not by a prefix. The same gpt-oss-20b is served by Vertex AI
and by your own vLLM, at different prices and with different credentials.
A profile declares the endpoint, the credential, the wire format, what the
server can do, and which models it serves.
Profiles come from core-models, the provider library mast shares with core-agent. Its built-ins work with nothing else configured:
mast --provider ollama --model qwen3:1.7b "what does CrashLoopBackOff mean?"GOOGLE_CLOUD_PROJECT=my-project \ mast --provider vertex-maas --model openai/gpt-oss-20b-maas "..."Your own go in .agents/providers/<name>.yaml, one profile per file. Most
start from a built-in with extends:
name: house-vllmextends: vllmbase_url: http://vllm.infra.svc:8000/v1usage: {cached_tokens: unreliable} # this vLLM build reports nonsense cached countsmodels: - {id: Qwen/Qwen3-Coder-Next, tier: mid, context_window: 262144} - {id: gpt-oss-20b, tier: small}tiers: {mid: Qwen/Qwen3-Coder-Next, small: gpt-oss-20b}Then run it with mast --provider house-vllm. That picks the profile’s
model for the task’s tier; --model picks one explicitly. Every field and
built-in is on the
profiles reference.
What mast does with a profile:
- It refuses early. An unknown profile, a model it doesn’t serve, a tier it doesn’t declare, or a missing API key or Google credential fails at startup, naming the profile. A misspelled key in the YAML is a load error.
- Tiers come from the profile. A specialist’s
tier: smallunder--provider house-vllmruns the profile’s small model. Gemini and Claude keep mast’s own tier tables; a profile only ever adds tiers for its own provider. - Usage is measured, and cost is priced where it can be. Every call
reports input and output tokens. Cached and reasoning tokens are reported
when the server sends them, and shown as not reported (never as zero) when
it doesn’t.
- Vertex AI partner models are priced from the shared catalog in core-models (published rates, regenerated from LiteLLM).
- A self-hosted model is priced only if its profile declares what it
costs you, under
models[].rates(USD per million tokens). Otherwise cost shows as$—and the meter counts the call as unpriced. - A
max_cost_usdceiling on an unpriced model is refused at startup, because it could never trip. Declare rates, or bound the run withmax_tokensormax_turns.
- KV-cache statistics, opt-in by profile. A self-hosted profile with a
metrics_url(vLLM, SGLang, llama.cpp) makes/usagecarry the server’s prefix-cache hit rate and peak KV utilization around each turn, labelled fleet-level, because it is the server’s whole traffic. See the model server’s KV cache. - No server-side built-ins. A bundle that turns on
builtin_toolsunder a profile is refused rather than run without them. - Which models are worth it. Five Vertex AI partner models have been through mast’s judged corpus against Claude. GLM 5.2 and Kimi K2 Thinking match it on intent coverage. Llama 4 Maverick is unsupported for tool use. The results, and what the numbers mean, are on core-models’ Tested models page.
- Retries honor the server. A 429 waits as long as
retry-after-msorretry-afterasks, at the HTTP layer, before mast’s own one outer retry.
Credentials
Section titled “Credentials”Mast takes credentials from the environment; it never reads a key out of a
workload bundle. Nothing here is a mast-specific variable — each backend
reads the same names its own SDK does, so a working gcloud/genai/Anthropic
environment already works.
| Backend | Environment |
|---|---|
| Gemini API | GEMINI_API_KEY (or GOOGLE_API_KEY; if both are set, GOOGLE_API_KEY wins) |
| Gemini on Vertex | --provider=vertex with GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION (or GOOGLE_CLOUD_REGION; defaults to global) — plus Application Default Credentials |
| Claude, first-party | ANTHROPIC_API_KEY |
| Claude on Vertex | ANTHROPIC_VERTEX_PROJECT_ID (falls back to GOOGLE_CLOUD_PROJECT) and CLOUD_ML_REGION (falls back to GOOGLE_CLOUD_LOCATION, then us-east5) — plus Application Default Credentials |
Prefer the alias to the environment variable for Gemini on Vertex.
GOOGLE_GENAI_USE_VERTEXAI=true still works and still flips the backend, but
without it or --provider=vertex a project alone does nothing: the run goes
to the API-key backend and fails asking for a key the deployment does not
have. --provider=vertex names the backend outright and fails at startup
naming the project variable if it is missing.
Credentials an egress gateway adds
Section titled “Credentials an egress gateway adds”Some sandboxes hold no Google credentials at all. On
Agent Substrate with egress
credential injection, for example, the egress gateway adds the real token to
each outbound request by replacing an Authorization header that has to be
there already. There are no Application Default Credentials to find, so
building a Vertex model would fail.
Set MAST_GOOGLE_AUTH=injected, the one mast-specific variable here, and mast
stops looking for ADC. It sends Authorization: Bearer placeholder for the
gateway to replace instead. It covers Gemini on Vertex, Claude on Vertex, and
MCP servers with google_oauth auth. The
project and location variables still apply. Leave it unset everywhere else:
with nothing replacing the placeholder, Google rejects it with a 401.
Vertex AI express mode
Section titled “Vertex AI express mode”Vertex AI also takes an API key instead of a project and ADC. mast runs a
gemini-* model that way when, with no --provider alias:
GOOGLE_GENAI_USE_VERTEXAI=true(orGOOGLE_GENAI_USE_ENTERPRISE=true);- an API key is set:
GOOGLE_API_KEY, orGEMINI_API_KEY(if both are set,GOOGLE_API_KEYwins); - and neither a project nor a location is in the environment.
This is the same rule the Google GenAI SDK applies, and the one mast
followed before it moved to core-models (whose vertex-express profile now
serves it). A project or location always wins over an API key:
GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION, or GOOGLE_CLOUD_REGION
(which counts unless GOOGLE_CLOUD_LOCATION is set, even to the empty
string) each turns express mode off, and the run then needs a project and
ADC. If that happens with a key set, the startup error names the variable
that ruled express mode out. --provider=vertex never uses express mode:
it is ADC by design.
The key has to be one Vertex AI accepts. A key restricted to the Gemini API
(Generative Language) is refused by Vertex AI with
403 API_KEY_SERVICE_BLOCKED.
For claude-* there is no such switch — a resolvable project is enough. That
asymmetry is worth remembering when one process runs both families.
On Cloud Run or GKE, prefer the service account’s Application Default Credentials over a key in the environment: with Workload Identity there is no key to leak, rotate, or forget to scope.
The offline fakes are load-bearing
Section titled “The offline fakes are load-bearing”echo, scripted, and toolactor are not toys. They are how the whole
loop — inject, dispatch, park on an approval, kill -9, resume — runs in
CI and on a laptop with no credentials at all, which is what the
quickstart does in five minutes. An
acceptance suite that needs a live provider is an acceptance suite that
gets skipped.
toolactor matters specifically because echo never emits tool calls:
testing the write gate needs a fake that actually tries to call something.
One recording, concurrent branches
Section titled “One recording, concurrent branches”scripted differs from the other two in one way that matters: it holds a
position in a transcript. Every specialist in the roster shares the one
instance, and fan-out
dispatch
runs its analysts
at the same time, so a single position walked by N branches at once
would hand recorded turns out by whichever branch got there first.
It doesn’t. A replay is per branch: each concurrent lane gets its own
cursor over its own decode of the same recording, and each starts at turn
0. So a three-analyst fan-out against a one-turn recording is three
analysts that each replay that turn — not one analyst that replays it and
two that fail with script exhausted.
Nothing changes for a sequential shape. A coordinator, a planner, a single-agent replay and a resumed run are all one lane, and they go on consuming one script in one order, exactly as recorded. Two things this does not do:
- One transcript cannot describe N different branches. Every branch replays the same turns, because a recorded turn does not name the agent it belongs to.
- Two invocations running at once in one process — a daemon handling two incidents — still share the unbranched replay. A recording describes one session.
Per-specialist models
Section titled “Per-specialist models”A specialist may name its own model, and it may name a different provider than the rest of the roster:
model: gemini-2.5-flash-lite # one cheap classifying turn
# specialists/OOMKilled.specialist.mdmodel: claude-sonnet-4-6 # the one that has to reasonOverrides are dispatched by model id, exactly like --model, and
resolution is memoized per id — eight specialists on one tier share one
client. Specialists that declare nothing inherit the process model.
This is the tiering knob that makes a twelve-specialist roster affordable: the classifier does not need the model the diagnoser does, and neither needs the model the change executor does. Pair it with per-specialist budgets.
Two behaviours to know before you tier a roster:
- An override that cannot be resolved fails startup. It is never quietly downgraded to the parent’s model. A bundle that reads as tiered while silently running everything on one tier is worse than one that refuses to boot — you would be paying for the expensive tier and believing you weren’t, or running the cheap one on work that needed better. Credentials for every provider the roster names must resolve at construction.
- Offline fakes collapse overrides. Under
echo,scripted, ortoolactor, every override resolves back to the fake — so a tiered bundle still runs credential-free in smoke and acceptance runs. The collapse is exact forechoandtoolactor, which are stateless.scriptedis a cursor, so it collapses to one instance but not to one position — see below.
Tiers: the portable way to say the same thing
Section titled “Tiers: the portable way to say the same thing”Naming a concrete model id binds the bundle to that provider, which is a real cost for a bundle other deployments fork. Usually what the bundle means is not “this step needs Haiku” but “this step is not worth the frontier model” — so say that:
tier: small # one cheap classifying turn
# specialists/OOMKilled.specialist.mdtier: mid # the one that has to reasonsmall, mid, frontier. Mast resolves the tier against whichever
provider it is actually running — the --provider alias if you passed
one, otherwise the root model id’s own prefix — so the roster reads the
same and costs the right thing on either backend:
| tier | Gemini | Anthropic |
|---|---|---|
small | gemini-3.5-flash-lite | claude-haiku-4-5 |
mid | gemini-3.5-flash | claude-sonnet-5 |
frontier | gemini-3.8-flash | claude-opus-5 |
A tier default names the latest model in its line, and it moves only
after that model has been run — not when the newer id appears in the
pricing catalog. gemini-3.8-flash was priced on 2026-09-09 and became
the frontier default on 2026-10-06, once it had passed both runs below.
claude-opus-5-5 and claude-sonnet-5-5 are priced as of 2026-10-06, and
claude-haiku-5-5 as of 2026-10-07; none is a default yet. Pin one with
model: if you want it today. Above 100K prompt tokens Anthropic charges
five times as much for claude-haiku-5-5, and since v1.0 the meter bills
that call at the higher rate too.
“Has been run” is a weekly job, not a judgement call: a candidate model runs the same 31-scenario judged corpus the incumbent default runs every night, graded by the incumbent so only one variable moves, and the two boards are diffed. Scoring within noise of the incumbent — and passing the outcome tier on the same model — is what promotes it.
Both behaviours above carry over unchanged: an unresolvable tier fails
startup, and the offline fakes collapse tiers back to the fake. Startup
logs each tier next to the id it became, so what a roster is spending is
readable without knowing the table. A root model whose provider cannot be
told from the alias or the id fails startup asking for --provider
instead of guessing a vendor.
A spec declares model: or tier:, never both — that is a load error,
not a precedence rule. Pin an id when the bundle has a reason to care
which vendor answers; declare a tier the rest of the time.
Server-side built-in tools are off unless the bundle asks
Section titled “Server-side built-in tools are off unless the bundle asks”Each provider ships tools that run on the vendor’s own servers — Gemini’s
google_search, url_context and code_execution, Anthropic’s
web_search. They are the one capability mast cannot see: a built-in never
comes back as a tool call, so the permissions gate, the write gate and the
effect outbox all look right past it, and a read_only specialist could be
reading the public internet with nothing in the transcript to say so.
So mast starts every provider with all of them off and lets the bundle turn one on:
builtin_tools: web_search: truemast’s baseline rather than each vendor’s is the deliberate part. Gemini
ships search and URL context on, Anthropic ships search off; inheriting
those would mean the same bundle reaching further on one provider than the
other, which is exactly what this page’s promise rules out. The startup
line reports what the constructed model will send —
builtin_tools=web_search — so a key that did not take is visible rather
than assumed. See
builtin_tools:.
Asking a model to think, or not to
Section titled “Asking a model to think, or not to”mast does not ask any model to think. Every provider decides that for
itself on the request mast sends, which for a reasoning model usually
means it thinks. The knob exists for library embedders who build the
request themselves: genai.ThinkingConfig, with a token budget and an
IncludeThoughts flag.
Since v0.9 that knob reaches Anthropic in the shape the target model
accepts, which is not one shape. Anthropic replaced the budget-carrying
thinking.type=enabled with thinking.type=adaptive at the 4-6
generation and removed the old one at 4-7, so a budget sent to
claude-opus-5 used to be a 400, not a degraded turn — and
claude-opus-5 is mast’s own default Claude model. mast now picks by
model: a budget goes to older models as a budget, and to newer ones as
“think”, because there is no field on the newer API that takes a number
of tokens.
A budget of zero is the one case worth knowing about. It reads as
“do not think”, and it is the one request that could not say so: mast
sent no thinking parameter at all and the model thought anyway. It now
sends thinking.type=disabled, which every Claude model accepts, and
the reasoning stops.
IncludeThoughts: true asks for the reasoning text itself. It is off by
default, and that is a cost decision as much as a privacy one: mast does
not publish a model’s thinking on any surface (see
interop), so paying for
text nothing reads would be waste. Older Claude models have no way to
express the request and ignore the flag, as they always have.
Spend is computed from each provider’s token accounting for the models
actually used, so a cross-provider roster still rolls up to one
max_cost_usd. See budgets.
Providers do not agree on which token buckets exist, and the record mast reads usage through is Google’s, so a count Anthropic reports and Gemini does not had nowhere to go. Since v0.9 each adapter also carries what it was actually told — cache reads, cache writes, reasoning and tool-use tokens, plus the model the backend says it ran — as a normalized record beside the usual one, distinguishing reported zero from not reported. That is what closed the cache-write under-billing described in budgets.
Vertex context caching is built, and nothing wires it
Section titled “Vertex context caching is built, and nothing wires it”mast carries a manager for an explicit Vertex context cache — create
it, extend its TTL, drop it when Vertex reaps it — and the Gemini
provider can stamp that cache onto each request. Nothing turns it
on. The mast binary never has: a daemon you start from the CLI runs
uncached. Until the packages outside the v1.0
surface moved under internal/
(#301), Go code could
construct the manager itself; it no longer can.
Nothing about it is configurable from a flag or a bundle, on purpose:
the cache is scoped to one system instruction and tool set, and
deciding when that is stable enough to cache is the embedding program’s
call, not ours. That makes it a candidate option for a custom main.go
rather than a flag. If you have a workload that wants it, open an issue
saying so — that is what brings it back.
When it is wired, an evicted or expired cache is recovered transparently — the turn that meets it retries uncached and the manager creates a fresh cache on the next one. There is no state to clear and no error your code sees.
Reference
Section titled “Reference”- CLI —
--model,--provider, and the environment variables each backend reads. - Workload bundle — per-specialist
model:andtier:.