Skip to content

Profiles

A profile declares how to reach models: the endpoint, the credential, the wire format, what the server can do, and which models it serves. Package profile owns the schema. Each product decides where operators write their profiles.

A self-hosted vLLM server, starting from the built-in vllm template:

[
{
"name": "house-vllm",
"extends": "vllm",
"base_url": "http://vllm.infra.svc:8000/v1",
"metrics_url": "http://vllm.infra.svc:8000/metrics",
"usage": {"cached_tokens": "unreliable"},
"models": [{"id": "Qwen/Qwen3-Coder-Next", "tier": "mid", "context_window": 262144}],
"tiers": {"mid": "Qwen/Qwen3-Coder-Next"}
}
]
FieldMeaning
nameWhat --provider and product config refer to. Lowercase letters, digits and -
extendsA built-in profile to start from. Every field set here overrides the built-in’s
dialectopenai-chat, openai-responses, anthropic or gemini
platformWhere the vendor’s API is hosted, when that changes the request. vertex routes an anthropic profile the way Vertex AI serves Claude: the model in the URL path, anthropic_version in the body. Set on anthropic-vertex and inherited by profiles extending it. sap-orchestration routes an openai-chat profile through SAP AI Core’s orchestration service (see SAP AI Core); set on sap-ai-core. Unset means the vendor’s own API. Said explicitly because neither the credential nor the backend name settles it: a proxy in front of Vertex may take a bearer token
base_urlThe API root. May contain {param} placeholders and ${VAR} references. Required for the OpenAI dialects. For anthropic with platform: vertex, the Vertex AI publisher prefix (…/publishers/anthropic/models)
paramsValues for base_url’s placeholders. Each may be ${VAR} or ${VAR:-default}, and a default may itself be a reference: ${A:-${B:-x}} is the first of A and B that is set, else x
auth.kindapi_key or bearer (with auth.env naming the variable that holds it), google_adc, oauth2_client_credentials, or none
auth.scopesOAuth scopes for google_adc (defaults to cloud-platform) or oauth2_client_credentials (none by default)
auth.token_urloauth2_client_credentials: the token endpoint. May be or contain ${VAR} and ${VAR:-default}
auth.client_id_env, auth.client_secret_envoauth2_client_credentials: the variables holding the client id and secret
auth.audienceoauth2_client_credentials: an audience for identity providers that require one. Optional
auth.client_authoauth2_client_credentials: basic (HTTP Basic, the default) or post (id and secret in the form body)
headersHeaders sent on every request to the model endpoint, for a gateway that needs one that isn’t the credential: SAP AI Core’s AI-Resource-Group, an API-management subscription key, a tenant or routing header, an anthropic-beta opt-in. Values may be ${VAR} or ${VAR:-default}, read at Resolve; a header whose value comes out empty isn’t sent. Credentials (Authorization, x-api-key, api-key, x-goog-api-key, x-goog-user-project) and the headers the dialects set (Content-Type, Accept, User-Agent, anthropic-version, …) are refused. Names are case-insensitive and sent in canonical form. See Headers
backendThe name prices are keyed on. Defaults to name, except that a profile extending vertex-maas, gemini or vertex keeps the built-in’s
pricingisolated for a backend that bills in its own units: prices come only from rows keyed <backend>/<model> (declared models[].rates, or such a row in pricing.json), never from a model’s first-party list price. Set on sap-ai-core and inherited; shared (the default) undoes it. The catalog applies built-in profiles’ settings itself, and yours when the product passes your profiles in pricing.Options.Profiles
open_modelsAccept any model id, not only the listed ones. Set on every self-hosted template
capabilitiesresponse_schema, reasoning_echo, parallel_tool_calls, server_tools, streaming, forced_tool_choice. Each is true, false, or unset. Unset reads as absent, except forced_tool_choice, which is assumed supported unless declared false
usage.cached_tokensunreliable records cached tokens as not reported even when the server sends them
reasoning_formatthink_tags for a server that puts reasoning inline as a leading <think>…</think> block (Ollama; vLLM or SGLang without a reasoning parser). The block becomes a reasoning part instead of answer text
metrics_urlThe server’s Prometheus endpoint, for the opt-in KV-cache sampler
metricsWhich server’s metric names metrics_url exposes: vllm, sglang, llamacpp or none. The vllm, sglang and ollama built-ins set it (ollama as none), and a profile extending one inherits it
extra_bodyVendor fields added to every request, for switches the dialect doesn’t model. For example, {"chat_template_kwargs": {"enable_thinking": true}} turns on Gemma 4’s thinking on vLLM. Can’t set a field the adapter sets itself (model, messages, tools, temperature, …)
modelsid, plus an optional tier (small, mid or frontier), context_window, and capabilities and extra_body that override the profile’s for that model only. context_window is the input window in tokens as this server serves it, which for a self-hosted model can be less than the model’s own (vLLM’s --max-model-len). Hosts read it with Profile.ContextWindowFor to size compaction, and fall back to their own table when it’s absent
models[].ratesA declared price, in USD per million tokens: input_per_mtok, output_per_mtok, cached_input_per_mtok, cache_write_per_mtok. For a server no published catalog covers, such as your own vLLM. Products price from their catalog first and fall back to these. An input or output price is required, because declaring a model free would make a cost ceiling that never trips
tiersThe model for each tier the profile can fill

Capabilities are per profile and refined per model, because two models behind one server can differ: one needs its reasoning echoed back, another rejects a forced tool call. With forced_tool_choice: false, a request that forces a tool call (required, or one named tool) is sent as auto instead, and the response carries core_models.tool_choice_downgraded.

extends merges params, headers and tiers key by key (headers case-insensitively, and an empty value drops an inherited header). It replaces models and auth as a whole, and overrides each capability on its own.

A gateway in front of a model often wants a header beside the credential. An Azure API Management front end, for example:

[
{
"name": "apim-vllm",
"extends": "vllm",
"base_url": "https://models.example.azure-api.net/vllm/v1",
"auth": {"kind": "bearer", "env": "APIM_TOKEN"},
"headers": {"Ocp-Apim-Subscription-Key": "${APIM_SUBSCRIPTION_KEY}", "X-Tenant": "${TENANT:-shared}"}
}
]
  • Open sends them on every attempt, retries included, in every dialect. A host that builds a dialect itself instead of through Open (for example gemini.New, to share one client with its context cache) wraps the HTTPClient it passes with Resolved.HeaderTransport.
  • They go to the model endpoint only: not to a token endpoint, and not to another host a redirect points to. A metrics_url gets them when it is on the API’s own scheme and host, by the same rule as the credential.
  • A header the request already carries keeps its value, so a per-call setting wins over the profile’s default.
  • A value may be a secret, so keep it in the environment and reference it. A value read from the environment never appears in an error or a printed Resolved; one written literally into the profile is part of the profile, like any other field. Resolved.HeaderNames lists the names for a host to show.
ProfileDialectEndpointAuthNotes
anthropicanthropichttps://api.anthropic.comAPI key from ANTHROPIC_API_KEY, sent as x-api-keyAny Claude id. No tiers: which model a tier means is the product’s call
anthropic-vertexanthropicVertex AI’s Claude publisher prefix for the project and regionGoogle ADCplatform: vertex. Project from ANTHROPIC_VERTEX_PROJECT_ID, then GOOGLE_CLOUD_PROJECT. Region from CLOUD_ML_REGION, then GOOGLE_CLOUD_LOCATION, default us-east5; global, us and eu get their own hosts. Backend anthropic-vertex. Ids as Vertex names them, such as claude-opus-4-5@20251101
geminigeminiThe Gemini Developer APIAPI key from GOOGLE_API_KEY, or GEMINI_API_KEYBackend gemini. Server-side tools (web_search, url_context, code_execution) are off unless asked for
vertexgeminiGemini on Vertex AI for the project and regionGoogle ADCBackend vertex. Project from GOOGLE_CLOUD_PROJECT. Region from GOOGLE_CLOUD_LOCATION, then GOOGLE_CLOUD_REGION, default global
vertex-expressgeminiGemini on Vertex AI in express mode: an API key, no projectAPI key (GOOGLE_API_KEY)Backend vertex, so it prices like vertex. platform: vertex routes the key to Vertex AI instead of the Developer API. The key has to be allowed to call the Vertex AI API
openaiopenai-responseshttps://api.openai.com/v1API key from OPENAI_API_KEYBackend openai. Stateless (store: false); a reasoning model’s encrypted reasoning is carried forward between turns. Any id is accepted: when a model refuses include, temperature or top_p, the request is retried once without it, the refusal is remembered for that model, and the response lists what was left off (core_models.dropped_params)
xaiopenai-chathttps://api.x.ai/v1API key from XAI_API_KEYBackend xai. To use xAI’s Responses API instead, extend it with "dialect": "openai-responses"
vertex-maasopenai-chatVertex AI’s OpenAI-compatible endpoint for the project and regionGoogle ADCProject from GOOGLE_CLOUD_PROJECT. Region from GOOGLE_CLOUD_LOCATION, default global. Ids are publisher-qualified, such as openai/gpt-oss-20b-maas. Lists the gpt-oss models with forced_tool_choice: false, per Google’s function-calling notes; any other id is still accepted
ollamaopenai-chathttp://localhost:11434/v1noneWorks with nothing set. reasoning_format: think_tags
vllmopenai-chattemplate: set base_urlnoneParallel tool calls declared
sglangopenai-chattemplate: set base_urlnoneParallel tool calls declared
openai-compatibleopenai-chattemplate: set base_urlnoneFor any other OpenAI-compatible endpoint; set auth too
sap-ai-coreopenai-chatSAP AI Core’s orchestration deployment, found at startupOAuth2 client credentials from AICORE_SERVICE_KEY, or AICORE_AUTH_URL, AICORE_CLIENT_ID, AICORE_CLIENT_SECRET and AICORE_BASE_URLplatform: sap-orchestration. Backend sap-ai-core, with no prices. Any id SAP’s orchestration serves, such as anthropic--claude-4.5-sonnet or gpt-5-mini. Not yet validated against a real tenant. See SAP AI Core

Opting a server into the Responses API. Any OpenAI-shaped profile can set "dialect": "openai-responses" to use POST {base_url}/responses instead of Chat Completions. vLLM serves it:

{"name": "house-vllm-responses", "extends": "vllm", "dialect": "openai-responses",
"base_url": "http://VLLM-HOST:8000/v1"}

Built-in capabilities are conservative. Structured output, for example, is declared absent on every self-hosted template, because vLLM and SGLang enforce it only with guided decoding enabled. A server that accepts the field and ignores it would turn a guaranteed-parseable answer into a paragraph at run time. If your server enforces it, declare it in your profile.

Some partner models are served from one region only. Llama 4 Maverick, for example, returns 404 at global and answers at us-east5. (It is also unsupported for tool use.) Give such a model a profile of its own rather than moving every model’s region:

{"name": "vertex-maas-us-east5", "extends": "vertex-maas", "params": {"region": "us-east5"}}

sap-ai-core reaches SAP’s Generative AI Hub through its orchestration service, the shape LiteLLM’s sap provider and SAP’s SDKs use. It reads the credentials SAP issues in either form:

  • A service key: the JSON from SAP BTP, in AICORE_SERVICE_KEY, as SAP issues it or nested under "credentials" as a Cloud Foundry binding holds it. It supplies the token endpoint, the client and the AI API URL.
  • Four values: AICORE_AUTH_URL, AICORE_CLIENT_ID, AICORE_CLIENT_SECRET and AICORE_BASE_URL (the service key’s url and serviceurls.AI_API_URL; SAP’s /oauth/token and /v2 paths are added when they’re missing).

Setting both is refused, naming the variables, rather than one silently winning. AICORE_RESOURCE_GROUP picks the resource group (default default), and AICORE_DEPLOYMENT_ID pins a deployment instead of the newest running orchestration one. Only client-secret service keys work; an X.509 key is refused with a message saying so.

Terminal window
export AICORE_SERVICE_KEY="$(cat sap-service-key.json)"
export AICORE_RESOURCE_GROUP=team-a

Orchestration’s own modules (content filtering, data masking, grounding, translation) are yours to configure. Put them under extra_body.modules, which is merged into the request’s config.modules; every other extra_body key goes to the model’s parameters:

{"name": "sap-masked", "extends": "sap-ai-core",
"extra_body": {"reasoning_effort": "low",
"modules": {"masking": {"providers": [{"type": "sap_data_privacy_integration",
"method": "anonymization", "entities": [{"type": "profile-email"}]}]}}}}

Two limits to know about:

  • The newest message in a request is rendered by orchestration’s template for {{?name}} placeholders. A literal {{? in it is sent as a placeholder whose value is {{?, so it should come back as written; earlier messages pass through untouched.
  • SAP bills in capacity units, and core-models ships no sap-ai-core prices. The profile sets pricing: isolated, so no first-party list price applies either: every model is unpriced until you declare models[].rates (or a sap-ai-core/<model> row in pricing.json) with your contract rate.

The request and response shapes follow SAP’s Orchestration v2 API specification and SAP’s Cloud SDK for AI, and are tested against a fake tenant. They have not yet been checked against a real SAP AI Core tenant.

p, err := coremodels.Open(ctx, prof, coremodels.Options{})
m, err := p.Model(ctx, "Qwen/Qwen3-Coder-Next") // an llm.LLM

Open resolves the profile and picks its dialect. Model refuses an id the profile doesn’t serve. A dialect that isn’t built yet fails at Open and says which phase brings it.

Both checks happen before any request is sent:

  • Validate checks the shape:

    • known dialect and a well-formed URL;
    • every placeholder has a param, and the auth block is consistent;
    • models are listed unless open_models is set;
    • every tier names a model the profile serves.

    It reports every problem at once, and each message names the profile.

  • Resolve checks the profile against the environment: every ${VAR} is set, and the API key or Google credentials are found. A missing value is named by its variable, never shown by its content.

The strict JSON reader DecodeJSON rejects unknown keys, so a misspelled base_ulr is an error instead of a silently missing value.