Skip to content

Approvals and the write gate

The question an SRE team asks about an autonomous agent is not “how good is its diagnosis”. It is “what can it do to my cluster, and who said it could?”

mast answers that with four boundaries, deliberately layered so that no single one has to be perfect. Three are enforced by the runtime; the fourth is your cluster’s, and mast is built to compose with it rather than replace it.

1. Capability — declared per specialist, checked at startup

Section titled “1. Capability — declared per specialist, checked at startup”

Every specialist declares what class of thing it is:

capability: read_only # the default
capability: change_executor # may reach mutating tools

At construction — before the daemon serves a single request — mast walks the roster and refuses to start if a read_only specialist can reach a mutating tool. Three shapes are refused, each with the offending file and tool named in the error:

  • it names a mutating tool in its allowlist,
  • it grants a whole MCP server (under default-deny-unknown, that is a grant of every mutating tool on it),
  • it declares no tools.mcp at all while a tool catalog exists — an unrestricted specialist is not a read-only one.

SingleTurn specialists are exempt: the mode carries no tools, so there is nothing to reach.

The point of this layer is that the read/write split is structural, not prompt-held. A prompt saying “you are a read-only diagnoser” is a suggestion to a language model. A construction-time refusal is not. Startup also logs every change_executor in the roster, so the answer to “what can change my cluster” is a log line, not an intersection of three files.

2. Mutation classification — declared, never guessed

Section titled “2. Mutation classification — declared, never guessed”

The gate can only stop calls it knows are mutating, so how a tool gets classified is load-bearing. Two parties can declare it, and unclassified means mutating if neither does.

The workload’s tool_catalog is read first and wins: a tool it names is classified the way it says, in either direction, and every un-gating is audit-logged at startup. Below that, mast reads the readOnlyHint annotation the MCP server publishes about its own tools, so a server that says a tool only reads is believed and that tool does not park. A tool neither party has classified is mutating — as is one the server annotates as not read-only, which on the wire is the same thing.

What mast will not do is guess from the verb in a tool’s name. The failure mode it chooses instead is the safe one — a tool nobody classified parks for an approval that may not have been needed, rather than a mutating tool firing unseen — and where a server’s word is not something you want to take, the tool_catalog override above is how you say so. Details, the full precedence order, and the reason a name-based heuristic was rejected: tools and MCP.

3. The write gate — one call, one operator, one answer

Section titled “3. The write gate — one call, one operator, one answer”

With the classification in hand, the gate is simple to state: a call that would change something stops before it fires.

hitl:
on_mutation: require_approval # default. also: apply | dry_run

The session parks with a durable interrupt carrying the specialist, the tool, and the exact arguments. The agent is not spinning; the process can die and the pause survives it (durability). An operator answers with one of three verdicts:

VerdictEffect
approvethe call fires, exactly as shown
rejectthe call does not fire, and the agent stops rather than looking for another way
editthe operator supplies corrected arguments, and those run

Three properties are worth calling out, because each closes a hole that looks small on paper:

  • Scope is once, unless the operator is looking at the whole set. A verdict approves the single call in front of them. The one exception is change_set, below: it authorizes the specific other calls quoted in the same question, and nothing else. Anything wider — “approve all scale operations for this session” — is refused and audited as approval_scope_refused, because that is the blanket consent the gate exists to prevent.
  • The approver is the authenticated caller, not the payload. Whatever name arrives in the request body is overwritten by the identity that actually authenticated. An audit trail an agent could write into is not an audit trail. The same rule governs who a resumed pause names — see who a resume names.
  • An edit is recorded durably. The substrate re-fires the original call on resume, so the edited arguments are persisted as their own record and re-applied — the thing that ran and the thing the operator authorized are the same thing, and the log shows both.

Every outcome lands in the audit log under a fixed vocabulary: awaiting_approval, denied_by_policy, denied_by_operator, approval_scope_refused, edit_unattributed, edit_refused, edit_applied, apply, dry_run, already_refused, refusal_loop, and the change-set outcomes change_set_approved, approved_by_change_set, change_set_scope_refused, change_set_refused.

Everything above assumes an operator finds out. Four things surface a park — GET /parks, the attach stream, mast sessions show, and the audit log — and all four are pulls. For an unattended workload that is a real gap: a fifteen-minute cadence can park at 03:00 and sit there until somebody thinks to look, and the whole point of a durable park is that nobody has to be watching when it opens.

--park-notify <conversation> closes it by pushing, through the same chat ingress --notify-url already configures. One message when the park opens, naming the session and the call and ending with the command that prints the resume line. Not repeated, budgeted at three then one per five minutes, and counted in mast_park_notifications_total.

It announces every park rather than only the ones nobody is watching. mast could check whether an operator is attached at the moment the park opens, but that answer has a short shelf life — the park outlives the process, and somebody who detaches thirty seconds later would get silence all night. The cost of announcing anyway is a redundant message to somebody already looking, which is the cheaper of the two mistakes.

Note what the message contains before choosing a channel: the park hint renders the call with its argument values, elided at 120 characters each. The full arguments stay in the confirmation record.

Whatever the chat ingress does — throttled, failed, misconfigured — the park itself is untouched. A failed announcement is an error on the daemon’s books and never on the turn’s.

A refusal the model cannot talk its way past

Section titled “A refusal the model cannot talk its way past”

“The agent stops rather than looking for another way” is what the refusal tells the model. The gate does not depend on it complying.

For the rest of the turn, the call an operator refused is suppressed: the model can propose it again, and the operator is not asked again. It comes back as already_refused. A different call still parks normally, so the suppression narrows nothing the operator has not already answered.

Three suppressed proposals and the turn ends, counted as refusal_loop. Nothing latches — no guardrail trips, there is nothing to reset, and the next turn starts clean, refusal and all. The scope is one turn on purpose: a refusal that outlived its turn would silently block a call the operator would have approved.

The behavioral watchdog would eventually catch the same loop, but it halts the session and makes an operator clear a guardrail to get the daemon working again. One “no” should not cost that, so the gate cuts first, at a lower threshold, and tells the watchdog to forget the calls it turned away — they never ran. The watchdog’s own halt is never cleared by this; an arm that could un-halt a session would let a looping agent overrule its operator.

Details and the exact wording the model gets: rejection stops the agent.

The other two modes exist for real situations, not as escape hatches: dry_run runs the whole loop and reports what would have happened, which is how you evaluate a roster before trusting it; apply is for already-bounded automation where the RBAC layer below is the real constraint. Both are opt-in, in the bundle, reviewable in a diff.

Field-level detail, request and response shapes: write gate reference.

4. RBAC — the boundary mast does not own

Section titled “4. RBAC — the boundary mast does not own”

The first three layers are mast’s, which means they are only as good as mast’s own correctness. The fourth is not: the shipped Kubernetes base mirrors the capability split into two service accounts, so a compromised or buggy read-only path has no credential to change anything with, whatever it believes about itself.

That is the layer to lean on hardest, because it holds when the others are wrong. See cluster permissions for the manifests and the current caveat on narrowing the GKE IAM binding.

The change set — approving the call, not the prose

Section titled “The change set — approving the call, not the prose”

The three layers above stop a call. This one decides which call the operator is being shown.

A finding that says “scale the api Deployment back to 2 replicas” is prose, and prose cannot be approved, only agreed with. If the change executor composes the actual tool call afterwards, from that sentence, on its own turn, then the thing that reached your cluster is not the thing anybody read.

So a report can carry the call. A workload’s report schema declares proposed_change, and a finding fills it with entries naming a tool from the workload’s own catalog:

"proposed_change": [
{"tool": "apply_change", "arguments": "{\"namespace\":\"prod\",\"replicas\":2}"}
]

Two checks run the moment the report is returned, before it becomes the specialist’s answer: the tool has to be one this workload declares, and the arguments have to fit that tool’s own input schema. A report that fails either comes back to the specialist as an error it can correct, rather than travelling on to an operator who has no way to tell. It is the same code that validates an operator’s edit at the gate, so the two cannot drift apart on what “valid arguments” means.

An empty list is a right answer. “Raise the memory limit, but I don’t know to what” is an honest diagnosis; a specialist that cannot name an exact call returns nothing and escalates, instead of inventing a plausible number to fill a field.

Under graph dispatch this closes the loop the first three layers left open. A finding with a non-empty change set that the operator approved is routed to the roster’s change executor, which is handed those calls verbatim — the edge is a structural rule about the finding, not an instruction in a prompt. Everything else about the gate is unchanged: each call still parks, still names its arguments, still needs one authenticated answer. A roster that declares no proposed_change, or a finding that proposes nothing, behaves exactly as it did before; and a roster carrying two change executors logs an error and leaves the edge unwired rather than guessing which was meant.

Field reference: the report contract.

Two gates, and how many questions they add up to

Section titled “Two gates, and how many questions they add up to”

Under graph dispatch with hitl.require_approval: true, a remediation meets two gates, and they are not one gate asked twice:

  • the finding gate asks whether a diagnoser’s report should be acted on at all. Answering it is what routes the change set to the executor.
  • the write gate asks about each call, with its arguments, as the executor makes it. Answering it is what changes the cluster.

So a single-call remediation costs an operator two answers, and that is the design rather than a redundancy: approving a finding is not approving a call, and hitl.change_set_ttl does not collapse them — a grant is minted at the write gate for calls 2..N of a set, so a set of one has nothing to grant.

What the executor does not add is a third question about its own report. By the time it has one, its calls have already been made, each past the write gate; approving the account afterwards would decide nothing.

The two gates also arrive on separate turns, and stay answered. A specialist parked at the write gate has not produced a finding yet, so its node does not raise the finding gate over a result that does not exist; and an answer given at either gate is recorded, so re-entering the graph on a later turn does not ask for it again. Earlier versions did both — the two parks landed on one turn, and answering either stranded the other (#269).

One answer for a set of calls — and what makes it go stale

Section titled “One answer for a set of calls — and what makes it go stale”

A remediation is often several calls: scale two Deployments, or patch a ConfigMap and restart what reads it. Parking each one separately means the operator who has already read the plan answers the same question three times, and the third answer is not a decision, it is a reflex.

So when a parked call belongs to a change set, the question says so and quotes the rest of the set. Answering with scope: change_set mints a grant for each remaining call:

Message: Approve mutating call scale_deployment(deployment=api, replicas=2)?
It is one of 2 calls in the change set ScaleUp proposed;
answer with scope=change_set to authorize all 2.
Change set: 2 call(s) proposed by ScaleUp
1. scale_deployment(deployment=api, replicas=2)
2. scale_deployment(deployment=worker, replicas=2)
freshness of scale_deployment: re-checked against get_deployment before the call fires
Approve all with: --response='{"verdict":"approve","scope":"change_set"}' (valid for 600s)

A grant is deliberately narrow:

  • it authorizes one exact (tool, arguments) pair — not the tool, not the verb, not “scaling” — so a call the model composes afterwards, however similar, still parks;
  • it is single-use and durable, so a restart between the answer and the call changes nothing, and a second identical call needs its own approval;
  • the deny policy still runs first, and spending a grant is still recorded as an allow-once decision in the audit log. A grant removes the question, never the accounting;
  • if any call in the set cannot be granted — an argument too large to render in the question, a precondition mast cannot evaluate — the whole verdict is refused with a reason rather than covering the part that fits. A partial “approve all of this” is the one outcome nobody chose.

What makes an approval stale is the cluster, not the clock. A set approved twenty seconds ago against a Deployment somebody has since scaled by hand is stale. One approved an hour ago against an untouched object is not. mast cannot work out which read tells it that — it does not know which tool reads the object a write is about, or which of the write’s arguments names it — so the bundle declares it, per tool, and mast runs it twice: once when the operator answers, once just before each granted call fires. If a watched field moved in between, the grant is voided and the call goes back to the operator with a question naming the field and both values.

Both of those reads run the tool itself, underneath the digest wrap. Digesting exists to shrink what a model reads; this read is mast’s own, and a digest of it would drop the very field the comparison is about.

hitl.change_set_ttl (default 10 minutes) is the backstop for everything a precondition cannot see. A tool that declares no precondition gets the TTL and nothing else — and its question says so, in those words, rather than implying a check mast is not making.

How to declare one, and the mistake to avoid on your first try (a precondition watching the field the set itself rewrites invalidates its own set): precondition: in the workload bundle.

Everything above is about whether a change happens. This is about what is true after it has.

An operator who approves a change is making a bet, and a fair bet needs a way to fold. Without one the position is the worst of the three available: you can approve a change, watch the incident get worse, and have no path back except reconstructing the old state by hand while the pager is going.

So when the workload declares it, the gate reads the target’s prior state before the call fires and records it — the old values, and the call that restores them. mast sessions show prints both.

The undo is a proposal, not a button. mast will not fire it. Running it sends it back through this same gate with a person answering, because a rollback mast decided on by itself is a mutating call nobody approved.

Two limits are worth knowing up front, because both are decisions rather than gaps. mast cannot work out the read or the inverse on its own — which tool reads the object a write is about is domain knowledge, and scale_deployment inverts by re-scaling while delete_pod does not invert at all — so the workload declares them, and a workload that declares no inverse gets a record of the old value with the undo marked undeclared. That is honest and still useful. And a capture proves a record exists, not that the revert would work; nothing executes one to check.

How to declare it: capture:. What it looks like in the transcript: what the change overwrote.

What the operator decided, after the incident is over

Section titled “What the operator decided, after the incident is over”

A verdict is expensive. Somebody stopped what they were doing, read a call, and made a judgement about a production system. Spending that once and keeping only a log line wastes the most informative thing a gated fleet produces: a case where the model’s proposal and a human’s answer are known to differ.

So every adjudication is a durable record beside the call it decided, and mast sessions export-decisions writes them out as JSON Lines — one row per decision, with what the model proposed, what actually executed, which of approve / reject / edit the operator chose, and whether the call ended up authorized, refused by the operator, or refused by mast. Rejections are in there for the same reason edits are: an export of nothing but approvals is a dataset that never shows the model being wrong.

Two things to know before pointing it at a production store. Approver identities are digested by default — a stable hash, so you can still group by approver or count how many people approved a class of change, without naming anyone; --include-approver opts out and the file records which mode produced it. Tool arguments are exported verbatim, because the proposed→executed diff is the whole label — which makes an export as sensitive as the arguments your tools take.

Mast captures and exports; it does not score, retrain, or read the file back. What it is for is the evaluation you run yourself. Field detail and the full record shape: exporting decisions.

LayerStopsDoes not stop
Capability splita read-only specialist ever reaching a mutating toola change_executor doing something wrong
Mutation predicatean unclassified tool sliding through as safea tool wrongly declared mutating: false
Write gateany classified mutation firing unseenan operator approving a bad call
RBACany credential-less change, whatever mast believesa change inside the granted scope

Nothing here removes the operator from the loop. That is the design: mast’s job is to make sure the operator is asked, with the actual call in front of them, and that the record of what they said survives the process that asked.

The fifth column of that table — what the four layers add up to against a deliberate attacker, and the places they are known not to reach — is the threat model.