not on service | | `gates` | namespaces a webhook gates, from namespaceSelector: all namespaces, or \/\ namespaces with up to 5 names | | `rules` | compact operations/resources summary of a webhook’s rules, e.g. “CREATE,UPDATE pods,deployments.apps” | | `object_selector` | a webhook’s objectSelector, when one is set | | `timeout` | webhook timeoutSeconds as \s (nil defaults to the API’s 10s) | | `triage_status` | triage state from the matched record (investigating\|triaged\|actioned\|escalated) — present only with —store on merged findings | | `triage_root_cause` | the incident agent’s root-cause hypothesis, from the matched triage-status record | | `triage_action` | the incident agent’s paper trail (PRs opened, escalations), from the matched triage-status record | | `triage_session` | incident session that wrote the matched triage-status record | | `triage_age` | how long ago the matched triage-status record was last updated | | `container` | container the finding is about (init containers prefixed init:) | | `image` | image reference that failed to pull | | `restarts` | container restart count | | `exit_code` | exit code of the container’s last termination | | `last_state` | reason of the container’s last termination (e.g. OOMKilled) | | `age` | how long the abnormal state has persisted | | `desired` | desired replica/scheduled count from spec | | `ready` | ready count from status | | `updated` | updated-to-current-revision count from status | | `available` | available count from status | | `failed` | failed pod count of a Job | | `schedule` | a CronJob’s spec.schedule | | `expected` | the activation a CronJob should have run and did not | | `missed_runs` | activations missed since the anchor; ≥N when the walk was capped | | `anchor` | what the missed count was measured from: last\_schedule or creation | | `time_zone` | a CronJob’s spec.timeZone, when set | | `last_schedule` | a CronJob’s status.lastScheduleTime, or never | | `active_jobs` | Jobs a CronJob still has running | | `condition` | node condition type that is abnormal | | `taint` | taint key indicating reclaim/drain | | `pods` | pods affected (behind a cordoned node or a PDB) | | `healthy` | currently healthy pods behind a PDB | | `required` | pods the PDB requires healthy | | `addon` | system add-on role: dns, proxy, cni, csi, metrics, connectivity | | `resource` | ResourceQuota resource name at or near its limit | | `used` | quota usage from status | | `hard` | quota hard limit from status | | `pct` | quota usage as percent of the hard limit | | `pack` | the pack this finding belongs to; also the summary-line note naming the pack that ran | | `metric` | the backend-neutral metric the query measured (pack\_unavailable: the absent metric) | | `verb` | apiserver request verb for this series (apiserver pack) | | `observed` | the worst (maximum) aligned value in the window, in the query’s unit — the breach basis | | `latest` | the newest aligned value in the window | | `threshold` | the crossed threshold: the critical one when severity=critical, else the warning one | | `window` | the lookback the series cover (—since, or the pack default); also a summary-line note | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout health lookout health --format=json --top=5 lookout health --namespace=prod ``` # Prometheus metrics > Every metric the sentinel serves on --metrics-addr, derived from the registered collectors. `lookout watch --metrics-addr=host:port` serves Prometheus metrics on `/metrics` (plus `/healthz` and `/readyz`). Every metric carries the `lookout_` prefix. Generation note (the documented choice): metric names and help strings are derived from the live collectors (`internal/watch.MetricsInventory`); the type and label columns are stamped per collector in that inventory because the Prometheus client does not expose them before first observation. A presence-check test (`TestMetricsInventoryComplete`) fails when a collector is added without an inventory row. The `lookout_leeway_*` block is the exception: those instruments are declared on the OpenTelemetry metric API and bridged into the same registry, so they have no collector to describe. Their rows are written out in `pkg/sources/topologydrift.MetricDocs` and pinned there against the real exporter, names, types and labels included. Rows marked **opt-in** are absent from a default scrape until a flag turns them on; see the [`lookout watch` flag table](/k8s-lookout/reference/watch/). | Metric | Type | Labels | Meaning | | ------------------------------------------------------------ | --------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `lookout_events_seen_total` | counter | `reason`, `namespace` | Total k8s events observed by the informer, before filter. | | `lookout_events_injected_total` | counter | `reason`, `namespace` | Total events that survived filter + dedup and were POSTed to the daemon. | | `lookout_events_deduped_total` | counter | `reason`, `namespace` | Total events suppressed by the rolling-window dedup cache. | | `lookout_events_filtered_total` | counter | `gate` | Total signals rejected by the engine filter before dedup, by the rule that rejected them (reason\_not\_allowed\|namespace\_excluded\|namespace\_not\_allowed\|unhealthy\_debounce\|crashloop\_debounce\|imagepull\_transient\_debounce). The leading-edge debounces deliberately swallow events; without this counter a gate tuned too tight is indistinguishable from a broken watcher. | | `lookout_inject_errors_total` | counter | `reason`, `http_code` | Total payload deliveries (or incident opens) against the configured sink that returned a non-2xx response or transport error. Counts sink operations regardless of —sink. | | `lookout_inject_shrinks_total` | counter | `shed` | Total payloads shrunk to fit —inject-max-bytes before delivery (issue #198), by what was shed (enrichment\|message\|member\_fingerprints\|watchboard\_entries). Identity is never dropped; a counted incident still routed. A rising enrichment count means —enrich-cap is set too high for the sink’s inject ceiling; member\_fingerprints means a storm outgrew the ceiling and its member list was cut to the earliest arrivals (#336); watchboard\_entries means —watchboard-batch is too large for it and the oldest warnings in the digest were dropped (#337). | | `lookout_session_creates_total` | counter | `outcome` | Total incident-open attempts against the configured sink (core-agent: POST /sessions; webhook: POST /incidents), labeled by outcome. | | `lookout_active_incidents` | gauge | — | Current number of incidents in the sidecar’s dedup cache. | | `lookout_recoveries_observed_total` | counter | `resolution` | Total kind=resolved outcome records emitted, by resolution (recovered\|object\_deleted). | | `lookout_recoveries_reverted_total` | counter | — | Total kind=resolved.reverted records emitted: symptom recurred within the revert window after a resolve. | | `lookout_recovery_tracking` | gauge | — | Current number of bound incidents the recovery tracker is watching for clearance. | | `lookout_recovery_drops_total` | counter | `cause` | Total resolved signals dropped instead of injected, by cause (unknown\_session: binding lost, e.g. restart without —dedup-persist). | | `lookout_storms_formed_total` | counter | — | Total kind=storm incidents opened by blast-radius correlation. | | `lookout_storms_resolved_total` | counter | — | Total storms resolved because every member incident cleared. | | `lookout_storms_active` | gauge | — | Currently open (unresolved) storms. | | `lookout_storm_members_total` | counter | `kind` | Total incidents folded into storms, by how they joined (suppressed: per-incident session never opened; superseded: pre-storm session pointed at the storm; attached: late arrival). | | `lookout_storm_updates_total` | counter | — | Total kind=storm.update size refreshes injected into storm sessions (membership grew past a reporting threshold: doubling or +10, max one per minute). | | `lookout_watchboard_entries_total` | counter | `kind` | Total warning-class signals buffered onto the shared watchboard digest, by signal kind. | | `lookout_watchboard_digests_total` | counter | — | Total kind=watchboard.digest injects flushed to the watchboard session. | | `lookout_watchboard_rotations_total` | counter | — | Total size-based watchboard session rotations: a fresh session opened after —watchboard-rotate digest injects. | | `lookout_watchboard_buffered` | gauge | — | Warning-class signals currently buffered awaiting the next watchboard digest flush. | | `lookout_watchboard_reattached_total` | counter | `kind` | Total buffered warnings delivered as a kind=family.member followup into an existing per-incident session sharing their blast-radius ancestor, instead of a digest entry (issue #220), by signal kind. | | `lookout_info_dropped_total` | counter | `kind` | Total info-severity signals routed to the stored-only class (no inject anywhere), by signal kind. With —store set they are persisted; without it they are dropped after counting. | | `lookout_findings_total` | counter | `kind`, `severity` | Total distinct findings the sentinel detected, by signal kind and severity — counted once per fresh dedup window, before severity routing, so stored-only and watchboard-batched findings count alongside injected ones. Always carries the cluster label. Namespace is deliberately absent (cardinality); rate() over this is the cluster’s health trend. | | `lookout_store_records_total` | counter | `route` | Total occurrences committed to the store, by routing outcome (injected\|suppressed\|storm\|storm-member\|watchboard\|info-stored\|resolved). | | `lookout_store_write_drops_total` | counter | `cause` | Total occurrence records LOST by the store’s write path, by cause (buffer\_full: the non-blocking writer buffer overflowed; write\_error: a batch insert failed). The store is telemetry, not a system of record — drops are loud, never blocking. | | `lookout_store_pruned_rows_total` | counter | `cause` | Total occurrence rows deleted by the prune loop, by cause (ttl: older than —store-ttl; size: oldest-first eviction after —store-max-mb was exceeded). | | `lookout_enrichments_total` | counter | `outcome` | Total enrichment runs, by outcome (ok: every stage succeeded; partial: some stage failed, the rest attached; failed: no section computed — the inject still fires, carrying enrichment\_error trailers; skipped: nothing to build, the incident object is not a workload and names none, so the inject fires with no bundle at all rather than one describing the resolver). | | `lookout_enrichment_bytes` | histogram | — | Size of the attached enrichment bundle in bytes, after the —enrich-cap prefix cut (the telemetry that will inform the fixed-vs-model-aware cap revisit). | | `lookout_enrichment_truncated_total` | counter | — | Total enrichment bundles the —enrich-cap byte budget truncated at a section boundary (dropped sections become overflow trailers naming the follow-up command). | | `lookout_enrichment_failures_total` | counter | `stage` | Total enrichment stage failures, by stage (resolve\|spec\|delta\|edges\|radius\|logs). Failures never block the inject; they surface as enrichment\_error trailers in the attached bundle. | | `lookout_memory_facts_total` | counter | `class` | Total distilled facts written (upserts included) by the scheduled distiller pass, by fact class. | | `lookout_distill_errors_total` | counter | — | Total failed distiller passes. A failed pass loses freshness only — the next pass re-derives every fact from the occurrence window. | | `lookout_triage_overrides_total` | counter | `action` | Total severity-routing decisions refined by an open triage-status record, by action (downgraded: agent’s severity\_override lowered the class; upgraded: it raised it; escalated: status=escalated pinned critical). | | `lookout_triage_resolved_flips_total` | counter | — | Total triage-status records flipped to resolved by recovery injects (the automatic lifecycle — resolved records join the corpus). | | `lookout_triage_regressed_total` | counter | — | Total kind=triage.regressed evidence followups: a downgraded incident’s dedup-window count reached —triage-regress-factor times its count at downgrade time. Evidence only, never a re-page. | | `lookout_cross_source_followups_total` | counter | `source` | Total dedup-window duplicates injected as followups because their source family differs from the incident’s opening source (leading/reactive joins made session-visible), by joining source family. | | `lookout_sink_info` | gauge | `sink` | The configured agent sink (—sink), value fixed at 1 on the active label (core-agent\|webhook). ADDITIVE metric: the sink is process-level config, so it rides this info gauge instead of a new label on the operation counters — existing scrapes keep their exact series identities. | | `lookout_runner_up` | gauge | — | 1 while this cluster’s watch loop is running, 0 otherwise. Always carries the cluster label; in a multi-cluster process (issue #208) one series per watched cluster reports that runner’s liveness independently. | | `lookout_runner_restarts_total` | counter | — | Total in-process restarts of this cluster’s runner by the supervisor after it exited while the process stayed up (multi-cluster fate isolation, issue #208). Stays zero in the single-cluster default, where a runner exit ends the process and the kubelet owns restart. | | `lookout_runner_terminal` | gauge | `reason` | 1 when the supervisor has GIVEN UP on this cluster: the runner exited for a reason no retry can fix (access\_denied — the authorizer refused a required permission), so it is no longer being watched and no longer being restarted (issue #383). ADDITIVE metric rather than a label on lookout\_runner\_up, which keeps its exact series identity. The alert to write: a series at 1 means a cluster in the fleet is dark until someone changes a grant. Stays absent in the single-cluster default, where such an exit ends the process instead. | | `lookout_source_denied` | gauge | `source`, `resource`, `required` | 1 when a permission this source held at STARTUP is denied now, confirmed over consecutive SelfSubjectAccessReview sweeps (—access-recheck, issue #385); back to 0 when the grant returns. required=true means the source cannot run at all and this cluster’s runner is stopping for it; required=false is one degraded dimension on a source that keeps going. The alert to write: any series at 1 means the sentinel has lost coverage it used to have — the silence from that source no longer means the cluster is healthy. | | `lookout_cluster_resolve_errors_total` | counter | `cluster`, `cause` | Total clusters this process was told to watch and did not, by cluster and cause (issues #388, #410). credentials: the cluster could not be resolved into a client. duplicate\_name: two clusters in the fleet share this name, which is the only handle the sentinel has on a cluster, so neither is watched. The cluster is SKIPPED, not fatal, so the rest of the fleet still runs — which means a non-zero value is a coverage gap: nothing is watching that cluster and its silence means nothing. Counted at startup, so it moves on process restart and on nothing else. | | `lookout_leeway_subjects_tracked` | gauge | `subject_kind` | Subjects with a tracked distribution, by kind. | | `lookout_leeway_node_groups_discovered` | gauge | — | Distinct node groups resolved from the node-group label precedence list, before the tracking bound is applied. Read it against lookout\_leeway\_subjects\_tracked with subject\_kind=NodeGroup: the two agree on a healthy cluster, and a large number here with none tracked is leeway refusing a precedence list that resolved to something per-node. Raise —topology-max-node-groups only once you believe the count. | | `lookout_leeway_domains_unavailable` | gauge | `topology_key` | Topology domains with no usable node, by axis (leeway). Reported per configured axis and zero when nothing is out, so the series exists before the first outage. This is the live reading; lookout\_leeway\_alert\_state with subject\_kind=Domain is the same fact after its dwell, and that is what leeway.domain\_unavailable is emitted from. | | `lookout_leeway_domain_ready_nodes` | gauge | `topology_key`, `domain` | Usable nodes per topology domain. | | `lookout_leeway_domain_objects` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key`, `domain`, `state` | **Opt-in.** Objects counted per subject, topology domain and scheduling state. By default `state` is collapsed to two values: `active` for an object holding the domain’s capacity (running, or terminating and not yet gone) and `waiting` for one that is not (pending or unschedulable). Pass —topology-per-domain-collapse-states=false for the four raw states, which share no label value with the two, so a query written for one mode returns nothing at all under the other rather than an undercount. | | `lookout_leeway_domain_series_withheld` | gauge | `reason` | Subjects whose per-domain breakdown is absent or incomplete, by the cardinality control responsible. Read this before concluding a subject has no objects in a domain: the absence of a domain\_objects row means either that or that leeway declined to export it, and this is which. `gate` is below the drift floor and not alerting, which is the intended standing state and is normally most of the estate; `namespace` is the allow or deny list; `key_cap` counts subjects that kept some axes and lost the rest, and is the one to watch, because it is the reading that says a dashboard is missing an axis rather than a subject. Counted in subjects for all three, so they are summable. | | `lookout_leeway_domain_expected` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key`, `domain` | **Opt-in.** Objects apportioned to each topology domain, the expectation domain\_objects is scored against. | | `lookout_leeway_observed_skew` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key` | S, the difference between the fullest and emptiest eligible domain (leeway). | | `lookout_leeway_excess_skew` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key` | E, observed skew beyond what the arithmetic and the declared bound allow (leeway). Zero is the normal reading: a subject that cannot be spread any more evenly than it already is scores zero here however lopsided S looks, which is the whole reason drift is not alerted on S. | | `lookout_leeway_drift` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key` | ρ, the fraction of a subject’s objects that would have to move to meet its expectation (leeway). | | `lookout_leeway_max_domain_share` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key` | The share of a subject’s objects held by its fullest domain (leeway). | | `lookout_leeway_relocation_distance` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key` | R, the number of objects that would have to move to meet the expectation (leeway). | | `lookout_leeway_intent_info` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key`, `mode`, `source`, `confidence`, `weighting`, `max_skew` | Placement intent inferred for a subject on one topology axis, as labels on a constant 1 (leeway). Only subjects that expressed an intent are present: a workload with no spread constraint, anti-affinity or affinity has no row here, which is what makes the series count a property of the estate’s declarations rather than of its size. A learned baseline is deliberately absent for the same reason, even though it scores and routes like any other intent: it is not something anybody declared, and it is eventually present for every subject. See lookout\_leeway\_baselines for those. `source` is what the intent was read from and `confidence` how much that source is worth — `assumed` means `k8s-lookout` guessed a cluster default it could not read, and every finding derived from it rests on that guess. | | `lookout_leeway_alert_state` | gauge | `namespace`, `subject`, `subject_kind`, `topology_key`, `tier`, `phase` | Where one subject-axis sits in the dwell machine: 1 pending, 2 firing. Only subjects with an open episode are present — a subject that is not drifting has no row rather than a zero, which keeps the series bounded by how much trouble a cluster is in rather than by how large it is. A resolving subject (clear, but inside the resolve dwell) still reads 2, because its finding is still outstanding. | | `lookout_leeway_transient_subjects` | gauge | `topology_key`, `transient` | Subject-axes whose judgement suppressed or relaxed, by the transient state responsible. This is the series to look at before believing a quiet estate: a fleet-wide `domain-outage` row is leeway declining to page four hundred workloads about one dead zone, and a `cluster-warmup` row that never clears is a sentinel that never synced. Only the axes under a transient are present, so zero rows is the healthy reading. | | `lookout_leeway_last_event_timestamp_seconds` | gauge | `resource` | Unix time of the last informer event leeway processed, per resource. | | `lookout_leeway_evaluation_duration_seconds` | histogram | `subject_kind` | Time spent evaluating one coalesced subject. | | `lookout_leeway_counter_mismatch_total` | counter | `subject_kind` | Subjects whose incremental distribution disagreed with a rebuild from the pod cache and were repaired in place, by kind (leeway). The alert to write: threshold zero. The two numbers are two computations of the same thing, so any non-zero rate is a BUG IN K8S-LOOKOUT and not a cluster condition — every finding derived from the drifted counters until it is fixed is wrong in the same direction. The repair keeps the next hour’s numbers usable; it is not a fix. | | `lookout_leeway_baselines` | gauge | `state` | Subject-axes with a learned baseline, by state: `learning` is still inside a maturity gate, `mature` is old enough and sampled enough to be scored against, and `frozen` is being held still because its subject is firing or under a transient. The three overlap — a frozen baseline is also learning or mature — so they do not sum to the total. This is the series to read before turning Tier C baseline signals on: `mature` is how many subjects would start being judged against what they normally do rather than against an even split, and a `mature` that never climbs means something is resetting the baselines — check the reset outcome on baseline\_samples\_total. | | `lookout_leeway_baseline_samples_total` | counter | `outcome` | Baseline samples, by what the estimator did with each (leeway). A healthy estate is almost all `applied`. A sustained `reset` rate is the failure mode worth alerting on: the domain set is the invalidation fingerprint, so something churning it — a zone label appearing and disappearing on nodes — restarts every affected baseline’s history and keeps it permanently immature, silently. `empty` is subjects with nothing to learn from (scaled to zero, or every pod Pending) and `held` is a frozen set, and neither is a problem. | | `lookout_leeway_preference_pod_time_seconds_total` | counter | `provider`, `axis`, `spec_hash`, `rank` | Pod-seconds accumulated at each preference rank of a compute class (leeway). This is the series the whole source exists for, and it is time-weighted on purpose: a ninety-second burst of rank-3 pods during a scale-up and three weeks parked on a spot fallback look identical to a gauge. `rank` is the derived preference TIER, not the raw ccc\_priority\_index — on a class that sets priorityScore those are different numbers and can even run in opposite directions. Three ranks are not numbers at all: `unknown` is a node whose rank is not yet resolved, `unsatisfiable` one GKE could not fit to any rule, and `off-axis` one provisioned outside the priority list entirely. `spec_hash` changes when the class is edited, which starts a new series rather than averaging two different definitions of rank 1 together. | | `lookout_leeway_preference_rank_weighted_time_seconds_total` | counter | `provider`, `axis`, `spec_hash` | Sum of rank times pod-seconds, over tier ranks only (leeway). Divide by the pod\_time summed over the same ranks to get mean achieved rank: 0.0 is an estate getting its first choice, and a number that climbs is capacity quietly draining out from under it. The unknown, unsatisfiable and off-axis buckets are excluded from both halves, because a mean over a bucket whose rank is `unsatisfiable` is not a mean of anything. | | `lookout_leeway_preference_pods` | gauge | `provider`, `axis`, `rank` | Pods currently occupying each preference rank. The supporting gauge to pod\_time, not a substitute for it. | | `lookout_leeway_preference_nodes` | gauge | `provider`, `axis`, `rank`, `rule_index`, `rule`, `source` | Nodes currently resolved to each preference rank, by where the rank came from. `source=annotation` is GKE’s own ccc\_priority\_index, `inferred` is `k8s-lookout` matching the node’s attributes against the class’s priority rules, and `none` is neither answering. `rule_index` is the raw list position and `rule` renders it for humans — both identity, never ordering. | | `lookout_leeway_preference_transitions_total` | counter | `provider`, `axis`, `from_rank`, `to_rank`, `lateral` | Nodes moving between placements on one axis (leeway). Node-level, not pod-level: spike S1 established that a compute-class fallback provisions a NEW node and the ReplicaSet creates a new pod on it, so no pod ever changes rank. `lateral=true` is a move between two rules sharing a tier — capacity churn within a preference level, counted because it shows where the estate is thrashing, and excluded from every degradation signal because an equal-score alternative is not a demotion. A from\_rank of `unknown` is the node’s annotation arriving, which happens once in the first minute of every node’s life and is not a fallback. | | `lookout_leeway_preference_unmatched` | gauge | `provider`, `axis` | Nodes no priority rule matched. The alert to write: threshold zero. Every one of these is a node whose rank rests on GKE’s undocumented annotation with nothing checking it, and a sustained non-zero reading means the matcher has fallen behind the rules people are actually writing. Counted even when the annotation answered, because the coverage gap that matters is the one the working primary path is hiding. | | `lookout_leeway_preference_ambiguous` | gauge | `provider`, `axis` | Nodes more than one priority rule matched. Two rules admitting the same node is legal and usually harmless — GKE takes the first — but it means the inferred rank is a guess between them, so the cross-check on these nodes is worth less than it looks. | | `lookout_leeway_preference_disagreement` | gauge | `provider`, `axis` | Nodes where the annotation and the inferred rank disagree. The other threshold-zero alert, and the more serious of the two: the annotation wins, so a non-zero reading is not a wrong rank — it is evidence that the matcher’s model of the rules is wrong, and therefore that unmatched and ambiguous cannot be trusted either. | | `lookout_leeway_preference_out_of_range` | gauge | `provider`, `axis` | Nodes whose annotation names a priority index the class no longer has. This is a class that was edited under a running node: GKE stamped index 3 and somebody has since deleted a rule. The node keeps running; its rank is simply unknowable, and it reads as rank `unknown` until it is replaced. | | `lookout_leeway_preference_axis_invalid` | gauge | `provider`, `axis` | Nodes on a class whose priorities are partially scored. A class where some rules set priorityScore and others do not has no well-defined order — list position and score would rank it differently — so `k8s-lookout` declines to score it at all rather than pick one. Fix the class: score every rule or none. | | `lookout_leeway_preference_no_rule_matching` | gauge | `provider`, `axis` | Nodes GKE itself could not fit to any priority rule (ccc\_no\_rule\_matching). GKE’s own verdict, not ours, and a different statement from `unmatched`: this is the provisioner saying the node it made satisfies nothing the class asked for. Reads as rank `unsatisfiable`. | | `lookout_leeway_preference_off_axis` | gauge | `provider`, `axis` | Nodes GKE provisioned outside the priority list (ccc\_scale\_up\_anyway). The class’s whenUnsatisfiable let the autoscaler ignore the priorities rather than leave pods Pending, so these nodes are not at any rank — they are off the axis. Reads as rank `off-axis`, and their time is still reported under pod\_time, because a week spent off-axis is itself the finding. | | `lookout_leeway_preference_rank_pending` | gauge | `provider`, `axis` | Nodes carrying a compute class with no rank resolved yet. Normal and brief: GKE writes ccc\_priority\_index 33 to 44 seconds after a node registers, and inference has nothing to say about a node whose labels have not all landed either. A reading that does not decay to zero is the one to look at. | | `lookout_leeway_preference_unsupported_rules` | gauge | `provider`, `axis`, `field` | Priority rules the matcher could not evaluate, by the field responsible. The matcher fails closed: a rule naming a field pkg/leeway does not model is excluded from inference rather than matched on the fields it does understand, because a matcher that ignores what it cannot read attributes nodes to the wrong rule and corrupts the cross-check into agreement with nothing. A new field appearing here is a feature request with the field name already filled in. | | `lookout_leeway_preference_unreadable_class_nodes` | gauge | `provider`, `class` | Nodes labelled with a compute class this source has no decoded spec for. Either the class object has not synced, which is brief, or it failed to decode, which is not — see decode\_errors\_total. These nodes are ranked against nothing and their pods accrue no pod-seconds, so a sustained reading means the rank shares are being computed over less than the whole estate. | | `lookout_leeway_preference_decode_errors_total` | counter | `provider`, `class` | ComputeClass objects this source refused to read, by name. A refusal is deliberate: a spec whose priorities are not a list, or whose rules are not objects, would produce a plausible-looking axis that is wrong. The row survives the object’s deletion, because an error counter that vanishes with the broken object hides the edit that broke it. | | `lookout_leeway_preference_tracker_underflows_total` | counter | — | Unbalanced pod departures in the rank accounting. The alert to write: threshold zero. This counts a pod leaving a rank that had nobody at it, which is a BUG IN K8S-LOOKOUT and not a cluster condition — every rank share is skewed by an unknown amount until it is fixed. The count is clamped rather than allowed to run negative, so the pod-second counters stay monotonic; that keeps them readable, it does not make them right. | | `lookout_leeway_preference_axis_info` | gauge | `provider`, `axis`, `spec_hash`, `ordering`, `rules`, `tiers`, `scale_up`, `active_migration` | One preference axis, as labels on a constant 1. `ordering` is how the tiers were derived — `list-position` is bare list order, `priority-score` is the field of that name (higher is more preferred, the OPPOSITE direction to list position), and `invalid` is a partially-scored class. `rules` is how many priorities the class declares and `tiers` how many distinct preference levels they collapse to; `tiers` of 1 means every node on the class is rank 0 by construction and nothing here can degrade. `scale_up` is whenUnsatisfiable as DECLARED — `unset` is not the same evidence as GKE’s documented default, and gates a finding on the difference. | | `lookout_leeway_preference_alert_state` | gauge | `provider`, `axis`, `rank_rule`, `tier`, `phase` | Where one rule on one axis sits in the dwell machine: 1 pending, 2 firing. Episodes are per RULE, not per axis: a class can be running almost entirely on its last rank and also have a tier nobody has touched in a month, and resolving the first must not close the second. Only rules with an open episode are present — a healthy class has no rows rather than zeroes. A resolving episode still reads 2, because its finding is still outstanding. | | `lookout_leeway_preference_wedged_pods` | gauge | `provider`, `class` | Unscheduled pods that named a compute class by nodeSelector and did not get one. The only symptom a wedged class has: on a DoNotScaleUp class no priority can be satisfied and the autoscaler will not provision outside the list, so the pods stay Pending and nothing in the rank distribution moves at all. Non-zero on a scale-up-anyway class is a different story — those pods are waiting on something else, which is why the finding is gated on whenUnsatisfiable being DECLARED and this gauge is not. | | `lookout_otlp_exports_total` | counter | `outcome` | **Opt-in.** Total OTLP metric export attempts, by outcome (ok: the collector accepted the batch; failed: it did not, and the batch was dropped). A rising failed rate means the OTLP backend is stale; the scrape endpoint is unaffected. | | `lookout_otlp_points_exported_total` | counter | — | **Opt-in.** Total metric data points the OTLP exporter delivered to the collector. | | `lookout_otlp_points_dropped_total` | counter | — | **Opt-in.** Total metric data points discarded because their OTLP export failed or ran out of time. There is no retry queue by design, so a dead collector costs samples rather than memory; temporality is cumulative, so what is lost is the sample and not the counter value. | | `lookout_otlp_export_last_success_timestamp_seconds` | gauge | — | **Opt-in.** Unix timestamp of the last OTLP metric export the collector accepted; zero until the first one. Alert on its age rather than on the failure counter alone, which stays flat when the export path stops running at all. | | `lookout_otlp_export_inflight` | gauge | — | **Opt-in.** 1 while an OTLP metric export is in flight, 0 otherwise. A reading stuck at 1 across scrapes is an export wedged against its deadline. | # lookout net probe > Actively confirm a network hypothesis — resolve DNS names, open TCP connections, GET HTTP(S) URLs — from wherever lookout runs (in a pod = the in-cluster view); zero cluster mutation, no pods spawned. Actively confirm a network hypothesis — resolve DNS names, open TCP connections, GET HTTP(S) URLs — from wherever `lookout` runs (in a pod = the in-cluster view); zero cluster mutation, no pods spawned. MCP tool: `k8s_net_probe` ## Usage [Section titled “Usage”](#usage) ```sh lookout net probe [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | ----------------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | | `--dns` | string | — | comma-separated names to resolve (e.g. api.prod.svc.cluster.local,db.example.com) | | `--tcp` | string | — | comma-separated host:port endpoints to connect to (e.g. api.prod.svc:8080,10.0.0.5:5432) | | `--http` | string | — | comma-separated http(s) URLs to GET; redirects are reported (3xx), not followed, and response bodies are never read into findings | | `--probe-timeout` | duration | `5s` | per-probe timeout; raise —timeout too when probing many slow targets (it caps the whole invocation) | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------ | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `probe.dns` | critical, warning, info | the result of one DNS resolution: info when it resolved, warning on timeout, critical otherwise — a probe result is always emitted, success included, because the answer to “can this be reached” is the point of the command | | `probe.tcp` | critical, warning, info | the result of one TCP connect: info when it connected, warning on timeout, critical otherwise | | `probe.http` | critical, warning, info | the result of one HTTP GET (redirects reported, not followed): info on success, warning on timeout or 4xx, critical otherwise | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ---------------- | ------------------------------------------------------------------------------------------------ | | `ips` | probe.dns: resolved addresses, sorted, comma-separated | | `latency` | how long the probe took: DNS resolution / TCP connect / full HTTP exchange | | `status` | probe.http: HTTP status code of the (unfollowed) response | | `content_length` | probe.http: Content-Length the server declared (body is discarded unread; omitted when unknown) | | `error_class` | failed probes: nxdomain\|timeout\|refused\|unreachable\|reset\|cert\|http\_4xx\|http\_5xx\|error | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout net probe --dns=api.prod.svc.cluster.local lookout net probe --tcp=db.prod.svc:5432 --probe-timeout=2s lookout net probe --http=https://api.prod.svc/healthz --format=json lookout net probe --dns=api.prod.svc --tcp=api.prod.svc:8080 --http=http://api.prod.svc:8080/readyz ``` # lookout perf probe > Control-plane and startup performance via metrics query packs: --pack=apiserver (p99 latency by verb/resource), apf (queue saturation + 429 rejects), etcd (WAL fsync p99 + DB size), startup (pod-first-ready p95 trend); apf/etcd need GKE control-plane metrics enabled — absence degrades to an explicit pack_unavailable finding. Control-plane and startup performance via metrics query packs: —pack=apiserver (p99 latency by verb/resource), apf (queue saturation + 429 rejects), etcd (WAL fsync p99 + DB size), startup (pod-first-ready p95 trend); apf/etcd need GKE control-plane metrics enabled — absence degrades to an explicit pack\_unavailable finding. MCP tool: `k8s_perf_probe` ## Usage [Section titled “Usage”](#usage) ```sh lookout perf probe [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | -------- | ------ | ------- | ----------------------------------------------------------------- | | `--pack` | string | — | which query pack to run (required): apiserver\|apf\|etcd\|startup | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ----------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | `perf.apiserver_p99` | critical, warning | apiserver request latency p99 crossed the pack threshold for a verb/resource — warning from 1s, critical from 4s | | `perf.apf_saturation` | critical, warning | an API Priority and Fairness level is holding a sustained queue — warning from 10 queued, critical from 100 | | `perf.apf_rejects` | critical, warning | APF is shedding load: the apiserver is returning 429s at a priority level | | `perf.etcd_fsync` | critical, warning | etcd WAL fsync p99 crossed the pack threshold — warning from 10ms, critical from 100ms | | `perf.etcd_db_size` | critical, warning | the etcd database is approaching its quota — warning from 4 GiB, critical from 5.5 GiB | | `perf.startup_p95` | critical, warning | pod first-ready p95 crossed the pack threshold — warning from 60s, critical from 300s | | `perf.pack_unavailable` | warning | a metric the requested pack needs is not in the metrics workspace, so part of the pack could not run; the rest still did (no coverage lies) | | `cloud.unavailable` | info | the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ---------------- | ------------------------------------------------------------------------------------------------------- | | `pack` | the pack this finding belongs to; also the summary-line note naming the pack that ran | | `metric` | the backend-neutral metric the query measured (pack\_unavailable: the absent metric) | | `verb` | apiserver request verb for this series (apiserver pack) | | `resource` | apiserver request resource for this series (apiserver pack) | | `priority_level` | APF priority level for this series (apf pack) | | `code` | the HTTP status code the query matched (apf pack: 429) | | `observed` | the worst (maximum) aligned value in the window, in the query’s unit — the breach basis | | `latest` | the newest aligned value in the window | | `threshold` | the crossed threshold: the critical one when severity=critical, else the warning one | | `window` | the lookback the series cover (—since, or the pack default); also a summary-line note | | `trend` | startup pack: second-half vs first-half mean delta of the window, e.g. “+34%” — the p95 trend direction | | `capability` | cloud.unavailable: the provider capability this command needed (metrics) | | `provider` | cloud.unavailable: the provider that was asked | | `unavailable` | summary-line note: why the metrics backend could not be served | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout perf probe --pack=apiserver lookout perf probe --pack=apf lookout perf probe --pack=etcd --since=6h lookout perf probe --pack=startup lookout perf probe --pack=apiserver --format=json ``` # lookout scan > Start here when you know something is wrong but not what: one call runs every target-free incident check across the cluster — broken workloads, dead admission webhooks, stuck volumes and PVCs, rejected Gateway routes, config drift — then drills into the dependency edges of whatever it flagged. Needs no target; `--include=audit` adds the posture sweep. Start here when you know something is wrong but not what: one call runs every target-free incident check across the cluster — broken workloads, dead admission webhooks, stuck volumes and PVCs, rejected Gateway routes, config drift — then drills into the dependency edges of whatever it flagged. Needs no target; `--include=audit` adds the posture sweep. MCP tool: `k8s_scan` (MCP profile: `triage`, `audit`) ## Usage [Section titled “Usage”](#usage) ```sh lookout scan [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | ----------------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--include` | string | — | additionally run these opt-in groups: audit,cloud,perf. Comma-separated, ‘all’ for every one, ’-’ to subtract (all,-cloud). Left out by default because they answer a different question (audit = posture, no incident) or need a provider build (cloud, perf) | | `--max-drilldown` | int | `20` | cap the stage-2 dependency-edge drill-down at this many workloads, worst severity first (0 disables it); the number dropped is reported as truncated= in the summary | | `--cert-warn` | duration | `720h` | report TLS certificates expiring within this window (drill-down stage; same meaning as `state edges --cert-warn`) | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `1m0s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ----------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `scan.check_skipped` | info | a stage declined this invocation because a zero-argument scan cannot supply something it needs — the coverage claim is smaller than it looks | | `scan.check_failed` | warning | a stage errored; the scan continued without it, so this run saw less than a whole cluster — unless EVERY stage failed and none read anything, which is a runtime error (exit 1) rather than a scan | | `scan.incomplete` | warning | the —timeout expired with stages still to run; not\_run names them | | `pod.crashloop` | critical | a container is crash looping | | `pod.imagepull` | critical | a container cannot pull its image | | `pod.waiting` | warning | a container is stuck in an error waiting state (CreateContainerConfigError, InvalidImageName, …) | | `pod.oomkilled` | warning | a container’s last termination was an OOM kill | | `pod.restarts` | warning | a container has restarted at least —restarts times | | `pod.notready` | warning | a container in a Running pod has been not-ready past the —pending-age grace | | `pod.failed` | warning | the pod reached phase Failed | | `pod.pending` | critical, warning | the pod has been Pending longer than —pending-age with no container-level diagnosis; critical when the scheduler has declared it Unschedulable, which is a capacity or constraint problem rather than latency | | `workload.replicafailure` | critical | the controller cannot create pods at all (quota, PodSecurity, admission) — no pod exists to diagnose | | `workload.stalled` | critical | a Deployment’s Progressing condition is False: the rollout has given up | | `workload.rollout` | critical, warning | replicas are short of desired; critical when nothing is serving at all | | `job.failed` | warning | a Job’s Failed condition is set | | `cron.missed` | critical, warning | an unsuspended CronJob’s schedule said to run more than —cron-grace ago and status says it did not; critical once several activations in a row are gone | | `cron.unparseable` | warning | a CronJob’s spec.schedule could not be parsed, so its activations cannot be judged at all | | `node.notready` | critical | the node’s Ready condition is not True | | `node.pressure` | critical | the node reports Memory/Disk/PID pressure | | `node.condition` | critical, warning | a non-standard node condition is True — NPD and its cousins publish problems that way | | `node.cordoned` | warning | the node is unschedulable but still holds pods: a stuck drain or a forgotten maintenance step | | `node.preempt` | critical, warning, info | a reclaim taint marks the node for termination; severity tracks how imminent | | `pdb.gridlocked` | critical, warning | the budget permits no disruptions; critical when healthy pods are already below the required minimum | | `addon.degraded` | critical, warning | a kube-system add-on (dns, proxy, cni, csi, metrics, connectivity) is short of replicas; critical when none are available | | `quota.near` | warning | a ResourceQuota resource is at or past —quota-warn percent of its hard limit | | `quota.exhausted` | critical | a ResourceQuota resource is at its hard limit: the next create is rejected | | `webhook.failing_closed` | critical | the webhook has no working backend and failurePolicy=Fail: every gated write is rejected cluster-wide | | `webhook.dead_backend` | warning | the webhook’s service backend is missing, has no ready endpoints, or does not serve the named port | | `webhook.slow_risk` | info | the webhook’s timeout is long enough to slow every gated write if the backend degrades | | `webhook.ca_expired` | critical | the webhook’s caBundle has expired: the API server cannot verify it | | `webhook.ca_expiring` | warning | the webhook’s caBundle expires within —cert-warn | | `volume.multi_attach` | critical | an RWO claim is wanted by pods on more than one node — the second pod never starts | | `volume.zone_conflict` | critical | the PV is locked to a zone the pod’s node is not in | | `volume.attach_error` | critical, warning | the attach or detach is failing; critical once it has been failing long enough to be stuck rather than slow | | `volume.orphaned_attachment` | info | a VolumeAttachment survives its PV or its node | | `storage.missing_class` | critical | the claim names a StorageClass that does not exist — it will stay Pending forever | | `storage.no_default_class` | critical | the claim names no class and the cluster has no default StorageClass | | `storage.no_provisioner` | warning | the claim’s class is static-only (kubernetes.io/no-provisioner) and no matching PV is available | | `storage.multiple_defaults` | warning | more than one StorageClass is annotated as the cluster default; which one wins is not defined | | `storage.pv_failed` | warning | a PersistentVolume is Failed: its reclaim did not complete, so the backing disk stays allocated and the volume cannot be reused | | `storage.pv_released` | info | a PersistentVolume is Released — retained on purpose, but its capacity is unusable until spec.claimRef is cleared | | `gateway.missing_class` | critical | the Gateway names a GatewayClass that does not exist — nothing will program it | | `gateway.class_not_accepted` | critical | the Gateway’s GatewayClass is not Accepted by its controller | | `gateway.not_accepted` | critical | the Gateway itself is not Accepted | | `gateway.not_programmed` | critical | the Gateway is Accepted but not Programmed: no data plane is carrying its traffic | | `gateway.listener_invalid` | warning | one listener of an otherwise working Gateway is not resolved or not programmed | | `route.missing_parent` | critical | the route’s parentRef names a Gateway that does not exist | | `route.not_accepted` | critical | the Gateway refused the route’s attachment (listener, hostname, or namespace policy) | | `route.missing_backend` | critical | the route’s backendRef Service does not exist | | `route.backend_port` | critical | the route’s backendRef Service exists but does not expose the named port | | `crd.unavailable` | info | the API group this check reads is not served by the cluster, so nothing was examined (no coverage lies) | | `wi.gsa_missing` | critical | the annotated Google service account does not exist — every GCP call from these pods fails | | `wi.unbound` | critical | the KSA annotates a GSA but the roles/iam.workloadIdentityUser binding is missing or malformed | | `wi.unannotated_use` | info | a pod sets GOOGLE\_APPLICATION\_CREDENTIALS but its ServiceAccount carries no Workload Identity annotation | | `cloud.unavailable` | info | the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence | | `drift.manual_edit` | critical, warning | a manager other than the GitOps controller owns spec fields on this object; critical when one of them is high blast radius (image, replicas, env) | | `audit.workload_identity_off` | warning | Workload Identity is off cluster-wide, or a node pool bypasses it — pods authenticate to the cloud as the node | | `audit.legacy_metadata` | warning | a node pool still serves the pre-v1 instance-metadata endpoints, which any pod can read | | `audit.public_control_plane` | warning, info | the control-plane endpoint is reachable from the internet; info when authorized networks narrow it | | `audit.exemption_expired` | warning | an exemption entry has lapsed: the findings it used to annotate are being reported unqualified again | | `audit.exemption_expiring` | info | an exemption entry lapses within —within — renew it or let it go deliberately | | `audit.privileged_container` | warning | a container runs privileged or holds a node-root capability (ALL, SYS\_ADMIN): a container escape is a node compromise | | `audit.host_namespace` | warning | the pod shares the node’s network, PID, or IPC namespace | | `audit.hostpath_mount` | warning, info | the pod mounts a host path; warning when it is writable, info when read-only | | `audit.default_sa_automount` | warning | the pod runs as the namespace’s default ServiceAccount with its token automounted, and something in the pod can use it | | `audit.podsecurity_gaps` | warning | the namespace enforces no Pod Security Admission level, so none of the above is prevented | | `audit.netpol_missing` | warning, info | nothing restricts this direction for the subject — a namespace with no policy at all, or a workload the covering policies’ selectors miss; info for the egress direction, where no policy is a defensible default | | `audit.version_behind` | warning, info | the control plane or a node pool is behind what the provider publishes, or a node pool has skewed from the control plane; info while the gap is still within the supported skew | | `audit.upgrade_unmanaged` | warning | nothing will close that gap on its own: no release channel, or node auto-upgrade/auto-repair off | | `audit.upgrade_blocked` | warning, info | an active maintenance exclusion, or a node image on the removed Docker runtime, will stop the upgrade when it comes | | `audit.upgrade_unattended` | info | upgrades will happen with nobody watching: no maintenance window, or no upgrade notifications | | `audit.no_pdb` | warning | the workload has no PodDisruptionBudget: a drain can take every replica at once | | `audit.single_replica` | warning | the workload runs a single replica, so any disruption is an outage | | `audit.no_readiness_probe` | warning | a container has no readiness probe, so traffic reaches it before it can serve | | `audit.no_liveness_probe` | info | a container has no liveness probe, so a wedged process is never restarted | | `audit.no_spread` | info | the workload’s replicas are not spread across nodes or zones | | `audit.rigid_scheduling` | warning, info | placement constraints pin the workload to too few nodes to survive losing one | | `audit.hpa_cannot_scale` | warning | the autoscaler structurally cannot scale: min equals max, the target is missing, or a container has no request for its utilization target to divide by | | `audit.suspended_cronjob` | warning | a CronJob has been suspended past —cron-suspended and has skipped activations because of it: whatever it does is not happening, and nothing else reports that | | `ipspace.range` | critical, warning, info | a pod/service/node range is at 80% of its CIDR or worse; critical from 95%, info for a range the cloud APIs cannot rate and for an —all row below the line | | `orphan.disk` | warning | a GCE disk has been unattached for at least —min-age and is still billing | | `orphan.lb` | warning | a forwarding rule or load balancer routes to zero endpoints and is still billing | | `quota.pressure` | critical, warning, info | a cloud quota is at or above —quota-warn percent of its limit; critical from 95%, info for an —all row below the line | | `stockout.zone` | warning | the cloud had no capacity for a machine type in this zone during the window — the reason a scale-up failed and pods stayed Pending | | `perf.apiserver_p99` | critical, warning | apiserver request latency p99 crossed the pack threshold for a verb/resource — warning from 1s, critical from 4s | | `perf.apf_saturation` | critical, warning | an API Priority and Fairness level is holding a sustained queue — warning from 10 queued, critical from 100 | | `perf.apf_rejects` | critical, warning | APF is shedding load: the apiserver is returning 429s at a priority level | | `perf.etcd_fsync` | critical, warning | etcd WAL fsync p99 crossed the pack threshold — warning from 10ms, critical from 100ms | | `perf.etcd_db_size` | critical, warning | the etcd database is approaching its quota — warning from 4 GiB, critical from 5.5 GiB | | `perf.startup_p95` | critical, warning | pod first-ready p95 crossed the pack threshold — warning from 60s, critical from 300s | | `perf.pack_unavailable` | warning | a metric the requested pack needs is not in the metrics workspace, so part of the pack could not run; the rest still did (no coverage lies) | | `edge.missing_ref` | critical | a referenced ConfigMap, Secret, ServiceAccount, TLS secret, IngressClass, StorageClass, or governing Service does not exist | | `edge.missing_key` | critical | the referenced key is absent from an existing ConfigMap/Secret | | `edge.invalid_ref` | warning | the referenced object exists but is the wrong type to serve the reference | | `edge.unclassed` | warning | the Ingress names no class and no IngressClass declares itself the cluster default — no controller will claim it | | `edge.selector_empty` | critical | a Service selector selects zero pods, so the service routes nowhere | | `edge.selector_unready` | critical, warning | the Service selects pods but some are not Ready; critical when none are | | `edge.endpoints_missing` | critical | a selecting Service has no EndpointSlices at all | | `edge.endpoints_orphaned` | warning | an endpoint targetRef names a pod that no longer exists | | `edge.endpoints_unready` | critical, warning | the endpoint ready-count disagrees with the selected pods (stale or lagging slices); critical at zero ready | | `edge.backend_missing` | critical | an Ingress backend service, or the port it names, does not exist | | `edge.cert_expired` | critical | a TLS certificate’s NotAfter is in the past | | `edge.cert_expiring` | warning | a TLS certificate expires within —cert-warn | | `edge.cert_invalid` | warning | tls.crt is missing or unparseable, or the secret is not kubernetes.io/tls | | `edge.rbac_dangling` | warning | a (Cluster)RoleBinding for the workload’s ServiceAccount points at a missing (Cluster)Role | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `check` | which registered command produced this finding — also the command to run for the full detail behind it | | `not_run` | on scan.incomplete: the checks the timeout left unrun, comma-separated | | `checks` | summary-line note: how many checks this scan actually ran | | `skipped` | summary-line note: opt-in groups this scan did NOT run (switch one on with —include=\) — stated so a quiet scan is never mistaken for a complete one | | `drilldown` | summary-line note: workloads the stage-2 dependency-edge drill-down covered | | `truncated` | summary-line note: drill-down candidates dropped by —max-drilldown | | `container` | container the finding is about (init containers prefixed init:) | | `image` | image reference that failed to pull | | `restarts` | container restart count | | `exit_code` | exit code of the container’s last termination | | `last_state` | reason of the container’s last termination (e.g. OOMKilled) | | `age` | how long the abnormal state has persisted | | `desired` | desired replica/scheduled count from spec | | `ready` | ready count from status | | `updated` | updated-to-current-revision count from status | | `available` | available count from status | | `failed` | failed pod count of a Job | | `schedule` | a CronJob’s spec.schedule | | `expected` | the activation a CronJob should have run and did not | | `missed_runs` | activations missed since the anchor; ≥N when the walk was capped | | `anchor` | what the missed count was measured from: last\_schedule or creation | | `time_zone` | a CronJob’s spec.timeZone, when set | | `last_schedule` | a CronJob’s status.lastScheduleTime, or never | | `active_jobs` | Jobs a CronJob still has running | | `condition` | node condition type that is abnormal | | `taint` | taint key indicating reclaim/drain | | `pods` | pods affected (behind a cordoned node or a PDB) | | `healthy` | currently healthy pods behind a PDB | | `required` | pods the PDB requires healthy | | `addon` | system add-on role: dns, proxy, cni, csi, metrics, connectivity | | `resource` | ResourceQuota resource name at or near its limit | | `used` | quota usage from status | | `hard` | quota hard limit from status | | `pct` | quota usage as percent of the hard limit | | `webhook` | admission webhook as \/\ | | `service` | service backend the webhook points at, as \/\ | | `backend` | why the backend is dead: service missing, no ready endpoints, or port \

not on service | | `gates` | namespaces the webhook gates, from namespaceSelector: all namespaces, or \/\ namespaces with up to 5 names | | `rules` | compact operations/resources summary of the webhook’s rules, e.g. “CREATE,UPDATE pods,deployments.apps” | | `object_selector` | the webhook’s objectSelector, when one is set | | `timeout` | webhook timeoutSeconds as \s (nil defaults to the API’s 10s) | | `subject` | CA-bundle certificate subject (CN when set); never key material | | `not_after` | CA-bundle certificate NotAfter, RFC 3339 | | `days_left` | whole days until NotAfter (negative = expired) | | `nodes` | distinct nodes those pods are scheduled on, sorted | | `access_modes` | the claim’s declared access modes | | `pv` | PersistentVolume behind the claim or attachment | | `pvc` | PersistentVolumeClaim the pod mounts (same namespace as the pod) | | `node` | node the attachment targets or the pod is scheduled on | | `attacher` | CSI driver responsible for the attachment (spec.attacher) | | `error` | the attach/detach error message, truncated to 200 chars | | `attached` | the attachment’s status.attached at scan time | | `pv_zones` | zones the PV’s node affinity allows, sorted | | `node_zone` | zone label of the node the pod is scheduled on | | `orphan` | which referenced side is gone: “pv missing”, “node missing”, or both | | `storage_class` | StorageClass the claim names, or the class the finding is about | | `classes` | StorageClasses the cluster does have, sorted (empty when there are none) | | `defaults` | StorageClasses annotated as the cluster default, sorted | | `provisioner` | the class’s spec.provisioner | | `phase` | the claim’s or volume’s status.phase at scan time | | `requested` | storage the claim requests (spec.resources.requests.storage) | | `capacity` | the volume’s spec.capacity.storage | | `reclaim_policy` | the volume’s spec.persistentVolumeReclaimPolicy | | `claim` | the claim the volume was bound to, as namespace/name | | `binding_mode` | the class’s volumeBindingMode (Immediate when unset) | | `gateway_class` | GatewayClass the Gateway names | | `controller` | the GatewayClass’s spec.controllerName — which implementation owns it | | `gateway` | Gateway the route attaches to, as namespace/name | | `listener` | listener name within the Gateway | | `port` | listener port, or the backendRef port the Service does not expose | | `protocol` | listener protocol | | `service_ports` | ports the backend Service does expose, sorted | | `api_group` | crd.unavailable: the API group-version this command needed | | `resources` | crd.unavailable: the resources it would have read | | `unavailable` | summary-line note: why the group could not be read (absent CRDs, or discovery denied) | | `gsa` | the cloud identity (GSA email) the ServiceAccount’s annotation claims | | `problem` | machine-matchable problem code from the provider (e.g. no-workload-identity-binding) | | `env` | the credential-file env var found (GOOGLE\_APPLICATION\_CREDENTIALS) | | `capability` | cloud.unavailable: the provider capability this command needed (workload-identity) | | `provider` | cloud.unavailable: the provider that was asked | | `manager` | on findings: the foreign manager string from managedFields (a tool name like kubectl-edit — never a user identity; see —identity); on the summary line: the resolved GitOps manager | | `detection` | summary note: how the GitOps manager was resolved — declared (—manager), majority (auto-detected recognized GitOps controller owning >50% of the spec leaf fields in scope), or none (no manager resolved; nothing emitted) | | `detection_reason` | summary note on detection=none, naming why: no-spec-fields-in-scope (nothing in scope owns a spec field), no-majority-manager (a leading candidate exists but owns 50% or less), or not-a-gitops-manager (the majority owner is not a recognized GitOps controller — e.g. kubeadm or a kubectl manager on a cluster with no GitOps at all) | | `candidate` | summary note on detection=none: the leading manager that fell short (of the majority, or of being a recognized GitOps controller) — pass it to —manager if it is in fact the GitOps controller | | `share` | summary note: the resolved manager’s (or, on detection=none, the candidate’s) percentage of every spec leaf field owned across the scanned objects, rounded. A declared manager with a low share means most findings are other legitimate owners | | `unmanaged` | summary note, omitted at zero: scanned objects the resolved GitOps manager owns no spec field on. Nothing is reported for them — an object the manager never applied cannot have drifted from it — so a high count next to zero findings means the manager’s scope is narrower than the scan’s | | `operation` | managedFields operation of the foreign manager’s last write: Apply or Update | | `tool` | client tool recognized from the manager string (kubectl for kubectl-edit/kubectl-patch/kubectl-\*) | | `fields` | compact spec paths the foreign manager owns (e.g. spec.template.spec.containers\[app].image), capped at 8 with a +N more tail | | `field_count` | total spec leaf fields the foreign manager owns on this object (uncapped) | | `principal` | —identity: the audited principal of the write nearest the drift time (GKE: principalEmail), or the explicit sentinel none-in-audit-window / no-write-time-anchor when the trail cannot answer | | `principal_agent` | —identity: the caller-supplied client string of that write (a kubectl or controller user-agent), when the trail records one; caller-controlled text, display-only | | `other_principals` | —identity: other distinct principals that wrote the object inside the audit window, capped at 8 with a +N more tail | | `identity` | summary note when —identity could not be served: the unavailable marker naming why (no provider / audit capability absent) | | `cluster` | on a node-pool finding: the cluster the pool belongs to, so the record stands alone | | `workload_pool` | the cluster-wide workload identity pool that this node pool’s pods bypass | | `metadata_mode` | how the node pool exposes instance metadata to pods: node-identity means any pod can mint tokens for the node’s service account | | `disable_legacy_endpoints` | the pool’s legacy-metadata setting as the provider records it: `enabled` when someone turned the pre-v1 endpoints back on, `unset` when the pool was never configured either way | | `node_pools` | summary note: node pools examined — the cluster itself is the other unit `scanned` counts | | `endpoint` | the control plane’s internet-facing address | | `authorized_networks` | how many source ranges the allow-list permits | | `authorized_network_cidrs` | those ranges, sorted as the provider returned them and capped at 8 with a +N more tail | | `gcp_public_cidrs` | whether the provider’s own public ranges are admitted in addition to the allow-list | | `exempt_kind` | the finding kind the entry covers — this is the entry’s `kind:` field, not this finding’s own kind | | `expires` | when the entry stops applying, RFC 3339 (a bare `YYYY-MM-DD` in the file resolves to 00:00:00Z that day) | | `expired_for` | how long ago the entry lapsed, rounded to whole days — only on audit.exemption\_expired | | `expires_in` | how long until the entry lapses, rounded to whole days — only on audit.exemption\_expiring | | `owner` | the entry’s `owner:` field, absent if it has none — which is itself worth fixing, since “expired, and nobody knows whose it was” is where these files end up | | `justification` | the entry’s `reason:` field: why the exempted finding was accepted. Distinct from the envelope’s exempt\_reason, which is the justification for THIS finding being exempt | | `containers` | containers implicated by the finding — those running privileged, or holding a node-root capability | | `container_names` | their names, capped at 8 with a +N more tail | | `total_containers` | containers in the pod template, init containers included, so `containers` reads as a fraction | | `capabilities` | the node-root capabilities added by those containers (ALL, SYS\_ADMIN), sorted and deduplicated | | `host_paths` | hostPath volumes the template mounts; a declared but unmounted hostPath volume grants no access and is not counted | | `host_path_names` | the paths on the node, sorted and capped at 8 | | `service_account` | the ServiceAccount the finding is about — always `default`, the one every pod gets when its template names none | | `mounting_workloads` | workloads in the namespace running as the default ServiceAccount without disabling automount at the pod level; the finding does not fire at 0 | | `mounting_workload_names` | their Kind/name, sorted and capped at 8 | | `pss_enforce` | the namespace’s pod-security.kubernetes.io/enforce label, omitted when unset | | `pss_warn` | its /warn label, omitted when unset — set without /enforce means the namespace is in dry-run | | `pss_audit` | its /audit label, omitted when unset — same dry-run meaning | | `workloads` | pod templates this pass judged in the namespace, so an unenforced namespace with nothing in it reads differently from a busy one | | `namespaces` | summary note: namespaces examined — the denominator for every namespace-subject claim, which `scanned` (pod templates) does not cover | | `policies` | NetworkPolicies in the namespace naming this direction in policyTypes; 0 on a namespace-subject finding, and the number that failed to select the subject on a workload one | | `total_policies` | NetworkPolicies in the namespace in either direction, so an egress-only namespace does not read as an empty one | | `host_network_workloads` | pod templates excluded because they use the node’s network namespace, where NetworkPolicy does not apply; omitted at 0 | | `covered_workloads` | pod templates in the namespace that ARE selected for this direction — the neighbours the subject fell out of step with | | `pod_labels` | the template’s own labels, which are what the policies’ selectors failed to match, sorted and capped at 8 | | `version` | the current version of the finding’s subject — the control plane’s, or the node pool’s | | `target_version` | the version the provider would move this cluster to: its channel’s upgrade target where one is published, otherwise the channel’s default | | `control_plane_version` | on a node-pool skew finding: the control-plane version the pool is measured against | | `minor_versions_behind` | how many minor releases separate the two versions | | `channel` | the release channel the cluster is subscribed to, and the one whose published versions the comparison used; `none` when it is subscribed to no channel | | `image_type` | the provider’s name for the node image the pool runs | | `exclusion` | the operator’s name for the maintenance exclusion currently in force | | `scope` | how much of the upgrade stream that exclusion holds back: all-upgrades, minor-upgrades or minor-and-node-upgrades | | `ends` | when the exclusion lifts, or `end-of-support` for one that runs until the cluster’s version leaves support | | `days_remaining` | how much longer the exclusion has left to run | | `replicas` | the workload’s spec.replicas (nil defaults to 1, matching the API server); absent on DaemonSets, whose replica count is the node count | | `namespace_pdbs` | PodDisruptionBudgets in the workload’s namespace — 0 says the namespace has no PDB culture at all, a non-zero value says this workload was missed | | `min_replicas` | the HPA’s spec.minReplicas (nil defaults to 1, matching the API server) | | `max_replicas` | the HPA’s spec.maxReplicas | | `metric` | the utilization metric the HPA cannot compute, comma-separated if more than one | | `scale_target` | the HPA’s scaleTargetRef as Kind/name | | `eligible_nodes` | nodes satisfying the workload’s REQUIRED placement constraint; an upper bound, since taints and cordons are not subtracted | | `cluster_nodes` | nodes in the cluster, so `eligible_nodes` reads as a fraction | | `constraint` | the label and field keys that narrow placement, sorted and capped at 8 | | `suspended_for` | how long spec.suspend has been true, rounded to whole days | | `suspended_since` | when the suspension is estimated to have started, RFC 3339 | | `pdbs` | summary note: PodDisruptionBudgets seen in scope | | `hpas` | summary note: HorizontalPodAutoscalers seen in scope | | `cidr` | the range’s CIDR block | | `purpose` | what the range allocates: pods, services, or nodes | | `zone` | orphan.disk: the disk’s zone | | `size_gb` | orphan.disk: provisioned size in GB (billed whether used or not) | | `disk_type` | orphan.disk: disk type short name (pd-ssd bills \~4x pd-standard idle) | | `unused_since` | orphan.disk: last detach (or creation, if never attached), RFC3339; omitted when the provider cannot date it | | `unused_for` | orphan.disk: how long the disk has been unattached; “unknown” when undatable | | `region` | orphan.lb: the forwarding rule’s region (“global” for global rules) | | `why` | orphan.lb: the provider’s orphan judgment (e.g. which backend resolved empty) | | `usage` | current usage in the quota’s own unit | | `limit` | the quota limit | | `unit` | the quota’s unit, when the provider names one | | `machine_type` | the exhausted machine type (omitted when the log record does not name one) | | `events` | stockout events for this zone/machine-type pair in the window | | `first_seen` | earliest event in the window (RFC3339) | | `last_seen` | latest event in the window (RFC3339) | | `reroute` | same-region zones active in the window with no stockout for this machine type, comma-separated; omitted when the window offers no clean candidate | | `window` | summary-line note: the lookback the events cover | | `pack` | the pack this finding belongs to; also the summary-line note naming the pack that ran | | `verb` | apiserver request verb for this series (apiserver pack) | | `priority_level` | APF priority level for this series (apf pack) | | `code` | the HTTP status code the query matched (apf pack: 429) | | `observed` | the worst (maximum) aligned value in the window, in the query’s unit — the breach basis | | `latest` | the newest aligned value in the window | | `threshold` | the crossed threshold: the critical one when severity=critical, else the warning one | | `trend` | startup pack: second-half vs first-half mean delta of the window, e.g. “+34%” — the p95 trend direction | | `workload` | the target the edges were traced from as \/\/\, stamped on every finding — a workload, or the Service itself when entered from the service side | | `likely_workload` | on a Service-entry edge.selector\_empty: the workload in that namespace whose pod labels best fit the broken selector, i.e. the one it was probably meant to select. Absent when two workloads fit equally well, because then naming one would be a guess | | `volume` | pod volume, or StatefulSet volumeClaimTemplate, whose reference is broken | | `key` | the referenced key that is missing from the ConfigMap/Secret | | `selector` | the Service label selector under scrutiny | | `selected` | pods the Service selector currently selects | | `endpoints` | total endpoints across the Service’s EndpointSlices | | `slices` | how many EndpointSlices back the Service | | `pod` | pod named by an orphaned endpoint targetRef | | `via` | how the broken reference is reached from the workload: mount, ingress, or imagePullSecret | | `ingress` | Ingress referencing the TLS secret, or the unserved Ingress itself | | `host` | Ingress rule host of the broken backend (empty for the default backend) | | `path` | Ingress rule path of the broken backend | | `role_ref` | dangling roleRef as \/\ | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout scan lookout scan --namespace=prod lookout scan --include=audit --format=json lookout scan --max-drilldown=0 ``` # Signal kinds > The signal-schema v1 kind catalog — every kind the sentinel can put on the wire, from the frozen ledger. The signal-schema v1 kind inventory (57 kinds), rendered from the SAME exported ledger (`pkg/inject/schema`) the freeze tests in `pkg/inject` pin (docs/signal-schema-v1.md). The schema is FROZEN: removing or renaming a kind or a frozen field is a v2 negotiation with fleet consumers, never a routine change; additions are v1-additive and extend the ledger, the field pins, and the schema doc in the same change. Every payload carries a stable incident-class `fingerprint` plus `cluster`/`project`/`zone` join dimensions — fleet rollup is a join, not a parsing project. The one exception is the frozen reactive pair (`k8s-event`, `k8s-event-followup`), whose wire shape stays byte-identical for playbook back-compat and never gains the identity fields. ## Cross-cutting kinds [Section titled “Cross-cutting kinds”](#cross-cutting-kinds) Emitted by the dispatcher itself (outcome records, storms, watchboard, triage evidence) plus the frozen reactive pair. | Kind | Wire struct | Role | | ------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `k8s-event` | `Payload` | Frozen reactive kind: the opening inject of a per-incident session; its wire shape is byte-identical for playbook back-compat. | | `k8s-event-followup` | `Payload` | Frozen reactive kind: a dedup-window recurrence injected into the already-open incident session. | | `resolved` | `ResolvedPayload` | Outcome record: the symptom stayed clear for —recovery-stable-for; carries resolution=recovered\|object\_deleted. | | `resolved.reverted` | `ResolvedPayload` | Outcome record: the symptom recurred within the revert window after a resolve. | | `storm` | `StormPayload` | Aggregate incident: opened when —storm-min incidents share a blast-radius key within —storm-window. | | `storm.member` | `StormMemberPayload` | Membership record injected into the storm session for each folded incident. | | `storm.member_superseded` | `StormMemberPayload` | Supersede pointer left in a pre-storm incident session that the storm absorbed. | | `storm.update` | `StormUpdatePayload` | Storm size refresh (latest wins): membership grew past a reporting threshold. | | `watchboard.digest` | `WatchboardDigestPayload` | Warning-class batch flushed to the shared watchboard session (—watchboard-batch / —watchboard-flush). | | `watchboard.rotated` | `WatchboardRotatedPayload` | Size-based rotation pointer naming the successor watchboard session after —watchboard-rotate digests. | | `triage.regressed` | `TriageRegressedPayload` | Regression evidence: a downgraded incident’s recurrence count reached —triage-regress-factor times its count at downgrade — evidence only, never a re-page. | | `family.member` | `FamilyMemberPayload` | Cross-source join: a signal from a different source family attached to this session’s incident (leading↔reactive) — at most one per source family per incident per dedup window; storm members never fan these out. | | `sentinel.access_revoked` | `Payload` | Coverage loss: a permission the sentinel held at startup is denied now, confirmed over consecutive SSAR sweeps — from here that source sees nothing, and its silence means less than it did. | ## Source-namespaced kinds [Section titled “Source-namespaced kinds”](#source-namespaced-kinds) All ride `inject.Payload` with the full identity stamped (`source`, `severity`, `fingerprint`, `project`, `zone`). The source column is the `--sources` name that emits the kind. | Kind | Source | Role | | -------------------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `objectstate.node_notready` | `object-state` | A Node’s Ready condition transitioned True→False/Unknown — workloads on that node are next. | | `objectstate.node_flapping` | `object-state` | A Node’s Ready condition flapped repeatedly within the flap window. | | `objectstate.progress_deadline` | `object-state` | A Deployment rollout made no progress with unready replicas — fired BEFORE the control plane’s ProgressDeadlineExceeded event. | | `objectstate.endpoints_empty` | `object-state` | A Service’s ready-endpoint count transitioned >0 → 0. | | `objectstate.pdb_gridlocked` | `object-state` | A PodDisruptionBudget’s disruptionsAllowed transitioned >0 → 0 while pods behind it exist — drains will stall. | | `objectstate.restart_burst` | `object-state` | A pod’s summed container restart count grew past the burst threshold — the leading edge of a crash loop, ahead of BackOff events. | | `objectstate.node_pressure` | `object-state` | A Node’s kubelet pressure condition (MemoryPressure/DiskPressure/PIDPressure) went False→True; escalates to critical when sustained or paired with eviction activity on the node. | | `objectstate.eviction_burst` | `object-state` | N pod evictions on one node within the burst window, folded into ONE node-scoped signal — the storm-off fallback for the per-pod Evicted event family. | | `rollout.stall` | `rollout` | A new revision made zero ready-count progress for —rollout-observe while the old revision stayed healthy. | | `workload.job_failed` | `workload` | A Job’s Failed condition went True (BackoffLimitExceeded, DeadlineExceeded, …) — batch failure with no crashlooping pod behind it. | | `workload.cron_missed` | `workload` | An unsuspended CronJob passed a scheduled activation without lastScheduleTime advancing; consecutive misses escalate to critical. | | `autoscaling.hpa_pinned` | `autoscaling` | An HPA sat at maxReplicas with its metric still over target past the sustain window — the autoscaler is out of headroom (escalates to critical when sustained longer). | | `autoscaling.hpa_metrics_dead` | `autoscaling` | An HPA’s metrics pipeline is broken (ScalingActive=False with a FailedGet\* reason, sustained) — autoscaling is silently dead. | | `saturation.forecast` | `saturation` | A linear-regression forecast says a resource dimension exhausts within —saturation-warn (critical below 15m). | | `degradation.capacity` | `degradation` | A Service’s ready-endpoint ratio declined stepwise across —degradation-window — capacity eroding before the outage. | | `degradation.probe_flap` | `degradation` | A pod’s readiness gate flipped repeatedly without ever sustaining failure long enough for the reactive Unhealthy path. | | `expiry.warning` | `expiry` | An expiry countdown (certificate/token) crossed a threshold: warning at —expiry-warn, critical at the design-fixed 72h. | | `capacity.pending` | `capacity` | A NotTriggerScaleUp event: the autoscaler declined a pending pod, with per-nodegroup rejection reasons. | | `capacity.scaleup` | `capacity` | A TriggeredScaleUp event: the autoscaler asked the cloud for nodes (info; stored context for later gaps). | | `capacity.scaledown` | `capacity` | The ScaleDown event family (info; warning for ScaleDownFailed). | | `capacity.scaleup_gap` | `capacity` | A nodegroup’s cloudProviderTarget exceeded its ready count beyond the sustain window — asked for a node, didn’t get one. | | `capacity.stockout` | `capacity` | A provider scale decision names a stockout: the zone/machine-type has no capacity. Remedy-disjoint from quota. | | `capacity.quota_blocked` | `capacity` | A provider scale decision names quota exhaustion: file a quota increase. Remedy-disjoint from stockout. | | `capacity.ip_exhausted` | `capacity` | A provider scale decision names IP exhaustion: new nodes/pods cannot get addresses. | | `capacity.pending-aged` | `capacity` | A pod stayed Pending+Unschedulable past —pending-age (critical past the design-fixed 15m). | | `capacity.cluster_forecast` | `capacity` | A scheduling domain’s pod-requests/node-allocatable ratio is on a linear trend to reach 1.0 — cluster full in \~N hours, before the first pod goes Pending. | | `ingress.sync_failed` | `ingress` | An ingress-gce Warning Sync event on an Ingress: GCLB programming is failing while the Ingress object looks fine. | | `ingress.translate_failed` | `ingress` | An ingress-gce Warning Translate event on an Ingress: the spec could not be translated into GCLB resources. | | `ingress.neg_failed` | `ingress` | A NEG-controller failure on a Service (sync/attach/detach/retry): endpoints are not reaching the load balancer. | | `gateway.programming_failed` | `gateway` | A Gateway (top-level or listener) held Programmed=False past the grace window: the load balancer/data plane is not being programmed. The Gateway-API analog of ingress.sync\_failed. | | `gateway.route_rejected` | `gateway` | A Gateway/listener or HTTPRoute parent held Accepted=False/ResolvedRefs=False past the grace window: the route config never became routable. The analog of ingress.translate\_failed. | | `quota.forecast` | `quota` | A GCP quota’s usage slope projects exhaustion (warning ETA<7d or usage>=90%; critical ETA<48h or >=98%), with a quota-increase draft attached. | | `notification.upgrade` | `notifications` | The provider announced a control-plane or node-pool upgrade starting — store-recorded evidence for incident-window correlation. | | `notification.upgrade_available` | `notifications` | The provider offered a new version for auto-upgrade. | | `notification.security_bulletin` | `notifications` | A provider security bulletin affects this cluster — batched to the watchboard. | | `token.burn` | `token-burn` | An agent session’s token rate ran at —burn-multiple times the cross-session baseline, or projects budget exhaustion within —burn-eta. | | `leeway.domain_unavailable` | `topology-drift` | A topology domain has no node anything can be scheduled onto — the subject is the domain and not a workload, so a dead zone is one signal rather than one per workload that drifted because of it. | | `leeway.contract_violated` | `topology-drift` | A declared topology contract (a DoNotSchedule spread constraint or a required anti-affinity) is being violated, sustained past the dwell window. | | `leeway.placement_drift` | `topology-drift` | A workload’s objects deviated from the placement its intent implies — declared, inferred, or, where nobody expressed one, an even apportionment over the domains it can reach (drift ρ over threshold), sustained past the dwell window. | | `leeway.baseline_breach` | `topology-drift` | A workload that declared nothing left the placement it has held all along — a domain’s share fell outside the band around its own learned normal, sustained past the dwell window. | | `leeway.rank_wedged` | `compute-class` | Pods are Pending against a compute class that told the autoscaler not to provision outside its priority list, so no capacity of any rank will arrive without a change to the class. | | `leeway.rank_degraded` | `compute-class` | A compute class is running below the priority it prefers — too little pod-time at rank 0, or too much at the least-preferred tier, sustained past the dwell window. The pods stay Running, which is why nothing else reports it. | | `leeway.rank_no_migration` | `compute-class` | A compute class that declared it would migrate workloads back to preferred capacity has not done so since that capacity became available again. | | `leeway.rank_tier_unused` | `compute-class` | A whole preference tier has never been occupied over the observation window — a dead rung on the priority ladder, or reserved capacity being paid for and never drawn on. Info, not a page: an unused tier is frequently the intended configuration. | Field-level detail (ordered json field lists, omitempty rules, the fingerprint recipe) lives in [`docs/signal-schema-v1.md`](https://github.com/go-steer/k8s-lookout/blob/main/docs/signal-schema-v1.md). # lookout stab drain > Before draining a node, list everything that will block the drain (PDBs at disruptionsAllowed=0) or be destroyed by it (bare pods, emptyDir data, single-replica workloads); --node details one node, -A means all nodes here (pods are always examined across all namespaces); scanned counts pods examined after the standard-drain skips (mirror/DaemonSet/completed pods). Before draining a node, list everything that will block the drain (PDBs at disruptionsAllowed=0) or be destroyed by it (bare pods, emptyDir data, single-replica workloads); —node details one node, -A means all nodes here (pods are always examined across all namespaces); scanned counts pods examined after the standard-drain skips (mirror/DaemonSet/completed pods). MCP tool: `k8s_drain_blockers` ## Usage [Section titled “Usage”](#usage) ```sh lookout stab drain [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | -------- | ------ | ------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `--node` | string | — | analyze one node in detail: every blocker on it becomes its own finding. Exactly one of —node or -A (all-nodes summary) is required. | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | --------------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------- | | `drain.pdb_gridlock` | critical | a PodDisruptionBudget covering pods on this node allows zero disruptions: the eviction API refuses and the drain hangs | | `drain.bare_pod` | warning | a pod on this node has no owner, so eviction deletes it permanently and nothing recreates it | | `drain.local_storage` | warning | a pod on this node has emptyDir volumes: the drain needs —delete-emptydir-data and the data is lost | | `drain.singleton` | warning | a pod on this node is the only replica of its controller — evicting it is an outage | | `drain.node` | critical, warning | the -A roll-up: this node is not cleanly drainable, with the blocker classes counted; critical when a PDB gridlock is among them | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | --------------------- | ----------------------------------------------------------------------------------- | | `node` | the node the blocker sits on (stamped on every —node-mode finding) | | `pods` | pods on the node covered by the gridlocked PDB | | `pod_names` | names of the covered pods, capped at 8 with a +N more tail | | `disruptions_allowed` | PDB status.disruptionsAllowed (always 0 in a gridlock finding) | | `current_healthy` | PDB status.currentHealthy | | `desired_healthy` | PDB status.desiredHealthy | | `volumes` | emptyDir volume names on the pod; memory-backed ones marked (medium=Memory) | | `workload` | the single-replica controller as \/\/\ | | `replicas` | the controller’s spec.replicas (always 1 in a singleton finding) | | `blockers` | total drain blockers on the node (also a —node-mode summary note) | | `pdb_gridlock` | gridlocked-PDB blocker count on the node (-A per-node finding; zero counts omitted) | | `bare_pods` | bare-pod blocker count on the node (-A per-node finding; zero counts omitted) | | `local_storage` | emptyDir blocker count on the node (-A per-node finding; zero counts omitted) | | `singletons` | single-replica blocker count on the node (-A per-node finding; zero counts omitted) | | `drainable` | summary note (—node mode): yes when the node has no blockers, else no | | `nodes` | summary note (-A mode): nodes examined | | `blocked` | summary note (-A mode): nodes with at least one blocker | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout stab drain --node=gke-prod-pool-a-x1z2 lookout stab drain -A lookout stab drain --node=gke-prod-pool-a-x1z2 --format=json ``` # lookout stab drift > Find spec fields of Deployments/StatefulSets/DaemonSets owned by a manager other than the GitOps controller (managedFields) — out-of-band kubectl edits and rogue co-managers. Reports manager strings (tool names, not people); --identity additionally resolves each drift write to the audited principal via the cloud provider's audit trail (GKE Cloud Audit Logs), reporting an explicit unavailable on clusters without one. Default scope: all namespaces; scanned counts workload objects examined. Find spec fields of Deployments/StatefulSets/DaemonSets owned by a manager other than the GitOps controller (managedFields) — out-of-band kubectl edits and rogue co-managers. Reports manager strings (tool names, not people); —identity additionally resolves each drift write to the audited principal via the cloud provider’s audit trail (GKE Cloud Audit Logs), reporting an explicit unavailable on clusters without one. Default scope: all namespaces; scanned counts workload objects examined. MCP tool: `k8s_gitops_drift` ## Usage [Section titled “Usage”](#usage) ```sh lookout stab drift [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | ------------ | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--manager` | string | — | the declared GitOps manager (e.g. argocd-controller); empty auto-detects it as the manager owning a strict majority (>50%) of the spec leaf fields summed across the scanned objects AND recognized as a GitOps controller (Argo CD, Flux, Helm, Config Sync, Fleet, kapp, Terraform, Pulumi). No manager clears both bars — the usual shape of a cluster with no GitOps controller at all — and the scan resolves to detection=none and emits nothing rather than measuring drift against a guess; the summary then names the leading candidate (ties to the lexicographically smallest) and its share, to pass back here if it is in fact the GitOps controller. A declared manager skips both bars: the operator knows their cluster | | `--identity` | bool | — | resolve each finding’s last drift write to the audited principal (who ran it) via the cloud provider’s audit trail; requires a provider with the audit capability (GKE: Cloud Audit Logs admin-activity read), otherwise the summary line reports an explicit unavailable | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `drift.manual_edit` | critical, warning | a manager other than the GitOps controller owns spec fields on this object; critical when one of them is high blast radius (image, replicas, env) | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `manager` | on findings: the foreign manager string from managedFields (a tool name like kubectl-edit — never a user identity; see —identity); on the summary line: the resolved GitOps manager | | `detection` | summary note: how the GitOps manager was resolved — declared (—manager), majority (auto-detected recognized GitOps controller owning >50% of the spec leaf fields in scope), or none (no manager resolved; nothing emitted) | | `detection_reason` | summary note on detection=none, naming why: no-spec-fields-in-scope (nothing in scope owns a spec field), no-majority-manager (a leading candidate exists but owns 50% or less), or not-a-gitops-manager (the majority owner is not a recognized GitOps controller — e.g. kubeadm or a kubectl manager on a cluster with no GitOps at all) | | `candidate` | summary note on detection=none: the leading manager that fell short (of the majority, or of being a recognized GitOps controller) — pass it to —manager if it is in fact the GitOps controller | | `share` | summary note: the resolved manager’s (or, on detection=none, the candidate’s) percentage of every spec leaf field owned across the scanned objects, rounded. A declared manager with a low share means most findings are other legitimate owners | | `unmanaged` | summary note, omitted at zero: scanned objects the resolved GitOps manager owns no spec field on. Nothing is reported for them — an object the manager never applied cannot have drifted from it — so a high count next to zero findings means the manager’s scope is narrower than the scan’s | | `operation` | managedFields operation of the foreign manager’s last write: Apply or Update | | `tool` | client tool recognized from the manager string (kubectl for kubectl-edit/kubectl-patch/kubectl-\*) | | `fields` | compact spec paths the foreign manager owns (e.g. spec.template.spec.containers\[app].image), capped at 8 with a +N more tail | | `field_count` | total spec leaf fields the foreign manager owns on this object (uncapped) | | `age` | how long ago the foreign manager last wrote (managedFields time); omitted when the API server recorded no time | | `principal` | —identity: the audited principal of the write nearest the drift time (GKE: principalEmail), or the explicit sentinel none-in-audit-window / no-write-time-anchor when the trail cannot answer | | `principal_agent` | —identity: the caller-supplied client string of that write (a kubectl or controller user-agent), when the trail records one; caller-controlled text, display-only | | `other_principals` | —identity: other distinct principals that wrote the object inside the audit window, capped at 8 with a +N more tail | | `identity` | summary note when —identity could not be served: the unavailable marker naming why (no provider / audit capability absent) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout stab drift lookout stab drift --namespace=prod --manager=argocd-controller lookout stab drift --workload=Deployment/prod/api --identity lookout stab drift --workload=Deployment/prod/api --format=json ``` # lookout state edges > Verify every dependency edge of one workload — ConfigMap/Secret keys, imagePullSecrets, Service selectors and endpoints, Ingress backends and class, StatefulSet governing Service and volume classes, ServiceAccount/RBAC references, TLS expiry — reporting only the broken ones. --workload also accepts Service// to enter from the service side, which is the direction the evidence arrives from when a service has no endpoints: it reports that service's selector, endpoints, ingresses and certificates, and names the workload the selector was probably meant for. Verify every dependency edge of one workload — ConfigMap/Secret keys, imagePullSecrets, Service selectors and endpoints, Ingress backends and class, StatefulSet governing Service and volume classes, ServiceAccount/RBAC references, TLS expiry — reporting only the broken ones. —workload also accepts Service/\/\ to enter from the service side, which is the direction the evidence arrives from when a service has no endpoints: it reports that service’s selector, endpoints, ingresses and certificates, and names the workload the selector was probably meant for. MCP tool: `k8s_state_edges` (MCP profile: `triage`) ## Usage [Section titled “Usage”](#usage) ```sh lookout state edges [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | ------------- | -------- | ------- | --------------------------------------------------- | | `--cert-warn` | duration | `720h` | report TLS certificates expiring within this window | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------- | | `edge.missing_ref` | critical | a referenced ConfigMap, Secret, ServiceAccount, TLS secret, IngressClass, StorageClass, or governing Service does not exist | | `edge.missing_key` | critical | the referenced key is absent from an existing ConfigMap/Secret | | `edge.invalid_ref` | warning | the referenced object exists but is the wrong type to serve the reference | | `edge.unclassed` | warning | the Ingress names no class and no IngressClass declares itself the cluster default — no controller will claim it | | `edge.selector_empty` | critical | a Service selector selects zero pods, so the service routes nowhere | | `edge.selector_unready` | critical, warning | the Service selects pods but some are not Ready; critical when none are | | `edge.endpoints_missing` | critical | a selecting Service has no EndpointSlices at all | | `edge.endpoints_orphaned` | warning | an endpoint targetRef names a pod that no longer exists | | `edge.endpoints_unready` | critical, warning | the endpoint ready-count disagrees with the selected pods (stale or lagging slices); critical at zero ready | | `edge.backend_missing` | critical | an Ingress backend service, or the port it names, does not exist | | `edge.cert_expired` | critical | a TLS certificate’s NotAfter is in the past | | `edge.cert_expiring` | warning | a TLS certificate expires within —cert-warn | | `edge.cert_invalid` | warning | tls.crt is missing or unparseable, or the secret is not kubernetes.io/tls | | `edge.rbac_dangling` | warning | a (Cluster)RoleBinding for the workload’s ServiceAccount points at a missing (Cluster)Role | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `workload` | the target the edges were traced from as \/\/\, stamped on every finding — a workload, or the Service itself when entered from the service side | | `likely_workload` | on a Service-entry edge.selector\_empty: the workload in that namespace whose pod labels best fit the broken selector, i.e. the one it was probably meant to select. Absent when two workloads fit equally well, because then naming one would be a guess | | `pods` | how many of the workload’s pods carry the broken reference | | `container` | container declaring the broken env/envFrom reference | | `env` | environment variable whose valueFrom reference is broken | | `volume` | pod volume, or StatefulSet volumeClaimTemplate, whose reference is broken | | `key` | the referenced key that is missing from the ConfigMap/Secret | | `selector` | the Service label selector under scrutiny | | `selected` | pods the Service selector currently selects | | `ready` | ready count (selected pods or serving endpoints, per finding kind) | | `endpoints` | total endpoints across the Service’s EndpointSlices | | `slices` | how many EndpointSlices back the Service | | `service` | the Service a slice, Ingress backend, or StatefulSet serviceName refers to | | `pod` | pod named by an orphaned endpoint targetRef | | `subject` | TLS certificate subject (CN when set); never key material | | `not_after` | TLS certificate NotAfter, RFC 3339 | | `days_left` | whole days until NotAfter (negative = expired) | | `via` | how the broken reference is reached from the workload: mount, ingress, or imagePullSecret | | `ingress` | Ingress referencing the TLS secret, or the unserved Ingress itself | | `host` | Ingress rule host of the broken backend (empty for the default backend) | | `path` | Ingress rule path of the broken backend | | `port` | Service port (name or number) the Ingress backend asks for | | `service_account` | ServiceAccount the RBAC finding is about, or the one contributing an imagePullSecret | | `role_ref` | dangling roleRef as \/\ | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout state edges --workload=Deployment/prod/api lookout state edges --workload=Pod/prod/api-6d5f8c-x2v9k --format=json lookout state edges --workload=StatefulSet/db/postgres --cert-warn=336h lookout state edges --workload=Service/prod/api ``` # lookout state gateway > When traffic through the Gateway API does not arrive — walk GatewayClass → Gateway → listener → HTTPRoute → Service and report every hop that is rejected, unprogrammed, or points at something that is not there. Silent, and cheap, on clusters without the Gateway API installed. When traffic through the Gateway API does not arrive — walk GatewayClass → Gateway → listener → HTTPRoute → Service and report every hop that is rejected, unprogrammed, or points at something that is not there. Silent, and cheap, on clusters without the Gateway API installed. MCP tool: `k8s_gateway_routes` ## Usage [Section titled “Usage”](#usage) ```sh lookout state gateway [flags] ``` ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ---------------------------- | -------- | ------------------------------------------------------------------------------------------------------- | | `gateway.missing_class` | critical | the Gateway names a GatewayClass that does not exist — nothing will program it | | `gateway.class_not_accepted` | critical | the Gateway’s GatewayClass is not Accepted by its controller | | `gateway.not_accepted` | critical | the Gateway itself is not Accepted | | `gateway.not_programmed` | critical | the Gateway is Accepted but not Programmed: no data plane is carrying its traffic | | `gateway.listener_invalid` | warning | one listener of an otherwise working Gateway is not resolved or not programmed | | `route.missing_parent` | critical | the route’s parentRef names a Gateway that does not exist | | `route.not_accepted` | critical | the Gateway refused the route’s attachment (listener, hostname, or namespace policy) | | `route.missing_backend` | critical | the route’s backendRef Service does not exist | | `route.backend_port` | critical | the route’s backendRef Service exists but does not expose the named port | | `crd.unavailable` | info | the API group this check reads is not served by the cluster, so nothing was examined (no coverage lies) | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | --------------- | ------------------------------------------------------------------------------------- | | `gateway_class` | GatewayClass the Gateway names | | `controller` | the GatewayClass’s spec.controllerName — which implementation owns it | | `gateway` | Gateway the route attaches to, as namespace/name | | `listener` | listener name within the Gateway | | `port` | listener port, or the backendRef port the Service does not expose | | `protocol` | listener protocol | | `condition` | the status condition that is not True | | `service` | backend Service the route names, as namespace/name | | `service_ports` | ports the backend Service does expose, sorted | | `classes` | GatewayClasses the cluster does have, sorted | | `api_group` | crd.unavailable: the API group-version this command needed | | `resources` | crd.unavailable: the resources it would have read | | `unavailable` | summary-line note: why the group could not be read (absent CRDs, or discovery denied) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout state gateway lookout state gateway --namespace=prod lookout state gateway --format=json ``` # lookout state storage > When a PersistentVolumeClaim sits Pending and the pod behind it will not schedule — name the reason: a StorageClass that does not exist, no class and no cluster default, a static-only class with nothing pre-provisioned, plus the default-class ambiguity and stranded volumes behind it. When a PersistentVolumeClaim sits Pending and the pod behind it will not schedule — name the reason: a StorageClass that does not exist, no class and no cluster default, a static-only class with nothing pre-provisioned, plus the default-class ambiguity and stranded volumes behind it. MCP tool: `k8s_storage_binding` ## Usage [Section titled “Usage”](#usage) ```sh lookout state storage [flags] ``` ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | --------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------- | | `storage.missing_class` | critical | the claim names a StorageClass that does not exist — it will stay Pending forever | | `storage.no_default_class` | critical | the claim names no class and the cluster has no default StorageClass | | `storage.no_provisioner` | warning | the claim’s class is static-only (kubernetes.io/no-provisioner) and no matching PV is available | | `storage.multiple_defaults` | warning | more than one StorageClass is annotated as the cluster default; which one wins is not defined | | `storage.pv_failed` | warning | a PersistentVolume is Failed: its reclaim did not complete, so the backing disk stays allocated and the volume cannot be reused | | `storage.pv_released` | info | a PersistentVolume is Released — retained on purpose, but its capacity is unusable until spec.claimRef is cleared | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ---------------- | ------------------------------------------------------------------------ | | `storage_class` | StorageClass the claim names, or the class the finding is about | | `classes` | StorageClasses the cluster does have, sorted (empty when there are none) | | `defaults` | StorageClasses annotated as the cluster default, sorted | | `provisioner` | the class’s spec.provisioner | | `phase` | the claim’s or volume’s status.phase at scan time | | `requested` | storage the claim requests (spec.resources.requests.storage) | | `capacity` | the volume’s spec.capacity.storage | | `reclaim_policy` | the volume’s spec.persistentVolumeReclaimPolicy | | `claim` | the claim the volume was bound to, as namespace/name | | `binding_mode` | the class’s volumeBindingMode (Immediate when unset) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout state storage lookout state storage --namespace=prod lookout state storage --format=json ``` # lookout state volumes > When pods hang in ContainerCreating with Multi-Attach or FailedAttachVolume events — join VolumeAttachment + PV/PVC + pods to name the exact conflict: RWO claims wanted on two nodes, attachments stuck in error, cross-zone PV locks, orphaned attachments. When pods hang in ContainerCreating with Multi-Attach or FailedAttachVolume events — join VolumeAttachment + PV/PVC + pods to name the exact conflict: RWO claims wanted on two nodes, attachments stuck in error, cross-zone PV locks, orphaned attachments. MCP tool: `k8s_volume_conflicts` ## Usage [Section titled “Usage”](#usage) ```sh lookout state volumes [flags] ``` ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ---------------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------- | | `volume.multi_attach` | critical | an RWO claim is wanted by pods on more than one node — the second pod never starts | | `volume.zone_conflict` | critical | the PV is locked to a zone the pod’s node is not in | | `volume.attach_error` | critical, warning | the attach or detach is failing; critical once it has been failing long enough to be stuck rather than slow | | `volume.orphaned_attachment` | info | a VolumeAttachment survives its PV or its node | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | -------------- | ----------------------------------------------------------------------------------- | | `pods` | scheduled pods referencing the conflicted claim, sorted (list capped, then +K more) | | `nodes` | distinct nodes those pods are scheduled on, sorted | | `access_modes` | the claim’s declared access modes | | `pv` | PersistentVolume behind the claim or attachment | | `pvc` | PersistentVolumeClaim the pod mounts (same namespace as the pod) | | `node` | node the attachment targets or the pod is scheduled on | | `attacher` | CSI driver responsible for the attachment (spec.attacher) | | `age` | how long the attach/detach error has persisted, truncated to seconds | | `error` | the attach/detach error message, truncated to 200 chars | | `attached` | the attachment’s status.attached at scan time | | `pv_zones` | zones the PV’s node affinity allows, sorted | | `node_zone` | zone label of the node the pod is scheduled on | | `orphan` | which referenced side is gone: “pv missing”, “node missing”, or both | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout state volumes lookout state volumes --namespace=prod lookout state volumes --format=json ``` # lookout state webhooks > When creates/updates hang or fail cluster-wide with "failed calling webhook", or before relying on a policy engine: audit every admission webhook — dead backends × failurePolicy (Fail + dead backend rejects every matching admission), the namespace/rule blast radius, timeout stall risk, CA-bundle expiry. The full check; health's webhooks category delegates here. When creates/updates hang or fail cluster-wide with “failed calling webhook”, or before relying on a policy engine: audit every admission webhook — dead backends × failurePolicy (Fail + dead backend rejects every matching admission), the namespace/rule blast radius, timeout stall risk, CA-bundle expiry. The full check; health’s webhooks category delegates here. MCP tool: `k8s_admission_webhooks` ## Usage [Section titled “Usage”](#usage) ```sh lookout state webhooks [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | ------------- | -------- | ------- | ----------------------------------------------------- | | `--cert-warn` | duration | `720h` | report webhook CA bundles expiring within this window | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------------------ | -------- | ----------------------------------------------------------------------------------------------------- | | `webhook.failing_closed` | critical | the webhook has no working backend and failurePolicy=Fail: every gated write is rejected cluster-wide | | `webhook.dead_backend` | warning | the webhook’s service backend is missing, has no ready endpoints, or does not serve the named port | | `webhook.slow_risk` | info | the webhook’s timeout is long enough to slow every gated write if the backend degrades | | `webhook.ca_expired` | critical | the webhook’s caBundle has expired: the API server cannot verify it | | `webhook.ca_expiring` | warning | the webhook’s caBundle expires within —cert-warn | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ----------------- | -------------------------------------------------------------------------------------------------------------------------- | | `webhook` | admission webhook as \/\ | | `service` | service backend the webhook points at, as \/\ | | `backend` | why the backend is dead: service missing, no ready endpoints, or port \

not on service | | `gates` | namespaces the webhook gates, from namespaceSelector: all namespaces, or \/\ namespaces with up to 5 names | | `rules` | compact operations/resources summary of the webhook’s rules, e.g. “CREATE,UPDATE pods,deployments.apps” | | `object_selector` | the webhook’s objectSelector, when one is set | | `timeout` | webhook timeoutSeconds as \s (nil defaults to the API’s 10s) | | `subject` | CA-bundle certificate subject (CN when set); never key material | | `not_after` | CA-bundle certificate NotAfter, RFC 3339 | | `days_left` | whole days until NotAfter (negative = expired) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout state webhooks lookout state webhooks --format=json --cert-warn=336h ``` # lookout state wi > When a GKE pod gets 403s or metadata-server errors calling GCP APIs, verify the Workload Identity chain — KSA annotation (iam.gke.io/gcp-service-account) → roles/iam.workloadIdentityUser binding on the GSA — reporting only the broken links; vanilla clusters report an explicit unavailable. When a GKE pod gets 403s or metadata-server errors calling GCP APIs, verify the Workload Identity chain — KSA annotation (iam.gke.io/gcp-service-account) → roles/iam.workloadIdentityUser binding on the GSA — reporting only the broken links; vanilla clusters report an explicit unavailable. MCP tool: `k8s_workload_identity` ## Usage [Section titled “Usage”](#usage) ```sh lookout state wi [flags] ``` ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | -------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------- | | `wi.gsa_missing` | critical | the annotated Google service account does not exist — every GCP call from these pods fails | | `wi.unbound` | critical | the KSA annotates a GSA but the roles/iam.workloadIdentityUser binding is missing or malformed | | `wi.unannotated_use` | info | a pod sets GOOGLE\_APPLICATION\_CREDENTIALS but its ServiceAccount carries no Workload Identity annotation | | `cloud.unavailable` | info | the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ------------- | ------------------------------------------------------------------------------------ | | `gsa` | the cloud identity (GSA email) the ServiceAccount’s annotation claims | | `pods` | how many in-scope pods run as the affected ServiceAccount | | `problem` | machine-matchable problem code from the provider (e.g. no-workload-identity-binding) | | `container` | container carrying the GOOGLE\_APPLICATION\_CREDENTIALS env var | | `env` | the credential-file env var found (GOOGLE\_APPLICATION\_CREDENTIALS) | | `capability` | cloud.unavailable: the provider capability this command needed (workload-identity) | | `provider` | cloud.unavailable: the provider that was asked | | `unavailable` | summary-line note: why the cloud read could not be served | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout state wi lookout state wi --namespace=prod lookout state wi --workload=Deployment/prod/api --format=json ``` # lookout triage changes > What changed around one workload in the window before onset — rollouts, config/secret updates, rescales, node ops — chronological, scoped to the target's graph neighborhood; full fidelity from a sentinel store, best-effort live otherwise. What changed around one workload in the window before onset — rollouts, config/secret updates, rescales, node ops — chronological, scoped to the target’s graph neighborhood; full fidelity from a sentinel store, best-effort live otherwise. MCP tool: `k8s_recent_changes` ## Usage [Section titled “Usage”](#usage) ```sh lookout triage changes /[/] [flags] ``` `/[/]` — the pod or workload at the center of the question: \/\/\, \/\ (namespace from —namespace, else “default”), or a bare pod name; kinds are case-insensitive with the usual short forms (po, deploy, rs, sts, ds, cj). —workload=\/\/\ is the flag-shaped alternative. ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | --------- | ---- | ------- | ----------------------------------------------------------------------------------------------------- | | `--depth` | int | `2` | neighborhood radius: graph edges followed per direction to decide which objects’ changes are in scope | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Point-in-time flags (graph-backed commands) [Section titled “Point-in-time flags (graph-backed commands)”](#point-in-time-flags-graph-backed-commands) This command answers from the topology graph and accepts the point-in-time flags: | Flag | Type | Default | Meaning | | ----------------- | ------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--at` | string | — | answer as of this instant instead of live: RFC3339 (2026-07-25T10:00:00Z) or a duration ago (20m). Requires —store. | | `--store` | string | — | path to a sentinel’s SQLite store (its —store file); source for —at point-in-time topology | | `--store-cluster` | string | — | read/write the store for THIS cluster, treating —store as the multi-cluster stem the sentinel was given: —store=/var/lib/lookout/lookout.db —store-cluster=prod-us opens /var/lib/lookout/lookout-prod-us.db (issue #410). Set it only against a sentinel running —clusters/—clusters-from; a single-cluster sentinel writes the literal —store path | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ----------------- | -------- | ------------------------------------------------------------------------------------------------ | | `change.rollout` | info | a workload’s pod template changed — a new image, container or mount, or a controller churn event | | `change.scale` | info | a workload’s replica count changed | | `change.config` | info | a ConfigMap in the neighborhood changed | | `change.secret` | info | a Secret in the neighborhood changed (names and shortened hashes only, never values —) | | `change.node` | info | a Node in the neighborhood changed | | `change.label` | info | only labels changed on a neighborhood object — enough to move it in or out of a selector | | `change.topology` | info | a neighborhood object appeared, disappeared, or changed in a way none of the other classes name | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ---------- | --------------------------------------------------------------------------------------------------------------------- | | `at` | when the change happened, RFC 3339 (also the summary-line note for the resolved —at instant) | | `relation` | the changed object’s place in the target’s neighborhood: self (the target or its pods), upstream, lateral, downstream | | `fields` | changed fields as path=from→to pairs — names, counts, and shortened hashes only, never values | | `origin` | where the change was seen: log (delta log), event (Kubernetes Event), api (reconstructed from current API state) | | `revision` | deployment.kubernetes.io/revision of a rollout’s ReplicaSet (live approximation) | | `image` | first container image of a rollout’s new pod template (live approximation) | | `window` | summary-line note: the (from, to] window the answer covers, RFC 3339 | | `source` | summary-line note: history (delta log from —store) or live-approximation (no store; see the fidelity gap in —help) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage changes Deployment/prod/api --store=/var/lib/lookout/lookout.db lookout triage changes Deployment/prod/api --since=1h --at=2026-07-25T10:00:00Z --store=/var/lib/lookout/lookout.db lookout triage changes payments-api-7d9c4b-x2n8p --namespace=prod ``` # lookout triage delta > Every abnormal object in one scan — the first call for "anything wrong in this cluster?": broken/pending pods, stalled rollouts, workloads blocked from creating pods at all, node pressure/NPD/preemption, gridlocked PDBs, degraded kube-system add-ons, quotas at their limits. Every abnormal object in one scan — the first call for “anything wrong in this cluster?”: broken/pending pods, stalled rollouts, workloads blocked from creating pods at all, node pressure/NPD/preemption, gridlocked PDBs, degraded kube-system add-ons, quotas at their limits. MCP tool: `k8s_triage_delta` (MCP profile: `triage`) ## Usage [Section titled “Usage”](#usage) ```sh lookout triage delta [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | --------------- | -------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------- | | `--only` | string | `pods,nodes,pdb,system,quota` | comma-separated finding classes to scan: any subset of pods,nodes,pdb,system,quota | | `--restarts` | int | `5` | flag containers restarted at least this many times | | `--pending-age` | duration | `5m` | flag Pending pods older than this; also the grace before a not-ready container in a Running pod is flagged | | `--quota-warn` | int | `90` | warn when a ResourceQuota resource reaches this percent of its hard limit (the hard limit itself is always critical) | | `--cron-grace` | duration | `5m` | how late a CronJob activation may be before it counts as missed; absorbs normal controller scheduling latency | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pod.crashloop` | critical | a container is crash looping | | `pod.imagepull` | critical | a container cannot pull its image | | `pod.waiting` | warning | a container is stuck in an error waiting state (CreateContainerConfigError, InvalidImageName, …) | | `pod.oomkilled` | warning | a container’s last termination was an OOM kill | | `pod.restarts` | warning | a container has restarted at least —restarts times | | `pod.notready` | warning | a container in a Running pod has been not-ready past the —pending-age grace | | `pod.failed` | warning | the pod reached phase Failed | | `pod.pending` | critical, warning | the pod has been Pending longer than —pending-age with no container-level diagnosis; critical when the scheduler has declared it Unschedulable, which is a capacity or constraint problem rather than latency | | `workload.replicafailure` | critical | the controller cannot create pods at all (quota, PodSecurity, admission) — no pod exists to diagnose | | `workload.stalled` | critical | a Deployment’s Progressing condition is False: the rollout has given up | | `workload.rollout` | critical, warning | replicas are short of desired; critical when nothing is serving at all | | `job.failed` | warning | a Job’s Failed condition is set | | `cron.missed` | critical, warning | an unsuspended CronJob’s schedule said to run more than —cron-grace ago and status says it did not; critical once several activations in a row are gone | | `cron.unparseable` | warning | a CronJob’s spec.schedule could not be parsed, so its activations cannot be judged at all | | `node.notready` | critical | the node’s Ready condition is not True | | `node.pressure` | critical | the node reports Memory/Disk/PID pressure | | `node.condition` | critical, warning | a non-standard node condition is True — NPD and its cousins publish problems that way | | `node.cordoned` | warning | the node is unschedulable but still holds pods: a stuck drain or a forgotten maintenance step | | `node.preempt` | critical, warning, info | a reclaim taint marks the node for termination; severity tracks how imminent | | `pdb.gridlocked` | critical, warning | the budget permits no disruptions; critical when healthy pods are already below the required minimum | | `addon.degraded` | critical, warning | a kube-system add-on (dns, proxy, cni, csi, metrics, connectivity) is short of replicas; critical when none are available | | `quota.near` | warning | a ResourceQuota resource is at or past —quota-warn percent of its hard limit | | `quota.exhausted` | critical | a ResourceQuota resource is at its hard limit: the next create is rejected | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | --------------- | ------------------------------------------------------------------- | | `container` | container the finding is about (init containers prefixed init:) | | `image` | image reference that failed to pull | | `restarts` | container restart count | | `exit_code` | exit code of the container’s last termination | | `last_state` | reason of the container’s last termination (e.g. OOMKilled) | | `age` | how long the abnormal state has persisted | | `desired` | desired replica/scheduled count from spec | | `ready` | ready count from status | | `updated` | updated-to-current-revision count from status | | `available` | available count from status | | `failed` | failed pod count of a Job | | `schedule` | a CronJob’s spec.schedule | | `expected` | the activation a CronJob should have run and did not | | `missed_runs` | activations missed since the anchor; ≥N when the walk was capped | | `anchor` | what the missed count was measured from: last\_schedule or creation | | `time_zone` | a CronJob’s spec.timeZone, when set | | `last_schedule` | a CronJob’s status.lastScheduleTime, or never | | `active_jobs` | Jobs a CronJob still has running | | `condition` | node condition type that is abnormal | | `taint` | taint key indicating reclaim/drain | | `pods` | pods affected (behind a cordoned node or a PDB) | | `healthy` | currently healthy pods behind a PDB | | `required` | pods the PDB requires healthy | | `addon` | system add-on role: dns, proxy, cni, csi, metrics, connectivity | | `resource` | ResourceQuota resource name at or near its limit | | `used` | quota usage from status | | `hard` | quota hard limit from status | | `pct` | quota usage as percent of the hard limit | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage delta lookout triage delta --namespace=prod --only=pods,quota lookout triage delta --only=nodes --format=json ``` # lookout triage events > Deduped chronological event timeline: kubectl get events, but collapsed by (object, reason family) over a workload's whole owner-reference tree, with HPA rescale-oscillation (thrash) detection. Deduped chronological event timeline: kubectl get events, but collapsed by (object, reason family) over a workload’s whole owner-reference tree, with HPA rescale-oscillation (thrash) detection. MCP tool: `k8s_event_timeline` ## Usage [Section titled “Usage”](#usage) ```sh lookout triage events [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | -------------- | -------- | ------- | ------------------------------------------------------------------------------------------- | | `--hpa-window` | duration | `30m` | report event.hpa\_thrash when enough scale-direction changes fall inside a window this long | | `--hpa-flips` | int | `2` | scale-direction changes within —hpa-window that count as thrash (2 = up→down→up) | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------------ | -------- | -------------------------------------------------------------------------------------------------------------------------- | | `event.warning` | warning | one collapsed timeline entry for a Warning-type event family on a subject | | `event.normal` | info | one collapsed timeline entry for a Normal-type event family — context for the warnings around it, not a problem on its own | | `event.hpa_thrash` | warning | an HPA changed scale direction at least —hpa-flips times inside —hpa-window: the autoscaler is fighting itself | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ------------ | --------------------------------------------------------------------------------------------------------------------------- | | `count` | events collapsed into this timeline entry: k8s per-event repeat counts summed across the entry’s reason family | | `first_seen` | RFC3339 timestamp of the entry’s oldest activity | | `last_seen` | RFC3339 timestamp of the entry’s newest activity (the timeline sort key) | | `source` | reporting component (kubelet, horizontal-pod-autoscaler, …) | | `variants` | raw Event.Reason values collapsed into this entry, comma-separated (present only when a reason family merged more than one) | | `replicas` | event.hpa\_thrash: the chronological replica sequence recovered from SuccessfulRescale events, e.g. 2->6->2->6 | | `flips` | event.hpa\_thrash: most scale-direction changes observed inside one —hpa-window | | `window` | event.hpa\_thrash: the —hpa-window the flips were counted in | | `target` | event.hpa\_thrash: the HPA’s scaleTargetRef as Kind/name (when the HPA object was readable) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage events --workload=Deployment/prod/api lookout triage events --workload=Pod/prod/api-6d5f8c-x2v9k --since=30m lookout triage events --namespace=prod lookout triage events -A --since=2h --format=json lookout triage events --workload=Deployment/prod/api --hpa-window=15m --hpa-flips=4 ``` # lookout triage list > List what EXISTS in a namespace — kubectl get across every kind at once, one line per object, leading with the // target the other read tools take. The first call for a namespace you have not enumerated: the health scans report only what is abnormal and name nothing when a namespace is clean, so they cannot tell you what is in one. An inventory, not a diagnosis — never guess an object's name, list the namespace. List what EXISTS in a namespace — kubectl get across every kind at once, one line per object, leading with the \/\/\ target the other read tools take. The first call for a namespace you have not enumerated: the health scans report only what is abnormal and name nothing when a namespace is clean, so they cannot tell you what is in one. An inventory, not a diagnosis — never guess an object’s name, list the namespace. MCP tool: `k8s_list_resources` (MCP profile: `triage`) ## Usage [Section titled “Usage”](#usage) ```sh lookout triage list [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | --------- | ------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--kinds` | string | — | comma-separated resource kinds to list, spelled as kubectl spells them (pods, deploy, certificates.cert-manager.io); empty lists the default set — deployments,statefulsets,daemonsets,cronjobs,jobs,pods,services,endpoints,ingresses,configmaps,secrets,persistentvolumeclaims,horizontalpodautoscalers,poddisruptionbudgets,serviceaccounts,networkpolicies,resourcequotas,limitranges — which is every namespaced kind an incident normally involves EXCEPT replicasets (one per Deployment revision; ask for them explicitly) | | `--max` | int | `500` | stop after this many objects; the summary line reports how many were left out (pass —kinds to narrow instead) | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ------------------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `inventory.object` | info | one object in scope, rendered as kubectl’s default columns for its kind — an aggregated `kubectl get`, so every row is emitted, healthy or not | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `target` | the object as \/\/\ (\/\ when cluster-scoped) — paste it into triage spec, state edges, triage radius or triage workload unchanged | | `ready` | ready over desired, as kubectl’s READY column: containers for a Pod, replicas for a workload | | `status` | kubectl’s STATUS column verbatim: a Pod’s phase or its blocking container reason, a Job’s condition, a Node’s readiness | | `restarts` | total container restarts of a Pod | | `up_to_date` | replicas on the current revision | | `available` | replicas counted available | | `completions` | a Job’s succeeded over requested completions | | `schedule` | a CronJob’s cron expression | | `timezone` | a CronJob’s spec.timeZone, when it sets one | | `suspend` | “true” on a suspended CronJob (omitted otherwise) | | `active` | a CronJob’s currently running Jobs | | `last_schedule` | how long ago a CronJob last created a Job | | `type` | a Service’s type or a Secret’s type | | `cluster_ip` | a Service’s cluster IP (“None” for a headless Service) | | `external_ip` | a Service’s provisioned load-balancer address, or “pending” for a LoadBalancer that has none yet | | `ports` | a Service’s ports as port\[:nodePort]/protocol | | `addresses` | how many endpoint addresses an Endpoints object holds (0 means nothing is behind the Service) | | `class` | an Ingress’s ingressClassName or a PVC/PV’s storage class | | `hosts` | an Ingress’s rule hosts | | `address` | an Ingress’s provisioned load-balancer address | | `keys` | how many keys a ConfigMap or Secret holds — Secret VALUES are never read, only counted | | `phase` | status.phase of a PVC, PV or Namespace | | `volume` | the PersistentVolume a PVC is bound to | | `capacity` | a PVC’s or PV’s storage capacity | | `access_modes` | a PVC’s or PV’s access modes, kubectl-abbreviated (RWO, ROX, RWX, RWOP) | | `claim` | the PVC a PersistentVolume is bound to, as \/\ | | `scale_target` | an HPA’s scaleTargetRef as \/\ | | `min` | an HPA’s minimum replicas | | `max` | an HPA’s maximum replicas | | `replicas` | an HPA’s current replica count | | `min_available` | a PDB’s spec.minAvailable (count or percentage) | | `max_unavailable` | a PDB’s spec.maxUnavailable (count or percentage) | | `allowed_disruptions` | how many pods a PDB currently allows to be evicted | | `pod_selector` | a NetworkPolicy’s spec.podSelector; “all” when it is empty, which selects every pod in the namespace | | `roles` | a Node’s node-role.kubernetes.io/\* labels, or “none” | | `version` | a Node’s kubelet version | | `age` | time since metadata.creationTimestamp, kubectl-style (45s, 3h20m, 12d) | | `kinds` | summary-line note: how many kinds the listing covered | | `truncated` | summary-line note: how many objects —max left out; they are the LAST kinds of the listing, which is ordered workloads → routing → configuration for this reason | | `skipped` | summary-line note: kinds that could not be listed and why, as \:\ (forbidden = the caller may not list it, so its absence from the output is a blind spot, not a fact) | | `namespace_absent` | summary-line note: “true” when the listing was empty because the namespace does not exist, which an empty listing alone cannot distinguish from an empty namespace | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage list --namespace=storefront lookout triage list --namespace=prod --kinds=pods,services,endpoints lookout triage list --namespace=prod --kinds=replicasets lookout triage list -A --kinds=ingresses --format=json ``` # lookout triage logs > kubectl logs, distilled: Drain-clusters raw lines into templates with counts (probe noise stripped, stack traces collapsed to top frames) — reach for this instead of reading logs whole. kubectl logs, distilled: Drain-clusters raw lines into templates with counts (probe noise stripped, stack traces collapsed to top frames) — reach for this instead of reading logs whole. MCP tool: `k8s_triage_logs` (MCP profile: `triage`) ## Usage [Section titled “Usage”](#usage) ```sh lookout triage logs [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | ----------------- | ------ | ------- | ------------------------------------------------------------------------------------------- | | `--pod` | string | — | read one pod by name (requires —namespace) | | `--container` | string | — | restrict to one container (default: all init + regular + ephemeral containers) | | `--previous` | bool | — | read the previous container instance (what a crashed container said before it died) | | `--tail` | int | `5000` | max lines fetched per container stream (0 = no limit) | | `--max-templates` | int | `40` | cap emitted template clusters; the low-count tail is summarized in one log.overflow finding | | `--keep-probes` | bool | — | keep health/readiness probe request lines instead of stripping them | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ----------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `log.template` | critical, warning, info | one distilled template and how many lines collapsed into it; severity is the guessed level — critical at fatal, warning for error-ish, info otherwise | | `log.stacktrace` | critical, warning, info | a template that is a Go panic, Java exception, or Python traceback, with its innermost frames | | `log.overflow` | info | the low-count tail —max-templates dropped, counted rather than discarded silently (no coverage lies) | | `log.probe_noise` | info | health/readiness probe request lines stripped before distillation, counted so the removal is visible | | `log.fetch_error` | warning | a container’s log stream could not be read, so its lines are missing from the distillation | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ------------------- | -------------------------------------------------------------------------------- | | `template` | log template; <\*> marks positions that varied across merged lines | | `count` | lines merged into this cluster (on log.probe\_noise: probe lines stripped) | | `pods` | distinct pods that emitted this template (present when >1) | | `level` | guessed log level (fatal\|error\|warn\|info\|debug) from token/field match | | `first_seen` | RFC3339 timestamp of the oldest merged line (from log timestamps when parseable) | | `last_seen` | RFC3339 timestamp of the newest merged line | | `lang` | stack-trace runtime on log.stacktrace findings: go\|java\|python | | `frames` | top stack frames on log.stacktrace findings, innermost first, ’ < ’ separated | | `sample` | one representative raw line, truncated and sanitized | | `container` | container the finding refers to (log.fetch\_error only) | | `omitted_templates` | clusters dropped by —max-templates (log.overflow only) | | `omitted_lines` | lines inside the dropped clusters (log.overflow only) | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage logs --workload=Deployment/prod/api --since=30m lookout triage logs --namespace=payments --previous --container=app lookout triage logs --pod=api-6d5f9c7b4-xk2p1 --namespace=prod --format=json ``` # lookout triage radius > Blast radius of one pod/workload — who is upstream (routes/owns/governs it), lateral (same node, shared config/volume), downstream (it depends on); --at answers it as of incident onset from a sentinel store. Blast radius of one pod/workload — who is upstream (routes/owns/governs it), lateral (same node, shared config/volume), downstream (it depends on); —at answers it as of incident onset from a sentinel store. MCP tool: `k8s_blast_radius` ## Usage [Section titled “Usage”](#usage) ```sh lookout triage radius /[/] [flags] ``` `/[/]` — the pod or workload at the center of the question: \/\/\, \/\ (namespace from —namespace, else “default”), or a bare pod name; kinds are case-insensitive with the usual short forms (po, deploy, rs, sts, ds, cj). —workload=\/\/\ is the flag-shaped alternative. ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | --------- | ---- | ------- | --------------------------------------------------------- | | `--depth` | int | `3` | graph edges followed per direction from the target’s pods | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Point-in-time flags (graph-backed commands) [Section titled “Point-in-time flags (graph-backed commands)”](#point-in-time-flags-graph-backed-commands) This command answers from the topology graph and accepts the point-in-time flags: | Flag | Type | Default | Meaning | | ----------------- | ------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--at` | string | — | answer as of this instant instead of live: RFC3339 (2026-07-25T10:00:00Z) or a duration ago (20m). Requires —store. | | `--store` | string | — | path to a sentinel’s SQLite store (its —store file); source for —at point-in-time topology | | `--store-cluster` | string | — | read/write the store for THIS cluster, treating —store as the multi-cluster stem the sentinel was given: —store=/var/lib/lookout/lookout.db —store-cluster=prod-us opens /var/lib/lookout/lookout-prod-us.db (issue #410). Set it only against a sentinel running —clusters/—clusters-from; a single-cluster sentinel writes the literal —store path | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ----------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------- | | `radius.neighbor` | info | one object in the target’s neighborhood, with its direction, relation, and hop distance — an enumeration of impact, not a defect | | `radius.missing` | warning | a neighbor the graph references but never observed, in a kind the snapshot does watch: the reference is dangling | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `direction` | neighbor’s direction from the target: upstream (routes/owns/governs it), lateral (shares a node/volume/config), downstream (the target points at it) | | `relation` | how the neighbor attaches: the edge kind (RoutesTo, Owns, Selects, Governs, RunsOn, Mounts) for upstream/downstream, shared-node\|shared-zone\|shared-config\|shared-secret\|shared-pvc for lateral | | `hop` | BFS depth from the target at which the neighbor was first reached (1 = direct edge) | | `shared` | on lateral neighbors: the shared object as \/\ | | `ready` | pod readiness (live mode only — history stores topology, not status) | | `source` | summary-line note: live (one-shot List pass) or history (reconstructed from —store) | | `at` | summary-line note: the resolved —at instant the history answer is as of, RFC 3339 | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage radius Deployment/prod/api lookout triage radius payments-api-7d9c4b-x2n8p --namespace=prod lookout triage radius --workload=StatefulSet/db/postgres --depth=2 lookout triage radius Deployment/prod/api --at=20m --store=/var/lib/lookout/lookout.db ``` # lookout triage spec > Read ONE resource's spec: kubectl describe, but token-dense, secret-safe, and default-elided — healthy conditions are omitted. Read ONE resource’s spec: kubectl describe, but token-dense, secret-safe, and default-elided — healthy conditions are omitted. MCP tool: `k8s_resource_spec` ## Usage [Section titled “Usage”](#usage) ```sh lookout triage spec /[/] [flags] ``` `/[/]` — the resource to read; Kind is case-insensitive, accepts the aliases po=Pod, deploy=Deployment, rs=ReplicaSet, sts=StatefulSet, ds=DaemonSet, svc=Service, cm=ConfigMap, pvc=PersistentVolumeClaim, ing=Ingress, netpol=NetworkPolicy, no=Node, and unlisted kinds (CRDs) resolve via API discovery (qualify as \.\ if ambiguous). Omit \ for cluster-scoped kinds, or to use —namespace (falling back to “default”). —workload=\/\/\ is the flag-shaped alternative. ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | -------- | ---- | ------- | ------------------------------------------------------------------------------------------------- | | `--diff` | bool | — | diff against the previous graph-history revision — requires a sentinel store; not yet implemented | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | ---------------- | -------- | ------------------------------------------------------------------------------------- | | `spec.resource` | info | the object itself: metadata, owner, and the kind-specific highlights (one per target) | | `spec.container` | info | one container of the target: image, resources, ports, probes, env (one per container) | | `spec.condition` | warning | a status condition of the target that is not in its nominal state | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ------------------ | ------------------------------------------------------------------------------------------------- | | `labels` | resource labels as sorted k=v pairs | | `owner` | controlling owner as Kind/name | | `phase` | status.phase, only when abnormal for the kind (zero nominal state) | | `node` | node the pod is scheduled on | | `service_account` | pod’s service account | | `volumes` | pod volumes as name:source (source names its referent, never its payload) | | `container` | container name (one spec.container finding per container) | | `init` | “true” when the container is an init container | | `image` | container image reference | | `requests` | resource requests as sorted k=v pairs | | `limits` | resource limits as sorted k=v pairs | | `ports` | container or service ports, compact (\[name:]port\[->target]\[/proto]) | | `liveness` | liveness probe one-liner (kind, target, non-default timings) | | `readiness` | readiness probe one-liner | | `env` | env vars; literal credential values are \[REDACTED], valueFrom entries render as named references | | `env_from` | envFrom sources as kind:name | | `replicas` | desired replica count | | `strategy` | rollout strategy summary (type + non-default knobs) | | `selector` | workload/service selector as sorted k=v pairs | | `type` | Service or Secret type, only when non-default | | `external_name` | ExternalName service target | | `session_affinity` | service session affinity, only when not None | | `keys` | ConfigMap/Secret data KEYS with byte sizes — values are never rendered | | `condition` | abnormal status condition as Type=Status | | `since` | the condition’s lastTransitionTime | | `spec` | kinds without a dedicated renderer: sanitized spec flattened to path=value pairs | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage spec Deployment/prod/api lookout triage spec po/payments-api-7d9c4b-x2n8p --namespace=prod lookout triage spec Node/gke-prod-pool-1-8f2a lookout triage spec Certificate/prod/api-tls --format=json lookout triage spec --workload=Deployment/prod/api ``` # lookout triage status > Write (or read back) the triage-status record for an incident — diagnosis, action taken, and your severity judgment — so health scans stop reporting it as a fresh unknown and the sentinel stops re-paging followups; the incident playbooks' closing move. Write (or read back) the triage-status record for an incident — diagnosis, action taken, and your severity judgment — so health scans stop reporting it as a fresh unknown and the sentinel stops re-paging followups; the incident playbooks’ closing move. MCP tool: `k8s_triage_status` ## Usage [Section titled “Usage”](#usage) ```sh lookout triage status [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | --------------------- | ------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--store` | string | — | path to the sentinel’s SQLite store (its —store file). Required: triage-status records live in the sentinel’s —store SQLite file; see docs/triage-status-write-design.md | | `--store-cluster` | string | — | read/write the store for THIS cluster, treating —store as the multi-cluster stem the sentinel was given: —store=/var/lib/lookout/lookout.db —store-cluster=prod-us opens /var/lib/lookout/lookout-prod-us.db (issue #410). Set it only against a sentinel running —clusters/—clusters-from; a single-cluster sentinel writes the literal —store path | | `--fingerprint` | string | — | the incident-class fingerprint from the inject payload or store row (sha256:…). Required to write; to read, this or —resource selects the record(s) | | `--resource` | string | — | resource key pinning the record to one object: \/\/\ (namespace segment empty for cluster-scoped objects, e.g. Node//gke-node-1). Required to write | | `--status` | string | — | triage state to record: investigating\|triaged\|actioned\|escalated (resolved is written by the sentinel’s recovery flip, never by agents). Empty = read mode: print the current record(s) instead of writing | | `--session` | string | — | incident session id that produced this record — the paper trail’s pointer back to the transcript | | `--root-cause` | string | — | root-cause hypothesis one-liner | | `--severity-override` | string | — | your routing judgment for further signals of this incident: critical\|warning\|info (empty = keep the signal’s own class). Honored by sentinel routing and health scans while the record is open | | `--action` | string | — | action taken / paper trail (“fix PR opened; config rollout pending”) | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `triage.status` | info | the triage record for an incident subject as it now stands — state, root-cause hypothesis, action, and who wrote it; a receipt, not a defect | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ------------------- | --------------------------------------------------------------------------------- | | `resource_key` | the record’s resource pin, as stored (\/\/\) | | `triage_status` | the record’s triage state (investigating\|triaged\|actioned\|escalated\|resolved) | | `triage_root_cause` | the recorded root-cause hypothesis | | `triage_action` | the recorded action / paper trail | | `triage_session` | the incident session that wrote the record | | `severity_override` | the recorded severity judgment (critical\|warning\|info), when one is set | | `updated` | when the record last changed, RFC 3339 | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage status --store=/var/lib/lookout/lookout.db --fingerprint=sha256:e2957792a0b3 --resource=Pod/prod/checkout-697567895d-2gglt --session=sess-0004 --status=triaged --severity-override=warning --root-cause="DB connection string invalid in checkout-config" --action="fix PR opened; config rollout pending" lookout triage status --store=/var/lib/lookout/lookout.db --fingerprint=sha256:e2957792a0b3 lookout triage status --store=/var/lib/lookout/lookout.db --resource=Pod/prod/checkout-697567895d-2gglt ``` # lookout triage top > Point-in-time CPU/memory saturation vs limits: kubectl top, but judged — usage-vs-limit percent per container with the OOM asymmetry built in (memory ≥95% of limit is critical, CPU caps at warning: it throttles, it does not kill); -A adds node usage vs allocatable. Trends/ETAs live in the sentinel's saturation source; --history adds window stats via the cloud provider. Point-in-time CPU/memory saturation vs limits: kubectl top, but judged — usage-vs-limit percent per container with the OOM asymmetry built in (memory ≥95% of limit is critical, CPU caps at warning: it throttles, it does not kill); -A adds node usage vs allocatable. Trends/ETAs live in the sentinel’s saturation source; —history adds window stats via the cloud provider. MCP tool: `k8s_resource_top` ## Usage [Section titled “Usage”](#usage) ```sh lookout triage top [flags] ``` ## Flags [Section titled “Flags”](#flags) | Flag | Type | Default | Meaning | | -------------------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--top-warn` | int | `80` | report a container/node only at or above this percent of its limit/allocatable (zero nominal state; memory additionally turns critical at 95%) | | `--all` | bool | — | exploratory dump: emit every sampled row regardless of —top-warn (info severity below it), sorted by pct descending; containers capped at —limit | | `--limit` | int | `50` | row cap for the —all container dump | | `--show-unlimited` | bool | — | list each container missing a cpu or memory limit individually (default: one aggregate count) | | `--show-unrequested` | bool | — | list each container missing a cpu or memory request individually (default: one aggregate count); a missing request is the scheduler-side half of the census, always a subset of —show-unlimited | | `--history` | duration | — | enrich container findings with max/avg/p95 usage-vs-limit over this window via the cloud provider metrics backend; no provider → explicit unavailable finding + summary marker, point-in-time output unaffected | ## Common flags (every `lookout` command) [Section titled “Common flags (every lookout command)”](#common-flags-every-lookout-command) | Flag | Type | Default | Meaning | | -------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--namespace` | string | — | limit the scan to one namespace | | `-A` | bool | — | scan all namespaces | | `--workload` | string | — | target one workload as \/\/\, e.g. Deployment/prod/api | | `--since` | duration | — | how far back to look (0 = command default) | | `--format` | string | `logfmt` | output format: logfmt\|json (one record per line either way) | | `--timeout` | duration | `10s` | abort the invocation after this long (exit 1) | | `--kubeconfig` | string | — | path to a kubeconfig file, instead of $KUBECONFIG / \~/.kube/config | | `--context` | string | — | kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=\ in the summary line | | `--exemptions` | string | — | path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=\ in the summary, never dropped | ## Finding kinds [Section titled “Finding kinds”](#finding-kinds) Every `kind=` this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the [finding-kind glossary](/k8s-lookout/reference/finding-kinds/) for the whole vocabulary. | Kind | Severity | Claim | | --------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `top.saturation` | critical, warning, info | a container’s usage is close to its limit — critical near the limit, info for an —all row below the threshold | | `top.node` | critical, warning, info | a node’s allocatable is close to committed — critical near the limit, info for an —all row below the threshold | | `top.unlimited` | info | how many containers in scope set no cpu/memory limit, and are therefore invisible to saturation analysis | | `top.unlimited_container` | info | one container that sets no cpu/memory limit (—show-unlimited) | | `top.unrequested` | info | how many containers in scope set no cpu/memory request, so the scheduler bin-packs them as zero | | `top.unrequested_container` | info | one container that sets no cpu/memory request (—show-unrequested) | | `cloud.unavailable` | info | the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence | ## Output fields [Section titled “Output fields”](#output-fields) Beyond the shared envelope fields (`kind`, `severity`, `namespace`, `kind_of_object`, `name`, `reason`, `message`, `fingerprint`, `exempt_reason`, `exempt_expires`): | Field | Meaning | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `resource` | the judged dimension: cpu or memory | | `usage` | current usage in the dimension’s natural unit (millicores for cpu, IEC bytes for memory) | | `limit` | the container’s configured limit (top.saturation), same unit as usage | | `allocatable` | the node’s allocatable capacity (top.node), same unit as usage | | `pct` | usage as a percent of the limit/allocatable, one decimal | | `container` | container name within the pod | | `node` | node the pod runs on (top.saturation; top.node carries the node as name) | | `pods` | top.unlimited/top.unrequested: pods in scope with at least one container missing a cpu or memory limit (resp. request) | | `containers` | top.unlimited/top.unrequested: containers in scope missing a cpu or memory limit (resp. request) | | `missing` | top.unlimited\_container/top.unrequested\_container: which dimensions are absent (cpu, memory, or both) | | `limitrange` | top.unlimited\_container/top.unrequested\_container: the namespace LimitRange(s) that default a dimension this container is missing — the pod predates them, so recreating it picks the value up | | `limitrange_defaulted` | top.unlimited/top.unrequested: how many of the counted containers sit in a namespace whose LimitRange now defaults the dimension they lack | | `max_pct` | —history: highest usage-vs-limit percent observed in the window | | `avg_pct` | —history: mean usage-vs-limit percent over the window | | `p95_pct` | —history: 95th-percentile usage-vs-limit percent over the window | | `capability` | cloud.unavailable: the provider capability —history needed (metrics) | | `provider` | cloud.unavailable: the provider that was asked | | `history` | summary-line note: the —history window the stats cover | | `unavailable` | summary-line note: why —history could not be served | ## Output contract [Section titled “Output contract”](#output-contract) Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage. ## Examples [Section titled “Examples”](#examples) ```sh lookout triage top --namespace=prod lookout triage top -A lookout triage top --workload=Deployment/prod/api lookout triage top --namespace=prod --all --limit=20 lookout triage top -A --top-warn=90 --show-unlimited lookout triage top -A --show-unrequested lookout triage top --namespace=prod --history=1h --format=json ``` # lookout watch > The resident per-cluster sentinel: every flag, derived from the live flag surface. `lookout watch` is the watch-path half of the binary: a resident per-cluster sentinel that turns leading indicators into per-incident agent sessions on a core-agent daemon. ## Usage [Section titled “Usage”](#usage) ```sh lookout watch [flags] ``` Signal sources are individually enabled via `--sources`; the default, `auto`, probes each portable source’s needs at startup and enables what the deployment supports. Flags of a disabled source are still validated — a nonsensical value is a config error in every mode. ## Flags [Section titled “Flags”](#flags) The table is generated from the sentinel’s real flag declarations (`internal/watch.FlagInventory`), sorted by name. The core flag surface is pinned by `TestFlagSurfaceFrozen`: removing or renaming one of those is a breaking change to running deployments, never a refactor. | Flag | Type | Default | Meaning | | ------------------------------------------ | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--access-recheck` | duration | `2m0s` | How often to re-run the SelfSubjectAccessReview probe over every enabled source’s declared access, so a grant revoked AFTER startup surfaces as a kind=sentinel.access\_revoked signal instead of a silently empty watch. A denial must repeat across two consecutive sweeps before it counts, so IAM propagation does not read as a revocation. Losing a REQUIRED permission stops this cluster’s runner (the same terminal path a startup refusal takes); an optional one degrades loudly and keeps running. 0 disables the re-check. | | `--backoff-min-count` | int | `3` | Require the crash-loop family (canonical CrashLoopBackOff — kubelet’s repeating BackOff cycle) to reach this Event.Count before firing, so a transient startup blip that self-heals does not open a noise session. Image-pull backoff is gated separately by —imagepull-transient-min-count. 1 fires on the first event. | | `--burn-eta` | duration | `30m0s` | Budget-exhaustion projection inside this window fires token.burn at critical (with the linear forecast); clearance requires the ETA to recede beyond 2x this threshold. Must be > 0. | | `--burn-multiple` | float | `4` | Session token rate at or above this multiple of the cross-session trailing-median baseline (sustained 2 polls) fires token.burn at warning. Must be > 1. | | `--capacity-poll` | duration | `1m0s` | Poll interval for the capacity source’s cluster-autoscaler-status ConfigMap read, provider scale-decision query, and pending-pod age sweep. Must be > 0. | | `--cluster-name` | string | — | Human-readable cluster name included in every inject payload. | | `--clusters` | string | — | Multi-cluster: comma-separated name=endpoint pairs to watch from one process, e.g. prod-us=abc.us-central1.gke.goog,prod-eu=def.europe-west1.gke.goog. A bare endpoint derives a short name from its first DNS label. Mutually exclusive with —clusters-from; needs a Fleet-capable provider (-tags gke). Leave empty for the one-sentinel-per-cluster default. | | `--clusters-from` | string | — | Multi-cluster: discover the clusters to watch instead of listing them. A project, or project/location, queried via the cloud provider’s cluster API (GKE: Container API ListClusters over the project; needs a Fleet-capable provider, -tags gke). Or the reserved value kubeconfig (optionally kubeconfig:\) to watch one cluster per context in a kubeconfig instead — no cloud API and no build tag, so EKS, on-prem and kind fleets work; with no path it follows client-go’s search ($KUBECONFIG, colon-separated and merged, else \~/.kube/config), which is also how you select a subset. Mutually exclusive with —clusters. | | `--compute-class-dwell` | duration | `10m0s` | How long a compute class’s rank verdict must persist before the source raises a finding. The resolve dwell (30m) and the flap guard are not separately tunable. Must be > 0. | | `--compute-class-infer` | bool | `true` | Cross-check GKE’s ccc\_priority\_index node annotation by independently matching each node against its compute class’s priority rules, and count every disagreement (lookout\_leeway\_preference\_disagreement). The annotation ALWAYS wins either way — this is a check on `k8s-lookout`’s model of the rules, not an override of GKE’s answer — so the only thing turning it off buys is silence on a cluster where the matcher is known to be behind the rules people write. Turning it off also blinds the unmatched and ambiguous counters, which are how that gap is meant to become visible. | | `--compute-class-last-rank-ceiling` | float | `0.9` | Raise leeway.rank\_degraded when more than this fraction of a class’s windowed pod-time ran at its LEAST-preferred priority. The default of 0.9 admits only two readings — capacity at every better priority is chronically unavailable, or the priority list is in the wrong order — and both are findings. Pass 0 to turn the rule off, which is the right answer for a class whose last rung is the cheap capacity it was always meant to run on. | | `--compute-class-rank0-floor` | float | — | Raise leeway.rank\_degraded when less than this fraction of a class’s windowed pod-time ran at its most-preferred priority. OFF by default (0), because on a class with three or more priorities the same reading also describes an estate that is merely mixed, and only somebody who knows what their class was for can say which. Set it when the first priority is the one that matters — a reservation, or the only family your licence covers. | | `--compute-class-tier-c-signals` | bool | — | Put Tier C compute-class findings on the wire. There is one: leeway.rank\_tier\_unused, a whole priority nothing has occupied for thirty days — a dead rung on the ladder, or reserved capacity being paid for and never drawn on. It is exported as a metric by default and not as a signal, because an unused priority is frequently the intended configuration and the finding is a bill to look at rather than a page. The rank\_wedged, rank\_degraded and rank\_no\_migration kinds always signal. | | `--compute-class-window` | duration | `1h0m0s` | How much recent history a compute class’s rank shares are a share of. The counters are cumulative, so a share has to be taken over a bounded window or a bad week in March goes on reporting in June. Longer smooths a class whose capacity comes and goes; shorter notices a fallback sooner and reads a rolling estate replacement as one. Composes with —compute-class-dwell rather than duplicating it: the window decides whether the condition is true now, the dwell decides whether it has been true long enough to say. Must be > 0. | | `--daemon-url` | string | — | Base URL of the core-agent daemon (http\://… or https\://…). Required. | | `--dedup-persist` | string | — | Optional path to persist dedup cache across sidecar restart. In multi-cluster mode this is a stem: each runner gets its own file, suffixed with the cluster’s project/location/name, so snapshots never clobber each other. | | `--dedup-window` | duration | `5m0s` | Rolling window for (uid,reason) dedup. | | `--degradation-drop` | float | `0.3` | Minimum ready-ratio decline from window start (with >= 2 distinct downward steps) that fires degradation.capacity. Must be in (0, 1]. | | `--degradation-window` | duration | `15m0s` | Trend window for the degradation source’s ready-ratio series and probe-flap counting. Must be > 0. | | `--distill-interval` | duration | `6h0m0s` | How often the distiller pass converts recurring occurrences into durable memory facts (requires —store; the pass reads the last 7d of occurrences). 0 disables distillation. Must be >= 0. | | `--dry-run` | bool | — | Watch the cluster for real (informers, sources, filter/dedup/routing all run) but print inject payloads to stdout instead of calling the daemon/sink. Needs cluster access like a normal run. | | `--enrich` | string | `critical` | Which severities get enrichment on their per-incident session’s initial inject: critical (default), warning (critical+warning), or off. | | `--enrich-cap` | int | `4096` | Byte budget for the attached enrichment bundle (fixed budget). Kept under —inject-max-bytes so the bundle plus the rest of the payload clears the daemon’s per-inject ceiling with headroom for the double-JSON envelope. Truncation happens at section boundaries; dropped sections become overflow trailers naming the `lookout` command that reproduces them. | | `--enrich-lists` | string | `all` | Which cluster resources the scoped-list enrichment fallback reads: ‘all’ (default), a comma-separated allowlist (pods,deployments), or subtractions (all,-secrets) to keep the watcher SA least-privilege. Denied or deselected lists degrade to a partial bundle with a skipped= note on the head, never a resolve failure. | | `--enrich-lists-preflight` | bool | — | Before the scoped-list pass, SelfSubjectAccessReview each selected resource and drop the denied ones proactively (fewer 403s in the watcher log); falls back to reactive Forbidden-skip if SSAR is not permitted. | | `--enrich-log-lines` | int | `200` | Log tail per container stream distilled into the enrichment bundle’s logs section. Must be >= 1. | | `--enrich-timeout` | duration | `5s` | Hard wall-clock budget for one enrichment run; on expiry the inject fires with whatever sections completed plus enrichment\_error trailers. Must be > 0. | | `--exclude-namespace` | string | — | Comma-separated deny-list of namespaces. Scopes the watch: the informers list and watch with a metadata.namespace!= field selector, so these namespaces never enter the cache. Cluster-scoped objects (nodes) are unaffected. | | `--expiry-interval` | duration | `1h0m0s` | Interval between expiry scans (periodic paged LISTs — deliberately no Secret informer). Must be > 0. | | `--expiry-namespaces` | string | — | Comma-separated namespaces the expiry scan LISTs secrets/serviceaccounts/Certificates in. Empty = all namespaces. Scopes the sensitive secrets-list grant — the startup RBAC probe verifies exactly this scope. | | `--expiry-warn` | duration | `336h0m0s` | Warning threshold for expiry.warning: certificates with notAfter inside this window fire at warning severity (critical at the design-fixed 72h). Must be >= 72h. | | `--gateway-grace` | duration | `5m0s` | How long a Gateway/HTTPRoute status condition (Programmed/Accepted/ResolvedRefs=False, reason != Pending) must be sustained — timed from its lastTransitionTime — before gateway.programming\_failed / gateway.route\_rejected fires. Absorbs normal LB provisioning latency. Must be > 0. | | `--graph-snapshot-interval` | duration | `5m0s` | How often to persist a compressed topology snapshot to —store (the per-delta change log is written continuously). Effective only with —store AND storm correlation on (the graph feed). Must be > 0. | | `--imagepull-transient-min-count` | int | `3` | Require an image-pull failure whose cause is RETRYABLE (registry 429/quota, 5xx, timeout, connection reset) to reach this Event.Count before firing, so a rate limit kubelet clears on its own does not open a noise session. Terminal causes (bad tag, denied, no space) and unrecognized ones still fire on the first event. 1 fires on the first event. | | `--in-cluster` | bool | — | Use in-cluster service account credentials. Auto-detected inside a pod. | | `--inject-max-bytes` | int | `8192` | Per-inject wire-body ceiling the dispatcher fits payloads to before POSTing (default matches the core-agent daemon’s 8192-byte limit). An over-limit payload is shrunk least-signal-first — enrichment dropped, then message truncated — never identity, so the incident still routes; without this the daemon 400s the whole inject and a new incident lands as an empty session (issue #198). | | `--kubeconfig` | string | — | Explicit kubeconfig path. Used outside a pod. | | `--log-level` | string | `info` | One of: debug, info, warn, error. | | `--metrics-addr` | string | — | Prometheus /metrics + /healthz + /readyz listener address (host:port). Empty = disabled. | | `--mode` | string | `per-incident` | Session routing mode: per-incident (create per (uid,reason)) or shared (all to —target-session). | | `--namespace` | string | — | Comma-separated allow-list of namespaces. Empty = all namespaces. A post-watch output filter, NOT a watch scope or a security boundary: every namespace is still listed, watched and cached. Use —exclude-namespace to shrink what is watched. | | `--notifications-subscription` | string | — | Subscription the notifications source reads (GKE: a Pub/Sub subscription on the cluster’s notificationConfig topic) — either projects/\

/subscriptions/\ or a bare name resolved against the provider project. Required when the notifications source is enabled. | | `--otel-exporter` | string | `none` | OpenTelemetry span exporter: none \| console \| otlp. The OTEL\_TRACES\_EXPORTER env var overrides this. | | `--owner` | string | — | X-Asserted-Caller value for POST /sessions in per-incident mode. Sidecar must be in daemon’s proxy\_identities. | | `--pending-age` | duration | `5m0s` | How long a pod must be Pending+Unschedulable before capacity.pending-aged fires at warning (critical at the design-fixed 15m, or at this value when set higher). Must be > 0. | | `--project` | string | — | Cloud project/account the cluster runs in, stamped into payloads. Empty = detect from the cloud provider’s metadata when a provider is compiled in; vanilla clusters can set it explicitly. | | `--quota-poll` | duration | `15m0s` | Poll interval for the quota source’s inventory read and per-watched-quota history query. Must be > 0. | | `--quota-warn` | float | `0.8` | Usage/limit ratio above which a quota is always watched (history fetched every poll) in addition to the top-10 nearest exhaustion. Must be in (0, 1). | | `--quota-window` | duration | `168h0m0s` | History window the quota usage slope is fitted over (the linear-\ confidence basis); a forecast needs usage points spanning at least half of it. Must be > 0. | | `--reason` | string | — | Comma-separated allow-list of Event.Reason values. Empty = shipped default set. | | `--recovery-stable-for` | duration | `5m0s` | How long a cleared symptom must stay clear before kind=resolved is injected into the incident’s session; recurrence within this window after a resolve fires kind=resolved.reverted. 0 disables recovery tracking. | | `--region` | string | — | Region the cluster runs in, stamped into payloads. Set for zonal and regional clusters alike. Empty = detect from the cloud provider’s metadata when a provider is compiled in; vanilla clusters can set it explicitly. Setting either —region or —zone stops metadata detection for BOTH — a half-flagged location would mix a flag’s region with a metadata zone somewhere else. | | `--rollout-observe` | duration | `3m0s` | How long a new revision must make zero ready-count progress (while the old revision stays healthy) before rollout.stall fires. Fired well before progressDeadlineSeconds. | | `--saturation-interval` | duration | `30s` | Sampling interval for the saturation source (metrics.k8s.io + kubelet volume stats). | | `--saturation-warn` | duration | `1h0m0s` | Forecast ETA below which saturation.forecast fires at severity warning (critical fires below 15m); clearance requires the ETA to recede beyond 2x this threshold. | | `--saturation-window` | duration | `1h30m0s` | Regression window for saturation forecasts; a forecast needs samples spanning at least half of it (the linear-\-window confidence basis). | | `--severity` | repeatable | — | Per-kind severity override(s): kind=level\[,kind=level…] with level one of critical\|warning\|info. Repeatable and additive; overrides the source-stamped default for that kind. Each kind may appear at most once. | | `--sink` | string | `core-agent` | Agent sink receiving incident payloads: core-agent (default: POST /sessions + /sessions/\/inject against —daemon-url) or webhook (generic receiver: POST \/incidents opens an incident with the schema-v1 payload JSON as the body; POST \/incidents/\/events appends follow-ups). | | `--sink-token-env` | string | — | Env var name holding the bearer token the webhook sink sends as Authorization: Bearer. Optional (unset = unauthenticated POSTs); only valid with —sink=webhook. | | `--sink-url` | string | — | Base URL of the generic webhook receiver (no trailing slash). Required with —sink=webhook. https is STRONGLY recommended: plain http is allowed (remote receivers are the point) but warns loudly at startup — incident payloads and the bearer token ride unencrypted. | | `--snapshot-interval` | duration | `30s` | How often to persist the dedup cache when —dedup-persist is set. 0 = only on shutdown. | | `--sources` | string | `auto` | Comma-separated signal sources to enable, or auto (the default): probe the portable sources’ needs at startup — RBAC via SelfSubjectAccessReview, plus metrics.k8s.io presence for saturation — and enable what this deployment supports, skipping misses with one loud line each (k8s-events must pass; a sentinel that cannot watch events is misdeployed). Known sources: k8s-events, object-state, rollout, workload, autoscaling, saturation, degradation, expiry, capacity, ingress, gateway, topology-drift, compute-class, quota, notifications, token-burn. quota (project tier), notifications (needs —notifications-subscription), and token-burn (core-agent cost stack) are never auto-enabled. An explicit list keeps semantics: a named source’s missing REQUIRED grant is fatal (optional dimensions — saturation’s nodes/proxy PVC read — still degrade loudly instead, issue #145). | | `--store` | string | — | Path to the sentinel-local SQLite occurrence store, e.g. /var/lib/lookout/lookout.db — put it on the —dedup-persist volume. Every emitted signal is recorded with its routing outcome; info-severity signals are persisted instead of dropped. In multi-cluster mode this is a stem: each runner opens its own file, suffixed with the cluster name, and —store-max-mb bounds EACH one. Reach a fleet file from the CLI with the same stem plus —store-cluster. Empty (default) disables the store. | | `--store-max-mb` | int | `512` | Size bound for the occurrence store in MiB; when exceeded, the oldest occurrences are pruned first (loudly). Bounds EACH store, so a multi-cluster fleet may use this much per cluster — a deliberate choice over a fleet budget divided N ways, which would make one cluster’s retention depend on how many clusters discovery found. Must be >= 1. | | `--store-ttl` | duration | `720h0m0s` | Retention for stored occurrences (default 30 days); the prune loop deletes older rows. Must be > 0. | | `--storm` | string | `auto` | Storm correlation: auto (the default — probe the graph informers’ grants at startup: pods/nodes/replicasets list+watch; all present resolves on, a miss resolves off with one loud line naming the grant), on (fatal at startup when a grant is missing), or off. true/false are aliases for on/off; bare —storm is no longer valid syntax. When on, new incidents sharing a blast-radius key (nearest common topology ancestor) group into one kind=storm session. | | `--storm-cluster-fallback` | bool | `true` | Group simultaneous NODE failures that nothing else groups (issue #334) under a synthetic Cluster ancestor, so a fleet-wide outage is one page instead of one per node. Applies only to nodes carrying no topology.kubernetes.io/zone label (a zone key is the modelled answer and always wins) and is deliberately expensive to trigger: a fifth of the fleet, at least 3 nodes, all inside 20s, and the storm expires after 5 idle minutes. Set false to keep one session per node. Requires storm correlation on. | | `--storm-min` | int | `3` | Minimum incidents sharing a blast-radius key within —storm-window to form a storm. Must be >= 2. | | `--storm-mine` | bool | — | Also correlate on DISCOVERED keys (issue #225): when —storm-mine-min incidents in the window share an exact image reference, node or container, group them into one storm even though no topology ancestor or external dependency connects them. Off by default — a mined key is circumstantial, so it needs more members than a modelled one, and every mined storm names what it grouped on. Requires storm correlation on. | | `--storm-mine-min` | int | — | Minimum incidents sharing a mined attribute value to form a storm. 0 (the default) means auto: the larger of 5 and —storm-min. An explicit value must be >= —storm-min — a discovered key must never be cheaper to form than a modelled one. Effective only with —storm-mine. | | `--storm-window` | duration | `1m0s` | Second-level correlation window for storm formation. 0 disables correlation even with —storm=on. | | `--target-session` | string | — | Required when —mode=shared: SessionID to post all injects to. | | `--token-budget-usd` | float | — | Per-session spend budget in USD for the token-burn source’s critical trigger; 0 (default) = unknown, budget trigger disarmed. Lookout-side config because core-agent v2.7.0 does not expose its CostCeiling over the attach API (TODO(core-agent) in pkg/sources/tokenburn). Must be >= 0. | | `--token-endpoint` | string | — | Override base URL for the core-agent cost stack (default: —daemon-url — the boundary rides the same daemon the injector talks to). No trailing slash. | | `--token-env` | string | — | Env var name holding the bearer token for the daemon. Required. | | `--token-poll` | duration | `1m0s` | Poll interval for the token-burn source’s cost-stack reads (core-agent GET /sessions + per-session /usage). Must be > 0. | | `--topology-baseline-band` | float | `4` | How many learned deviations wide a baseline’s tolerance band is — the false-positive knob for Tier C. A subject breaches when a domain’s share leaves the band around what it learned. Raise it if learned baselines are noisy on your cluster; the default is deliberately wide, because Tier C is the tier where nobody asked to be watched. Must be > 0. | | `--topology-baseline-half-life` | duration | `12h0m0s` | How long a learned baseline takes to half-absorb a step change in placement. Shorter follows a cluster that is legitimately rebalancing and stops calling it drift; longer keeps a longer memory of normal and so keeps noticing a slow slide that a short half-life would quietly adopt as the new normal. Must be > 0. | | `--topology-capacity-ratio` | float | `1.25` | How unequal a subject’s eligible zones have to be, as a max/min ratio of allocatable CPU, before an even split stops being the expectation and capacity does. Three zones where one is a quarter the size of the others cannot hold a third of anything; scoring them evenly reports drift on a cluster behaving exactly as its shape requires. Applies only where nobody declared a weighting — a policy that names one is always honoured. Raise it to keep the even expectation on mildly uneven clusters; below 1 it fires on any inequality at all. Must be > 0. | | `--topology-cluster-defaults` | string | — | Your cluster’s kube-scheduler PodTopologySpread defaultConstraints, as `key=maxSkew[:DoNotSchedule\|ScheduleAnyway]` comma-separated — they are not readable from a managed control plane, so leeway cannot find them out. THREE STATES: leave this unset and the upstream system defaults are ASSUMED (every intent from them is labelled source=cluster-default-assumed and can never raise a critical finding); pass “none” to assert your cluster configures none; or name them to be scored against your real numbers. These only ever apply to pods that declare no topologySpreadConstraints of their own. | | `--topology-domain-unavailable-keys` | string | `topology.kubernetes.io/zone,topology.kubernetes.io/region` | Comma-separated topology axes on which a domain with no usable node raises leeway.domain\_unavailable. The subject is the domain, so a zone that goes away is one finding for the cluster instead of one per workload that drifted because of it, and the workloads stay suppressed. Defaults to zone and region. Only name axes whose domains hold many nodes: an axis that is unique per node — kubernetes.io/hostname is one, and is a topology key — would raise a finding per NotReady node, which is objectstate’s job and not this one. Pass an empty string to turn the detector off. | | `--topology-dwell` | duration | `10m0s` | How long a placement breach must persist before the topology-drift source raises a finding. Placement is rebuilt constantly — by rollouts, by the descheduler, by a drain — so the dwell is what separates drift from motion. Shorter pages you during a routine rollout; the resolve dwell (30m) and the flap guard are not separately tunable. | | `--topology-keys` | string | `topology.kubernetes.io/zone,topology.kubernetes.io/region` | Comma-separated node labels the topology-drift source treats as topology axes, in precedence order. The defaults are the two standard well-known labels; a cluster that partitions on something else (a rack or cell label) names it here. | | `--topology-learn-baselines` | bool | `true` | Learn each workload’s normal placement, so that a workload which declared no spread constraint is scored against what it actually does instead of against an even split. Learning is passive and cheap: it samples every subject once a minute, matures after 6h, and only ever applies where nothing else expressed an intent — it cannot override a declared constraint. Turn it off to score every undeclared workload against an even apportionment, which is what happened before this existed. | | `--topology-max-node-groups` | int | `200` | How many node groups may be tracked before leeway tracks none of them. Past the bound nothing is scored and nothing is exported except the discovered count, which is the reading that sends you to —topology-node-group-keys; truncating instead would score an arbitrary subset that changes every pass. Pass a negative value to turn node-group subjects off entirely. | | `--topology-node-group-keys` | string | `cloud.google.com/compute-class,karpenter.sh/nodepool,eks.amazonaws.com/nodegroup,cloud.google.com/gke-nodepool,kops.k8s.io/instancegroup,agentpool` | Comma-separated node labels a node group’s name is read from, in precedence order, first match wins (FR-3). Node groups are tracked as subjects of their own, so a pool that was configured for three zones and has all its nodes in one is one finding naming the pool rather than one per workload riding it. Compute class comes before node pool in the default because auto-provisioned pools are named per machine type and are numerous and short-lived. A label that is unique per node turns every node into a group — see —topology-max-node-groups, which is what stops that reaching your metrics. | | `--topology-per-domain-collapse-states` | bool | `true` | Halve the per-domain series count by folding the four scheduling states onto two labels: state=“active” for an object holding the domain’s capacity (running or terminating) and state=“waiting” for one that is not (pending or unschedulable). The distinction needs between the four is upstream of the metric and is unaffected — this changes the label, not the count or any finding. Turn it off to get running/pending/unschedulable/terminating back, at twice the series. | | `--topology-per-domain-exclude-namespaces` | string | — | Comma-separated namespaces whose subjects never export a per-domain breakdown, applied AHEAD of —topology-per-domain-namespaces — naming a namespace in both excludes it. This is the knob for the one churning namespace that dominates the series count; it does not stop the namespace being watched, scored or alerted on, only its per-domain breakdown being exported. | | `--topology-per-domain-max-keys` | int | `4` | How many topology axes one subject may contribute a per-domain breakdown on. A cluster that names five or six axes in —topology-keys multiplies every admitted subject’s series by that many, and the axes past the first two or three are almost never the one being read. Which survive is the —topology-keys precedence order, so the cap is stable across scrapes rather than following whichever axis drifted. What it dropped is counted by lookout\_leeway\_domain\_series\_withheld{reason=“key\_cap”}. Pass a negative value for no cap. | | `--topology-per-domain-min-drift` | float | `0.05` | Drift (ρ, the fraction of a subject’s objects that would have to move) at which a subject’s per-domain breakdown is exported anyway. The default keeps the breakdown for the subjects somebody is about to investigate and withholds it for the rest, which is what makes the standing cost the aggregate one. Pass a negative value for every scored subject; —topology-per-domain-series overrides this entirely. | | `--topology-per-domain-namespaces` | string | — | Comma-separated namespaces whose subjects may export a per-domain breakdown. Empty — the default — admits every namespace, so this is the narrowing knob for a cluster that wants the breakdown standing for the namespaces it cares about and the aggregate everywhere else. Cluster-scoped subjects (node groups, and the domains themselves) are never filtered by this: they belong to no namespace, and dropping them would silently remove the cluster-wide reading. What it dropped is counted by lookout\_leeway\_domain\_series\_withheld{reason=“namespace”}. | | `--topology-per-domain-series` | bool | — | Export lookout\_leeway\_domain\_objects and lookout\_leeway\_domain\_expected for EVERY tracked subject, not only the drifting ones. OFF by default because the count is multiplicative: roughly 480k series on a 20k-subject cluster, against \~3.5k for every other leeway metric combined. See —topology-per-domain-min-drift for what you get without it. Turn this on to debug one cluster’s placement, not as a standing posture. | | `--topology-tier-c-signals` | bool | — | Put Tier C topology-drift findings on the wire. Tier C is the tier where nobody declared anything: the workload expressed no spread constraint or anti-affinity, so it was scored against an even apportionment over the domains it can reach — or, once one is learned, against its own baseline — and a breach says “this changed” rather than “this is wrong”. Those findings are exported as metrics only by default. Tiers A and B — a declared contract, or an intent inferred from what the workload does say — always signal. | | `--triage-regress-factor` | int | `3` | A downgraded incident (severity\_override) whose dedup-window count reaches this multiple of its count at downgrade time gets ONE kind=triage.regressed evidence followup into its bound session — never an automatic re-page (docs/triage-status-write-design.md). Must be >= 2; 0 disables. | | `--unhealthy-min-count` | int | `3` | Require this many consecutive Unhealthy events before firing. | | `--watchboard-batch` | int | `5` | Buffered warning-class signals that trigger a watchboard digest flush (per-incident mode). Must be >= 1. | | `--watchboard-flush` | duration | `1m0s` | Maximum age of a buffered warning before the watchboard digest flushes regardless of batch size. Must be > 0. | | `--watchboard-rotate` | int | `200` | Digest injects per watchboard session before size-based rotation opens a fresh session. Must be >= 1. | | `--zone` | string | — | Zone the cluster runs in, stamped into payloads. Set it only for a ZONAL cluster: a regional cluster has no zone of its own (its nodes are spread across the region’s zones), and leaving it empty there is the correct answer, not a gap. The failure domain — this zone when set, else —region — is what enters the signal fingerprint hash, so a deployment that stamps neither produces domain-less fingerprints: stable, but cross-cluster joins within a failure domain need one stamped. | See also: [Signal kinds](/k8s-lookout/reference/signal-kinds/) — everything the sentinel can inject; [Prometheus metrics](/k8s-lookout/reference/metrics/) — the `--metrics-addr` surface.