lookout health

“Any issues with this cluster?” in one call: a ten-category scorecard (control-plane, nodes, crash loops, pending, rollouts, storage, add-ons, quotas, certs, webhooks) — every category answers healthy|degraded|unavailable, degraded ones with details. With —store, findings merge the sentinel’s open triage-status records: a scan mid-incident reports the diagnosis and the agent’s severity judgment, not a fresh unknown.

MCP tool: k8s_cluster_health

Usage

lookout health [flags]

Flags

Flag	Type	Default	Meaning
--top	int	3	how many findings to name inline on a degraded category’s scorecard line
--cert-warn	duration	720h	report TLS certificates expiring within this window (certs category)
--store	string	—	path to a sentinel’s SQLite store (its —store file); merges open triage-status records so findings carry triage_* fields and severity reflects the agent’s override
--store-cluster	string	—	read/write the store for THIS cluster, treating —store as the multi-cluster stem the sentinel was given: —store=/var/lib/lookout/lookout.db —store-cluster=prod-us opens /var/lib/lookout/lookout-prod-us.db (issue #410). Set it only against a sentinel running —clusters/—clusters-from; a single-cluster sentinel writes the literal —store path

Common flags (every lookout command)

Flag	Type	Default	Meaning
--namespace	string	—	limit the scan to one namespace
-A	bool	—	scan all namespaces
--workload	string	—	target one workload as <Kind>/<namespace>/<name>, e.g. Deployment/prod/api
--since	duration	—	how far back to look (0 = command default)
--format	string	logfmt	output format: logfmt|json (one record per line either way)
--timeout	duration	10s	abort the invocation after this long (exit 1)
--kubeconfig	string	—	path to a kubeconfig file, instead of $KUBECONFIG / ~/.kube/config
--context	string	—	kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=<name> in the summary line
--exemptions	string	—	path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=<n> in the summary, never dropped

Finding kinds

Every kind= this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the finding-kind glossary for the whole vocabulary.

Kind	Severity	Claim
health.category	critical, warning, info	one scorecard line: how this category answered — healthy, degraded, or unavailable. The scorecard always answers, so healthy is explicit rather than silent; the line carries the worst severity found inside the category
pvc.pending	warning	a PersistentVolumeClaim is not bound; pods mounting it cannot start
pvc.lost	critical	a PersistentVolumeClaim’s bound volume is lost
cert.expired	critical	a TLS secret’s certificate has expired
cert.expiring	warning	a TLS secret’s certificate expires within —cert-warn
cert.invalid	warning	a TLS secret’s tls.crt does not contain a parseable X.509 certificate
pod.crashloop	critical	a container is crash looping
pod.imagepull	critical	a container cannot pull its image
pod.waiting	warning	a container is stuck in an error waiting state (CreateContainerConfigError, InvalidImageName, …)
pod.oomkilled	warning	a container’s last termination was an OOM kill
pod.restarts	warning	a container has restarted at least —restarts times
pod.notready	warning	a container in a Running pod has been not-ready past the —pending-age grace
pod.failed	warning	the pod reached phase Failed
pod.pending	critical, warning	the pod has been Pending longer than —pending-age with no container-level diagnosis; critical when the scheduler has declared it Unschedulable, which is a capacity or constraint problem rather than latency
workload.replicafailure	critical	the controller cannot create pods at all (quota, PodSecurity, admission) — no pod exists to diagnose
workload.stalled	critical	a Deployment’s Progressing condition is False: the rollout has given up
workload.rollout	critical, warning	replicas are short of desired; critical when nothing is serving at all
job.failed	warning	a Job’s Failed condition is set
cron.missed	critical, warning	an unsuspended CronJob’s schedule said to run more than —cron-grace ago and status says it did not; critical once several activations in a row are gone
cron.unparseable	warning	a CronJob’s spec.schedule could not be parsed, so its activations cannot be judged at all
node.notready	critical	the node’s Ready condition is not True
node.pressure	critical	the node reports Memory/Disk/PID pressure
node.condition	critical, warning	a non-standard node condition is True — NPD and its cousins publish problems that way
node.cordoned	warning	the node is unschedulable but still holds pods: a stuck drain or a forgotten maintenance step
node.preempt	critical, warning, info	a reclaim taint marks the node for termination; severity tracks how imminent
pdb.gridlocked	critical, warning	the budget permits no disruptions; critical when healthy pods are already below the required minimum
addon.degraded	critical, warning	a kube-system add-on (dns, proxy, cni, csi, metrics, connectivity) is short of replicas; critical when none are available
quota.near	warning	a ResourceQuota resource is at or past —quota-warn percent of its hard limit
quota.exhausted	critical	a ResourceQuota resource is at its hard limit: the next create is rejected
webhook.failing_closed	critical	the webhook has no working backend and failurePolicy=Fail: every gated write is rejected cluster-wide
webhook.dead_backend	warning	the webhook’s service backend is missing, has no ready endpoints, or does not serve the named port
webhook.slow_risk	info	the webhook’s timeout is long enough to slow every gated write if the backend degrades
webhook.ca_expired	critical	the webhook’s caBundle has expired: the API server cannot verify it
webhook.ca_expiring	warning	the webhook’s caBundle expires within —cert-warn
perf.apiserver_p99	critical, warning	apiserver request latency p99 crossed the pack threshold for a verb/resource — warning from 1s, critical from 4s
perf.pack_unavailable	warning	a metric the requested pack needs is not in the metrics workspace, so part of the pack could not run; the rest still did (no coverage lies)

Output fields

Beyond the shared envelope fields (kind, severity, namespace, kind_of_object, name, reason, message, fingerprint, exempt_reason, exempt_expires):

Field	Meaning
category	scorecard category the finding belongs to (on health.category: which category this line scores)
status	category status: healthy|degraded|unavailable (the scorecard always answers — healthy is explicit)
total	findings in a degraded category
top	worst findings of a degraded category inline, as kind[ namespace/name]; capped by —top
subject	TLS certificate subject (CN when set); never key material
not_after	TLS certificate NotAfter, RFC 3339
days_left	whole days until NotAfter (negative = expired)
phase	PersistentVolumeClaim phase on storage findings (Pending or Lost)
webhook	admission webhook as <configuration>/<webhook name>
service	service backend a webhook points at, as <namespace>/<name>
backend	why a webhook backend is dead: service missing, no ready endpoints, or port <p> not on service
gates	namespaces a webhook gates, from namespaceSelector: all namespaces, or <matched>/<total> namespaces with up to 5 names
rules	compact operations/resources summary of a webhook’s rules, e.g. “CREATE,UPDATE pods,deployments.apps”
object_selector	a webhook’s objectSelector, when one is set
timeout	webhook timeoutSeconds as <n>s (nil defaults to the API’s 10s)
triage_status	triage state from the matched record (investigating|triaged|actioned|escalated) — present only with —store on merged findings
triage_root_cause	the incident agent’s root-cause hypothesis, from the matched triage-status record
triage_action	the incident agent’s paper trail (PRs opened, escalations), from the matched triage-status record
triage_session	incident session that wrote the matched triage-status record
triage_age	how long ago the matched triage-status record was last updated
container	container the finding is about (init containers prefixed init:)
image	image reference that failed to pull
restarts	container restart count
exit_code	exit code of the container’s last termination
last_state	reason of the container’s last termination (e.g. OOMKilled)
age	how long the abnormal state has persisted
desired	desired replica/scheduled count from spec
ready	ready count from status
updated	updated-to-current-revision count from status
available	available count from status
failed	failed pod count of a Job
schedule	a CronJob’s spec.schedule
expected	the activation a CronJob should have run and did not
missed_runs	activations missed since the anchor; ≥N when the walk was capped
anchor	what the missed count was measured from: last_schedule or creation
time_zone	a CronJob’s spec.timeZone, when set
last_schedule	a CronJob’s status.lastScheduleTime, or never
active_jobs	Jobs a CronJob still has running
condition	node condition type that is abnormal
taint	taint key indicating reclaim/drain
pods	pods affected (behind a cordoned node or a PDB)
healthy	currently healthy pods behind a PDB
required	pods the PDB requires healthy
addon	system add-on role: dns, proxy, cni, csi, metrics, connectivity
resource	ResourceQuota resource name at or near its limit
used	quota usage from status
hard	quota hard limit from status
pct	quota usage as percent of the hard limit
pack	the pack this finding belongs to; also the summary-line note naming the pack that ran
metric	the backend-neutral metric the query measured (pack_unavailable: the absent metric)
verb	apiserver request verb for this series (apiserver pack)
observed	the worst (maximum) aligned value in the window, in the query’s unit — the breach basis
latest	the newest aligned value in the window
threshold	the crossed threshold: the critical one when severity=critical, else the warning one
window	the lookback the series cover (—since, or the pack default); also a summary-line note

Output contract

Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage.

Examples

lookout health
lookout health --format=json --top=5
lookout health --namespace=prod