lookout scan

lookout scan is the entry point for “something is wrong and I do not know what”: no target, no flags, nothing deployed. It runs every target-free incident check in one invocation, then drills into the dependency edges of whatever it flagged.

Every finding is stamped check=<command> — which is also the command to run for the detail behind it. That is why this page is grouped by stage rather than alphabetically: the heading is the follow-up call.

A kind absent from a run means the check looked and found nothing. A check that could not run says so explicitly, in the stream — see what scan says about itself at the bottom.

Stage 1 — the target-free incident checks

These run on every scan, in this order: the broadest check first, so the thing that is wrong is usually named before you finish reading.

lookout triage delta

Every abnormal object in one scan — the first call for “anything wrong in this cluster?”: broken/pending pods, stalled rollouts, workloads blocked from creating pods at all, node pressure/NPD/preemption, gridlocked PDBs, degraded kube-system add-ons, quotas at their limits.

Kind	Severity	What it means
pod.crashloop	critical	a container is crash looping
pod.imagepull	critical	a container cannot pull its image
pod.waiting	warning	a container is stuck in an error waiting state (CreateContainerConfigError, InvalidImageName, …)
pod.oomkilled	warning	a container’s last termination was an OOM kill
pod.restarts	warning	a container has restarted at least —restarts times
pod.notready	warning	a container in a Running pod has been not-ready past the —pending-age grace
pod.failed	warning	the pod reached phase Failed
pod.pending	critical, warning	the pod has been Pending longer than —pending-age with no container-level diagnosis; critical when the scheduler has declared it Unschedulable, which is a capacity or constraint problem rather than latency
workload.replicafailure	critical	the controller cannot create pods at all (quota, PodSecurity, admission) — no pod exists to diagnose
workload.stalled	critical	a Deployment’s Progressing condition is False: the rollout has given up
workload.rollout	critical, warning	replicas are short of desired; critical when nothing is serving at all
job.failed	warning	a Job’s Failed condition is set
cron.missed	critical, warning	an unsuspended CronJob’s schedule said to run more than —cron-grace ago and status says it did not; critical once several activations in a row are gone
cron.unparseable	warning	a CronJob’s spec.schedule could not be parsed, so its activations cannot be judged at all
node.notready	critical	the node’s Ready condition is not True
node.pressure	critical	the node reports Memory/Disk/PID pressure
node.condition	critical, warning	a non-standard node condition is True — NPD and its cousins publish problems that way
node.cordoned	warning	the node is unschedulable but still holds pods: a stuck drain or a forgotten maintenance step
node.preempt	critical, warning, info	a reclaim taint marks the node for termination; severity tracks how imminent
pdb.gridlocked	critical, warning	the budget permits no disruptions; critical when healthy pods are already below the required minimum
addon.degraded	critical, warning	a kube-system add-on (dns, proxy, cni, csi, metrics, connectivity) is short of replicas; critical when none are available
quota.near	warning	a ResourceQuota resource is at or past —quota-warn percent of its hard limit
quota.exhausted	critical	a ResourceQuota resource is at its hard limit: the next create is rejected

lookout state webhooks

When creates/updates hang or fail cluster-wide with “failed calling webhook”, or before relying on a policy engine: audit every admission webhook — dead backends × failurePolicy (Fail + dead backend rejects every matching admission), the namespace/rule blast radius, timeout stall risk, CA-bundle expiry. The full check; health’s webhooks category delegates here.

Kind	Severity	What it means
webhook.failing_closed	critical	the webhook has no working backend and failurePolicy=Fail: every gated write is rejected cluster-wide
webhook.dead_backend	warning	the webhook’s service backend is missing, has no ready endpoints, or does not serve the named port
webhook.slow_risk	info	the webhook’s timeout is long enough to slow every gated write if the backend degrades
webhook.ca_expired	critical	the webhook’s caBundle has expired: the API server cannot verify it
webhook.ca_expiring	warning	the webhook’s caBundle expires within —cert-warn

lookout state volumes

When pods hang in ContainerCreating with Multi-Attach or FailedAttachVolume events — join VolumeAttachment + PV/PVC + pods to name the exact conflict: RWO claims wanted on two nodes, attachments stuck in error, cross-zone PV locks, orphaned attachments.

Kind	Severity	What it means
volume.multi_attach	critical	an RWO claim is wanted by pods on more than one node — the second pod never starts
volume.zone_conflict	critical	the PV is locked to a zone the pod’s node is not in
volume.attach_error	critical, warning	the attach or detach is failing; critical once it has been failing long enough to be stuck rather than slow
volume.orphaned_attachment	info	a VolumeAttachment survives its PV or its node

lookout state storage

When a PersistentVolumeClaim sits Pending and the pod behind it will not schedule — name the reason: a StorageClass that does not exist, no class and no cluster default, a static-only class with nothing pre-provisioned, plus the default-class ambiguity and stranded volumes behind it.

Kind	Severity	What it means
storage.missing_class	critical	the claim names a StorageClass that does not exist — it will stay Pending forever
storage.no_default_class	critical	the claim names no class and the cluster has no default StorageClass
storage.no_provisioner	warning	the claim’s class is static-only (kubernetes.io/no-provisioner) and no matching PV is available
storage.multiple_defaults	warning	more than one StorageClass is annotated as the cluster default; which one wins is not defined
storage.pv_failed	warning	a PersistentVolume is Failed: its reclaim did not complete, so the backing disk stays allocated and the volume cannot be reused
storage.pv_released	info	a PersistentVolume is Released — retained on purpose, but its capacity is unusable until spec.claimRef is cleared

lookout state gateway

When traffic through the Gateway API does not arrive — walk GatewayClass → Gateway → listener → HTTPRoute → Service and report every hop that is rejected, unprogrammed, or points at something that is not there. Silent, and cheap, on clusters without the Gateway API installed.

Kind	Severity	What it means
gateway.missing_class	critical	the Gateway names a GatewayClass that does not exist — nothing will program it
gateway.class_not_accepted	critical	the Gateway’s GatewayClass is not Accepted by its controller
gateway.not_accepted	critical	the Gateway itself is not Accepted
gateway.not_programmed	critical	the Gateway is Accepted but not Programmed: no data plane is carrying its traffic
gateway.listener_invalid	warning	one listener of an otherwise working Gateway is not resolved or not programmed
route.missing_parent	critical	the route’s parentRef names a Gateway that does not exist
route.not_accepted	critical	the Gateway refused the route’s attachment (listener, hostname, or namespace policy)
route.missing_backend	critical	the route’s backendRef Service does not exist
route.backend_port	critical	the route’s backendRef Service exists but does not expose the named port
crd.unavailable	info	the API group this check reads is not served by the cluster, so nothing was examined (no coverage lies)

lookout state wi

When a GKE pod gets 403s or metadata-server errors calling GCP APIs, verify the Workload Identity chain — KSA annotation (iam.gke.io/gcp-service-account) → roles/iam.workloadIdentityUser binding on the GSA — reporting only the broken links; vanilla clusters report an explicit unavailable.

Kind	Severity	What it means
wi.gsa_missing	critical	the annotated Google service account does not exist — every GCP call from these pods fails
wi.unbound	critical	the KSA annotates a GSA but the roles/iam.workloadIdentityUser binding is missing or malformed
wi.unannotated_use	info	a pod sets GOOGLE_APPLICATION_CREDENTIALS but its ServiceAccount carries no Workload Identity annotation
cloud.unavailable	info	the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence

lookout stab drift

Find spec fields of Deployments/StatefulSets/DaemonSets owned by a manager other than the GitOps controller (managedFields) — out-of-band kubectl edits and rogue co-managers. Reports manager strings (tool names, not people); —identity additionally resolves each drift write to the audited principal via the cloud provider’s audit trail (GKE Cloud Audit Logs), reporting an explicit unavailable on clusters without one. Default scope: all namespaces; scanned counts workload objects examined.

Kind	Severity	What it means
drift.manual_edit	critical, warning	a manager other than the GitOps controller owns spec fields on this object; critical when one of them is high blast radius (image, replicas, env)

Stage 2 — the dependency-edge drill-down

Every workload stage 1 flagged at warning or above then has its dependency edges verified: one cluster List pass and N in-memory state edges evaluations, each rolled up to its outermost controller, so twenty crashlooping pods of one Deployment are one drill-down rather than twenty. --max-drilldown bounds it (default 20) and the summary reports what it dropped.

Kind	Severity	What it means
edge.missing_ref	critical	a referenced ConfigMap, Secret, ServiceAccount, TLS secret, IngressClass, StorageClass, or governing Service does not exist
edge.missing_key	critical	the referenced key is absent from an existing ConfigMap/Secret
edge.invalid_ref	warning	the referenced object exists but is the wrong type to serve the reference
edge.unclassed	warning	the Ingress names no class and no IngressClass declares itself the cluster default — no controller will claim it
edge.selector_empty	critical	a Service selector selects zero pods, so the service routes nowhere
edge.selector_unready	critical, warning	the Service selects pods but some are not Ready; critical when none are
edge.endpoints_missing	critical	a selecting Service has no EndpointSlices at all
edge.endpoints_orphaned	warning	an endpoint targetRef names a pod that no longer exists
edge.endpoints_unready	critical, warning	the endpoint ready-count disagrees with the selected pods (stale or lagging slices); critical at zero ready
edge.backend_missing	critical	an Ingress backend service, or the port it names, does not exist
edge.cert_expired	critical	a TLS certificate’s NotAfter is in the past
edge.cert_expiring	warning	a TLS certificate expires within —cert-warn
edge.cert_invalid	warning	tls.crt is missing or unparseable, or the secret is not kubernetes.io/tls
edge.rbac_dangling	warning	a (Cluster)RoleBinding for the workload’s ServiceAccount points at a missing (Cluster)Role

What --include adds

Three groups are left out of a bare scan, each for a reason that is a property of the whole group. --include=all takes every one; - subtracts (all,-cloud).

--include=audit

Best-practice posture: the absence of a safety net around a workload or cluster that is currently healthy — a different claim from the incident groups, which is why it is a different group

Its 24 kinds have their own page — what lookout audit checks.

--include=cloud

GCP-side reads: stockouts, orphaned resources, IP space, quota

lookout cloud ipspace

Pod/Service/node CIDR utilization per subnet, judged: warning at 80%, critical at 95% — IP space is incompressible, an exhausted range fails the next node or pod block outright. Consumption rate/ETA lives in the sentinel’s capacity source.

Kind	Severity	What it means
ipspace.range	critical, warning, info	a pod/service/node range is at 80% of its CIDR or worse; critical from 95%, info for a range the cloud APIs cannot rate and for an —all row below the line
cloud.unavailable	info	the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence

lookout cloud orphans

Billing-active cloud leftovers: unattached GCE disks older than —min-age and forwarding rules/LBs routing to zero endpoints — cost and hygiene sweep, not an incident read.

Kind	Severity	What it means
orphan.disk	warning	a GCE disk has been unattached for at least —min-age and is still billing
orphan.lb	warning	a forwarding rule or load balancer routes to zero endpoints and is still billing
cloud.unavailable	info	the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence

lookout cloud quota

Per-project cloud quota usage vs limit, ranked nearest-to-exhaustion: findings from —quota-warn (default 80%), critical at 95% — quota is incompressible (scale-ups fail at the limit) and increases need lead time. Trend/ETA lives in the quota source.

Kind	Severity	What it means
quota.pressure	critical, warning, info	a cloud quota is at or above —quota-warn percent of its limit; critical from 95%, info for an —all row below the line
cloud.unavailable	info	the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence

lookout cloud stockout

GCE capacity stockouts (ZONE_RESOURCE_POOL_EXHAUSTED) per zone/machine-type over —since (default 24h), with event-derived reroute candidates — the cloud-side why behind pods stuck Pending on failed scale-ups.

Kind	Severity	What it means
stockout.zone	warning	the cloud had no capacity for a machine type in this zone during the window — the reason a scale-up failed and pods stayed Pending
cloud.unavailable	info	the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence

--include=perf

Control-plane and startup performance via Cloud Monitoring query packs

lookout perf probe

Control-plane and startup performance via metrics query packs: —pack=apiserver (p99 latency by verb/resource), apf (queue saturation + 429 rejects), etcd (WAL fsync p99 + DB size), startup (pod-first-ready p95 trend); apf/etcd need GKE control-plane metrics enabled — absence degrades to an explicit pack_unavailable finding.

Kind	Severity	What it means
perf.apiserver_p99	critical, warning	apiserver request latency p99 crossed the pack threshold for a verb/resource — warning from 1s, critical from 4s
perf.apf_saturation	critical, warning	an API Priority and Fairness level is holding a sustained queue — warning from 10 queued, critical from 100
perf.apf_rejects	critical, warning	APF is shedding load: the apiserver is returning 429s at a priority level
perf.etcd_fsync	critical, warning	etcd WAL fsync p99 crossed the pack threshold — warning from 10ms, critical from 100ms
perf.etcd_db_size	critical, warning	the etcd database is approaching its quota — warning from 4 GiB, critical from 5.5 GiB
perf.startup_p95	critical, warning	pod first-ready p95 crossed the pack threshold — warning from 60s, critical from 300s
perf.pack_unavailable	warning	a metric the requested pack needs is not in the metrics workspace, so part of the pack could not run; the rest still did (no coverage lies)
cloud.unavailable	info	the cloud capability this check needs is unavailable, so nothing was examined — an explicit degradation record, never silence

What scan says about itself

A scan that could not run something reports that, rather than reporting a smaller cluster. These kinds are the coverage claim, and they are why an empty scan means “nothing is wrong” and not “nothing ran”.

Kind	Severity	What it means
scan.check_skipped	info	a stage declined this invocation because a zero-argument scan cannot supply something it needs — the coverage claim is smaller than it looks
scan.check_failed	warning	a stage errored; the scan continued without it, so this run saw less than a whole cluster — unless EVERY stage failed and none read anything, which is a runtime error (exit 1) rather than a scan
scan.incomplete	warning	the —timeout expired with stages still to run; not_run names them

See also

• lookout scan reference — flags, output fields, the full kind table in one list.
• What lookout audit checks — the posture half.
• What the sentinel watches — the things a one-shot command structurally cannot see.