Skip to content

lookout stab drain

Before draining a node, list everything that will block the drain (PDBs at disruptionsAllowed=0) or be destroyed by it (bare pods, emptyDir data, single-replica workloads); —node details one node, -A means all nodes here (pods are always examined across all namespaces); scanned counts pods examined after the standard-drain skips (mirror/DaemonSet/completed pods).

MCP tool: k8s_drain_blockers

Terminal window
lookout stab drain [flags]
FlagTypeDefaultMeaning
--nodestring—analyze one node in detail: every blocker on it becomes its own finding. Exactly one of —node or -A (all-nodes summary) is required.
FlagTypeDefaultMeaning
--namespacestring—limit the scan to one namespace
-Abool—scan all namespaces
--workloadstring—target one workload as <Kind>/<namespace>/<name>, e.g. Deployment/prod/api
--sinceduration—how far back to look (0 = command default)
--formatstringlogfmtoutput format: logfmt|json (one record per line either way)
--timeoutduration10sabort the invocation after this long (exit 1)
--kubeconfigstring—path to a kubeconfig file, instead of $KUBECONFIG / ~/.kube/config
--contextstring—kubeconfig context to read, instead of its current-context. Selects a cluster for THIS invocation only — nothing is written back — so concurrent invocations can target different clusters. Reported as context=<name> in the summary line
--exemptionsstring—path to a git-reviewed exemption file (YAML); covered findings are ANNOTATED with their reason and expiry and counted as exempt=<n> in the summary, never dropped

Every kind= this command can emit, and the severities it carries them at. Nothing else appears in its output; a kind absent from a run means the check looked and found nothing. See the finding-kind glossary for the whole vocabulary.

KindSeverityClaim
drain.pdb_gridlockcriticala PodDisruptionBudget covering pods on this node allows zero disruptions: the eviction API refuses and the drain hangs
drain.bare_podwarninga pod on this node has no owner, so eviction deletes it permanently and nothing recreates it
drain.local_storagewarninga pod on this node has emptyDir volumes: the drain needs —delete-emptydir-data and the data is lost
drain.singletonwarninga pod on this node is the only replica of its controller — evicting it is an outage
drain.nodecritical, warningthe -A roll-up: this node is not cleanly drainable, with the blocker classes counted; critical when a PDB gridlock is among them

Beyond the shared envelope fields (kind, severity, namespace, kind_of_object, name, reason, message, fingerprint, exempt_reason, exempt_expires):

FieldMeaning
nodethe node the blocker sits on (stamped on every —node-mode finding)
podspods on the node covered by the gridlocked PDB
pod_namesnames of the covered pods, capped at 8 with a +N more tail
disruptions_allowedPDB status.disruptionsAllowed (always 0 in a gridlock finding)
current_healthyPDB status.currentHealthy
desired_healthyPDB status.desiredHealthy
volumesemptyDir volume names on the pod; memory-backed ones marked (medium=Memory)
workloadthe single-replica controller as <Kind>/<namespace>/<name>
replicasthe controller’s spec.replicas (always 1 in a singleton finding)
blockerstotal drain blockers on the node (also a —node-mode summary note)
pdb_gridlockgridlocked-PDB blocker count on the node (-A per-node finding; zero counts omitted)
bare_podsbare-pod blocker count on the node (-A per-node finding; zero counts omitted)
local_storageemptyDir blocker count on the node (-A per-node finding; zero counts omitted)
singletonssingle-replica blocker count on the node (-A per-node finding; zero counts omitted)
drainablesummary note (—node mode): yes when the node has no blockers, else no
nodessummary note (-A mode): nodes examined
blockedsummary note (-A mode): nodes with at least one blocker

Output: one finding per line (logfmt; —format=json for one JSON object per line), keys in fixed order; healthy resources emit nothing. The final line is always the summary: scanned= findings= elapsed= — findings=0 with a summary present means “scanned and healthy”; a stream without a summary line is void. Exit 0 data, 1 runtime error (diagnostics on stderr only), 2 usage.

Terminal window
lookout stab drain --node=gke-prod-pool-a-x1z2
lookout stab drain -A
lookout stab drain --node=gke-prod-pool-a-x1z2 --format=json