Runtime Protection Best Practices

Pre-deploy scanning stops known-bad images, but it cannot see exploitation, drift, or zero-days once a container runs. This list covers how to detect and contain what scanning misses.

How to Use This List

Work top to bottom the first time; each group builds on the one before it.

Treat these as a maturity path, not a checklist to complete in a day.

Groups A and B are the prevention baseline. Groups C through E are the runtime detection and response layers that assume something got through.

A - Harden Before You Detect

• Run as non-root with a read-only root filesystem. Set runAsNonRoot: true and readOnlyRootFilesystem: true so an exploited process cannot write to the image or escalate easily.
• Drop all Linux capabilities, then add back only what is needed. capabilities.drop: ["ALL"] shrinks the attack surface before any detection runs.
• Enforce restricted Pod Security Standards. Label namespaces with pod-security.kubernetes.io/enforce: restricted so unsafe Pods never admit in the first place.
• Apply a seccomp profile. Set seccompProfile.type: RuntimeDefault to block dangerous syscalls at the kernel, reducing what a compromised container can attempt.
• Use default-deny NetworkPolicies. Start every namespace with deny-all ingress and egress, then allow only required flows - drift then stands out clearly.

B - Keep Scanning, But Continuously

• Do not rely only on pre-deploy scans. A scan is a point-in-time result; new CVEs appear against images you already shipped, so re-scan running workloads.
• Run a continuous in-cluster scanner. The Trivy Operator scans deployed images and writes vulnerabilityreports you can query and alert on.
• Gate admission on fresh scan results. Use Kyverno or Gatekeeper to deny Pods whose images exceed your critical-CVE threshold at deploy time.
• Keep SBOMs for every image. When a new CVE lands, query SBOMs to find affected running images instead of re-scanning the whole fleet.
• Verify signatures at admission. Require cosign-signed images so a clean-but-tampered image cannot slip past scanning.

C - Detect Behavioral Drift at Runtime

• Deploy Falco as a per-node DaemonSet. One eBPF agent per node observes process, file, and syscall events across every container without per-Pod sidecars.
• Alert on shells in production containers. An interactive shell in a service that should never spawn one is high-signal drift worth paging on.
• Watch for writes to sensitive paths. File writes under /etc, /bin, or /usr in a running container almost always indicate compromise or misconfiguration.
• Flag unexpected outbound connections. A workload connecting to a new external host can indicate exfiltration or a command-and-control channel.
• Detect use of the container runtime socket. Access to the containerd or Docker socket from a workload is a classic escape vector - alert on it.

D - See the Network, Not Just the Host

• Use Cilium Hubble for flow visibility. Its eBPF datapath exports every flow with a verdict, so you can tell a policy drop from a DNS or routing failure.
• Investigate incidents by verdict. Filter Hubble flows to DROPPED and read the drop reason before assuming the application is at fault.
• Enable L7 visibility only where needed. Scope HTTP/gRPC/DNS parsing to the services you are debugging, since L7 inspection adds overhead.
• Correlate network and host signals. Pair Hubble network flows with Falco host events to reconstruct what an attacker did end to end.

E - Tune, Route, and Respond

• Tune rules with exceptions, not deletions. Silence known-good activity using scoped macros and exception conditions rather than disabling whole rules.
• Route alerts by priority. Send CRITICAL and WARNING to a pager via Falcosidekick; send informational events to a log sink you review later.
• Watch the detectors themselves. Monitor Falco's dropped-event counters and Hubble Relay health so a blind spot does not go unnoticed.
• Rehearse a containment runbook. Decide in advance how you cordon a node, isolate a Pod with a deny-all policy, and capture forensics before restarting.
• Feed runtime findings back into shift-left. Every real detection should tighten a scan rule, an admission policy, or a base image so the same gap does not recur.

When You Are Done

You should be able to answer three questions quickly.

Can we prove what a container did last night - which processes ran and which hosts it contacted?

Would we get paged, not just logged, if a shell opened in a production Pod?

When a new CVE drops, can we list affected running images from SBOMs within minutes?

If any answer is no, revisit the group that covers it before adding more tooling.

FAQs

Isn't scanning enough if it's strict? No. Scanning only knows published CVEs. Zero-days, stolen credentials, and logic flaws pass clean scans but show up as runtime drift.

Does runtime protection prevent attacks or just detect them? Falco and Hubble primarily detect and alert. For enforcement, add seccomp, AppArmor, read-only filesystems, or an enforcing tool like Tetragon.

How do I avoid alert fatigue? Scope rules to namespaces, add exceptions for known-good behavior, and route by priority so only actionable events page a human.

Where do SBOMs fit in runtime protection? They bridge shift-left and runtime: when a CVE appears, SBOMs tell you which running images are affected without re-scanning everything.

Do these tools need privileged access? The node agents need elevated privileges to load eBPF programs. Treat them as trusted infrastructure and restrict who can edit those DaemonSets.

Related

• Shift-Left vs Runtime Security
• Scanning Basics
• Runtime Security
• eBPF Observability

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).