How the Docker Engine Works

Summary

Docker Engine is a client-server system: the docker CLI sends requests over a REST API to a long-running daemon (dockerd) that does the real work.
Insight: the CLI is just a thin API client. Nothing prevents you from calling the same API from a remote host, a script, or another tool.
Key Concepts: dockerd (the daemon), containerd (container lifecycle), runc (the OCI runtime that clones the process), BuildKit (the build backend), and shim processes that keep containers alive independent of the daemon.
When to Use: understanding this split is essential before you configure remote engines, rootless mode, or debug why a container survives a daemon restart.
Limitations/Trade-offs: the daemon runs as root by default and is a single point of control on the host. Rootless mode reduces that blast radius at some feature cost.
Related Topics: Docker contexts, engine configuration, and the containerd runtime that Kubernetes uses on nodes.

Foundations

Docker Engine is not one program. It is a stack of cooperating components with a clear division of labor.

At the top is the docker CLI, the binary you type commands into. It contains almost no container logic.

Below it sits dockerd, the Docker daemon. This long-running process exposes a REST API and owns image storage, networks, and volumes.

The CLI turns docker run nginx into an HTTP request against that API. By default the transport is a Unix socket at /var/run/docker.sock.

This means the client and the engine are decoupled. The CLI can target a local socket, a remote TCP endpoint over TLS, or an SSH tunnel to another host.

Underneath dockerd is containerd, a CRI-compatible daemon that manages the full container lifecycle: pulling images, unpacking snapshots, and supervising running containers.

containerd does not itself create the container process. It delegates that to runc, a small OCI-compliant runtime that sets up namespaces and cgroups and then executes the entrypoint.

Mechanics & Interactions

Walk through docker run -d nginx to see the pieces interact.

The CLI serializes the request and posts it to the daemon's /containers/create and /containers/{id}/start endpoints.

dockerd checks whether the nginx image exists locally. If not, it asks containerd to pull it from a registry.

containerd fetches the image layers, verifies digests, and stores them as content-addressed blobs, then prepares a snapshot (a writable filesystem view built from the read-only layers).

To start the container, containerd invokes runc. runc creates the Linux namespaces (PID, network, mount, UTS, IPC), applies cgroup limits, drops capabilities, and calls execve on the entrypoint.

Here is the subtle part: runc exits after the container process is running.

A small shim process (containerd-shim-runc-v2) stays behind as the container's parent. The shim holds the container's stdout/stderr and reports exit status back to containerd.

Because the shim is the parent, the container keeps running even if you restart dockerd. The daemon reattaches to the shim on startup rather than killing everything.

This is why systemctl restart docker does not necessarily stop your containers, and why the daemon can be upgraded with live containers using the right configuration.

Networking and volumes are handled by dockerd through pluggable drivers. The default bridge network is a Linux bridge with NAT rules the daemon programs into iptables/nftables.

Builds take a separate path. docker build hands the work to BuildKit, the default build backend in modern Engine. BuildKit computes a dependency graph of build steps, runs independent steps in parallel, and caches aggressively.

Advanced Considerations & Applications

The client-daemon split is the foundation for remote engines. A DOCKER_HOST pointing at ssh://user@buildhost runs all builds on a beefier machine while you type on a laptop.

Rootless Docker runs dockerd and the containers inside a user namespace, so the daemon does not need real root. This shrinks the attack surface but limits some networking and cgroup features.

The containerd boundary matters enormously for Kubernetes. Kubernetes never talks to Docker Engine. Kubelet talks directly to containerd (or CRI-O) through the Container Runtime Interface (CRI).

Docker Engine bundles its own containerd, but that instance is managed by dockerd, not by a kubelet. dockershim, the old adapter that let Kubernetes drive Docker, was removed in Kubernetes 1.24.

So the practical mental model is: use Docker for building images and local development, and let containerd via CRI run your pods in production.

Because both build and runtime speak OCI, an image built by BuildKit runs unchanged on a containerd node. The engine you build with and the runtime you deploy to are deliberately interchangeable.

For observability, remember every action is an API call. dockerd emits events you can stream with docker events, and its logs (journalctl -u docker) are where daemon-level failures surface.

Common Misconceptions

"Docker runs containers." Docker Engine orchestrates them, but runc actually creates the process and containerd supervises it.

"If I restart the daemon, my containers die." With live restore and the shim architecture, running containers survive a daemon restart.

"Kubernetes uses Docker to run pods." It has not since dockershim was removed. Nodes run containerd or CRI-O directly.

"A container is a lightweight VM." No. It is an ordinary Linux process isolated by namespaces and cgroups, sharing the host kernel.

"The CLI does the heavy lifting." The CLI is a thin API client. All state and work live in the daemon.

FAQs

Is Docker Engine the same as Docker Desktop? No. Engine is the Linux daemon and CLI. Desktop wraps a Linux VM plus a GUI for macOS and Windows.

Why does the daemon run as root? It manages namespaces, cgroups, and networking that traditionally require privileges. Rootless mode is the alternative.

What is the shim for? It decouples container lifetime from daemon lifetime, so upgrades and restarts do not kill workloads.

Can I use Docker images on Kubernetes? Yes. They are OCI images, which containerd on nodes runs natively.

Does BuildKit need the daemon? BuildKit can run inside the daemon or as a standalone buildkitd, but by default docker build uses the daemon's BuildKit.

How do I talk to a remote engine? Set DOCKER_HOST or use a Docker context pointing at an SSH or TLS endpoint.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

How the Docker Engine Works

Summary

Docker Engine is a client-server system: the docker CLI sends requests over a REST API to a long-running daemon (dockerd) that does the real work.
Insight: the CLI is just a thin API client. Nothing prevents you from calling the same API from a remote host, a script, or another tool.
Key Concepts: dockerd (the daemon), containerd (container lifecycle), runc (the OCI runtime that clones the process), BuildKit (the build backend), and shim processes that keep containers alive independent of the daemon.
When to Use: understanding this split is essential before you configure remote engines, rootless mode, or debug why a container survives a daemon restart.
Limitations/Trade-offs: the daemon runs as root by default and is a single point of control on the host. Rootless mode reduces that blast radius at some feature cost.
Related Topics: Docker contexts, engine configuration, and the containerd runtime that Kubernetes uses on nodes.