How Image Layers & Union Filesystems Work

Summary

An image is an ordered stack of read-only layers; a union filesystem merges them into one view, and the container adds a writable top layer.
Insight: Layers are shared by content digest, so ordering decides both cache hit rate and how much unique data you ship.
Key Concepts: OverlayFS, lowerdir/upperdir/merged, copy-on-write, whiteout files, content-addressed layers.
When to Use: Reach for this model when shrinking images, speeding up builds, or explaining why a "deleted" file still bloats an image.
Limitations/Trade-offs: Copy-on-write is cheap to start but expensive on large in-place writes; deep layer stacks add lookup overhead.
Related Topics: How a Dockerfile becomes an image, multi-stage builds, distroless bases.

Foundations

Every image is a stack of layers. Each layer is a tarball of filesystem changes relative to the layer below it.

A layer records additions, modifications, and deletions - not a full filesystem each time. Stacking them reconstructs the complete rootfs.

Because each layer is addressed by the SHA-256 digest of its content, two images that share a base share the exact same layer bytes on disk and over the network. You pull a shared layer once.

The runtime presents this stack as a single directory tree using a union filesystem. On modern Linux that is almost always OverlayFS, exposed by containerd as the overlayfs snapshotter.

OverlayFS merges directories using four roles. The lowerdir entries are the read-only image layers, the upperdir is the writable container layer, a workdir is scratch space overlay needs, and merged is the unified view the process sees.

When the process reads a file, OverlayFS searches from the top layer down and returns the first match. Upper layers shadow lower ones, so a later layer's copy of a file wins.

Copy-on-write governs writes. Reading a file touches the lower layer directly; writing one first copies the whole file up into the writable layer, then edits the copy.

That copy-up is why writing one byte of a 2 GB file materializes the entire 2 GB in the container layer. It also means the original file still exists, untouched, in the image layer below.

Deletion uses whiteout files. Removing a file from a lower layer creates a special marker in the upper layer that hides it, rather than freeing the underlying bytes.

This is the single most important consequence for image size. If you add a large file in one layer and rm it in a later layer, both the file and its whiteout ship, so the image grows even though the file is invisible.

# Anti-pattern: secret and its removal both persist as layers
RUN curl -o /tmp/big.tar.gz https://example.com/big.tar.gz \
 && tar xf /tmp/big.tar.gz -C /opt
RUN rm /tmp/big.tar.gz   # whiteout only; the 500MB is still in the earlier layer

The fix is to add and remove within the same RUN, so the net change committed to that one layer excludes the temporary file.

RUN curl -o /tmp/big.tar.gz https://example.com/big.tar.gz \
 && tar xf /tmp/big.tar.gz -C /opt \
 && rm /tmp/big.tar.gz

Advanced Considerations & Applications

Layer ordering drives cache economics. The build cache reuses a layer only if its instruction and inputs are unchanged and every parent layer above it in the stack also matched.

So you place stable content low (base image, OS packages, dependencies) and volatile content high (application source). Editing source then invalidates only the top layers.

Shared base layers cut registry storage and node disk. A hundred services on node:22-slim store that base once per node, and containerd unpacks it once into the snapshotter.

Deep stacks are not free. Each read walks the layers, so hundreds of layers add path-lookup overhead, and Docker historically capped stacks around 127 layers. Consolidating related RUN steps keeps stacks shallow.

At runtime, emptyDir and volumes bypass the union filesystem entirely. Write-heavy workloads should target a volume so hot writes do not thrash copy-on-write in the container layer.

On Kubernetes nodes, containerd manages these layers through its snapshotter. The mechanics are identical to Docker because both consume OCI images and both default to OverlayFS on Linux.

Common Misconceptions

"Deleting a file shrinks the image." Only if the deletion happens in the same layer that added the file. Otherwise a whiteout hides it while the bytes remain below.

"The writable layer is where reads happen." Reads come from wherever the file first appears top-down, usually a read-only image layer. Only writes populate the upper layer.

"More layers always means a bigger image." Size is the sum of unique layer content, not the layer count. Splitting content across more layers can even improve sharing.

"Squashing is a free win." Squashing removes intermediate history but also removes shared and cached layers, so it can raise pulls and rebuild cost.

"Volumes are just another layer." Volumes sit outside the union filesystem and are not part of the image at all.

FAQs

Why did my image grow after I deleted a big file? Because the delete created a whiteout in a later layer while the file stayed in an earlier one. Remove it in the same RUN that created it.

What union filesystem does Docker use? OverlayFS via the overlay2 storage driver on Linux, which is the default on all supported kernels.

Does copy-on-write slow my app? Only for writes, and only the first write to each file pays the copy-up. Put write-heavy paths on a volume.

How do I inspect layer sizes? Use docker history myimage for per-layer sizes, or dive for an interactive layer explorer.

Do containerd and Docker layer images the same way? Yes. Both follow the OCI image spec and default to OverlayFS, so layers behave identically.

Is layer order really that important? Yes. It determines cache hits on rebuild and how much unique data each image adds on top of shared bases.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

How Image Layers & Union Filesystems Work

Summary

An image is an ordered stack of read-only layers; a union filesystem merges them into one view, and the container adds a writable top layer.
Insight: Layers are shared by content digest, so ordering decides both cache hit rate and how much unique data you ship.
Key Concepts: OverlayFS, lowerdir/upperdir/merged, copy-on-write, whiteout files, content-addressed layers.
When to Use: Reach for this model when shrinking images, speeding up builds, or explaining why a "deleted" file still bloats an image.
Limitations/Trade-offs: Copy-on-write is cheap to start but expensive on large in-place writes; deep layer stacks add lookup overhead.
Related Topics: How a Dockerfile becomes an image, multi-stage builds, distroless bases.

Foundations

Every image is a stack of layers. Each layer is a tarball of filesystem changes relative to the layer below it.

A layer records additions, modifications, and deletions - not a full filesystem each time. Stacking them reconstructs the complete rootfs.

Because each layer is addressed by the SHA-256 digest of its content, two images that share a base share the exact same layer bytes on disk and over the network. You pull a shared layer once.

The runtime presents this stack as a single directory tree using a union filesystem. On modern Linux that is almost always OverlayFS, exposed by containerd as the overlayfs snapshotter.

Mechanics & Interactions

When the process reads a file, OverlayFS searches from the top layer down and returns the first match. Upper layers shadow lower ones, so a later layer's copy of a file wins.

Copy-on-write governs writes. Reading a file touches the lower layer directly; writing one first copies the whole file up into the writable layer, then edits the copy.

That copy-up is why writing one byte of a 2 GB file materializes the entire 2 GB in the container layer. It also means the original file still exists, untouched, in the image layer below.

Deletion uses whiteout files. Removing a file from a lower layer creates a special marker in the upper layer that hides it, rather than freeing the underlying bytes.

# Anti-pattern: secret and its removal both persist as layers
RUN curl -o /tmp/big.tar.gz https://example.com/big.tar.gz \
 && tar xf /tmp/big.tar.gz -C /opt
RUN rm /tmp/big.tar.gz   # whiteout only; the 500MB is still in the earlier layer

The fix is to add and remove within the same RUN, so the net change committed to that one layer excludes the temporary file.

RUN curl -o /tmp/big.tar.gz https://example.com/big.tar.gz \
 && tar xf /tmp/big.tar.gz -C /opt \
 && rm /tmp/big.tar.gz

Advanced Considerations & Applications

Layer ordering drives cache economics. The build cache reuses a layer only if its instruction and inputs are unchanged and every parent layer above it in the stack also matched.

So you place stable content low (base image, OS packages, dependencies) and volatile content high (application source). Editing source then invalidates only the top layers.

Shared base layers cut registry storage and node disk. A hundred services on node:22-slim store that base once per node, and containerd unpacks it once into the snapshotter.

At runtime, emptyDir and volumes bypass the union filesystem entirely. Write-heavy workloads should target a volume so hot writes do not thrash copy-on-write in the container layer.

On Kubernetes nodes, containerd manages these layers through its snapshotter. The mechanics are identical to Docker because both consume OCI images and both default to OverlayFS on Linux.

Common Misconceptions

"Deleting a file shrinks the image." Only if the deletion happens in the same layer that added the file. Otherwise a whiteout hides it while the bytes remain below.

"The writable layer is where reads happen." Reads come from wherever the file first appears top-down, usually a read-only image layer. Only writes populate the upper layer.

"More layers always means a bigger image." Size is the sum of unique layer content, not the layer count. Splitting content across more layers can even improve sharing.

"Squashing is a free win." Squashing removes intermediate history but also removes shared and cached layers, so it can raise pulls and rebuild cost.

"Volumes are just another layer." Volumes sit outside the union filesystem and are not part of the image at all.

FAQs

Why did my image grow after I deleted a big file? Because the delete created a whiteout in a later layer while the file stayed in an earlier one. Remove it in the same RUN that created it.

What union filesystem does Docker use? OverlayFS via the overlay2 storage driver on Linux, which is the default on all supported kernels.

Does copy-on-write slow my app? Only for writes, and only the first write to each file pays the copy-up. Put write-heavy paths on a volume.

How do I inspect layer sizes? Use docker history myimage for per-layer sizes, or dive for an interactive layer explorer.

Do containerd and Docker layer images the same way? Yes. Both follow the OCI image spec and default to OverlayFS, so layers behave identically.

Is layer order really that important? Yes. It determines cache hits on rebuild and how much unique data each image adds on top of shared bases.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

How Image Layers & Union Filesystems Work

Buscar en la documentación

How Image Layers & Union Filesystems Work