Separating Config and Secrets from Images

Summary

Definition: Separating config from images means the container image carries only code and runtime, while every value that varies between environments is injected at run time from ConfigMaps, Secrets, or an external store.
Insight: A single, immutable image should promote unchanged from dev to prod. Only the injected configuration differs.
Key Concepts: immutable image, 12-factor config, environment injection, build-time vs run-time, config drift.
When to Use: Always for services deployed to more than one environment, and any time a value would otherwise force a rebuild to change.
Limitations/Trade-offs: Externalized config adds moving parts (ConfigMaps, Secrets, operators) and requires a reload or restart strategy when values change.
Related Topics: ConfigMaps, Kubernetes Secrets, External Secrets Operator, config reload patterns.

Foundations

The mental model comes from the 12-factor app methodology, factor III: store config in the environment.

Config is everything that varies between deploys - database URLs, feature flags, log levels, API endpoints, and credentials.

Code is everything that does not vary between deploys. That is what belongs in the image.

The litmus test is simple. If you could open-source the image tomorrow without leaking anything, your secrets are correctly externalized.

An image is a build artifact. It should be built once, tagged with an immutable digest, and promoted through environments without change.

Baking a staging database hostname into an image means you need a different image for production. That breaks the "build once, run anywhere" contract.

Baking a password into an image is worse. It lands in a registry layer, gets cached on every node that pulls it, and survives in image history even after you overwrite the file.

The alternative is late binding. The image stays generic, and the platform injects the environment-specific values when the container starts.

Mechanics & Interactions

In Kubernetes, non-secret config lives in a ConfigMap and sensitive values live in a Secret. Both are namespaced key-value objects.

A Pod consumes them one of two ways. As environment variables, or as files mounted from a volume.

Environment variables are simplest. You reference a ConfigMap or Secret key with valueFrom, and the value appears in the container process environment.

env:
  - name: LOG_LEVEL
    valueFrom:
      configMapKeyRef:
        name: app-config
        key: log_level
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: app-db
        key: password

Volume mounts are the other path. The kubelet projects each key as a file, which suits config files, TLS certificates, and large blobs.

The critical difference between the two: environment variables are captured at process start and never change, while mounted files are updated in place by the kubelet after the source object changes.

This distinction drives the entire reload story later. Env vars need a restart to pick up new values. Mounted files can be re-read by a running process.

Secrets deserve a caveat. By default they are stored in etcd base64-encoded, not encrypted. Base64 is an encoding, not protection.

Production clusters enable encryption at rest for Secret resources and scope access with RBAC so only the workloads and operators that need a Secret can read it.

The build-time versus run-time boundary is the whole point. ENV and ARG in a Dockerfile set defaults and build inputs, but they must never carry per-environment secrets.

Advanced Considerations & Applications

Digest pinning reinforces immutability. Reference base images and your own images by @sha256:... digest so the exact bytes are fixed and cannot be silently replaced by a moved tag.

For secrets that must never touch etcd in plaintext form, teams push the source of truth into a dedicated store - AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager - and sync into Kubernetes with the External Secrets Operator.

GitOps changes how config flows. ConfigMaps are version-controlled in Git and reconciled by Argo CD or Flux, giving you an audit trail and easy rollback.

Raw Secret YAML cannot go in Git. Sealed Secrets or an external store solves that, so the Git repo holds only encrypted or referenced material.

A subtle failure mode is config drift. When someone edits a live ConfigMap with kubectl edit, the cluster no longer matches Git, and the next reconcile either reverts it or conflicts.

The Docker side matters for local development. Compose injects config through environment: and env_file:, and Docker BuildKit secret mounts keep credentials out of image layers during builds.

# BuildKit secret mount - the token never lands in a layer
docker build --secret id=npmtoken,src=$HOME/.npmrc -t app:dev .

Scale amplifies the payoff. One image across fifty services and three environments means one artifact to scan, sign, and promote, with configuration as the only variable.

Common Misconceptions

"Base64 in a Secret means it is encrypted." False. Base64 is reversible with a single command. Real protection comes from encryption at rest plus RBAC.

"Environment variables are a security boundary." They are not. Env vars are visible in the process listing, in /proc, and often in crash dumps and logs. Prefer mounted files for high-sensitivity material.

"A ConfigMap update instantly reaches my app." Only if the app reads mounted files and re-reads on change. Env-var consumers keep the old value until the Pod restarts.

"I need one image per environment." No. The correct pattern is one image plus per-environment config injected at run time.

"Secrets in Git are fine if the repo is private." Private is not encrypted. A cloned repo, a leaked token, or a misconfigured mirror exposes everything. Never commit raw Secret YAML.

FAQs

Should I use env vars or mounted files for config? Use env vars for a handful of scalar values, and mounted files for config files, certificates, and anything you want a running process to reload.

Are Kubernetes Secrets actually secret? Only when you enable encryption at rest and restrict access with RBAC. Out of the box they are just base64-encoded in etcd.

Where do defaults belong? Sensible non-secret defaults can ship in the image, but any value that differs between environments must be injectable and must override the default.

Can I put secrets in a Dockerfile ENV? No. ENV values persist in image layers and history. Use BuildKit secret mounts at build time and Kubernetes Secrets at run time.

How do I keep config in Git without leaking secrets? Version ConfigMaps directly, and handle secrets with Sealed Secrets or the External Secrets Operator so Git holds only encrypted or referenced values.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

Separating Config and Secrets from Images

Summary

Definition: Separating config from images means the container image carries only code and runtime, while every value that varies between environments is injected at run time from ConfigMaps, Secrets, or an external store.
Insight: A single, immutable image should promote unchanged from dev to prod. Only the injected configuration differs.
Key Concepts: immutable image, 12-factor config, environment injection, build-time vs run-time, config drift.
When to Use: Always for services deployed to more than one environment, and any time a value would otherwise force a rebuild to change.
Limitations/Trade-offs: Externalized config adds moving parts (ConfigMaps, Secrets, operators) and requires a reload or restart strategy when values change.
Related Topics: ConfigMaps, Kubernetes Secrets, External Secrets Operator, config reload patterns.