Docker

"It works on my machine." — every engineer, right before the outage

There is a sentence that has cost the software industry more time than any bug: "but it works on my machine." It is true, and that is exactly the problem. The code ran on your laptop — with your version of Python, your system libraries, your one lucky environment variable — and then it met the CI server, or the production box, or a teammate's machine, and it fell apart. Docker exists to end that sentence forever. It is the tool that lets you package an application together with everything it needs to run, so that "my machine" and "the server" become, for the first time, the same machine.

Core Concepts

The problem: environment drift

Software does not run in a vacuum. Your application depends on a language runtime (Node 20, say), a pile of libraries at specific versions, a handful of system packages, some environment variables, and assumptions about the operating system underneath. That entire invisible scaffold is the environment — and the environment on your laptop is never quite the environment anywhere else. Your machine has Node 20.11; the server has 18.4. You installed a library globally two months ago and forgot. The CI runner is on a different Linux distribution with older shared libraries. Each difference is small. Together they are the reason a build that is green on your desk goes red the moment it leaves it. This slow divergence between where code is written, tested, and run is called environment drift, and before containers the industry fought it with pages of setup documentation, "works for me" shrugs, and heroic debugging sessions that produced no feature.

The honest framing of Docker is this: it does not make your code better. It makes your code's environment portable, reproducible, and disposable. That turns out to be almost as valuable.

What a container actually is

A container is a running process — your application — wrapped in its own isolated slice of the operating system. From inside, the process sees what looks like a private machine: its own filesystem, its own network interface, its own set of process IDs, its own view of memory limits. From outside, on the host, it is just another process in the list, sharing the same kernel as everything else. That double nature — feels like a whole machine, costs like a process — is the entire trick.

Crucially, the container carries its own filesystem with it: your app plus its runtime plus its libraries plus its system packages, all bundled together. Ship that bundle and you ship the environment. The server no longer needs Node installed, or the right library versions, or any setup at all — it only needs to be able to run a container. "Works on my machine" dies here, because now the machine travels with the code.

Containers vs virtual machines — and why containers are light

Before containers, the standard way to get an isolated, reproducible environment was a virtual machine. A VM works by virtualizing hardware: a hypervisor pretends to be a whole computer — CPU, disk, network card — and on top of that fake computer you install a complete guest operating system, kernel and all. Your app then runs inside that guest OS. It works, and the isolation is strong, but you are now running an entire second operating system just to host one application. That costs gigabytes of disk, hundreds of megabytes of RAM, and tens of seconds to boot — per VM.

A container skips the guest OS entirely. Recall from the Operating Systems chapters that the kernel is the part of the OS that talks to hardware and manages processes and memory. A container shares the host's kernel and isolates only the process. There is no second OS to boot because there is no second OS — there is your process, fenced off, using the kernel that is already running. That is why a container starts in milliseconds and measures in megabytes where a VM measures in gigabytes.

The fencing is done with two Linux kernel features:

• Namespaces control what a process can see. A PID namespace makes the container's process think it is process 1 with no siblings; a network namespace gives it its own interfaces; a mount namespace gives it its own filesystem view. Isolation of visibility.
• cgroups (control groups) control what a process can use — how much CPU, memory, and I/O it is allowed. Isolation of resources, so one greedy container can't starve its neighbours.

Docker did not invent either of these. It composed them into something usable, which we will return to at the end of the chapter.

Read the density row again. On one server you might run a dozen VMs; you can run hundreds of containers. That ratio is why the cloud runs on containers — and why the next chapter, on Kubernetes, exists to herd them.

Images vs containers

These two words are constantly confused, and the distinction is worth getting exactly right.

An image is a blueprint: an immutable, read-only bundle containing your application, its dependencies, and a recipe for how to start. It is a file on disk (well, a stack of files — more below). Nothing runs. It is a template, like a class in code, or a .iso you haven't booted.

A container is a running instance of an image: what you get when you take the blueprint and actually start it, like an object instantiated from a class. You can start many containers from one image — ten identical copies of your web server from a single image — the same way you create many objects from one class definition.

The Dockerfile and why layer order matters

You describe an image in a Dockerfile — a plain-text recipe read top to bottom. Four instructions carry most of the weight:

# Start FROM an existing base image — here, Node 20 on slim Debian
FROM node:20-slim

# Set the working directory inside the image
WORKDIR /app

# COPY files from your machine into the image
COPY package.json package-lock.json ./

# RUN a command at build time — install dependencies
RUN npm ci

# Now copy the rest of the source
COPY . .

# CMD is the command run when a CONTAINER starts (not at build)
CMD ["node", "server.js"]

FROM picks a starting point so you are not building an OS from scratch. COPY moves files in. RUN executes a command while building the image and bakes the result in. CMD is different — it does nothing at build time; it declares the command to run when a container starts. Confusing RUN (build-time) with CMD (run-time) is one of the most common beginner mistakes.

Now the part that separates people who use Docker from people who understand it: layer caching. Each instruction in a Dockerfile creates a layer — a saved snapshot of the filesystem changes that instruction made. Images are these layers stacked on top of each other. When you rebuild, Docker reuses a cached layer if nothing that layer depends on has changed — and here is the rule: the moment one layer's inputs change, that layer and every layer after it must be rebuilt.

That is why the Dockerfile above copies package.json and runs npm ci before copying the rest of the source. Your dependencies change rarely; your source code changes constantly. By putting the slow, rarely-changing step (installing hundreds of packages) first, Docker can reuse that cached layer on every rebuild where you only edited source — turning a two-minute install into an instant cache hit. Flip the order — COPY . . before npm ci — and every one-character code change invalidates the copy layer, forcing a full reinstall every single time. Same image, wildly different build speed. Put rarely-changing steps first.

The registry: build once, run anywhere

Once built, an image needs to get from your machine to the server. That is what a registry is for — a shared store for images. Docker Hub is the public default; cloud providers run their own (AWS's ECR, Google's Artifact Registry, GitHub's GHCR). You docker push an image up and docker pull it down. This is the payoff of the whole model: you build the image once — on your laptop or, better, in CI — push it to a registry, and every environment pulls the exact same bytes. The image tested in CI is bit-for-bit the image running in production. There is no rebuild-on-the-server step to drift.

Image size discipline and multi-stage builds

Images travel over networks and sit on disk, so size is not vanity — a 1.2 GB image is slow to push, slow to pull, slow to start, and carries more software that can harbour security holes. Two disciplines keep images lean.

First, small base images. node:20 weighs close to a gigabyte because it includes a full Debian userland and build tools. node:20-slim trims it hard; Alpine-based images (node:20-alpine) go smaller still by using a minimal Linux distribution. Pick the smallest base your app actually needs.

Second, and more powerful, the multi-stage build. Many apps need a heavy toolchain to build — compilers, dev dependencies, bundlers — but none of that to run. A multi-stage build uses a fat image to build the artifact, then copies only the finished artifact into a slim final image, throwing the build tools away:

# ---- Stage 1: build (fat image, full toolchain) ----
FROM node:20 AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build          # produces the compiled output

# ---- Stage 2: run (slim image, artifact only) ----
FROM node:20-slim
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY package*.json ./
RUN npm ci --omit=dev          # production dependencies only — no build/test tooling
CMD ["node", "dist/server.js"]

The final image contains the built app and its runtime dependencies — and none of the compilers, source, or dev tooling that stage one needed. The build machinery is left behind in a stage that never ships. This routinely turns a gigabyte into a couple of hundred megabytes.

Statelessness and volumes

A container's filesystem is ephemeral. Write a file inside a running container, then stop and remove it, and that file is gone — the next container from the same image starts from the pristine image again. This is a feature, not a bug: it forces the discipline that containers should be stateless, holding no important data of their own, so that any container is as good as any other and can be killed and replaced freely. (That freedom is exactly what makes the auto-scaling and self-healing of the next chapter possible.)

But real applications do have state — a database's files, user uploads. That state must live outside the container's ephemeral layer, in a volume: a piece of the host's storage (or a networked disk) mounted into the container. The container comes and goes; the volume and its data persist. The rule of thumb: keep the container disposable, push everything worth keeping into a volume or an external service like a managed database.

Ports and networking, briefly

A container has its own private network namespace, so a process listening on port 3000 inside the container is not reachable from the host until you say so. You publish a port to map a host port to a container port — docker run -p 8080:3000 sends traffic arriving at the host's 8080 into the container's 3000. This explicit mapping is what lets you run three containers that all listen on port 3000 internally, published to 8080, 8081, and 8082 on the host, without collision.

Common Mistakes

• Copying source before installing dependencies. It invalidates the cache on every code change and forces a full dependency reinstall each build. Copy the manifest and install first, source last.
• Shipping the build toolchain to production. Compilers, dev dependencies, and source have no business in the running image. Use a multi-stage build and copy only the artifact.
• Storing important data inside the container. The filesystem is ephemeral — it vanishes when the container is removed. Persistent data belongs in a volume or an external service.
• Confusing RUN with CMD. RUN executes at build time and bakes into a layer; CMD declares what runs when a container starts. Mixing them up gives you either a broken build or a container that does nothing.
• Reaching for a heavy base image out of habit. node:20 when node:20-slim or -alpine would do means slower pulls, bigger attack surface, and wasted registry space.
• Treating a container like a tiny VM. SSHing in to patch a live container produces a "pet" whose state exists nowhere in code. Fix the Dockerfile, rebuild, redeploy.

Interview Corner

Chapter Summary

• Docker solves environment drift: it packages an app with its dependencies so it runs identically on your laptop, CI, and production — ending "works on my machine."
• A container is an isolated process that shares the host kernel; a VM virtualizes hardware and ships a whole guest OS. That's why containers are megabytes and milliseconds where VMs are gigabytes and seconds.
• The isolation comes from kernel primitives: namespaces (what a process can see) and cgroups (what it can use).
• An image is an immutable blueprint built from a Dockerfile; a container is a running instance of it — class versus object.
• Dockerfiles build in layers; order matters — put rarely-changing steps (dependency installs) first so code changes hit the cache.
• Keep images small with slim/Alpine bases and multi-stage builds; keep containers stateless, with persistent data in volumes; build once, push to a registry, run anywhere.

Editorial reflection — most engineers learn Docker as a set of commands to memorise: build, run, push, repeat. That's the smallest part of it. The real lesson is a mental shift — from treating environments as fragile, hand-tended places you hope stay consistent, to treating them as code you build, version, and throw away without a second thought. Once "the environment is just another artifact" clicks, the whole modern cloud — CI/CD, orchestration, immutable infrastructure — stops looking like a pile of tools and starts looking like one idea, followed to its conclusion.