HowItWorks:Docker

Explains how things actually work — part of our technology dissections collection.

Docker is an open platform for developing, shipping, and running applications. It packages an application — code, runtime, libraries, configuration — into a container: a loosely isolated environment that runs the same everywhere, on a developer's laptop, a data centre server, or in the cloud. Docker is written in the Go programming language and builds on Linux kernel features — most importantly namespaces — to deliver that isolation.

This page dissects Docker the way it is built today: the container concept, the components (client, daemon, containerd, runc), the build and run flows, and the design decisions behind them.

What is a container?

A container is an isolated process with all of the files it needs to run. Docker's own definition: containers are "isolated processes for each of your app's components" — self-contained (no reliance on host pre-installed dependencies), isolated (minimal influence on the host and other containers), independent (each is managed separately), and portable (the same container runs on a laptop, in a data centre, or in the cloud).

The crucial difference from a virtual machine (VM) is where the isolation happens:

Virtual machine	Container
A full guest operating system with its own kernel, hardware drivers and applications	An isolated process with the files it needs, sharing the host's kernel
Heavy: every VM duplicates an entire OS	Lightweight: many containers share one kernel, so more workloads fit on the same hardware
Isolated by a hypervisor	Isolated by Linux namespaces, resource-limited by cgroups

As Docker's docs put it: "A container is simply an isolated process with all of the files it needs to run. If you run multiple containers, they all share the same kernel, allowing you to run more applications on less infrastructure." In practice the two are often combined: cloud providers give you VMs, and a container runtime on each VM runs many containerized applications.

A short history

Docker was created by Solomon Hykes, founder of dotCloud, as an internal project that was open-sourced in 2013 — Hykes himself dates the project's public life from "2013-2014", when "pioneers started to use containers and collaborate in a monolithic open source codebase, Docker". Docker 1.0, the first production-ready release, shipped on June 9, 2014.

Three milestones turned Docker the project into the container ecosystem:

• 2015 — Open Container Initiative (OCI). Launched on June 22, 2015 by Docker, CoreOS and other container industry leaders under the Linux Foundation, the OCI writes the open standards for container formats and runtimes. Docker donated its container format and its runtime, runc, to serve as the cornerstone of the effort.
• 2016 — containerd. Docker spun its core container runtime out into a standalone project called containerd and donated it to the Cloud Native Computing Foundation (CNCF). containerd had been embedded in Docker since Docker Engine 1.11 in April 2016; it graduated as a CNCF project on February 28, 2019.
• 2017 — Moby Project. Docker moved all its open-source collaboration into the Moby Project, "a new open-source project to advance the software containerization movement", providing a library of components and a framework for assembling them into custom container systems. Docker itself remains an open-source product built on Moby.

The company behind all of it, Docker, Inc. — the company whose founder created the project at dotCloud — still develops Docker; all of Docker's open-source collaboration now lives in the Moby project.

Components and interactions

Docker uses a client-server architecture: the Docker client talks to the Docker daemon (dockerd), which does "the heavy lifting of building, running, and distributing" containers, over a REST API on a UNIX socket or network interface. The daemon manages Docker objects — images, containers, networks, volumes, plugins — and can even communicate with other daemons (for Docker services). Another client, besides the CLI, is Docker Compose, for multi-container applications.

Modern Docker Engine is not one monolith but a stack of cooperating components:

The responsibilities are split roughly as follows:

• dockerd — the API server and object manager. It receives every request, holds the state of images/containers/networks/volumes, and orchestrates the rest of the stack.
• containerd — "an industry-standard container runtime", a daemon that "manages the complete container lifecycle of its host system, from image transfer and storage to container execution and supervision to low-level storage to network attachments and beyond". Since Docker Engine 29, the containerd image store is the default for fresh installs.
• runc — the OCI runtime that actually creates and runs container processes, applying namespaces, cgroups and filesystem isolation; it is the reference implementation of the OCI runtime specification.
• containerd-shim — the small per-container process that keeps the container's stdio open and lets the daemons restart without killing containers.
• The Linux kernel — the isolation primitives themselves: namespaces (each container gets its own set), cgroups for resource limits, OverlayFS for layered filesystems, seccomp and capabilities for security.
• A registry — stores and distributes images; Docker Hub is the default, and docker pull/docker run fetch from it while docker push uploads to it.
• BuildKit — the builder backend that executes image builds; since its adoption it is "the default builder for Docker Desktop and Docker Engine users".

Docker in action

Two flows cover most of what Docker does: building an image from a Dockerfile, and running a container from an image.

Image build

To build your own image you write a Dockerfile, "a text document that contains all the commands a user could call on the command line to assemble an image". Each instruction — FROM, RUN, COPY, CMD, ENTRYPOINT, ... — becomes a layer of the image. Because layers are immutable and cached, "when you change the Dockerfile and rebuild the image, only those layers which have changed are rebuilt" — the main reason images are so lightweight and builds so fast compared with other virtualization technologies.

Underneath the classic docker build command sits BuildKit. Its core is a Low-Level Build (LLB) format: "a content-addressable dependency graph" that lets it detect and skip unused build stages, run independent stages in parallel, and transfer only the changed files of the build context. A "frontend" — itself distributed as an image — turns the human-readable Dockerfile into LLB, so the Dockerfile language can evolve without rebuilding the engine.

Container run

A container is "a runnable instance of an image", "defined by its image as well as any configuration options you provide to it when you create or start it". When you run docker run -i -t ubuntu /bin/bash, the documented sequence is:

1. If the ubuntu image is not present locally, Docker pulls it from the configured registry.
2. Docker creates a new container (as with docker container create).
3. Docker allocates a read-write filesystem to the container as its final layer — the only layer a running container may write to.
4. Docker creates a network interface connecting the container to the default network and assigns it an IP address.
5. Docker starts the container and executes /bin/bash, attached to your terminal.
6. When you type exit, the container stops but is not removed — you can start it again or delete it.

That last point is important: "when a container is removed, any changes to its state that aren't stored in persistent storage disappear." Data that must survive goes into volumes or bind mounts; ephemeral state simply dies with the container.

Key design decisions

Containers share the host kernel

The founding decision: a container is not a mini-VM. Docker uses Linux namespaces to give each container an isolated view (processes, networking, filesystem, users, and more), and cgroups to limit CPU, memory and I/O. Because every container on a host shares the same kernel, there is no guest OS to boot, so containers start in seconds and many more fit on a machine than VMs would.

The trade-off is architectural: isolation is as good as the kernel's — and the admin's — configuration. Docker historically defaulted to cgroups v1; the ecosystem has since moved to cgroups v2, and as of Docker Engine 29 cgroup v1 is deprecated (supported at least until May 2029). On macOS and Windows, Docker Desktop bundles the daemon, the client, Docker Compose, Kubernetes and more, so the same Linux-based engine is available everywhere.

Images are immutable, layered and content-addressed

An image is a "read-only template", usually "based on another image, with some additional customization" — e.g. the Ubuntu image plus the Apache web server plus your application. Its layers are immutable: "each of these layers, once created, are immutable" — a layer is a set of filesystem changes (additions, deletions, modifications).

Two mechanisms make layers workable:

• Content-addressable storage — layers are stored and identified by a hash of their content, so identical layers (e.g. the same base image) are stored once and shared between images, saving storage and bandwidth.
• Union filesystems — at run time, layers are stacked into a single unified view; the container's root is set to that view with chroot. On top of the read-only image layers, a directory is created for the running container, so the container writes copy-on-write without ever touching the image.

Reuse is the payoff: a second Python app reuses the same Python base layers, "making builds faster and reducing the amount of storage and bandwidth required to distribute the images".

The Dockerfile is a declarative build recipe

Rather than mutating a running machine (and hoping the result is reproducible), Docker builds images from a declarative Dockerfile — "all the commands a user could call on the command line to assemble an image". A valid Dockerfile must start with FROM, which "initializes a new build stage and sets the base image". The key instructions:

• RUN — "execute any commands to create a new layer on top of the current image".
• COPY / ADD — copy files from the build context (or another stage) into the image.
• CMD / ENTRYPOINT — define the command executed when a container starts; ENTRYPOINT configures the container to run "as an executable", CMD provides default arguments.
• Multiple FROM lines enable multi-stage builds — build in one stage, copy only the artifacts into a minimal final stage.

Because each instruction is a layer, the instruction sequence is also the cache key: RUN layers are reused on rebuild unless invalidated — a changed COPY or ADD invalidates the cache — so instructions placed early in the file are the ones that get reused across builds. BuildKit hardened this model by tracking "the checksums of build graphs" instead of heuristics, making the cache faster, more precise and portable — it can even be exported to a registry.

A daemon-centric client-server design

Docker chose a single privileged daemon that owns all state, fronted by a deliberately thin CLI. Every docker command is a REST call to the Engine API; the daemon performs it and reports back. This concentrates complexity in one place (good for security auditing and for the object model) and makes the engine scriptable — CI systems, orchestrators and tools like Compose and Docker Desktop are just API clients.

The cost: a single point of failure and a long-lived process, which is precisely what the containerd extraction (below) addressed — the low-level container lifecycle moved out of the daemon, so the daemon can restart without killing containers.

The Engine API is versioned and stable

Like MediaWiki's parser stability, Docker's contract with the outside world is its versioned Engine API. Clients and daemons negotiate the API version, so a newer CLI can drive an older daemon and vice versa. Docker Engine 29 keeps this discipline hard: the daemon now requires API version v1.44 or later, and the client dropped support for negotiating versions below v1.44 — old enough that tooling has years of runway to migrate. Breaking changes are announced in the release notes and take effect per API version, not per install.

Standardize the primitives, assemble the rest

The most consequential decision is arguably the unbundling. Rather than keeping the whole stack proprietary to Docker, the low-level pieces became open standards and neutral projects:

• OCI (2015) — runtime, image and distribution specifications, with Docker's runc as the reference runtime. Any OCI-compliant engine can run OCI images; that is why docker pull images run under other runtimes.
• containerd (2016) — the core runtime spun out and donated to the CNCF, as part of "a multi-year effort to break up the Docker platform into a more modular architecture of loosely coupled components".
• Moby (2017) — the open-source framework of components and assemblies on which Docker itself is built, letting others assemble their own container systems from the same parts.

Docker Engine is thus best understood as one particular assembly of standardized, swappable components — "batteries included", but with every battery standardized and replaceable. Compose followed the same arc: born in 2014 as a Python tool (docker-compose), rewritten in Go as Compose v2 (2020) integrated into the CLI as docker compose, and released as Compose v5 (2025) with an official Go SDK — all defined by the rolling, version-less Compose Specification.

Try it yourself

The commands below follow Docker's official documentation, verified on 2026-08-31. No Docker daemon was available in the authoring environment, so run them on your own machine with Docker installed; the outputs shown are those documented by Docker.

# 1. Confirm the welcome-to-docker image really exists on Docker Hub (live check, 2026-08-31)
curl -s "https://hub.docker.com/v2/repositories/docker/welcome-to-docker/" | grep -o '"pull_count":[0-9]*'

Output (verified live):

"pull_count":6104168

# 2. Start a web server container, publishing port 8080 on the host to port 80 in the container
docker run -d -p 8080:80 docker/welcome-to-docker

"The output from this command is the full container ID." The container now serves a small website at http://localhost:8080.

# 3. List running containers
docker ps

Documented output (IDs and names vary):

CONTAINER ID   IMAGE                      COMMAND                  CREATED          STATUS          PORTS                      NAMES
a1f7a4bb3a27   docker/welcome-to-docker   "/docker-entrypoint.…"   11 seconds ago   Up 11 seconds   0.0.0.0:8080->80/tcp       gracious_keldysh

# 4. Stop the container
docker stop a1f7a4bb3a27

"Only enough of the ID to make it unique" is needed — docker stop a1f would do.

# 5. Run an interactive Ubuntu shell in a container
docker run -i -t ubuntu /bin/bash

If the image isn't local, Docker pulls it; then it creates the container, allocates its writable layer, attaches it to the default network, starts it and runs /bin/bash — type exit and the container stops but is not removed.

# 6. Build your own image from a Dockerfile
# Dockerfile:
#   FROM ubuntu
#   RUN apt-get update && apt-get install -y curl
#   COPY app.sh /app.sh
#   CMD ["/app.sh"]
docker build -t myapp .

Docker reads the Dockerfile's instructions in order; each creates a layer, and unchanged layers are reused from the build cache on the next build.

Further reading

• What is Docker? — the official overview
• Docker Engine — engine documentation
• Dockerfile reference — the build language
• BuildKit — the builder backend
• Docker Compose — multi-container applications
• containerd — the container runtime
• Open Container Initiative — the container standards
• Moby Project — the open framework behind Docker
• moby/moby — the engine's source code