Why containers?

Objective

Understand the problem that Docker solves and what a container really is. Master the basic vocabulary: image, container, registry, Docker Engine and OCI. This lesson requires no installation.

Key points

• A container is an isolated Linux process. It has its own view of files, of the network and of other processes, but it uses the kernel of the host machine.
• An image is a frozen, read-only template (the program, its libraries, its configuration). You can start as many containers as you want from the same image.
• A registry stores and distributes images (Docker Hub, GitHub Container Registry, a private registry).
• Docker Engine is the engine: a daemon (dockerd) that builds images and runs containers, driven by the docker command.
• The OCI standard (Open Container Initiative) describes the image format and how to run images. An image built with Docker also runs with Podman, containerd or Kubernetes.

The problem: "it works on my machine"

Imagine an online shop in Casablanca. The developer wrote the site with Node.js 22 on an Ubuntu laptop. The production server runs Debian with Node.js 18. On launch day, a library refuses to start: the Node version does not match, a system package is missing, an environment variable was forgotten.

This problem has three classic causes:

1. Dependencies differ from one machine to another (language versions, system libraries).
2. Configuration is not written down: it lives in someone's head.
3. Several applications on the same server get in each other's way (one wants PHP 8.1, the other PHP 8.3).

A container fixes these three points. The application travels with everything it needs, the recipe is written in a file (the Dockerfile), and each application has its own space, isolated from the others.

Virtual machine versus container

A virtual machine (VM) simulates a complete computer: a virtual processor, a virtual disk and, above all, its own operating system with its own kernel. A hypervisor (VMware, KVM, Hyper-V, VirtualBox) shares the hardware between the VMs.

A container simulates nothing. It is a normal process of the host system, which the Linux kernel confines with two mechanisms:

• namespaces: the process has its own process list (pid), its own network (net), its own mount points (mnt), its own hostname (uts), its own users (user), and so on;
• control groups (cgroups): the kernel limits the memory, CPU and I/O that this process can consume.

The two are not opposites. In real life, you very often rent a VM (a VPS from a hosting provider) and run containers on it.

Windows and macOS have no Linux kernel. Docker Desktop therefore starts a small Linux VM in the background, and your containers run inside it.

You can see the namespaces of any Linux process, even without Docker:

ls /proc/self/ns          # namespaces of the current shell
# cgroup ipc mnt net pid pid_for_children time time_for_children user uts
cat /proc/self/cgroup     # the control group it belongs to
uname -r                  # the kernel: the SAME for the host and all its containers

Image and container

The best mental model is the class and the object in programming, or the mould and the cake:

• the image is the mould: immutable, versioned, shareable;
• the container is a cake taken out of the mould: a living instance that starts, works, stops and can be deleted.

An image is made of stacked layers. Each layer adds, changes or deletes files compared with the previous one. When a container starts, Docker places a thin writable layer on top, specific to that container. This has a major consequence:

Everything a container writes to its layer disappears when you delete it. Data you want to keep (a database, files uploaded by customers) goes into a volume. You will see this in detail in level 2.

Two principles guide how you use images:

1. You do not modify an image. To change something, you build a new image (a new version), delete the old container and start a new one.
2. One container, one role. The website in one container, the PostgreSQL database in another, the Redis cache in a third. You then assemble them with Docker Compose.

Registries and image names

A registry is a server that stores images, a bit like an apt or dnf package repository. Docker Hub is the default public registry. There are many others: GitHub Container Registry (ghcr.io), cloud registries (Amazon ECR, Azure ACR, Google), GitLab, Harbor… and you can host your own.

The full name of an image has three parts:

ghcr.io / startup-rabat/api : 2.3.1
└──┬──┘   └──────┬────────┘   └─┬─┘
registry     repository        tag

When a part is missing, Docker fills it in:

• No registry: docker.io (Docker Hub).
• No namespace: library/, the folder of official images (nginx, postgres, redis, node, debian…).
• No tag: latest. This word does not mean "the most recent". It is just the default tag. In production, always write a specific tag.

Docker Engine: who does what?

When you type docker run nginx, several programs work together:

you ──> docker (client, the command)
            │  REST API on /var/run/docker.sock
            ▼
         dockerd (the Docker Engine daemon)
            │  images, networks, volumes, builds (BuildKit)
            ▼
         containerd ──> runc ──> the isolated process (your container)

• docker is the client. It runs nothing itself: it sends orders to the daemon's API. It can even talk to a remote daemon.
• dockerd is the daemon. It manages images, networks, volumes and builds.
• containerd manages the container lifecycle, and runc actually creates the isolated process (namespaces, cgroups).

The daemon runs as root. Anyone who can talk to its socket can therefore do anything on the machine. You will see in level 4 what this means for security.

OCI: an open standard

In 2015, Docker and other players created the Open Container Initiative. It publishes open specifications:

• image-spec: the format of an image (layers, manifest, configuration);
• runtime-spec: how to run a container from an unpacked image;
• distribution-spec: how a registry receives and distributes images.

Thanks to OCI, you are not locked into one tool. An image built by Docker on a developer's laptop can be started by Podman on a Red Hat server, by containerd in Kubernetes, or stored in any compatible registry.

Why Docker became essential

A concrete example: the registration application of a school in Fès consists of a website, an API and a PostgreSQL database. With Docker, a new developer clones the project, types a single command (docker compose up), and gets the same stack as production within a minute.

A bit of history

Containers were not born with Docker. They are the result of twenty years of additions to the Unix world and the Linux kernel:

Docker's real revolution was not technical. It was to package these mechanisms into a simple tool, with a portable image format and a registry where everyone shares their images.

Docker, Podman, containerd: which one to use?

You will come across several tools. They all speak the same image language:

This course uses Docker Engine and Docker Compose, the most widespread at hosting providers and in job offers. Everything you learn about images and Dockerfiles also applies to Podman.

Vocabulary at a glance

Common pitfalls

• Thinking a container is a small VM: it has no kernel of its own. A kernel module or a global sysctl setting is configured on the host.
• Storing important data inside the container: it disappears when the container is deleted. Use a volume.
• Using latest in production: you no longer know which version is running, and a simple restart can change the version.
• Pulling any image from Docker Hub: prefer official images or images from verified publishers, just as you would choose a trusted package repository.

Lab

Docker is not installed in the lab: it is not needed here. You will:

1. list the namespaces of a process, the basic building block of a container;
2. record the kernel version, the kernel that all containers on the machine would share;
3. write the full name of an official image;
4. break down the name of an image hosted on GitHub Container Registry.

Quiz

1. What is the fundamental difference between a virtual machine and a container?
2. What are the two Linux kernel mechanisms that isolate and limit a container?
3. What happens to a file that a container wrote to its own layer when you delete the container?
4. What full name does Docker use for redis?
5. What is the OCI standard useful for, for a company?

Reference sources: official Docker documentation (docs.docker.com, Apache 2.0 license). Original MasterLinux text.