📖 Chapter 04 — Docker and Containers
The practical Docker: Compose, networks, volumes, the patterns that make containers manageable.
Learning Objectives
- Write a docker-compose.yml for a multi-container app
- Manage container data with named volumes and bind mounts
- Connect containers with Docker networks
- Update, backup, and restore a containerized service
Introduction
You have Docker installed. You know what a container is (Chapter 03). Now the question is: how do you actually run things?
For a single container, the answer is docker run. For a multi-container app (a web server + a database + a cache, for example), the answer is Docker Compose. This chapter is about Docker Compose: how to write the file, how to manage the services, how to keep them running.
The docker-compose.yml pattern
A Docker Compose file describes a "project" — a group of containers that work together. The file is in YAML. Example:
version: "3.9"
services:
immich-server:
image: ghcr.io/immich-app/immich-server:release
restart: unless-stopped
ports:
- "3001:3001"
volumes:
- /mnt/tank/Lab/immich/upload:/usr/src/app/upload
environment:
- DB_HOSTNAME=immich-db
- DB_USERNAME=immich
- DB_PASSWORD=${DB_PASSWORD}
depends_on:
- immich-db
- redis
immich-db:
image: postgres:15
restart: unless-stopped
volumes:
- /mnt/tank/Lab/immich/postgres:/var/lib/postgresql/data
environment:
- POSTGRES_USER=immich
- POSTGRES_PASSWORD=${DB_PASSWORD}
- POSTGRES_DB=immich
redis:
image: redis:7
restart: unless-stoppedThis is one Compose file describing three containers: the Immich server, the PostgreSQL database, and Redis. The file says:
- What image to use for each container
- What ports to expose (immich-server's 3001 is published to the host's 3001)
- What host paths to mount as volumes (the actual data on the NAS)
- What environment variables to set (from the shell, with
${DB_PASSWORD}) - What depends on what (immich-server waits for immich-db and redis to start)
- What restart policy to use (always restart, unless stopped)
With this file in ~/projects/immich/docker-compose.yml, one command starts everything:
docker compose up -dAnd one command stops and removes everything:
docker compose downOne file. One command. Three containers, working together.
Volumes: where the data lives
Containers are ephemeral. When you delete a container, its filesystem goes with it. To keep data across container restarts and updates, you mount volumes.
Two kinds:
- Named volumes: managed by Docker. Docker creates them, stores them in
/var/lib/docker/volumes/, and removes them when you tell it to. Convenient, but the data is hidden in Docker's internal storage. - Bind mounts: a host path mounted into the container. The data lives where you put it; you can see it, back it up, manage it like any other file. For a NAS-based home lab, bind mounts are the right default — your data is on the NAS dataset, not in Docker's internal storage.
Example bind mount from the file above:
volumes:
- /mnt/tank/Lab/immich/upload:/usr/src/app/uploadThis mounts /mnt/tank/Lab/immich/upload on the host (the NAS) to /usr/src/app/upload in the container. The container sees its upload directory on the host's filesystem. Backups of the host path are backups of the container's data.
Networks: how containers talk
By default, Docker Compose creates a network for each project. The containers on that network can talk to each other by name. In the example above, immich-server can reach immich-db at hostname immich-db, and redis at hostname redis.
Containers on different networks can't talk to each other by default. To connect two projects, you put them on the same network (or use network_mode: host on a container to bypass networking entirely).
Environment variables and secrets
For passwords, API keys, and other secrets, the right pattern is an .env file in the same directory as docker-compose.yml:
DB_PASSWORD=correct-horse-battery-staple
IMMICH_ADMIN_PASSWORD=another-secret
CLOUDFLARE_API_TOKEN=...Reference these in the Compose file with ${DB_PASSWORD}. Docker Compose reads the .env file and substitutes the values.
For secrets that should never be in version control, add .env to .gitignore. The .env file is on the host, not in the container image. It's protected by the host's filesystem permissions, not by Docker.
For an extra layer of protection, use Docker secrets (Compose's secrets: key) or an external secret manager (HashiCorp Vault, Bitwarden, etc.). For most home labs, the .env file is sufficient.
The restart policy
Most containers should restart automatically if they crash. The right restart policy is unless-stopped:
restart: unless-stoppedThis restarts the container if it crashes, if the host reboots, if Docker restarts. It only stops if you explicitly stop it with docker compose down or docker stop.
Don't use always — that prevents you from stopping a misbehaving container. Don't omit the restart policy — the default is no, which means a crashed container stays crashed.
Updating containers
To update a container to a new version:
docker compose pull # pull the new image
docker compose up -d # restart the container with the new imageDocker Compose pulls the new image, then recreates the container. The data volumes are preserved. The container's downtime is the time it takes to restart, usually seconds.
For apps that store data in a database, this is straightforward. For apps with a complex data model (Immich, Nextcloud, Paperless), check the release notes for breaking changes. Some updates require running a migration; some require a specific version of the database; some are incompatible with older versions of the app.
The pattern: snapshot before you update (Chapter 09 of Volume 1). If the update breaks, restore the snapshot and try again.
Backing up container data
With bind mounts, the data is on the host filesystem. Back up the host path. That's it.
tar -czf /mnt/tank/Lab/backups/immich-$(date +%F).tar.gz /mnt/tank/Lab/immichFor a Compose project with multiple containers and multiple bind mounts, write a small script that backs up each one. Schedule the script with cron.
For containers that use named volumes (Docker-managed storage), the data is in /var/lib/docker/volumes/. Back up the volumes directory, or use docker run --rm -v <volume>:/data -v $(pwd):/backup alpine tar czf /backup/volume.tar.gz /data to extract a volume to a tarball.
The "single Compose file per service" pattern
For a home lab with several self-hosted services, the right pattern is one Compose file per service, each in its own directory:
Each service is independent. You can update, restart, or remove one without affecting the others. The README in each directory documents the service: what it is, how to access it, what to do when it breaks.
The "shared network" pattern
For services that need to talk to each other (e.g., a web app that uses a database, a monitoring stack that watches everything), the pattern is to share a network:
networks:
default:
name: shared
app-net:
name: app-netOr use the external network pattern: create a network once (docker network create shared), reference it from multiple Compose files with networks: shared: external: true. The containers on different Compose projects can talk to each other.
The tradeoff: shared networks are convenient but reduce isolation. A compromised container can scan the shared network and find other services. For most home labs, the convenience wins; for security-sensitive setups, keep networks isolated.
The "watchtower" pattern
Watchtower is a container that watches other containers and updates them automatically when a new image is released. The conversation's recommendation: don't use Watchtower for production services. The risk of an automatic update breaking a critical service is real. Update manually, on your schedule, after reading the release notes.
Watchtower is fine for non-critical lab services where "always latest" is more important than "definitely working."
The "what to do when a container is broken" pattern
Five steps, in order:
docker compose ps— what's running? What's exited?docker compose logs <service>— what does the log say?docker compose restart <service>— does a restart fix it?docker compose down <service> && docker compose up -d <service>— does a fresh start fix it?- Read the docs, the GitHub issues, the project's Discord. You might be hitting a known issue.
Most container problems are fixed by steps 1-3. The rest are investigation.
Engineering Note
Compose files are documentation. The docker-compose.yml file describes what the service is, what it depends on, how it runs. Six months from now, when you forget how the service is set up, the Compose file is the source of truth. Write it like documentation. Add comments. Keep it up to date. Future-you will be grateful.
Summary
Docker Compose: one YAML file per service, describing containers, networks, volumes, environment. Use bind mounts for data on the NAS dataset. Use the project's default network for container-to-container communication. Use .env for secrets. Use restart: unless-stopped for resilience. Update with docker compose pull && docker compose up -d. Snapshot before updates. Back up the data paths. One service per directory. Don't use Watchtower for production services.
Checklist
- ⬜ Set up a
~/projects/directory for Compose files - ⬜ One Compose file per service, with a README
- ⬜ Use bind mounts for data on
/mnt/tank/Lab/... - ⬜ Use
.envfiles for secrets, not hardcoded values - ⬜ Set
restart: unless-stoppedon all production containers - ⬜ Document the backup command for each service in its README
Looking Ahead
Chapter 05 is VMs with KVM. When a container isn't enough — a different OS, kernel access, strong isolation — VMs are the right tool. The chapter covers creating VMs, managing them, and the common home lab VM patterns (a dev VM, a Windows VM, a Linux desktop VM).