Home · Volume 6 · Chapter 07

📖 Chapter 07 — The Runbook

The documentation that turns operations into a habit. The single source of truth.

v0.1 · draft Vol 6 · Ch 07
~10 min

Learning Objectives

Introduction

Six months from now, you want to add a new app. You forgot the steps. You search the chat history, the documentation, the half-finished notes in your files. You spend 30 minutes figuring out what you already knew.

The runbook fixes this. The runbook is a single document that describes the home NAS: the hardware, the software, the configuration, the procedures, the contacts. The runbook is the single source of truth. When you forget how something works, you read the runbook. When you change something, you update the runbook.

This chapter covers the runbook: what goes in it, how to write it, how to keep it current.

What goes in the runbook

For a home NAS, the runbook sections:

  1. System overview: the hardware, the network, the storage layout. The 30,000-foot view.
  2. Services and apps: what's installed, where it's configured, how to access it.
  3. Users and access: who has access, with what permissions.
  4. Credentials and secrets: where passwords, API keys, and tokens are stored (Vaultwarden, presumably).
  5. Backup and recovery: the 3-2-1 setup, the recovery procedures.
  6. Monitoring and alerting: the alerts, the runbooks for each.
  7. Maintenance schedule: the monthly, quarterly, annual tasks.
  8. Disaster playbooks: drive failure, ransomware, fire. The step-by-step.
  9. Change log: what changed, when, why.

Each section is short. The runbook is a reference, not a novel.

The format

For a home NAS, the runbook is a markdown file (or a collection of files in a directory). The benefits of markdown:

Store the runbook in the Lab dataset (tank/Lab/docs/runbook/), in a Git repository on the NAS, or in a dedicated notes tool (Obsidian, Notion). The location doesn't matter as much as the discipline: the runbook exists, it gets updated.

The system overview

The first section. The 30,000-foot view of the system. An example:

# Home NAS — System Overview

## Hardware
- CPU: AMD Ryzen 5 5600G
- RAM: 64 GB DDR4
- Boot: 2× 500 GB SSD (mirror)
- Data: 6× 12 TB HDD (RAIDZ2, ~44 TB usable)
- Network: 1 GbE (with 10 GbE upgrade planned)

## Network
- Router: 192.168.1.1
- NAS: 192.168.1.10 (static)
- Lab VMs: 192.168.1.50-100
- Tailscale: 100.x.y.z (per-device IPs)

## Storage
- Pool: tank (single vdev, RAIDZ2, 6×12 TB)
- Datasets: Family, Photos, Videos, Documents,
            Private, PhoneUploads, Apps, Media/{Active,Archive}, Lab
- Snapshots: per-dataset policy (see Chapter 09 of V1)
- Scrub: monthly, 2nd Sunday 2 AM
- SMART: weekly short, monthly long

## Services
- Immich (photos): port 3001
- Jellyfin (media): port 8096
- Vaultwarden (passwords): port 8000
- Paperless-ngx (documents): port 8001
- Nextcloud (calendar/contacts): port 8002
- Home Assistant: port 8123
- Tailscale: 100.x.y.z
- Uptime Kuma: port 3001 (different host)

## Backup
- Local: external USB drive, weekly replication
- Offsite: rotated weekly with friend
- Cloud: Backblaze B2 (Media/Archive only)
- 3-2-1: verified quarterly

The overview is the answer to "what is this system." When you forget, you read this first.

The services and apps section

For each app, the minimum useful information:

Example:

## Immich
- Purpose: family photo and video library
- URL: http://192.168.1.10:3001 (local), http://100.x.y.z:3001 (Tailscale)
- Data: /mnt/tank/Apps/immich/{upload,postgres,cache}
- Update: TrueNAS Apps → Immich → Update (snapshot before)
- Restore: from /mnt/tank/Lab/backups/immich/ (manual backup)
- Common issues: see alerts runbook below

For 10+ services, this section gets long. That's fine; the runbook is a reference.

The credentials and secrets section

For each credential, the location. NOT the value (store the value in Vaultwarden). The location is enough:

## Credentials
- TrueNAS admin: Vaultwarden → Infrastructure → TrueNAS admin
- TrueNAS root: Vaultwarden → Infrastructure → TrueNAS root (sealed envelope in safe as backup)
- Tailscale auth key: Vaultwarden → Infrastructure → Tailscale
- Cloudflare API token: Vaultwarden → Infrastructure → Cloudflare
- Backblaze B2: Vaultwarden → Infrastructure → Backblaze
- Immich admin: Vaultwarden → Apps → Immich
- ...

With the credentials in Vaultwarden and the locations in the runbook, the system is auditable: you know where every credential lives.

The monitoring and alerting runbook

For each alert, the runbook. The format:

## Alert: Immich is down
Severity: high
What it means: The Immich web UI is not responding.
What to do:
  1. Check if the container is running: TrueNAS → Apps → Immich
  2. If stopped, start it
  3. If it won't start, check the logs: TrueNAS → Apps → Immich → Logs
  4. Common cause: the database (PostgreSQL) is not running
  5. If the database is the issue, restart it first, then Immich
  6. If neither helps, restore from the last good snapshot
False positives: Restart the app (brief blips can trigger the alert)
Documentation: https://immich.app/docs

One page per critical alert. The runbook is the answer to "what do I do when this fires." When the alert fires at 3 AM, you read the runbook. You don't think; you follow the steps.

The maintenance schedule

One section with the calendar of regular tasks:

## Maintenance schedule

### Daily
- (none — let the automation run)

### Weekly
- Saturday: check Ntfy for any alerts that fired
- Sunday: review the Uptime Kuma dashboard for any service issues

### Monthly
- 1st Sunday: long SMART test
- 1st Monday: review disk usage
- 1st Monday: review the Uptime Kuma monitors (add new, remove unused)
- 2nd Sunday 2 AM: pool scrub (automated)
- 2nd Monday: review the runbook; update if anything changed
- 2nd Tuesday: apply OS and app updates (snapshot before)

### Quarterly
- Restore a project from backup; verify
- Review the user list; remove anyone who shouldn't have access
- Review the security posture; check for advisories
- Update the TrueNAS config backup
- Test the offsite backup; retrieve and verify

### Annually
- Replace any aging drives (year 5+)
- Review the capacity plan; forecast the next 12 months
- Major TrueNAS version upgrade (if available)
- Review the runbook end-to-end; update as needed
- Practice the disaster recovery playbooks

The schedule is the rhythm. Once you have it, the maintenance becomes a habit.

The disaster playbooks

From Chapter 5: the playbooks for drive failure, two-drive failure, pool loss, ransomware, fire. Each is a step-by-step procedure. The runbook has them all, ready to be followed.

The change log

The runbook is a living document. Every change to the system should be reflected in the runbook, and the change log records it. The format:

## Change log

### 2026-07-20
- Added Home Assistant to the apps list
- Updated the network section to include the Home Assistant VM

### 2026-07-15
- Migrated from Docker Compose to TrueNAS Apps for Immich
- Updated the Immich section in services

### 2026-07-01
- Initial runbook created

The change log is the history. When something breaks and you're not sure when it changed, the change log answers the question.

How to keep it up to date

The hardest part of a runbook is keeping it current. The discipline:

  1. Update after every change. When you add a service, change a configuration, replace a drive — update the runbook in the same session.
  2. Review monthly. Once a month, read the runbook end-to-end. Anything that's drifted from reality, fix.
  3. Use the runbook. When you need to know something, check the runbook first. If the runbook is wrong, fix it. If the runbook is right, you saved time.

The cost: 10-15 minutes per change, 30-60 minutes per month for the review. The benefit: never being lost in your own system.

The runbook's location

For TK's build, the runbook lives in the Lab dataset:

All markdown. All version-controlled (in a Git repo, optionally pushed to a remote). All on the NAS, where the data is, where the runbook is most useful.

The runbook as the project's spine

The runbook is the spine. Every other operational practice (backups, monitoring, updates, capacity planning) hangs off it. The runbook is the answer to "how does this system work, and what do I do when something breaks."

Without the runbook, the system is a collection of apps, datasets, and configurations. With the runbook, it's a documented system that you (or someone else) can understand, maintain, and recover.

Engineering Note

The runbook is the most important document on the NAS. Not the photos. Not the videos. Not the documents. The runbook. The runbook is what lets the system survive you, survive the family, survive the years. The 30 minutes a month to maintain it is the most important 30 minutes of operations.

Summary

The runbook is a single source of truth for the home NAS. Sections: system overview, services, users, credentials, backup, monitoring, maintenance schedule, disaster playbooks, change log. Markdown, in the Lab dataset, version-controlled. Updated after every change. Reviewed monthly. Used daily. The runbook is the spine of operations. The 30 minutes a month is the most important 30 minutes of maintenance.

Checklist

Volume 6 is complete

3-2-1 backups, monitoring, security hardening, capacity planning, disaster recovery, updates, the runbook. The home NAS is now an operational system, not a build. The 30 minutes a month is the discipline that makes it last. Volume 6 is the last volume of The TK Home Infrastructure Handbook. The build is done. The system is documented. The discipline is the runbook.

Previous← Ch 06 · Update Strategy Back to start🏠 Home →

Ch 07 · v0.1 · drafted from the original ChatGPT conversation, July 2026