📖 Chapter 07 — Monitoring
Knowing when something is broken, before the family notices.
Learning Objectives
- Set up uptime monitoring with Uptime Kuma
- Collect metrics with Prometheus
- Visualize metrics with Grafana
- Configure alerts so you know when something is broken
Introduction
You have a NAS. You have apps. You have a home lab. Something is going to break. The question is: will you know before the family notices, or will you find out when Mimi says "why can't I see the photos?"
Monitoring is the discipline of knowing. Three layers: uptime (is the service reachable?), metrics (how is the service performing?), and alerts (notify me when something is wrong).
The three layers
For a home lab, the three layers serve different purposes:
- Uptime monitoring: "Is the service responding?" Check every 30-60 seconds. If it stops responding, alert. This is the smoke detector.
- Metrics: "How is the service performing?" CPU, RAM, disk, network, request rate, latency. This is the dashboard.
- Logs: "What happened?" Detailed event records. The black box.
For a home lab, the right balance is: solid uptime monitoring (smoke detector), some metrics (dashboard), and occasional log diving when something's wrong. Don't over-engineer.
Uptime Kuma
Uptime Kuma is a self-hosted uptime monitoring tool. The killer feature: it's a single Docker container, with a clean web UI, and it can monitor any HTTP/HTTPS endpoint, TCP port, DNS record, or ping. It's the right default for the home lab.
Setup:
version: "3.9"
services:
uptime-kuma:
image: louislam/uptime-kuma:1
restart: unless-stopped
ports:
- "3001:3001"
volumes:
- /mnt/tank/Lab/uptime-kuma/data:/app/data
After installation, open the web UI and add monitors. For each service:
- Immich: HTTP monitor, URL
http://<nas>:3001/api/server/ping, every 60 seconds - Jellyfin: HTTP monitor, URL
http://<nas>:8096/System/Info/Public - Vaultwarden: HTTP monitor, URL
http://<nas>:8000/alive - Nextcloud: HTTP monitor, URL
http://<nas>:<port>/status.php - The NAS itself: ping monitor (if your network allows ICMP) or HTTP monitor on the TrueNAS UI
Alerting
Uptime Kuma alerts you when a monitor goes down. The alert channels:
- Email: SMTP to your email address. Simple, works everywhere.
- Slack / Discord / Telegram: webhooks to the chat service. Alerts appear in a channel.
- Pushover / Ntfy / Gotify: push notifications to your phone. Best for immediate alerts.
- SMS via Twilio: expensive but reliable. For when you absolutely must know.
For a home lab, Ntfy is a great default. It's a self-hosted (or cloud) push notification service. Install the Ntfy app on your phone, set up a topic, configure Uptime Kuma to send to the topic. Alerts appear as phone notifications.
Prometheus and Grafana
For metrics — CPU, RAM, disk, network — the right tools are Prometheus (the collector) and Grafana (the visualizer). They're the standard for self-hosted metrics.
Setup:
- Install Prometheus. It scrapes metrics endpoints from various services (the "exporters") on a schedule.
- Install Grafana. It connects to Prometheus and visualizes the metrics.
- Install exporters:
node_exporterfor the host's CPU/RAM/disk,cAdvisorfor Docker containers, plus service-specific exporters (e.g., the Postgres exporter for PostgreSQL). - Configure Grafana dashboards. Use a community dashboard (there are thousands on Grafana's dashboard library) for each exporter.
The full Prometheus + Grafana stack is heavy. For a small home lab, a lighter alternative is Netdata: a single container that collects metrics, visualizes them, and alerts. Less powerful but much simpler.
What to monitor
The minimum useful monitoring for a home lab:
- Every service the family uses: Immich, Jellyfin, Vaultwarden, Paperless, Nextcloud. If any of these stops responding, the family notices.
- The NAS itself: CPU, RAM, disk usage, network. If the NAS is overloaded or running out of disk, services will start failing.
- The pool: ZFS status, capacity, last scrub. (TrueNAS has its own alerts for these; Uptime Kuma is for the services, not the pool.)
- The network: the router, the internet connection. If the internet is down, the services are still running but unreachable from outside.
The "alert fatigue" problem
If you configure alerts for every minor thing, you get alert fatigue: you start ignoring the alerts because most of them are noise. The mitigation:
- Alert on user-visible problems. "Immich is down" is user-visible. "Jellyfin took 200ms longer to respond" is not.
- Set thresholds thoughtfully. A 90% disk alert is useful. A 50% disk alert is noise.
- Test alerts. Make sure they actually fire when they should.
- Maintain the alerts. When you stop caring about a metric, delete the alert.
For a home lab, the right number of alerts is 3-5: the most critical services, the most critical resources. Anything more is noise.
Logging
Logs are the third layer. For most home lab services, the logs are in the container's docker logs output. For TrueNAS, the logs are in the web UI. For the NAS's system, the logs are in /var/log/ on the TrueNAS shell.
For a small home lab, you don't need a centralized logging system. When something's wrong, docker logs <container> shows you what happened. The TrueNAS UI shows system events. The router's admin shows network events.
For a larger home lab (10+ services, multiple VMs), a centralized logging system like Loki or ELK is worth the setup. For a typical home lab, it's overkill.
The "what to do when an alert fires" runbook
For each alert, document what to do when it fires. The runbook:
- Alert: Immich is down
- Check: is the container running?
docker ps | grep immich - If not, restart it:
docker compose up -d immich-server - If it won't start, check the logs:
docker logs immich-server - Common cause: the database isn't running. Check immich-db.
- Check: is the container running?
- Alert: NAS disk usage > 90%
- Check: what's using the disk? (TrueNAS → Storage)
- Clean: delete old snapshots, large files you forgot about
- Or: add capacity (Volume 3, Chapter 7)
- Alert: Jellyfin is slow
- Check: is hardware transcoding enabled?
- Check: is the network saturated?
- Check: is the source media too heavy (4K HDR over 1 GbE)?
For each critical service, write a one-page runbook. Keep it in the Lab dataset's docs/ folder. Reference it when the alert fires.
The "is it worth it" question
For a tiny home lab (1-2 services), monitoring is overhead. The "I think it's working" check (occasionally visit the service) is enough.
For a real home lab (5+ services), monitoring is worth it. The first time an alert wakes you up to a problem that would have been a 4-hour outage without it, you understand the value.
The conversation's recommendation: start with Uptime Kuma only. Add Prometheus and Grafana later if you want metrics. Add Loki/ELK later if you want centralized logs. Layer the monitoring as the lab grows.
Engineering Note
Monitoring is the difference between a system you trust and a system you hope. A system you trust: you know it's working, you know when it's not, and you have the runbook to fix it. A system you hope: you assume it's working until it isn't. Hope is not a strategy. Monitoring turns hope into knowledge.
Summary
Uptime Kuma for service-level monitoring. Prometheus + Grafana for metrics (or Netdata for a lighter option). Ntfy for phone alerts. The right number of alerts is 3-5, the most critical. For each alert, write a runbook. Don't over-engineer: start with Uptime Kuma, add metrics when you need them. Monitoring is what turns "I think it's working" into "I know it's working."
Checklist
- ⬜ Install Uptime Kuma
- ⬜ Add monitors for every service the family uses
- ⬜ Configure Ntfy (or similar) for phone alerts
- ⬜ Write a one-page runbook for each critical service
- ⬜ After a month, review alerts: which fired for real, which were noise?
- ⬜ Adjust thresholds; delete alerts that don't earn their keep
Looking Ahead
Chapter 08 is home automation. Home Assistant, MQTT, Zigbee, Z-Wave — the smart home that runs on the NAS, doesn't depend on the cloud, and respects the family's privacy. The chapter that turns the NAS into the brain of the house.