📖 Chapter 02 — Monitoring and Alerting

Knowing what's working, knowing when something breaks.

Learning Objectives

• Set up monitoring for the services, the hardware, and the network
• Configure alerts that fire on real problems, not noise
• Build a dashboard for the most important metrics
• Know when something is wrong before the family notices

Introduction

From Volume 4, Chapter 7: Uptime Kuma, Prometheus, Grafana. This chapter goes deeper on the operational side: the discipline of monitoring, the patterns that work, the alerts that earn their keep. The goal: know when something is broken, ideally before the family notices, and have a runbook for what to do.

What to monitor

For a home NAS, three categories of monitoring:

• Services: are the apps the family uses (Immich, Jellyfin, Vaultwarden, etc.) reachable and responding?
• Hardware: is the NAS itself healthy? CPU, RAM, disk, temperature, network.
• Storage: is the pool healthy? Capacity, snapshot retention, scrub status, SMART status.

Each category has different tools. Services: Uptime Kuma. Hardware and storage: TrueNAS's built-in monitoring + Prometheus + Grafana (or Netdata for simplicity).

Service monitoring with Uptime Kuma

From Volume 4, Chapter 7: Uptime Kuma is the right default for service monitoring. The setup:

1. Install Uptime Kuma in a container (or via the TrueNAS Apps catalog).
2. Add a monitor for each service: HTTP, TCP, or ping.
3. Set the check interval: 60 seconds is a good default.
4. Set the retry interval: how long before a failure is real (vs a transient blip).
5. Configure the alert channels: email, Ntfy, Slack, Discord, etc.

For each service, define what "down" means. For Immich, "down" might be the web UI not responding for more than 2 minutes. For the NAS itself, "down" might be the TrueNAS web UI not responding for more than 5 minutes. The right threshold depends on the service's normal restart time and how often it has transient blips.

The "what to monitor" checklist

For a complete home lab setup, monitor:

• Family services: Immich, Jellyfin, Vaultwarden, Paperless, Nextcloud (if used). The apps the family actually uses.
• Lab services: Uptime Kuma itself, the reverse proxy, the monitoring stack, the data science stack (Jupyter, MLflow).
• Infrastructure: the NAS's web UI, the NAS's SSH, the router's web UI, the internet connection (via an external monitor like UptimeRobot or a ping to 8.8.8.8).

For each, an HTTP or TCP monitor with a reasonable check interval. For most home labs, 10-30 monitors is enough.

Alert channels

For a home lab, the right alert channels:

• Ntfy: a self-hosted (or cloud) push notification service. Install the app, set up a topic, configure Uptime Kuma to send alerts to the topic. Alerts appear as phone notifications.
• Email: SMTP to your email address. Simple, works everywhere. Good for less-urgent alerts (SMART warnings, snapshot failures).
• Discord/Slack: webhooks to a chat channel. Good for team environments; less useful for a single-person home lab.

For most home labs, Ntfy is the right default for immediate alerts, email for daily digests. Ntfy for "Immich is down right now"; email for "your weekly backup succeeded" or "the pool is at 85% capacity."

Hardware monitoring

For the NAS's hardware, the right tools:

• TrueNAS's built-in monitoring: shows CPU, RAM, disk, network, pool status. Configure email alerts for SMART failures, scrub errors, and other critical events.
• Netdata: a single-container monitoring tool. Real-time metrics for the host and its containers. Easier than Prometheus + Grafana, less powerful.
• Prometheus + Grafana: the standard for serious monitoring. Powerful, flexible, heavier. Worth the setup for a complex home lab.

For most home labs, TrueNAS's built-in monitoring + Netdata is enough. Prometheus + Grafana is for when you have multiple hosts, complex queries, or specific dashboard needs.

What to alert on

For hardware, alert on:

• SMART failures or warnings (a drive is reporting problems)
• Scrub errors (a drive returned bad data during the scrub)
• Pool degradation (a vdev has lost redundancy)
• Disk capacity > 85% (you have time to clean up before it gets critical)
• CPU temperature > 80°C (the system is overheating)

For services, alert on:

• Service unreachable for more than 2-5 minutes
• Service returning 5xx errors for more than 5 minutes

For storage, alert on:

• Snapshot task failed
• Replication task failed
• SMART test failed

The "alert fatigue" problem

If you configure alerts for every minor thing, you get alert fatigue. You start ignoring the alerts because most are noise. The mitigation:

• Alert on user-visible problems. "Immich is down" is user-visible. "Immich took 200ms longer to respond" is not.
• Set thresholds thoughtfully. 90% disk is useful. 50% is noise.
• Test alerts. Make sure they actually fire when they should.
• Maintain the alerts. When you stop caring about a metric, delete the alert.

For a home lab, the right number of alerts is 5-10. Any more is noise. The discipline: every alert has a runbook. The alert fires, you follow the runbook, you fix the problem, you update the runbook if needed.

Dashboards

For visual monitoring, a dashboard. The minimum useful dashboard for a home NAS:

• Service status: green if all up, red if any down (Uptime Kuma has a public status page)
• CPU and RAM usage (last 24 hours)
• Network throughput (last 24 hours)
• Disk usage per pool (current)
• Last scrub date (countdown to next)

For most home labs, the dashboard is in Netdata or Grafana. The dashboard is for you; it's not for the family. The family doesn't need to see the CPU usage. They need the services to work.

Logging

Logs are the third monitoring layer. For a small home lab, the logs are where you go when something's wrong:

• Container logs: docker logs <container> or the TrueNAS Apps UI.
• TrueNAS system logs: the TrueNAS web UI, System → Audit.
• TrueNAS middleware logs: the shell, /var/log/middlewared.log.

For a more complex setup, a centralized logging system (Loki, ELK) is worth the setup. For a small home lab, the in-place logs are enough.

The "is monitoring worth it" question

For a small home lab (1-2 services), monitoring is overhead. The "I think it's working" check (occasionally visit the service) is enough.

For a real home lab (5+ services), monitoring is worth it. The first time an alert wakes you up to a problem that would have been a 4-hour outage, you understand the value.

The discipline: start with Uptime Kuma + the TrueNAS built-in monitoring. Add Netdata if you want metrics. Add Prometheus + Grafana later if you need it. Layer the monitoring as the lab grows.

Engineering Note

Summary

Monitor services (Uptime Kuma), hardware (Netdata or TrueNAS built-in), storage (TrueNAS built-in). Alert on real problems: service down, SMART failure, pool degraded, disk over 85%. Use Ntfy for immediate alerts, email for digests. Set 5-10 alerts, not 50. Each alert has a runbook. Review monthly. The discipline: monitoring is a habit, not a setup task. The 30 minutes a month is what makes it work.

Checklist

• ⬜ Install Uptime Kuma and add monitors for every critical service
• ⬜ Configure Ntfy for immediate alerts
• ⬜ Configure TrueNAS email alerts for SMART, scrub, and pool events
• ⬜ Write a one-page runbook for each critical alert
• ⬜ Test each alert: verify it fires when the service is down
• ⬜ Monthly: review the alerts, update the runbooks, remove noise

Looking Ahead

Chapter 03 is security hardening. The layers that protect against attackers: the network, the OS, the apps, the credentials, the monitoring. Defense in depth. The chapter that turns "I have a NAS on the internet" into "I have a NAS on the internet that I've made as hard to break into as I reasonably can."

Ch 02 · v0.1 · drafted from the original ChatGPT conversation, July 2026