📖 Chapter 04 — Datasets

The most important concept in the system. Smart folders with their own rules.

Learning Objectives

• State the seven things a dataset can do that a regular folder cannot
• Decide when to use a dataset vs a regular folder inside a dataset
• Design a dataset tree for a family NAS
• Pick the right compression, encryption, and quota settings

Introduction

If you remember nothing else from this volume, remember this: datasets are the most important concept in ZFS. Not pools, not vdevs, not RAIDZ2 — datasets. The reason a ZFS system feels modern and a regular Windows share doesn't is almost entirely because of datasets.

Once you have the dataset model in your head, every question about "where do I put X" or "who can see Y" gets easier.

What a dataset is

A dataset is a folder inside a pool, but it has its own:

• Permissions (who can see it, who can write to it)
• Compression settings (how the bytes are squeezed)
• Snapshot schedule (how often a copy is taken)
• Quota (the maximum size it can grow to)
• Reservation (the minimum space it's guaranteed)
• Encryption (whether the bytes are scrambled at rest)
• Replication rules (where else it's mirrored to)
• SMB share configuration (how it's exposed to the network)

That list is the entire reason datasets exist. A regular folder can't do any of that. A dataset can do all of it, independently for every dataset you create.

The slogan

Dataset = Folder with superpowers.

Datasets vs folders vs nested folders

This is the most common design question. When do you make a new dataset, and when do you just make a regular folder inside an existing one?

Rule of thumb from the conversation:

• Make a dataset when you need different rules for different content. Different permissions, different snapshot schedules, different quotas, different encryption.
• Make a regular folder when the content is logically grouped but rules-wise identical. Photos from 2024 vs photos from 2025 don't need different snapshot policies — they're both just photos.

For TK's family NAS, the rule "make a dataset when the rules differ" produces this design:

Notice the deliberate choices:

• `Apps` is its own dataset because apps need their own permissions and don't need user access.
• `Media` is one dataset with two children (`Active`, `Archive`) because they share the same rules but represent different phases of work.
• Photos and Videos are separate datasets because Videos are huge and don't compress, so they need different compression and possibly different snapshot retention than Photos.

Compression

Compression is "essentially free on modern CPUs." Leave it on. Use lz4 for almost everything.

• Photos and videos don't compress much (they're already compressed by the camera). But leaving lz4 on doesn't hurt.
• Documents compress a lot (text has lots of repeated patterns). 50% savings is common.
• zstd is heavier compression but uses more CPU. Only worth it for cold archive data, if at all.

The decision is: just enable lz4 on every dataset you create. Move on.

Quotas and reservations

Two related concepts that work in opposite directions:

• Quota: the maximum size a dataset can grow to. Useful for: keeping `PhoneUploads` from filling the whole pool, giving each family member a `Private` dataset with a personal limit, capping experimental VMs.
• Reservation: the minimum space the dataset is guaranteed. Useful for: ensuring critical datasets (like `Documents`) can never run out of space because someone uploaded 4TB of vacation videos.

For TK's build, the conversation recommends:

• Quota on each private dataset (e.g. 500GB for Tom, 500GB for Mimi)
• Quota on `PhoneUploads` (e.g. 200GB) to keep it from becoming an unbounded dumping ground
• No reservation on shared datasets (the family can share the remaining space freely)

Encryption

Datasets can be encrypted at rest. The bytes are scrambled on the drives; even someone who physically steals the drives can't read the data without the key.

For most home NAS setups, encryption is optional but increasingly recommended:

• Pros: protects against physical theft, protects against drive disposal mistakes, protects data in off-site backups.
• Cons: if you lose the encryption key, the data is gone. Period. No recovery.

TrueNAS lets you set encryption per dataset. You can encrypt the whole pool (encrypts every dataset in it), or encrypt specific datasets (the rest are unencrypted). The conversation recommends per-dataset encryption for the most sensitive data (the `Private` family datasets) and leaving shared data unencrypted for simplicity.

Permissions — preview

Datasets are where permissions get interesting. Each dataset has its own owner, group, and permission bits. Chapter 07 covers this in detail; for now, just know that when you create a dataset, you'll be asked "who owns this and what can they do?" and your answer is per-dataset.

The PhoneUploads pattern (revisited)

From earlier in the conversation, the recommended pattern for phone uploads:

Why this works:

• Phone auto-uploads land in `PhoneUploads/Tom/IMG_0001.jpg` etc.
• You periodically (monthly, weekly, whenever) move keepers to `Photos/` or `Media/Active/`.
• PhoneUploads gets a short snapshot retention (7 days) and is excluded from off-site backups (covered in Volume 6).
• Photos gets a long snapshot retention (years) and is included in off-site backups.

TK decided not to use this pattern (manual uploads only) because he didn't want to use bandwidth on automatic junk. That's a valid call. The pattern is here so you can use it if your situation changes.

Snapshot retention per dataset

Datasets can have different snapshot policies. Here's the conversation's recommendation:

Engineering Note

Summary

Datasets are smart folders. They have their own permissions, compression, snapshots, quotas, reservations, encryption, and replication. The rule for when to make a new dataset: when the rules differ. Use lz4 compression everywhere. Use quotas to bound potentially-unbounded content (uploads, personal data, experiments). Encrypt sensitive data and back up the key. Per-dataset snapshot policies let you balance "long retention" against "lots of disk space" dataset by dataset.

Checklist

• ⬜ Sketch your dataset tree before you create the pool. You'll thank yourself later.
• ⬜ For each dataset, decide: compression, quota (if any), encryption (if any), snapshot policy.
• ⬜ If you encrypt anything, back up the encryption key NOW, not later.

Looking Ahead

Chapter 05 covers SMB shares — how Windows and Mac actually see your datasets. Datasets are a ZFS concept; SMB shares are the bridge from ZFS to the rest of the world.

Ch 04 · v0.1 · drafted from the original ChatGPT conversation, July 2026