📖 Chapter 07 — Reproducibility

Doing the work in a way that can be repeated. The discipline that makes analysis trustworthy.

Learning Objectives

• Define reproducibility in the data science context
• Use Git, conda/pip, and Docker for reproducibility
• Apply DVC for data versioning
• Build a reproducible analysis workflow

Introduction

Six months from now, you want to re-run an analysis. Can you? You have the notebook. You have the data. You have the code. But the Python version is different. The package versions are different. The data file is the wrong version. The result is different.

This is the reproducibility problem. The analysis was correct when you ran it, but you can't reproduce it. The discipline of reproducibility is the discipline of preventing this.

What reproducibility means

For a data scientist, reproducibility means: given the same inputs (code, data, environment), the same person (or a different person) gets the same output. The analysis can be re-run by anyone with the right tools and the right artifacts.

Three levels of reproducibility:

• Repeatable: the same person, on the same machine, gets the same result. The minimum.
• Reproducible: a different person, on a different machine, gets the same result. The standard.
• Replicable: a different team, with different data, gets a qualitatively similar result. The hardest; for science, not for home data science.

For home data science, the standard is "reproducible." The same code, the same data, the same environment, run by anyone with the right setup, gets the same result.

The four ingredients

A reproducible analysis has four ingredients:

1. Code: the scripts and notebooks that did the analysis. In version control (Git).
2. Data: the input data. Versioned (with DVC, or with a hash in the filename).
3. Environment: the Python version, the package versions, the system libraries. Captured in a file (requirements.txt, environment.yml, Dockerfile).
4. Documentation: a README, a methodology note, a description of what the analysis does and how to run it.

Without any of these, the analysis is not reproducible. With all four, the analysis can be re-run by anyone.

Git for code

Git is the standard for code versioning. The discipline:

• Every project is a Git repository.
• Every change is a commit with a meaningful message.
• Tags mark releases (v1.0, v1.1, etc.) and the state of the code at the time of a specific result.
• Branches separate work-in-progress from the main code.

For a home data science project, a single repository per project is enough. The repository lives in projects/<name>/ on the NAS. Push to a remote (GitHub, GitLab, or a self-hosted Gitea) for backup.

DVC for data

Git is great for code; it's not great for data. Large data files bloat the repository; binary data doesn't diff well; the data isn't really versioned in the Git sense.

DVC (Data Version Control) is the standard for data versioning. DVC works alongside Git:

• The data files live in a DVC-managed storage (local, S3, GCS, etc.).
• Git tracks small .dvc files that point to the data.
• git checkout + dvc checkout retrieves the data at that version.

For home data science, DVC is overkill for small datasets. For datasets over 1 GB, or for analysis where the exact data version matters, DVC is the right answer.

conda or pip for the environment

For Python environments, the two standard tools:

• conda: a package manager that handles Python and non-Python dependencies (NumPy, BLAS, CUDA). More powerful; slightly more complex.
• pip: the standard Python package manager. Simpler; doesn't handle non-Python dependencies.

For most home data science, pip is enough. The discipline: pip freeze > requirements.txt captures the exact versions. The requirements.txt goes in Git. To recreate the environment: pip install -r requirements.txt.

For projects with complex dependencies (PyTorch + CUDA, for example), conda is the right answer. The discipline: conda env export > environment.yml. The environment.yml goes in Git. To recreate: conda env create -f environment.yml.

Docker for the full environment

For maximum reproducibility, Docker captures the entire environment: the OS, the system libraries, the Python version, the packages, the code. A Dockerfile describes it all. The container runs the same way anywhere.

For a home data science project, the Dockerfile is overkill. But for a project you want to share with others, or for a service you want to deploy, Docker is the right answer.

The pattern: a Dockerfile that starts from a base image (e.g., jupyter/scipy-notebook), copies the code, installs the requirements, sets the entry point. The container is the analysis.

The README

For every project, a README. The minimum:

• What the project does (one paragraph)
• How to set it up (clone, install, configure)
• How to run it (the command to run the analysis)
• Where the data is (the data path or how to download it)
• Where the results are (the output path)

For a model: the metric, the data version, the environment, the model artifact location. For a dashboard: how to start it, how to configure it. For a report: how to generate it.

The README is the entry point. Without it, the project is opaque. With it, the project is self-explanatory.

The methodology note

For a non-trivial analysis, a methodology note (in docs/) explains the approach:

• What data was used and why
• What methods were applied and why
• What assumptions were made
• What the limitations are

The methodology note is what makes the analysis trustworthy. The reader can see the reasoning, not just the result. Future-you can remember why a particular choice was made.

The "is this reproducible" checklist

For a project, run through this checklist:

• ⬜ Is the code in Git?
• ⬜ Is the data versioned (DVC, or a hash in the filename)?
• ⬜ Is the environment captured (requirements.txt, environment.yml, Dockerfile)?
• ⬜ Is there a README explaining how to run it?
• ⬜ Can a different person (or future-you) re-run it and get the same result?

If the answer to all five is yes, the project is reproducible. If any answer is no, the project has a reproducibility gap.

The cost of reproducibility

Reproducibility costs time. The 30 minutes to write a README, the 10 minutes to set up Git, the 5 minutes to run pip freeze — they add up. The benefit is also real: you can re-run the analysis in 6 months, you can hand it off to a collaborator, you can debug a problem by re-running with the same conditions.

The conversation's principle: reproducibility is a habit, not a task. You do it from the start of a project, not as a cleanup at the end. The cost is small when it's part of the workflow; the cost is high when it's a separate effort.

Engineering Note

Summary

Reproducibility means: same code, same data, same environment, same result. The four ingredients: code in Git, data versioned (DVC), environment captured (requirements.txt or environment.yml), documentation (README + methodology note). The cost is small when it's a habit. The benefit is re-runnable, shareable, durable work. Reproducibility is the difference between analysis and anecdote.

Checklist

• ⬜ For each new project, start with a Git repository
• ⬜ For each project, capture the environment with pip freeze or conda env export
• ⬜ For projects with large or versioned data, use DVC
• ⬜ Write a README for each project: what, how, where
• ⬜ Quarterly: pick a project, re-run it from scratch, verify the result

Volume 5 is complete

The data scientist's NAS, Jupyter, data formats, pipelines, ML, sharing, reproducibility. The data scientist's home lab is now a real platform. Volume 6 is the last volume: the operations, the security, the disaster recovery. The volume that turns "I built it" into "I can keep it running for years."

Ch 07 · v0.1 · drafted from the original ChatGPT conversation, July 2026