📖 Chapter 05 — Machine Learning

The ML lifecycle on the NAS. Training, tracking, deploying, monitoring.

Learning Objectives

• Train ML models on the NAS (scikit-learn, PyTorch)
• Track experiments with MLflow
• Save and version model artifacts
• Deploy models for inference (locally or via an API)

Introduction

You have data, you have pipelines, you have analysis. Now: machine learning. The ML lifecycle is different from analysis: training a model takes time and resources, the model artifact is the output (not the chart), and you need to know if the model is still working in production.

This chapter covers the ML lifecycle on the NAS. For most home data science, the workload is: train a model on a CPU, save the model, use it for predictions (locally or via a small API). For heavier workloads (deep learning, large models), a GPU server (Volume 4, Chapter 9).

The ML lifecycle

The standard ML lifecycle has five steps:

1. Data preparation: clean, transform, and split into train/test/validation sets.
2. Training: fit a model on the training data. Choose the algorithm, the hyperparameters, the features.
3. Evaluation: score the model on the test set. Compare to baselines. Decide if the model is good enough.
4. Deployment: save the trained model. Make it available for predictions (an API, a batch job, a local function).
5. Monitoring: in production, track the model's inputs and outputs. Detect when the model's performance degrades (data drift, concept drift).

For home data science, the lifecycle is similar to a production setup, but with smaller scale and less strict SLAs. The principles are the same.

Tooling: the right ones for the right job

• scikit-learn: the standard for classical ML. Linear models, tree-based models, clustering, preprocessing, model selection. Pure Python, CPU-only, fast for small-to-medium datasets.
• PyTorch: the standard for deep learning. Neural networks, computer vision, NLP, LLMs. CPU and GPU support. Heavier than scikit-learn.
• MLflow: the standard for experiment tracking. Records parameters, metrics, artifacts. Has a UI to compare runs.
• Hugging Face Transformers: for pre-trained models (LLMs, vision models). Loads models from the Hub, fine-tunes them, deploys them.

For most home ML, scikit-learn + MLflow is the right stack. PyTorch for deep learning. Hugging Face when you need a pre-trained model.

Training a model

Example with scikit-learn:

import polars as pl
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import mlflow
import mlflow.sklearn

# Load the data
df = pl.read_parquet("/mnt/tank/Lab/data/processed/features.parquet")

# Split
X = df.drop("target").to_numpy()
y = df["target"].to_numpy()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Train
with mlflow.start_run():
    model = RandomForestClassifier(n_estimators=100, max_depth=10)
    model.fit(X_train, y_train)

    # Evaluate
    y_pred = model.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    mlflow.log_metric("accuracy", accuracy)

    # Log the model
    mlflow.sklearn.log_model(model, "model")

    # Log parameters
    mlflow.log_params({"n_estimators": 100, "max_depth": 10})

The with mlflow.start_run() block creates a "run" in MLflow. The parameters, metrics, and model are all recorded. The model artifact is saved to MLflow's storage, which can be on the NAS.

MLflow on the NAS

MLflow has two parts:

• MLflow Tracking: the part that records runs. Has a UI for comparing runs.
• MLflow Registry: the part that stores model versions, transitions models through stages (Staging → Production), and serves models.

Install MLflow on the NAS:

version: "3.9"

services:
  mlflow:
    image: ghcr.io/mlflow/mlflow:latest
    restart: unless-stopped
    ports:
      - "5000:5000"
    volumes:
      - /mnt/tank/Lab/mlflow:/mlflow
    command: mlflow server --host 0.0.0.0 --backend-store-uri sqlite:///mlflow/mlflow.db --default-artifact-root /mlflow/artifacts

Set the tracking URI in your training script:

import mlflow
mlflow.set_tracking_uri("http://<nas>:5000")

Now all runs are recorded on the NAS, with the UI at http://<nas>:5000.

Why MLflow matters

Without MLflow, "I trained a model" is a vague claim. You trained it with what parameters? What was the accuracy? What data? Can you reproduce it?

With MLflow, "I trained a model" is a record. The run has the parameters, the metrics, the model artifact, the git commit hash of the training code, the environment. Future-you can re-run the same training and get the same result.

The discipline: every model is an MLflow run. Every run has parameters, metrics, and the model. The code, the data, the model — all linked.

GPU for training

For most home ML, CPU is enough. A modern CPU can train a scikit-learn model on a small-to-medium dataset in minutes to hours. For deep learning, GPU helps a lot.

For a GPU server, the right approach (from Volume 4, Chapter 9): a separate machine with one or more GPUs. Pass the GPU through to the training container. The data is on the NAS, mounted via NFS.

For most home data science, a CPU is enough. The conversation's recommendation: start with CPU. Buy a GPU when you have a specific workload that needs it (deep learning, LLM fine-tuning, Stable Diffusion training).

Deploying a model

For most home data science, the model is used in one of three ways:

• Batch predictions: the model is loaded by a script, predictions are made on a batch of inputs, the results are saved. Used for reports, batch jobs, scheduled work.
• Interactive predictions: the model is loaded in a Jupyter notebook, the user calls it on inputs they specify. Used for exploration, ad-hoc analysis.
• An API: the model is served as an HTTP endpoint, other services call it. Used when predictions are needed in real-time (e.g., a chat app that uses an LLM).

For batch and interactive, the model is loaded with scikit-learn or PyTorch's standard API. For an API, the right tools:

• FastAPI: a Python web framework. Easy to set up an inference endpoint. Run as a Docker container.
• MLflow serve: MLflow can serve a registered model as an API. mlflow models serve -m models:/my-model/production.
• vLLM (for LLMs): a high-throughput LLM serving framework. For serving large language models efficiently.

For most home data science, FastAPI is the right default. The endpoint is a few lines of code, runs in a container, and is reachable from the network.

Monitoring in production

For a model in production, the question is: is it still good? The monitoring:

• Input drift: the distribution of inputs changes. The model sees data that's different from what it was trained on.
• Output drift: the distribution of predictions changes. The model's behavior is different.
• Performance degradation: the model's accuracy on labeled data decreases.

For most home data science, monitoring is a "nice to have" — the model is used occasionally, and degradation is caught when someone notices. For a production model, set up a monitoring system: log inputs and predictions, compare distributions, alert on drift.

The "what to save" question

For a trained model, save:

• The model artifact: the trained model file (scikit-learn's joblib, PyTorch's state_dict, etc.)
• The training script: the code that trained the model. In Git, with a commit hash.
• The data version: which version of the dataset was used. (DVC for this, or a hash of the Parquet file.)
• The environment: the Python packages used (requirements.txt or environment.yml).
• The metrics: accuracy, precision, recall, whatever matters for the use case.

MLflow captures all of this. The run record is the model lineage: "this model was trained on this data, with this code, in this environment, achieving these metrics."

Engineering Note

Summary

The ML lifecycle on the NAS: data → training → evaluation → deployment → monitoring. scikit-learn for classical ML, PyTorch for deep learning. MLflow for experiment tracking. FastAPI for serving models. CPU for most home workloads, GPU when needed. The discipline: every model is an MLflow run, with parameters, metrics, code, data, and the model artifact. The run is the lineage.

Checklist

• ⬜ Install MLflow on the NAS
• ⬜ For each model, log parameters, metrics, the model, and the data version
• ⬜ Save the training script in Git, with a commit hash in the MLflow run
• ⬜ For deployed models, set up FastAPI (or MLflow serve) for the inference endpoint
• ⬜ For production models, set up monitoring for input/output drift

Looking Ahead

Chapter 06 is sharing and collaboration. Notebooks, datasets, results. How to share analytical work with other people: securely, with the right access, and with the right context. The chapter that turns "I have an analysis" into "the team can review, reproduce, and build on the analysis."

Ch 05 · v0.1 · drafted from the original ChatGPT conversation, July 2026