Nobody can reproduce the model that is in production
The regulator asked how a decision was made. The answer took eleven days and was 'approximately'.
The system
- A credit-scoring model in production for 14 months at a lender.
- Trained in a notebook on a data-science workstation, exported as a pickle, handed to engineering.
- Training data was 'the customer table as of some date in March'.
- Library versions were whatever the environment had; there is no lock file.
What you observe
- A regulator requests the exact inputs and logic behind a specific declined application.
- Re-running the notebook produces a model whose predictions differ from production on ~7% of cases.
- The original training snapshot no longer exists; the source table is mutable and has been updated since.
- The pickle will not load under the current library versions.
Stop and answer before reading on
What do you check first, and why that before anything else? Write down three checks in order. The ordering is what is being graded — anyone can list plausible causes.
Diagnostic order
- 1
Establish what is actually recoverable, and write it down.
Separate 'we have it', 'we can reconstruct it with stated assumptions', and 'it is gone'. This distinction is the entire deliverable to a regulator; hedging across all three is what turns a bad situation into a compliance finding.
- 2
Pin down the artifact that was actually serving.
Whatever the notebook does now, the deployed binary is the ground truth for what happened. Retrieve it from the deployment system, along with its container image and dependency manifest if any.
- 3
Reconstruct the training snapshot from immutable sources.
Transaction logs, warehouse time-travel, backups — anything append-only. If the source table is mutable and undated, the honest answer is that the training set is not recoverable, and that is the finding.
- 4
Quantify the reproduction gap rather than claiming success.
Report the 7% disagreement explicitly and characterise where it concentrates. A reproduction that is 93% faithful, honestly stated, is defensible. One presented as exact is not.
Root cause
There was never a lineage record. The model was an artifact with no recorded link to a data version, a code commit, an environment, or a set of hyperparameters. Nothing failed at any single point — the system was simply never built to answer the question, and by the time it was asked, mutable data had made the answer unrecoverable.
The fix
- Immediate: freeze and archive the production artifact, its container image, and the current source data snapshot.
- Register every future model in a model registry with data version, code commit, environment hash, hyperparameters and evaluation report attached.
- Move training data onto immutable, versioned snapshots (warehouse time travel, or DVC/LakeFS over object storage).
- Replace notebook training with a pipeline that runs in CI from a commit, so 'how was this built' has a URL as its answer.
What should have caught it
- Treat reproducibility as a release gate: a model that cannot be rebuilt from recorded inputs does not ship.
- Model cards recording intended use, training data, evaluation slices and known limitations, written at training time rather than during an audit.
- Retention policy for training snapshots that matches the regulatory retention requirement for the decisions the model makes.
- Periodic reproduction drills — pick a production model at random and rebuild it. The first drill always fails; that is the point.
Follow-ups you should expect
- What is the minimum lineage metadata you would mandate, given engineering time is finite?
- How do you version a 4 TB training set without copying it?
- Non-determinism from GPU kernels and data-loader shuffling remains. How close is close enough?
- How does this change if the model is fine-tuned from a third-party base model?