Manuscript · 2026 · In preparation
Silent Ranking Changes Under Dependency Upgrades in MovieLens Recommender Pipelines
Abstract
Dependency upgrades can change a recommender’s outputs while it continues to run. We evaluate 35 upgrade transitions across 201 single-threaded MovieLens 1M cases spanning eleven pipeline configurations with shallow models or preprocessing. The design uses same-environment controls, frozen-model inference, paired-seed retraining and a separately pinned evaluator. Of the 201 cases, 164 retain identical checkpoint digests, 14 change a checkpoint, 16 fail at runtime and 7 are inapplicable. The changes occur in 7 of 35 transitions, and 9 cases change a top-10 list. Interventions link the changes to four mechanisms: exact-tie selection, OpenBLAS core selection, a bundled OpenBLAS binary and the last-bit output of log1p. All completed real ranking comparisons meet the declared absolute NDCG@10 equivalence margins. The results support measuring checkpoint changes, ranking changes, contract failures and relevance loss separately.
Research questions
- What installation, runtime, behavioural and quality outcomes arise under dependency upgrades, and what are the corresponding denominators?
- Which pipeline stages and user activity groups are affected, and which mechanisms account for the observed changes?
- Which inexpensive check distinguishes benign list churn from material offline quality loss?
Reproducing it
Every table, figure and number in the paper is generated from a results ledger by scripts in the repository. Protocols were frozen by git tag before held-out runs. Requirements are uv and Docker; MovieLens is downloaded and checked against pinned checksums, never redistributed.