Samrand Hassan
English

Manuscript · 2026 · In preparation

Silent Ranking Changes Under Dependency Upgrades in MovieLens Recommender Pipelines

Samrand M. Hassan

Read the PDF Code and data

Abstract

Dependency upgrades can change a recommender’s outputs while it continues to run. We evaluate 35 upgrade transitions across 201 single-threaded MovieLens 1M cases spanning eleven pipeline configurations with shallow models or preprocessing. The design uses same-environment controls, frozen-model inference, paired-seed retraining and a separately pinned evaluator. Of the 201 cases, 164 retain identical checkpoint digests, 14 change a checkpoint, 16 fail at runtime and 7 are inapplicable. The changes occur in 7 of 35 transitions, and 9 cases change a top-10 list. Interventions link the changes to four mechanisms: exact-tie selection, OpenBLAS core selection, a bundled OpenBLAS binary and the last-bit output of log1p. All completed real ranking comparisons meet the declared absolute NDCG@10 equivalence margins. The results support measuring checkpoint changes, ranking changes, contract failures and relevance loss separately.

Research questions

  1. What installation, runtime, behavioural and quality outcomes arise under dependency upgrades, and what are the corresponding denominators?
  2. Which pipeline stages and user activity groups are affected, and which mechanisms account for the observed changes?
  3. Which inexpensive check distinguishes benign list churn from material offline quality loss?

Reproducing it

Every table, figure and number in the paper is generated from a results ledger by scripts in the repository. Protocols were frozen by git tag before held-out runs. Requirements are uv and Docker; MovieLens is downloaded and checked against pinned checksums, never redistributed.

Get in touch

Research collaboration, a talk, or a hard engineering problem — write to me.