Generalizing Query Performance Prediction under Retriever and Concept Shifts via Data-driven Correction

Query Performance Prediction (QPP) aims to estimate the effectiveness of an information retrieval (IR) system without access to ground-truth relevance judgments. Existing supervised QPP methods typically follow a regression model framework that maps query-document representations to target metrics such as RR@10 or nDCG@10. However, these approaches often suffer from degraded performance under concept shift, where the distribution of relevance given a query-document pair changes between training and test datasets. This paper proposes a novel classification-based framework, QPP-MLC (QPP Multi-Label Classification), which formulates QPP as a multi-label classification task. QPP-MLC infers the relevance of each document among the top-k retrieved results and aggregates these document-level relevance predictions to predict the overall query performance. As a result, QPP-MLC provides a diagnosis tool for the concept shift and a correction method under the concept shift by modulating a threshold level of classification tasks. Experiments on MS MARCO and TREC DL benchmarks show that QPP-MLC achieves strong prediction accuracy and outperforms traditional regression-based QPP methods.