Exploiting Calibration as a Multi-Objective Recommender System

Diego Correa da Silva

Exploiting Calibration as a Multi-Objective Recommender System

Collaborative Recommender Systems generate personalized recommendations by analyzing users' past interactions. However, traditional approaches often prioritize relevance, leading to issues such as super-specialization, popularity bias, and class imbalance. These limitations can result in recommendation lists that fail to represent the full spectrum of a user’s interests fairly. In this sense, Calibrated Recommendations address this problem by balancing relevance with fairness (calibration), ensuring that the distribution of recommended items aligns more closely with the user’s preference distribution. For example, when the user's profile comprises 80% Adventure and 20% Sci-fi, the calibrated recommendation seeks to generate a list following this distribution. Relevance and calibration are two distinct goals that the system should achieve. This multi-objective is reached through a trade-off balance approach. This thesis addresses calibrated recommendations as a multi-objective recommendation system, aiming to measure and improve the calibration of the recommendation list using the user's preferences as a target. Thus, we divide the goals of this thesis into studies, and inside each study, research questions were raised and answered. In the first study, we systematically benchmark 57 fairness measures, introducing novel methods for extracting user preference distributions and refining relevance estimation. As a result, four measures achieve the same four best performances. In the second study, we explored the broader impact of calibration on key recommendation objectives, including novelty, coverage, personalization, unexpectedness, and serendipity. Our findings indicate that calibration enhances item coverage and personalization while maintaining high recommendation utility. In the third study, we investigate the structural properties of the distributions used in calibrated recommendations. Unlike traditional recommender systems that operate in a one-dimensional space, calibrated recommendations involve high-dimensional user preference distributions. Our analysis shows that calibrated recommendation lists naturally form distinct user clusters, a phenomenon best understood through outlier detection models. In the fourth study, we propose two novel approaches for modeling user preferences to enhance the accuracy and adaptability of calibration techniques. The first method incorporates time-sensitive weighting to discount outdated preference information. The second method introduces an entropy-based approach to better capture user preferences in domains where item features are set-valued, such as movies with multiple genres. Experimental evaluations confirm that these approaches effectively reduce miscalibration while maintaining recommendation accuracy. Overall, this thesis advances the field of calibrated recommendations by providing a comprehensive evaluation of fairness measures, proposing novel calibration techniques, and analyzing the structural properties of user preference distributions.

Frederico Araújo Durão

Marcelo Garcia Manzato (ICMC/USP)

http://lattes.cnpq.br/8598262641668520

Adriano César Machado Pereira (DCC/UFMG)

http://lattes.cnpq.br/6813736989856243

Rodrigo Rocha Gomes e Souza (PGCOMP/UFBA)

http://lattes.cnpq.br/7697794806460975

Bruno Pereira dos Santos (PGCOMP/UFBA)

http://lattes.cnpq.br/0092226104911153

Leandro Balby Marinho (UFCG)

http://lattes.cnpq.br/3728312501032061

Tiago Januario (Boston University)

http://lattes.cnpq.br/7975984954243833

Rafael Augusto de Melo (PGCOMP/UFBA)

http://lattes.cnpq.br/4117373032501782

Cássio Vinicius Serafim Prazeres (PGCOMP/UFBA)

http://lattes.cnpq.br/5075736089100544

18 Jun, 2025

8:30 AM