Rashomon Alignment

arXiv (Cornell University)

Abstract

We propose Rashomon Alignment (RA), a new measure to assess functional similarity between two models. Existing functional similarity measures are distributional, quantifying differences between outputs of models applied to real-world data. However, these measures can be regarded as ecologically valid only for regions in the input space represented by the available data. We introduce a geometrical perspective on functional model similarity, which estimates it across the entire data space, offering a comprehensive view of decision boundary alignment independent of any specific data distribution. We also propose geometric Rashomon Alignment as a measure of geometrical similarity, which is computed using data uniformly sampled from the instance space. We perform an experimental analysis on more than 90 datasets, examining critical cases where model alignment diverges from predictive accuracy. Our results show that geometrical and distributional alignment provide different and complementary perspectives on the similarity between models and algorithms. RA can be used for multiple purposes, including model selection, ensemble construction, and enhanced interpretability of machine learning models and algorithms.

Authors 4

1. Moisés Santos

   Universidade do Porto · Artificial Intelligence in Medicine (Canada)

   Affiliation as printed

   Artificial Intelligence and Computer Science Lab (LIACC) , Portugal

   Faculty of Engineering , University of Porto , Portugal

2. Peter van der Putten Aachen

   Leiden University

   Affiliation as printed

   LIACS , Leiden University , Leiden , The Netherlands

3. Bernhard Pfahringer

   University of Waikato

   Affiliation as printed

   School of Computing and Mathematical Sciences , University of Waikato , Hamilton , New Zealand

4. Carlos Soares