What Do Temporal Graph Learning Models Learn?

arXiv (Cornell University)

Abstract

Learning on temporal graphs has become a central topic in graph representation learning, with numerous benchmarks indicating the strong performance of state-of-the-art models. However, recent work has raised concerns about the reliability of benchmark results, noting issues with commonly used evaluation protocols and the surprising competitiveness of simple heuristics. This contrast raises the question of which characteristics of the underlying graphs temporal graph learning models actually use to form their predictions. We address this by systematically evaluating eight models on their ability to capture eight fundamental characteristics related to the link structure of temporal graphs. These include structural characteristics such as density, temporal patterns such as recency, and edge formation mechanisms such as homophily. Using both synthetic and real-world datasets, we analyze how well models learn these characteristics. Our findings reveal a mixed picture: models capture some characteristics well but fail to reproduce others. With this, we expose important limitations. Overall, we believe that our results provide practical insights for the application of temporal graph learning models and motivate more interpretability-driven evaluations in graph learning research.

Authors 3

1. Abigail J. Hayes

   University of Mannheim

   Affiliation as printed

   University of Mannheim ,

2. Tobias Schumacher Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

3. Markus Strohmaier

   University of Mannheim · GESIS - Leibniz Institute for the Social Sciences · Complexity Science Hub

   Affiliation as printed

   University of Mannheim , GESIS -Leibniz Institute for the Social Sciences , and Complexity Science Hub Vienna