The Surprising Power of Graph Neural Networks with Random Node Initialization

Abstract

Graph neural networks (GNNs) are effective models for representation learning on relational data. However, standard GNNs are limited in their expressive power, as they cannot distinguish graphs beyond the capability of the Weisfeiler-Leman graph isomorphism heuristic. In order to break this expressiveness barrier, GNNs have been enhanced with random node initialization (RNI), where the idea is to train and run the models with randomized initial node features. In this work, we analyze the expressive power of GNNs with RNI, and prove that these models are universal, a first such result for GNNs not relying on computationally demanding higher-order properties. This universality result holds even with partially randomized initial node features, and preserves the invariance properties of GNNs in expectation. We then empirically analyze the effect of RNI on GNNs, based on carefully constructed datasets. Our empirical findings support the superior performance of GNNs with RNI over standard GNNs.

Authors 4

1. Ralph Abboud

   University of Oxford

   Affiliation as printed

   University of Oxford

2. İsmail İlkan Ceylan

   University of Oxford

   Affiliation as printed

   University of Oxford

3. Martin Grohe Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

4. Thomas Lukasiewicz

   University of Oxford

   Affiliation as printed

   University of Oxford

   [Department of Computer Science, University of Oxford]