Can Persistent Homology provide an efficient alternative for Evaluation of Knowledge Graph Completion Methods?

arXiv (Cornell University)

Abstract

In this paper we present a novel method, $\textit{Knowledge Persistence}$ ($\mathcal{KP}$), for faster evaluation of Knowledge Graph (KG) completion approaches. Current ranking-based evaluation is quadratic in the size of the KG, leading to long evaluation times and consequently a high carbon footprint. $\mathcal{KP}$ addresses this by representing the topology of the KG completion methods through the lens of topological data analysis, concretely using persistent homology. The characteristics of persistent homology allow $\mathcal{KP}$ to evaluate the quality of the KG completion looking only at a fraction of the data. Experimental results on standard datasets show that the proposed metric is highly correlated with ranking metrics (Hits@N, MR, MRR). Performance evaluation shows that $\mathcal{KP}$ is computationally efficient: In some cases, the evaluation time (validation+test) of a KG completion method has been reduced from 18 hours (using Hits@10) to 27 seconds (using $\mathcal{KP}$), and on average (across methods & data) reduces the evaluation time (validation+test) by $\approx$ $\textbf{99.96}\%$.

Authors 6

1. Anson Bastos

   Indian Immunologicals (India) · Indian Institute of Technology Hyderabad

   Affiliation as printed

   IIT , Hyderabad India

2. Kuldeep Singh
   Affiliation as printed

   Zerotha Research and Cerence GmbH Germany

3. Abhishek Nadgeri Aachen Zerotha Research and

   RWTH Aachen University

   Affiliation as printed

   Zerotha Research and RWTH Aachen Germany

4. Johannes Hoffart

   Systems, Applications & Products in Data Processing (Germany)

   Affiliation as printed

   SAP Germany

5. Toyotaro Suzumura

   The University of Tokyo

   Affiliation as printed

   The University of Tokyo Japan

6. Manish Singh

   Indian Institute of Technology Hyderabad

   Affiliation as printed

   IIT Hyderabad India