Revisiting Incremental Skip-gram for Dynamic Network Embedding

Abstract

Network embedding methods represent nodes of a graph in a low-dimensional vector space while preserving structural information, enabling the training of machine learning models for network analysis tasks such as link prediction, node classification, and anomaly detection. Random-walk–based approaches using the Skip-gram with Negative Sampling (SGNS) model have become widely used due to their scalability and effectiveness. To handle dynamic networks, Peng et al. [15] proposed the incremental training with a negative sampling strategy that updates node embeddings as the network changes, avoiding the cost of retraining embeddings from scratch. In this work, we conduct a reproducibility analysis of the proposed method, examining both its theoretical formulation and the provided implementation. Our analysis identifies several issues in the presented proof. We provide a correct proof with their assumptions and show that the convergence behavior is governed by the drift between negative-sampling distributions across snapshots, rather than by the total corpus size as claimed. We also analyze the source code and find that essential artifacts for reproducibility are missing, and the code is misaligned with the method described in the manuscript. Overall, this study highlights important theoretical and implementation inconsistencies in the proposed method, proposes open directions, and emphasizes the need for careful validation to ensure reproducibility in dynamic network embedding research.

Authors 2

1. Akrati Saxena Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Leiden, Netherlands

2. Ella Has Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Leiden, Netherlands