Double descent: When do neural quantum states generalize?

Physical review. E, vol. 113, pp. 045303

Abstract

Neural quantum states (NQS) provide flexible and compact wave-function parametrizations for numerical studies of quantum many-body physics. In particular, NQS aim to circumvent the exponential scaling of the Hilbert space by compressing quantum many-body wave functions with a tractable amount of parameters. While inspired by deep learning, it remains unclear to what extent NQS share characteristics with neural networks used for standard machine learning tasks. We demonstrate that, in a simplified supervised setting, NQS exhibit the double descent phenomenon, a key feature of modern deep learning, where generalization worsens as network size increases before improving again in an overparameterized regime. Notably, we find the second descent to occur only for network sizes much larger than the Hilbert space dimension, i.e., network sizes that are out of reach for problems of practical interest. Within our setting, this observation places typical NQS in the underparameterized regime. We also observe that the optimal network size in the underparameterized regime depends on the number of unique training samples. While the double descent phenomenon does indeed translate to the NQS setting, potential practical consequences of our findings point more toward the need for symmetry-aware, physics-informed architecture design, rather than directly adopting machine learning heuristics.

Authors 6

1. M. Schuyler Moss

   University of Waterloo · Perimeter Institute

   Affiliation as printed

   Perimeter Institute

   University of Waterloo

2. Alev Orfi

   Flatiron Institute · New York University

   Affiliation as printed

   Flatiron Institute

   New York University

3. Antoine Georges

   University of Geneva · École Polytechnique · Collège de France · Centre de Physique Théorique · Flatiron Institute

   Affiliation as printed

   CPHT

   Collège de France

   Flatiron Institute

   Université de Genève

   École Polytechnique

4. Dries Sels

   Flatiron Institute · New York University

   Affiliation as printed

   Flatiron Institute

   New York University

5. Anna Dawid Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science

   Universiteit Leiden

6. Agnes Valenti

   Flatiron Institute

   Affiliation as printed

   Flatiron Institute