Optimization-Free Diffusion Model -- A Perturbation Theory Approach

arXiv (Cornell University)

Abstract

Diffusion models have emerged as a powerful framework in generative modeling, typically relying on optimizing neural networks to estimate the score function via forward SDE simulations. In this work, we propose an alternative method that is both optimization-free and forward SDE-free. By expanding the score function in a sparse set of eigenbasis of the backward Kolmogorov operator associated with the diffusion process, we reformulate score estimation as the solution to a linear system, avoiding iterative optimization and time-dependent sample generation. We analyze the approximation error using perturbation theory and demonstrate the effectiveness of our method on high-dimensional Boltzmann distributions and real-world datasets.

Authors 3

1. Yuehaw Khoo

   University of Chicago

   Affiliation as printed

   CCAM and Department of Statistics , University of Chicago , 5747 S Ellis Avenue , Chicago , 60637 , IL , United States

2. Mathias Oster Aachen

   RWTH Aachen University

   Affiliation as printed

   IGPM , RWTH Aachen , Templergraben 55 , Aachen , 52062 , NRW , Germany

3. Yifan Peng

   University of Chicago

   Affiliation as printed

   CCAM , University of Chicago , 5747 S Ellis Avenue , Chicago , 60637 , IL , United States