January 01, 2022 preprint Open access Aachen paper

Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias Towards Low Rank

SSRN Electronic Journal

DOI: 10.2139/ssrn.4139166 Full text (OA) OpenAlex

Authors 4

Hung-Hsu Chou corresponding Aachen

RWTH Aachen University

Affiliation as printed

RWTH Aachen University, Templergraben 55, 52056 Aachen, 52056, Germany

RWTH Aachen University
Carsten Gieshoff Aachen

RWTH Aachen University

Affiliation as printed

RWTH Aachen University, Templergraben 55, 52056 Aachen, 52056, Germany

RWTH Aachen University
Johannes Maly
Holger Rauhut Aachen

RWTH Aachen University

Affiliation as printed

RWTH Aachen University, Templergraben 55, 52056 Aachen, 52056, Germany

RWTH Aachen University

Cited by 5 stored of 5

Convergence of gradient descent for learning linear neural networks
2024 Advances in Continuous and Discrete Models article Computer Science Stochastic Gradient Optimization Techniques Open access

10

citations
Compressive Sensing and Neural Networks from a Statistical Learning Perspective
2022 Applied and numerical harmonic analysis book-chapter Engineering Sparse and Compressive Sensing Techniques Open access

12

citations
Continuous vs. Discrete Optimization of Deep Neural Networks
2021 arXiv (Cornell University) preprint Computer Science Stochastic Gradient Optimization Techniques Open access

4

citations
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction
2021 Publication Server of the Catholic University Eichstätt-Ingolstadt (Catholic University of Eichstätt-Ingolstadt) book-chapter Engineering Sparse and Compressive Sensing Techniques Open access

13

citations
Implicit Regularization in Tensor Factorization
2021 arXiv (Cornell University) preprint Mathematics Tensor decomposition and applications Open access

1

citations

5 results

No patents citing this paper on Lens.org (checked 2026-10-06).

References 26

Principal Component Analysis
2005 Encyclopedia of Statistics in Behavioral Science reference-entry Chemistry Spectroscopy and Chemometric Analyses

14,489

citations
Understanding the difficulty of training deep feedforward neural networks
2010 article Computer Science Neural Networks and Applications

12,459

citations
Mastering the game of Go without human knowledge
2017 Nature article Computer Science Artificial Intelligence in Games

9,334

citations
Approximation capabilities of multilayer feedforward networks
1991 Neural Networks article Computer Science Machine Learning and ELM

6,251

citations
Deep Image Prior
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) conference-paper Computer Science Image and Signal Denoising Methods

2,010

citations
Neural networks and principal component analysis: Learning from examples without local minima
1989 Neural Networks article Computer Science Neural Networks and Applications

1,410

citations
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
2018 arXiv (Cornell University) preprint Computer Science Neural Networks and Applications Open access

1,340

citations
On Early Stopping in Gradient Descent Learning
2007 Constructive Approximation article Computer Science Stochastic Gradient Optimization Techniques

1,005

citations
Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
2018 arXiv (Cornell University) preprint Computer Science Image and Signal Denoising Methods Open access

181

citations
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
2018 arXiv (Cornell University) preprint Computer Science Stochastic Gradient Optimization Techniques Open access

124

citations
Geometry of Optimization and Implicit Regularization in Deep Learning
2017 arXiv (Cornell University) preprint Computer Science Stochastic Gradient Optimization Techniques Open access

81

citations
Stable low-rank matrix recovery via null space properties
2016 Information and Inference A Journal of the IMA article Engineering Sparse and Compressive Sensing Techniques Open access

73

citations
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
2020 arXiv (Cornell University) preprint Computer Science Stochastic Gradient Optimization Techniques Open access

61

citations
Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks
2019 Neural Computation article Computer Science Stochastic Gradient Optimization Techniques Open access

48

citations
Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks
2019 arXiv (Cornell University) preprint Computer Science Stochastic Gradient Optimization Techniques Open access

22

citations
The Implicit Bias of Depth: How Incremental Learning Drives Generalization
2019 arXiv (Cornell University) preprint Computer Science Stochastic Gradient Optimization Techniques Open access

22

citations
Low-rank regularization and solution uniqueness in over-parameterized matrix sensing.
2020 International Conference on Artificial Intelligence and Statistics conference-paper Engineering Sparse and Compressive Sensing Techniques

8

citations
W2963959597
details pending

0

citations
W2963446085
details pending

0

citations
W2899476926
details pending

0

citations