The Spectrum of Nonlinear Random Matrices for Ultra-Wide Neural Networks

Tuesday, Dec. 7, 2021 11:20 am11:35 am

Add to Calendar


Yizhe Zhu (University of California, Irvine)


Calvin Lab Auditorium

We obtain limiting spectral distribution of empirical conjugate kernel and neural tangent kernel matrices for two-layer neural networks with deterministic data and random weights. When the width of the network grows faster than the size of the dataset, a deformed semicircle law appears. In this regime, we also calculate the asymptotic test and training errors for random feature regression. Joint work with Zhichao Wang (UCSD).

PDF icon yizhezhu.pdf2.27 MB