Summer 2019

How To Beat Empirical Risk Minimization

Wednesday, Jul. 31, 2019 11:00 am11:30 am PDT

Add to Calendar


Ali Rahimi


Room 116

We explore this interpretation for regularization: there is inherent statistical inefficiency in empirical risk minimization and good regularizers reduce this inefficiency. This viewpoint is in contrast to regularization to reduce model complexity, or to introduce a priori knowledge into the model. We show sufficient conditions under which there is a Stein phenomenon in empirical risk minimization, and we correct for this phenomenon using a shrinkage operator, which we interpret as a regularizer.

Joint work with Alan Mackey.