Summer 2019

The singular values of convolutional layers

Monday, Jun. 10, 2019 11:00 am12:00 pm PDT

Add to Calendar


Phil Long, Google


Room 116


We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation.  This characterization also leads to an algorithm for projecting a convolutional layer onto an operator-norm ball. We show that this is an effective regularizer; for example, it improves the test error of a deep residual network on CIFAR-10 from 6.2% to 5.3%.

This is joint work with Hanie Sedghi and Vineet Gupta.