Feature Learning: From Optimization to Modern Generalization

What are features? Do neural networks learn features efficiently? Why is learning features important for modern regimes of generalization?

I'll first discuss some of my previous work on efficient feature learning via gradient descent in 2-layer neural networks. I'll conclude with some more abstract feature learning directions I'm interested in, touching on the other questions above.