Foundations of Machine Learning

T​he goal of this program was to grow the reach and impact of computer science theory within machine learning.

One central component of the program was ​formalizing basic questions in developing areas of practice​ and gaining fundamental insights into these. Target areas of particular interest were ​interactive learning​ and representation learning. Interactive learning consists of scenarios in which the communication between human and learner is richer than a one-­way transmission of labeled examples; this happens, for instance, in teaching, or explanation-based learning, and in crowdsourcing. Representation learning studies intermediate-­ or higher-level representations of data that facilitate learning. Questions of interest include the learnability of deep architectures and how much of it can be accomplished unsupervised, representations that allow generative abilities, and reasoning based on learned intermediate-­level features.

A second component of the program was advancing the algorithmic frontier of machine learning​. Target areas included ​Bayesian statistics​, in which many of the core algorithmic problems bear similarity to problems that have been studied intensively in the theoretical computer science community, and ​large-­scale optimization​, in which a host of interesting challenges arise at the interface of theory and practical deployment.

A final component of the program was ​understanding heuristics​: what works in practice and why. The most popular algorithms for a variety of basic statistical tasks — clustering, embedding, and so on — behave in a manner that is not fully understood. Some, like principal component analysis, have strong properties but are used in ways that cannot directly be justified by appealing to these properties. Others, like k-­means, have obvious failure modes in a worst-­case setting and yet are quite successful on many types of data. The program brought together theoreticians and practitioners who were interested in teasing apart these issues and expanding the useful formal characterizations of such procedures.