Questions for Theory in the New Age of Machine Learning

Not long ago, two reasonable assumptions about machine learning were: (1) the primary mechanism to achieve learning is to tune parameters, and (2) because we have little prior knowledge to provide a strong inductive bias, learning must rely on big data and sophisticated statistics. Today, both assumptions seem out of date when one considers architecting learning agents that employ LLMs as subroutines. We will explore this new style of LLM-based learning agents, as well as theoretical questions they raise.