Sasha Rush | Polylogues

On April 10, Simons Institute Science Communicator in Residence Anil Ananthaswamy sat down with Sasha Rush, an associate professor at Cornell Tech working on natural language processing and machine learning, with a focus on deep learning text generation, language modeling, and structured prediction.

This episode of Polylogues explores a significant shift in the last year in how large language models are trained and used. Anil and Sasha discuss neural scaling laws, which were empirically discovered a handful of years ago, and have had a major influence on companies’ decisions to build increasingly large models. The past year has seen both a concern that we are seeing diminishing returns with scale, and an important change in how computing resources are deployed in building and using bigger models.