All talks listed in Pacific Time. Schedule subject to change.

–
Coffee and Check-In
–
Opening Remarks
–
Transformers, parallel computation, and logarithmic depth
Daniel Hsu (Columbia University)
–
Break
–
Capabilities and limitations of Transformer in sequential reasoning
Bingbin Liu (Carnegie Mellon University)
–
Break
–
The Parallelism Tradeoff: Understanding Transformer Expressivity Through Circuit Complexity
Will Merrill (New York University)
–
Lunch (on your own)
–
Emergence and grokking in "simple" architectures
Misha Belkin (UCSD)
–
Break
–
Iterated Models: Expressive Power, Learning, and Chain of Thought
Nati Srerbo (Toyota Technological Institute at Chicago)
–
Reception
–
Coffee and Check-In
–
The emergence of clusters in self-attention dynamics
Philippe Rigollet (MIT)
–
Break
–
Computational Benefits and Limitations of Transformers and State-Space Models
Eran Malach (Kempner Institute, Harvard University)
–
Break
–
Transformer Expressivity and Formal Logic
David Chiang (University of Notre Dame)
–
Lunch (on your own)
–
Interpretability Agents
Sarah Schwettmann (MIT)
–
Break
–
Recent Efficiency Improvements to Transformers
David Woodruff (Carnegie Mellon University)
–
Break
–
Panel Discussion: What can theory offer to the design and use of LLMs?
Jon Kleinberg, Philippe Rigollet, Nati Srebro, and Naomi Saphra
–
Coffee and Check-In
–
Language Generation in the Limit
Jon Kleinberg (Cornell University)
–
Break
–
Do Large Language Models Perform Latent Reasoning? (Remote Talk)
Mor Geva (Tel Aviv University)
–
Break
–
Language Acquisition in Language Models
Naomi Saphra (Kempner Institute at Harvard University)
–
Lunch (on your own)
–
What was Revolutionized by the "Transformer Revolution?
Stella Biderman (ElutherAI)
–
Break
–
Poster Session (2nd Floor Interactive Area)
–
Coffee and Check-In
–
Learning to Reason with LLMs
Noam Brown (OpenAI)
–
Break
–
Understanding and Improving Efficient Language Models
Simran Arora (Stanford University)
–
Break
–
Using recurrence to achieve weak to strong generalization
Tom Goldstein (University of Maryland)
–
Lunch (on your own)
–
Towards Understanding Modern Alchemy
Ekin Akyurek (MIT)
–
Break
–
A Retrieval-based Language Model at Scale (Remote Talk)
Sewon Min (UC Berkeley & AI2)
–
Break
–
Panel Discussion: Are Transformers the end game?
Simran Arora, Stella Biderman, Jitendra Malik, Andrew Wilson
–
Coffee and Check-In
–
On the Tradeoffs of State Space Models
Albert Gu (Carnegie Mellon University)
–
Break
–
Re-thinking Transformers: Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
Andrew Gordon Wilson (New York University)
–
Break
–
Exact solutions to the geometric dynamics of signal propagation through transformers predict their trainability
Surya Ganguli (Stanford University)
–
Lunch (on your own)
–
Using Algorithms to Understand Transformers (and Using Transformers to Understand Algorithms)
Vatsal Sharan (University of Southern California)
–
Break
–
Associative memories as a building block in Transformers
Alberto Bietti (Flatiron Institute)
–
Hike