Surface Data vs. Deep Data

In this talk from the workshop on Topics in Intelligence: World Models and Social Reasoning, Alyosha Efros (UC Berkeley) argues that models trained on surface (scraped) data will not transcend mimicry and pastiche. He suggests that to achieve full intelligence, we need deep data, mined by the models through curiosity-driven exploration, driven by intrinsic motivation rather than rewards.