Validation by Listening

In this talk from the recent workshop on Decoding Communication in Nonhuman Species IV, Adam Kalai (OpenAI) explores how to evaluate machine translation systems in the absence of ground-truth reference translations, focusing on the extreme case where only acoustic outputs are available, without contextual or visual grounding. Kalai presents a “reference-free” method that tests coherence in translated text, offering a way to distinguish genuine signal from hallucination in settings such as animal communication.