Results 41 - 50 of 24600
Modern AI systems are increasingly deployed in scientific and safety-critical applications, making reliability a fundamental challenge. Yet, despite its importance, the notion of reliable AI remains only partially understood from a mathematical perspective. In this talk, I will discuss some thoughts on what reliable AI should mean and argue that quantifying reliability is a central ingredient. Within this broader perspective, I will present two complementary lines of recent work: the detection and quantification of hallucinations in image reconstruction using feature-based conformal techniques, and uncertainty quantification through probabilistic regression with diffusion models. Placing these developments in a broader framework for reliable AI, I will discuss how they contribute to assessing when model outputs can be trusted and highlight several open mathematical questions towards more principled notions of reliability.
Language model hallucinations arise not only from gaps in knowledge, but also from failures in how models retrieve, select, and integrate information. In this talk, I will examine these failures from two complementary perspectives. First, I will present a mechanistic account of non-factual hallucinations, identifying distinct failures in knowledge enrichment and answer extraction within the model’s factual recall process. I will then extend this analysis to long-context language models, examining how parametric knowledge interacts, and sometimes conflicts, with information provided in context. The results show that stronger contextual retrieval does not necessarily lead to more reliable knowledge use and may even interfere with parametric recall. Together, these studies characterize hallucination as a failure of knowledge coordination and highlight implications for model interpretability, evaluation, mitigation, and design.
No abstract available.
Interaction with AI systems is now happening at massive scale. People engage in multi-round exchanges with AI to guide real-world decisions, and increasingly, AI agents act on our behalf, drawing on humans and tools for support. Yet despite the prevalence of these interactions, we still lack fundamental principles that should govern them: what makes an interaction trustworthy, and how can we ensure it reliably improves the resulting decisions?
The first part of the talk takes a human-centric view of collaboration. The human chooses whether to engage with the AI in the first place and remains accountable for the final decision, so trustworthy collaboration must be built around them. From this view, I identify two fundamental levers governing the interaction. AI should preserve the judgments the human would have made correctly on their own, while providing meaningful help where the human is likely to err. This perspective changes the role of uncertainty quantification. Rather than describing the AI’s uncertainty in isolation, uncertainty becomes a mechanism for governing the effect of AI on the human’s decision. I will show how these principles lead to algorithms for both one-shot predictions and multi-round interactions, with distribution-free guarantees that hold for any human paired with any AI model, without requiring a model of human behavior.
The second part turns to agentic AI, where the traditional decision-support paradigm is reversed: the agent acts on our behalf, while humans and tools provide support. This shift requires rethinking uncertainty as a joint property of the agent and its available support, capturing whether that support can materially improve the agent’s decision. I will show how this perspective unifies diverse support settings within a common framework and provides a principled foundation for designing reliable agent-support interactions.
Across both settings, the talk develops a common language for trustworthy interaction and algorithms that translate these principles into guarantees across different humans, AI models, and forms of collaboration.
No abstract available.