Results 2121 - 2130 of 23898
In this talk, I outline techniques for learning and teaching specifications - in the form of automata - using expert demonstrations and natural language. This covers modeling the likelihood of a demonstration given a task specification automaton, guiding searching the space of automata using natural language and counter-factual demonstrations, and finding pedagogic demonstrations.
Towards capturing and measuring robustness of computations wrt to a desired specification, we look into the following questions:
1. Can we have a notion of an edit distance between infinite words, and if so, can it be efficiently computed?
2. Can we distill from an omega-regular language the natural preference relation that it induced?
The quest of answering the first question goes via a discussion on normalized edit distance between finite words. In particular, we show that the normalized edit distance (NED) of [Marzal and Vidal, 93] over uniform weights is a metric. We then provide a necessary and sufficient condition on a cost function d on edit operations so that the resulting measure NED_d between words would be a metric.
We further show that NED can be extended to a notion that applies to infinite words, call it \omega-NED, and that computing \omega-NED between words/languages can be done in polynomial time.
To answer the second question we show that we can refine the 5-valued semantics of [Tabuada & Neider, 2016] into an infinite-valued semantics that captures robustness. Towards this aim we provide natural colors to individual letters of an infinite word capturing whether that letter contributes positively or negatively to the robustness of the infinite word wrt to the language.
Based on joint works with Joshua Grogin, Oded Margalit, Elina Sudit, Ilay Tzarfati and Gera Weiss.
Formal task specifications, such as those expressible in temporal logic or as automata, have gained interest as an alternative means of specifying objectives for deep reinforcement learning (RL) agents. Unlike traditional Markovian rewards, formal specifications are composable, easily transferred across environments, and offer a precise notion of satisfaction. Despite these appeals, formal specifications have limited adoption across RL, in large part due to the difficulty of expressing formal specifications as readily-optimizable objectives. In this talk, I will discuss two directions where formal specifications can provide a distinct benefit over traditional reward functions in training RL agents. First, I will demonstrate how the precision of specifications allows us to disambiguate between correctness and optimality conditions, which can help ensure that learned policies achieve desired behavior. I will then discuss how the compositionality of specifications can enable more efficient learning in cooperative multi-agent settings. Throughout the talk, I will address how we can mitigate common pitfalls of specification-guided RL, such as optimization difficulty and reward sparsity. I will close with a brief discussion on when we should consider formal specifications in general, when we should avoid them, and the role alternative reward objectives can play in RL.
I will discuss prediction-powered inference – a strategy for creating confidence intervals using machine-learning predictions and a small amount of ground-truth labels. When the predictions are accurate, the intervals shrink, yielding better statistical power. The validity of the intervals holds regardless of the machine-learning algorithm. Intervals can be computed for many estimands, such as means, quantiles, and linear and logistic regression coefficients. We demonstrate the benefits of prediction-powered inference with data sets from proteomics, genomics, electronic voting, remote sensing, census analysis, and ecology. The talk will draw from the following jointly authored papers: [1], [2], [3].
The talk will summarize research findings in https://www.jmlr.org/papers/v24/23-0838.html, https://ojs.aaai.org/index.php/AAAI/article/view/30018/31790, https://ojs.aaai.org/index.php/AAAI/article/view/20670, https://dl.acm.org/doi/10.1145/3576841.3585931, and https://www.computer.org/csdl/proceedings-article/wacv/2025/108300j193/… that describe the uncertainty quantification techniques for detecting out-of-distribution and out-of-context inputs in the Trinity neuro-symbolic architecture developed at SRI.