Results 11 - 20 of 24254

A major bottleneck of standard auto-regressive large language models is that their inference process is inherently sequential, resulting in very long and costly inference times. To circumvent this, practitioners proposed a class of language models called diffusion language models, of which the masked diffusion model (MDM) is one of the most promising and successful. The MDM is able to sample out-of-order and, ostensibly, many tokens at once and in parallel. However, there is very limited rigorous understanding of how much parallel sampling these models can perform without noticeable degradation in their sampling performance. In this work, we give a new, exact characterization of the expected divergence between the true distribution and the sampled distribution, for any distribution and any unmasking schedule for the sampler, showing an elegant connection between MDM sampling and the classical theory of univariate function approximation.

Diffusion model is a prevailing paradigm for generative AI, and this talk will briefly report two rigorous results that are centered around its performance.

First, I will quantify the generalization capability of the classical diffusion model, i.e. the one for Euclidean data. The question is, when the generative model is not memorizing the training data, what new samples will it generate? This question is not only pertinent to privacy and copyright considerations, but also important for understanding whether/how diffusion model creates new knowledge. The inductive bias of diffusion model’s generation will be examined, leading to a quantification of how diffusion model generalizes. This quantification is purely based on the empirical distribution without considering any population limit.

Then I will switch gear and describe a new test-time scaling method that improves masked diffusion model for discrete data, so that its output can maximize a given reward function, without any fine-tuning of the model weights. The method, MDM-VGB, is a discrete diffusion sampler that augments unmasking generation with principled reward-guided remasking. It is inspired by a recent success, VGB, that leverages the classical Jerrum-Sinclair backtracking Markov chain for the reward-tilted generation of autoregressive model; meanwhile, the Any-Order AutoRegressive nature of MDM-VGB allows more efficient error corrections. Besides strong empirical performance, we also prove that MDM-VGB is robust to process verifier noise, achieving a quadratic complexity, while popular test-time heuristics like best-of-N suffer from an exponential complexity due to accumulative errors.

Abstract not available.

Given snapshot observations of a stochastic process, how can one reconstruct a smooth dynamical evolution consistent with all observed marginals? A natural object of interest is the probability spline (P-spline): a smooth path in the space of probability measures that interpolates prescribed marginals in direct analogy with classical cubic splines. Existing approaches to learning such paths are typically formulated through multi-marginal Schrödinger bridge problems or flow-matching objectives. While powerful, these methods often rely on simulation-based procedures, intricate preprocessing pipelines, or numerically delicate training objectives. In this work, we introduce **Acceleration Matching**, a new framework for learning probability splines through the estimation of a conditional acceleration field. Our approach is entirely simulation-free and is trained via a simple, explicit regression objective requiring virtually no preprocessing. Conceptually, it shifts the focus from matching velocities or transport maps to directly learning the second-order structure governing probabilistic dynamics. Across a range of benchmarks, Acceleration Matching achieves performance competitive with—and frequently superior to—state-of-the-art alternatives while substantially reducing training time. These results suggest that second-order learning principles may provide a scalable and effective foundation for interpolation and generative modeling in probability space.

I will talk about Signed Rectified Flow (Signed RF), a generalization of Rectified Flow that targets a signed measure of the form (1+a)p_+ - ap_-, where a>0, p_+ represents the distribution to promote, and p_- represents the distribution to suppress. Although sampling from a signed measure is not well-defined, Signed RF induces a valid generative process that concentrates on the positive region of the signed measure while provably excluding regions dominated by the negative component. This yields a principled framework for incorporating negative information and exclusion constraints into generative modeling. Theoretically, we analyze the underlying signed continuity equation and explain how negative mass creates exclusion barriers through a charged-particle interpretation. Empirically, we find that Signed RF enables practical algorithms for injecting both positive and negative information into generative flows, leading to higher-quality, less memorized, and safer generation.

Abstract not available.

Rare events such as conformational changes in biomolecules, phase transitions, and chemical reactions are central to the behavior of many physical systems, yet they are extremely difficult to study computationally because unbiased simulations seldom produce them. Transition Path Theory (TPT) provides a rigorous statistical framework for analyzing such events: it characterizes the ensemble of reactive trajectories between two designated metastable states (reactant and product), and its central object--the committor function, which gives the probability that the system will next reach the product rather than the reactant--encodes all essential kinetic and thermodynamic information. We introduce a framework that casts committor estimation as a stochastic optimal control (SOC) problem. In this formulation the committor defines a feedback control--proportional to the gradient of its logarithm--that actively steers trajectories toward the reactive region, thereby enabling efficient sampling of reactive paths. To solve the resulting hitting-time control problem we develop two complementary objectives: a direct backpropagation loss and a principled off-policy Value Matching loss, for which we establish first-order optimality guarantees. We further address metastability, which can trap controlled trajectories in intermediate basins, by introducing an alternative sampling process that preserves the reactive current while lowering effective energy barriers. On benchmark systems, the framework yields markedly more accurate committor estimates, reaction rates, and equilibrium constants than existing methods. The talk will mostly cover the manuscript https://arxiv.org/abs/2604.13213.