Results 11 - 20 of 24273
Rare events such as conformational changes in biomolecules, phase transitions, and chemical reactions are central to the behavior of many physical systems, yet they are extremely difficult to study computationally because unbiased simulations seldom produce them. Transition Path Theory (TPT) provides a rigorous statistical framework for analyzing such events: it characterizes the ensemble of reactive trajectories between two designated metastable states (reactant and product), and its central object--the committor function, which gives the probability that the system will next reach the product rather than the reactant--encodes all essential kinetic and thermodynamic information. We introduce a framework that casts committor estimation as a stochastic optimal control (SOC) problem. In this formulation the committor defines a feedback control--proportional to the gradient of its logarithm--that actively steers trajectories toward the reactive region, thereby enabling efficient sampling of reactive paths. To solve the resulting hitting-time control problem we develop two complementary objectives: a direct backpropagation loss and a principled off-policy Value Matching loss, for which we establish first-order optimality guarantees. We further address metastability, which can trap controlled trajectories in intermediate basins, by introducing an alternative sampling process that preserves the reactive current while lowering effective energy barriers. On benchmark systems, the framework yields markedly more accurate committor estimates, reaction rates, and equilibrium constants than existing methods. The talk will mostly cover the manuscript https://arxiv.org/abs/2604.13213.
Trajectory inference is a fundamental problem in many scientific domains: given a collection of unpaired snapshots of observations at discrete time points, the goal is to generate smooth trajectories that best resemble and interpolate the data. Existing algorithms exhibit computational challenges: they either rely on preprocessing subroutines to enforce smoothness or on simulation-based training objectives, both of which can be expensive. In order to overcome these limitations, we propose a new algorithm called Acceleration Matching (AM). Our approach consists of lifting the original interpolation problem to phase space and then regressing onto an explicit conditional acceleration field that induces random, smooth trajectories that agree with the prescribed marginals. Importantly, our resulting training algorithm only requires positional data, avoids trajectory simulation during training, and is devoid of expensive preprocessing. We provide ample numerical evidence suggesting that AM is competitive with or superior to existing algorithms on several benchmark problems in the literature. This is joint work with Bartolo Dazzini, Alain Durmus, and Giovanni Conforti.
Deep neural networks can learn powerful probability models of images, as demonstrated by the high-quality samples produced by score-based diffusion models. Yet it remains unclear how these networks capture complex global image statistics without suffering from the curse of dimensionality. I will present two complementary studies of this question in diffusion U-Nets. First, using a simplified multiscale U-Net, we show that coarse-to-fine conditioning in the decoder makes fine-scale image structure locally predictable: once conditioned on coarser-scale information, fine-scale features can be modeled using stationary local Markov models with small receptive fields. This transforms a high-dimensional modeling problem into a sequence of lower-dimensional conditional problems. Global structure, however, is handled differently at the U-Net bottleneck, where the spatial resolution is small enough for receptive fields to cover the entire image. In the second work, we investigate the representation that emerges in this bottleneck. We find that the middle block represents each image through a sparse subset of active channels, whose spatial averages form a nonlinear representation of the underlying clean image. Euclidean distances in this representation space reflect semantic similarity, even though the model is trained without external conditioning such as text or class labels. We further introduce a self-guided reconstruction method that uses a representation extracted by the unconditional model to condition its own stochastic synthesis, revealing the image features encoded in the bottleneck. Together, these results suggest that U-Nets manage image dimensionality by combining compact global representations with low-dimensional conditional models of local detail.
Diffusion model is a prevailing paradigm for generative AI, and this talk will briefly report two rigorous results that are centered around its performance.
First, I will quantify the generalization capability of the classical diffusion model, i.e. the one for Euclidean data. The question is, when the generative model is not memorizing the training data, what new samples will it generate? This question is not only pertinent to privacy and copyright considerations, but also important for understanding whether/how diffusion model creates new knowledge. The inductive bias of diffusion model’s generation will be examined, leading to a quantification of how diffusion model generalizes. This quantification is purely based on the empirical distribution without considering any population limit.
Then I will switch gear and describe a new test-time scaling method that improves masked diffusion model for discrete data, so that its output can maximize a given reward function, without any fine-tuning of the model weights. The method, MDM-VGB, is a discrete diffusion sampler that augments unmasking generation with principled reward-guided remasking. It is inspired by a recent success, VGB, that leverages the classical Jerrum-Sinclair backtracking Markov chain for the reward-tilted generation of autoregressive model; meanwhile, the Any-Order AutoRegressive nature of MDM-VGB allows more efficient error corrections. Besides strong empirical performance, we also prove that MDM-VGB is robust to process verifier noise, achieving a quadratic complexity, while popular test-time heuristics like best-of-N suffer from an exponential complexity due to accumulative errors.
A major bottleneck of standard auto-regressive large language models is that their inference process is inherently sequential, resulting in very long and costly inference times. To circumvent this, practitioners proposed a class of language models called diffusion language models, of which the masked diffusion model (MDM) is one of the most promising and successful. The MDM is able to sample out-of-order and, ostensibly, many tokens at once and in parallel. However, there is very limited rigorous understanding of how much parallel sampling these models can perform without noticeable degradation in their sampling performance. In this work, we give a new, exact characterization of the expected divergence between the true distribution and the sampled distribution, for any distribution and any unmasking schedule for the sampler, showing an elegant connection between MDM sampling and the classical theory of univariate function approximation.
Scalable approaches to learning dynamic measure transport maps have shaped the forefront of generative modeling, leading to paradigm-defining methods for simulation-free training such as diffusion models and flow matching. The natural evolution of this trajectory is to demand the same modeling power at a fraction of the inference cost---that is, to directly learn the \emph{flow-map} operator, or integrator, associated with a generative dynamical system. In this paper, we model such flow-map operators using invertible neural networks, forming a new class of exactly invertible models termed Normalizing Flow Maps (NFM). NFMs generalize classical Normalizing Flows (NFs) by learning time-indexed transport maps between arbitrary pairs of times, while retaining efficient exact likelihood evaluation through change of variables. In contrast to standard flow-map approaches such as Consistency Models and Mean Flow models, NFMs provide both fast few-step sampling and tractable density estimation, making them suitable for domains where likelihoods are essential, including Boltzmann Generators.
To train NFMs scalably, we develop a family of flow-map consistency objectives that operate at both the pointwise and distributional levels. Beyond regression-based Lagrangian, Eulerian, and self-distillation losses, we introduce likelihood-compatible distributional consistency objectives, including forward- and reverse-KL formulations over intermediate marginals and interpolant conditionals. These objectives exploit the exact change-of-variables structure of NFM to regularize the learned flow maps as density-preserving transports, bridging regression-based flow-map training with maximum-likelihood principles. Empirically, we demonstrate that NFM achieve strong performance on image generation, yielding state-of-the-art sample quality among normalizing-flow models while enabling fast sampling in either direction via the time-inverse structure of flow maps. We further show that NFMs outperform prior normalizing-flow baselines, including RegFlow-style methods, for equilibrium sampling of small peptide systems in Cartesian coordinates. Together, these results establish normalizing flow maps as a practical framework for exact-likelihood generative modeling with fast, consistent, and bidirectional sampling.