Results 551 - 560 of 24762
Deep neural networks can learn powerful probability models of images, as demonstrated by the high-quality samples produced by score-based diffusion models. Yet it remains unclear how these networks capture complex global image statistics without suffering from the curse of dimensionality. I will present two complementary studies of this question in diffusion U-Nets. First, using a simplified multiscale U-Net, we show that coarse-to-fine conditioning in the decoder makes fine-scale image structure locally predictable: once conditioned on coarser-scale information, fine-scale features can be modeled using stationary local Markov models with small receptive fields. This transforms a high-dimensional modeling problem into a sequence of lower-dimensional conditional problems. Global structure, however, is handled differently at the U-Net bottleneck, where the spatial resolution is small enough for receptive fields to cover the entire image. In the second work, we investigate the representation that emerges in this bottleneck. We find that the middle block represents each image through a sparse subset of active channels, whose spatial averages form a nonlinear representation of the underlying clean image. Euclidean distances in this representation space reflect semantic similarity, even though the model is trained without external conditioning such as text or class labels. We further introduce a self-guided reconstruction method that uses a representation extracted by the unconditional model to condition its own stochastic synthesis, revealing the image features encoded in the bottleneck. Together, these results suggest that U-Nets manage image dimensionality by combining compact global representations with low-dimensional conditional models of local detail.
Diffusion model is a prevailing paradigm for generative AI, and this talk will briefly report two rigorous results that are centered around its performance.
First, I will quantify the generalization capability of the classical diffusion model, i.e. the one for Euclidean data. The question is, when the generative model is not memorizing the training data, what new samples will it generate? This question is not only pertinent to privacy and copyright considerations, but also important for understanding whether/how diffusion model creates new knowledge. The inductive bias of diffusion model’s generation will be examined, leading to a quantification of how diffusion model generalizes. This quantification is purely based on the empirical distribution without considering any population limit.
Then I will switch gear and describe a new test-time scaling method that improves masked diffusion model for discrete data, so that its output can maximize a given reward function, without any fine-tuning of the model weights. The method, MDM-VGB, is a discrete diffusion sampler that augments unmasking generation with principled reward-guided remasking. It is inspired by a recent success, VGB, that leverages the classical Jerrum-Sinclair backtracking Markov chain for the reward-tilted generation of autoregressive model; meanwhile, the Any-Order AutoRegressive nature of MDM-VGB allows more efficient error corrections. Besides strong empirical performance, we also prove that MDM-VGB is robust to process verifier noise, achieving a quadratic complexity, while popular test-time heuristics like best-of-N suffer from an exponential complexity due to accumulative errors.
A major bottleneck of standard auto-regressive large language models is that their inference process is inherently sequential, resulting in very long and costly inference times. To circumvent this, practitioners proposed a class of language models called diffusion language models, of which the masked diffusion model (MDM) is one of the most promising and successful. The MDM is able to sample out-of-order and, ostensibly, many tokens at once and in parallel. However, there is very limited rigorous understanding of how much parallel sampling these models can perform without noticeable degradation in their sampling performance. In this work, we give a new, exact characterization of the expected divergence between the true distribution and the sampled distribution, for any distribution and any unmasking schedule for the sampler, showing an elegant connection between MDM sampling and the classical theory of univariate function approximation.
Scalable approaches to learning dynamic measure transport maps have shaped the forefront of generative modeling, leading to paradigm-defining methods for simulation-free training such as diffusion models and flow matching. The natural evolution of this trajectory is to demand the same modeling power at a fraction of the inference cost---that is, to directly learn the \emph{flow-map} operator, or integrator, associated with a generative dynamical system. In this paper, we model such flow-map operators using invertible neural networks, forming a new class of exactly invertible models termed Normalizing Flow Maps (NFM). NFMs generalize classical Normalizing Flows (NFs) by learning time-indexed transport maps between arbitrary pairs of times, while retaining efficient exact likelihood evaluation through change of variables. In contrast to standard flow-map approaches such as Consistency Models and Mean Flow models, NFMs provide both fast few-step sampling and tractable density estimation, making them suitable for domains where likelihoods are essential, including Boltzmann Generators.
To train NFMs scalably, we develop a family of flow-map consistency objectives that operate at both the pointwise and distributional levels. Beyond regression-based Lagrangian, Eulerian, and self-distillation losses, we introduce likelihood-compatible distributional consistency objectives, including forward- and reverse-KL formulations over intermediate marginals and interpolant conditionals. These objectives exploit the exact change-of-variables structure of NFM to regularize the learned flow maps as density-preserving transports, bridging regression-based flow-map training with maximum-likelihood principles. Empirically, we demonstrate that NFM achieve strong performance on image generation, yielding state-of-the-art sample quality among normalizing-flow models while enabling fast sampling in either direction via the time-inverse structure of flow maps. We further show that NFMs outperform prior normalizing-flow baselines, including RegFlow-style methods, for equilibrium sampling of small peptide systems in Cartesian coordinates. Together, these results establish normalizing flow maps as a practical framework for exact-likelihood generative modeling with fast, consistent, and bidirectional sampling.
We view drifting from a Wasserstein gradient flow (WGF) perspective and propose a conservative method for one-step generative modeling, replacing the original displacement-based velocity with a KDE-gradient field given by the difference between kernel-smoothed data and model scores. Using a joint-entropy identity, we establish continuous-time finite-particle bounds on R^d, including a root residual-velocity rate of N^{-1/(d+4)} under bandwidth-uniform quadrature regularity. We also analyze the original non-conservative Laplace-kernel method through a sharp companion-kernel decomposition and show how the resulting residual-velocity bounds yield explicit one-step generation guarantees in terms of the drift size.
Diffusion models generate high-quality synthetic data. They operate by defining a continuous-time forward process which gradually adds Gaussian noise to data until fully corrupted. The corresponding reverse process progressively "denoises" a Gaussian sample into a sample from the data distribution. However, generating high-quality outputs requires many discretization steps to obtain a faithful approximation of the reverse process. This is expensive and has motivated the development of many acceleration methods. We propose to accomplish sample generation by learning the posterior {\em distribution} of clean data samples given their noisy versions, instead of only the mean of this distribution. This allows us to sample from the probability transitions of the reverse process on a coarse time scale, significantly accelerating inference with minimal degradation of the quality of the output. This is accomplished by replacing the standard regression loss used to estimate conditional means with a scoring rule. We validate our method on image and robot trajectory generation, where we consistently outperform standard diffusion models at few discretization steps.