Results 491 - 500 of 24339

A number of Latent Variable Models in Machine Learning (including Mixture Models, Topic Models, Stochastic block models and Mixed Membership Community Mod els) can be abstracted to the geometric problem of learn ing a latent polytope K given data points, each obtained by randomly perturbing a latent point in K. The challenge is that perturbations are typically much larger than the dimensions of K and so data points lie (far) outside K. To tackle this, we introduce the “Subset Smoothed” polytope K′ which is the convex hull of (n/k) points, each obtained by averaging a k− subset of the n data points. [k is a parameter.] We will observe that K′ ≈ K under reasonable assumptions on data. We will also observe that K′ has a polynomial time optimization oracle. These simple observations are the starting point of our provable algorithm for learning K which the talk will describe.

Joint Work with Chiranjib Bhattacharyya, Amit Kumar

In current Large Language Models, we can trust the production of smoothly flowing prose on the basis of the principles of machine learning. However, there is no comparably principled basis to justify trust in the content of the text produced. It appears to be conventional wisdom that addressing this issue by adding more principled reasoning is not computationally affordable.

Here we propose a principled approach to this problem that is efficient enough to be practical for large language models. Further, the method allows the retention of much of the currently used software and hardware base. Our method for improving the functioning of large language models consists of a first stage of preprocessing that recodes the data to a Unary Relational Integracode that is more explicit about the relationships among the objects described in the text, followed as a second stage by a standard but possibly streamlined machine learning process that then also learns to predict these relationships.

The method may be viewed as realizing a world model that applies beyond natural language, to vision and actions, for example, where the multiple properties of an object referred to in an input are brought together explicitly, rather than remaining distributed in the various references to it in the input. We articulate its advantages in terms of Robust Logic, a system for performing principled chaining on learned, and hence uncertain, information. We show that this recoding has the surprising and fortuitous property that, while succinct, it makes a core problem in learning relational rules that hold in the world described in the training data polynomial time learnable, the polynomial depending on the complexity of the rule. This gives heuristic support for sound reasoning within each call of the learned classifier as well as between multiple calls.

We explain how hallucinations in language models are rise in the first place. Prior work has shown how to modify models to reduce hallucinations. We also argue that the reason hallucinations persist is that benchmarks inadvertently reward guessing when unsure, and we discuss incentive compatible ways to modify the benchmarks.

Joint work with Santosh Vempala, Ofir Nachum and Edwin Zhang.

Not long ago, two reasonable assumptions about machine learning were: (1) the primary mechanism to achieve learning is to tune parameters, and (2) because we have little prior knowledge to provide a strong inductive bias, learning must rely on big data and sophisticated statistics. Today, both assumptions seem out of date when one considers architecting learning agents that employ LLMs as subroutines. We will explore this new style of LLM-based learning agents, as well as theoretical questions they raise.