Results 781 - 790 of 24429

Speaker: Rashida Hakim (Columbia University)

Title: Fair Data-Exchange Mechanisms

Abstract: We study data exchange among strategic agents without monetary transfers, motivated by domains such as research consortia and healthcare collaborations where payments are infeasible or restricted. The central challenge is to reap the benefits of data-sharing while preventing free-riding that would otherwise lead agents to under invest in data collection. We introduce a simple fair-exchange contract in which, for every pair of agents, each agent receives exactly as many data points as it provides, equal to the minimum of their two collection levels. We show that the game induced by this contract is supermodular under a transformation of the strategy space. This results in a clean structure: pure Nash equilibria exist, they form a lattice, and can be computed in time quadratic in the number of agents. In addition, the maximal equilibrium is truthfully implementable under natural enforcement assumptions and is globally Pareto-optimal across all strategy profiles. In a graph-restricted variant of the model supermodularity fails, but an adaptation of the construction still yields efficiently computable pure Nash equilibria and Pareto-optimal outcomes. Overall, fair exchange provides a tractable and incentive-aligned mechanism for data exchange in the absence of payments.

Speaker: Chhavi Yadav (CMU, UC Berkeley)

Title: ExpProof : Operationalizing Explanations for Confidential Models with ZKPs , FairProof : Confidential and Certifiable Fairness for Neural Networks

Abstract: In principle, explanations are intended as a way to increase trust in machine learning models and are often obligated by regulations. However, many circumstances where these are demanded are adversarial in nature, meaning the involved parties have misaligned interests and are incentivized to manipulate explanations for their purpose. As a result, explainability methods fail to be operational in such settings despite the demand (Bordt et al., 2022). In this paper, we take a step towards operationalizing explanations in adversarial scenarios with Zero-Knowledge Proofs (ZKPs), a cryptographic primitive. Specifically we explore ZKP-amenable versions of the popular explainability algorithm LIME and evaluate their performance on Neural Networks and Random Forests.

Machine learning models are increasingly used in societal applications, yet legal and privacy concerns demand that they very often be kept confidential. Consequently, there is a growing distrust about the fairness properties of these models in the minds of consumers, who are often at the receiving end of model predictions. To this end, we propose Fairproof -- a system that uses Zero-Knowledge Proofs (a cryptographic primitive) to publicly verify the fairness of a model, while maintaining confidentiality. We also propose a fairness certification algorithm for fully-connected neural networks which is befitting to ZKPs and is used in this system. We implement Fairproof in Gnark and demonstrate empirically that our system is practically feasible.

Modern data marketplaces and data sharing consortia increasingly rely on incentive mechanisms to encourage agents to contribute data. However, schemes that reward agents based solely on the quantity of submitted data are vulnerable to manipulation, as agents may submit fabricated or low-quality data to inflate their rewards. Existing approaches for incentivizing truthful data contributions typically rely on strong assumptions about the underlying data distribution or restrictive models of strategic behavior, limiting their practical applicability.

In this talk, I will present our recent work addressing these challenges. I will begin with our approach to incentivizing truthful data contributions in normal mean estimation problems. I will then describe a more general reward mechanism built on a novel two-sample test inspired by the Cramér–von Mises statistic. We show that truthful reporting constitutes a Nash equilibrium under our framework. I will also briefly highlight some empirical results on real-world language and image datasets demonstrating that our methods effectively promote truthful data sharing.

Relevant papers:
- Chen et al 2023: Mechanism Design for Collaborative Normal Mean Estimation
- Clinton et al 2025, Collaborative Mean Estimation Among Heterogeneous Strategic Agents: Individual Rationality, Fairness, and Truthful Contribution
- Clinton et al 2025, A Cramér-von Mises Approach to Incentivizing Truthful Data Sharing

When designing compound AI systems, a common approach is to query multiple copies of the same model and aggregate the responses to produce a synthesized output. Given the homogeneity of these models, this raises the question of whether aggregation unlocks access to a greater set of outputs than querying a single model. In this talk, we investigate the power and limitations of aggregation within a stylized principal-agent framework. This framework models how the system designer can partially steer each agent's output through its reward function specification, but still faces limitations due to prompt engineering ability and model capabilities. Our analysis uncovers three natural mechanisms -- feasibility expansion, support expansion, and binding set contraction -- through which aggregation expands the set of outputs that are elicitable by the system designer. We prove that any aggregation operation must implement one of these mechanisms in order to be elicitability-expanding, and that strengthened versions of these mechanisms provide necessary and sufficient conditions that fully characterize elicitability-expansion. Finally, we provide an empirical illustration of our findings for LLMs deployed in a toy reference-generation task. Altogether, our results take a step towards characterizing when compound AI systems can overcome limitations in model capabilities and in prompt engineering.

Based on joint work with Nivasini Ananthakrishnan.

Empirical conclusions depend not only on data but on analytic decisions made throughout the research process. Many-analyst studies have quantified this: independent teams testing the same hypothesis on the same dataset often reach conflicting conclusions. But such studies require costly coordination and are rarely conducted. We show that fully autonomous AI analysts built on large language models (LLMs) can cheaply and at scale replicate this structured analytic diversity. In our framework, each AI analyst executes a complete analysis pipeline on a fixed dataset and hypothesis, while a separate AI auditor screens runs for methodological validity. Across three datasets, AI-generated analyses exhibit substantial dispersion in effect sizes, p-values, and conclusions, driven by systematic differences in preprocessing, model specification, and inference across LLMs and personas. Critically, outcomes are steerable: changing the analyst persona or model shifts the distribution of results even among valid analyses.
These findings highlight a central challenge for AI-automated empirical science: when defensible analyses are cheap, evidence becomes abundant and vulnerable to selective reporting. But the same capability suggests a solution: treating results as distributions makes analytic uncertainty visible, and deploying AI analysts on a fixed specification can reveal disagreement from underspecified choices. We therefore argue for new transparency norms: multiverse-style reporting and prompt disclosure, alongside code and data.

Joint work with Martin Bertran and Riccardo Fogliato

Attributing LLM outputs to the training examples that causally influence their behavior can give us visibility into LLMs’ opaque reasoning and help us understand subtle persona changes. Unfortunately, finding training data attribution algorithms which are both accurate and scalable has remained an elusive goal. I argue for separately studying an Estimation Problem (accurately estimating the causal effect of a training example) and a Retrieval Problem (efficiently finding the highest-scoring training examples). I then present a generic retrieval method for influential sequences which can be paired with a wide range of influence estimators (including EKFAC) and for which one can obtain high confidence about recall. I discuss how causal training data attribution can be used as a tool to assure LLM alignment.