Tutorials

Presenters: Shubhanshu Shekhar ([email protected]) 

equential anytime-valid inference involves the study of statistical procedures whose validity guarantees hold uniformly over time, and therefore remain valid under continuous monitoring and data-dependent stopping. Such procedures are important in many applications including A/B testing, financial auditing, evaluation of AI models, and clinical trials. This tutorial develops an information-theoretic perspective on their design and analysis. In particular, we will emphasize the role of reverse information projections, the associated minimum divergence terms, and their variational representations in both characterizing the fundamental limits of performance and suggesting constructive procedures that achieve those limits.

We will begin with the technically simple finite-alphabet setting, using it to introduce a likelihood-based methodology for constructing near-optimal test martingales and e-processes combining ideas from universal coding and classical sequential analysis. We will then explain how to extend the same strategy to more abstract alphabets with compact composite null distribution classes. We will discuss how the Donsker–Varadhan variational representation of relative entropy and Sion’s minimax theorem can be combined to produce a useful dual representation of the minimum divergence to the null distribution class, and how this representation guides likelihood-free constructions in nonparametric problems. Through representative examples, we will identify the main conditions needed for these procedures to achieve optimal growth-rate and expected stopping time performance. The tutorial will conclude with open questions concerning stronger notions of optimality, computationally tractable implementations, and extensions beyond i.i.d. observations.

Presenters: Narayanan Rengaswamy ([email protected])

Quantum computing has a potential advantage over classical computing in terms of computational speedup, owing to the principles of superposition and entanglement. However, qubits and quantum operations are inherently faulty, and quantum error correction plays a central role in enabling fault tolerant quantum computing. The fault models vary based on technologies, and some of the leading quantum technologies are superconducting qubits, trapped-ions, neutral atoms and photonic/bosonic systems. The design of codes and decoders for various fault models and methods to implement fault tolerant circuits on the encoded information is central to enabling scalable and reliable quantum computing. In this tutorial, we will cover the basics of quantum error-correcting codes (QECC) starting with the 9-qubit Shor code. We will develop the general framework of stabilizer codes and CSS (Calderbank-Shor-Steane) codes, touching upon examples of the Steane code and toric/surface code. We will discuss the construction of fault tolerant logical operators on such codes. Then, we will discuss two important classes of quantum low-density parity-check (LDPC) codes known as hypergraph product and lifted product codes. Subsequently, we will present efficient decoding algorithms for different codes and error models. We will conclude with a discussion on quantum architectures and open problems in the area.

The tentative outline of the tutorial is as follows: Stabilizer and CSS codes, logical operators of codes, hypergraph and Lifted Product quantum LDPC codes, decoders. Time permitting, we will also demonstrate some decoders in software using public repositories.

Presenters: Yao Xie ([email protected]) and Xiuyuan Cheng ([email protected])

Modern generative models are often introduced as tools for producing realistic samples. This tutorial takes a different view: diffusion and flow-based generative models can be used as constructive tools for inference in probability space. In this view, the goal is not only to generate new data, but to learn, represent, transform, condition, and sample from probability distributions that arise in statistical inference, uncertainty quantification, inverse problems, and learning under limited or indirect information.

The tutorial will introduce flow- and diffusion-based generative models through a common mathematical language based on transport maps, velocity fields, stochastic dynamics, continuity equations, Fokker-Planck equations, and Wasserstein geometry. We will emphasize connections to information-theoretic and statistical quantities such as entropy, KL divergence, Fisher divergence, likelihood, and optimal transport distances. The tutorial is designed to be accessible to graduate students in information theory, statistics, signal processing, and machine learning, and will highlight open questions on stability, sample complexity, high-dimensional limits, and error transfer from learned generative models to downstream inference tasks.

Presenters: Sanghamitra Dutta ([email protected]), Pasan Dissanayake ([email protected])

Classical information-theoretic measures such as mutual information capture the entire dependency between two random variables but fail to capture how this dependency is distributed among the input features, i.e., the structure of the multivariate information. To bridge this gap, Partial Information Decomposition (PID) has emerged as a powerful framework to quantify the information that several random variables provide about another random variable, either individually (unique information), redundantly (shared/redundant information), or only jointly (synergistic information). This tutorial provides a comprehensive introduction to the foundations of PID and showcases its critical role in addressing three pillars of modern machine learning: efficiency, interpretability, and ethics. Our session will be structured into three main parts: (i) Foundations & Toy Examples: We begin by defining unique, redundant, and synergistic information using intuitive examples, establishing their clear distinction over classical measures. (ii) Model Efficiency via Distillation: We deep dive into how PID quantifies the fundamental limits of knowledge distillation, which is the process of training compact "student'' models from complex "teacher'' models for resource-constrained environments (e.g., edge devices, smartphones, and IoT). We theoretically demonstrate that for capacity-constrained student models, task-relevant transferred knowledge is succinctly captured by redundant information, leading to a novel alternating optimization framework of Redundant Information Distillation. (iii) Interpretability and Ethics: We demonstrate how PID systematically identifies sources of disparities, evaluates trade-offs between local and global fairness, and demystifies spurious correlations. Lastly, we conclude by reviewing practical estimation techniques for PID measures and discussing open challenges and future research directions.

The unprecedented surge of AI comes with ever-increasing model size and complexity, leading to an unprecedented demand for energy. Knowledge distillation plays a key role in building efficient machine learning models by enabling compact student models to match the performance of larger teacher models while remaining computationally efficient. Despite these advantages, existing work offers limited understanding of how effectively knowledge distillation can support a student model’s learning process by transferring only the task-relevant information needed for downstream performance. Redundant information distillation addresses this gap by leveraging PID as a principled information-theoretic framework to analyze and quantify this transfer, leading to a more robust and effective distillation technique. On the other hand, AI is also rapidly integrating into high-stakes domains like hiring, finance, education, and healthcare. While these models excel at pattern recognition, blindly capturing all statistical correlations can lead to severe consequences, such as propagating historical biases, reinforcing harmful stereotypes, or making uninterpretable decisions. Consequently, embedding ethics and interpretability into machine learning is of utmost importance for informing policy, protecting institutional reputation, and preventing societal harm. Here, PID offers a unique mathematical framework to bridge this gap: where classical information-theoretic measures fall short, PID successfully disentangles complex sources of bias, clarifies local versus global fairness trade-offs, and quantifies exact feature contributions. In essence, PID techniques play a pivotal role in accelerating the adoption of AI that aligns with the pillars of efficiency, interpretability, and ethics.