Approximate Message Passing for Matrix Regression
type: CreativeWork
description: Approximate message passing (AMP) algorithms have become popular in various structured high-dimensional statistical problems. Previous AMP algorithms for generalized linear models (GLM) typically require the signal to be in the form of a vector, and the design matrix to have independent and identically distributed Gaussian entries. In this thesis, we explore estimation problems where previous assumptions of AMP algorithms are no longer satisfied. To relax the vector signal assumption, we introduce the matrix GLM model which accounts for matrix signals, and propose an AMP algorithm for estimation and rigorously characterize its performance in the high-dimensional limit. We consider prominent cases of the matrix GLM model to illustrate its usefulness, namely mixed linear regression, max-affine regression, and mixture-of-experts. For max-affine regression, we propose an algorithm that combines AMP with expectation-maximization to estimate intercepts of the model along with the signals. To relax the Gaussian design assumption, we extend the matrix GLM to have a generalized white noise design matrix (instead of Gaussian), and propose an AMP algorithm for estimating the matrix signal and rigorously characterize its performance. We apply this model to the pooled data and the quantitative group testing (QGT) problem, where the design matrix is binary-valued. For the noiseless pooled data setting, we show that the studied AMP algorithm is equivalent to one recently proposed by El Alaoui et al. Our results provide a rigorous version of their performance guarantees, previously obtained via non-rigorous techniques. For comparison, we propose estimators based on convex relaxation and iterative thresholding, without providing theoretical guarantees. To further improve the performance of AMP algorithms for QGT and pooled data, we introduce the spatially coupled Bernoulli test matrix and an AMP algorithm. We rigorously characterize its asymptotic performance in both the noiseless a…
abstract: Approximate message passing (AMP) algorithms have become popular in various structured high-dimensional statistical problems. Previous AMP algorithms for generalized linear models (GLM) typically require the signal to be in the form of a vector, and the design matrix to have independent and identically distributed Gaussian entries. In this thesis, we explore estimation problems where previous assumptions of AMP algorithms are no longer satisfied. To relax the vector signal assumption, we introduce the matrix GLM model which accounts for matrix signals, and propose an AMP algorithm for estimation and rigorously characterize its performance in the high-dimensional limit. We consider prominent cases of the matrix GLM model to illustrate its usefulness, namely mixed linear regression, max-affine regression, and mixture-of-experts. For max-affine regression, we propose an algorithm that combines AMP with expectation-maximization to estimate intercepts of the model along with the signals. To relax the Gaussian design assumption, we extend the matrix GLM to have a generalized white noise design matrix (instead of Gaussian), and propose an AMP algorithm for estimating the matrix signal and rigorously characterize its performance. We apply this model to the pooled data and the quantitative group testing (QGT) problem, where the design matrix is binary-valued. For the noiseless pooled data setting, we show that the studied AMP algorithm is equivalent to one recently proposed by El Alaoui et al. Our results provide a rigorous version of their performance guarantees, previously obtained via non-rigorous techniques. For comparison, we propose estimators based on convex relaxation and iterative thresholding, without providing theoretical guarantees. To further improve the performance of AMP algorithms for QGT and pooled data, we introduce the spatially coupled Bernoulli test matrix and an AMP algorithm. We rigorously characterize its asymptotic performance in both the noiseless a…
author: Tan, Thong Cai Nelvin
language: eng
keywords: Machine Learning, High-Dimensional Statistics, Information Theory
license: https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/603ccd87-d523-4242-9f65-bb7ac63ef582/download
license: https://creativecommons.org/licenses/by/4.0/
date published: 2024-08-30
date modified: 2026-07-22T22:24:00Z
provider: Cambridge University
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: University of Cambridge

PropertyValue
property ID: DOI
value: 10.17863/CAM.113622