Information-theoretic limitations of distributed information processing

In a generic distributed information processing system, a number of agents connected by communication channels aim to accomplish a task collectively through local communications. The fundamental limits of distributed information processing problems depend not only on the intrinsic difficulty of the task, but also on the communication constraints due to the distributedness. In this thesis, we reveal these dependencies quantitatively under information-theoretic frameworks. We consider three typical distributed information processing problems: decentralized parameter estimation, distributed function computation, and statistical learning under adaptive composition. For the first two problems, we derive converse results on the Bayes risk and the computation time, respectively. For the last problem, we first study the relationship between the generalization capability of a learning algorithm and its stability property measured by the mutual information between its input and output, and then derive achievability results on the generalization error of adaptively composed learning algorithms. In all cases, we obtain general results on the fundamental limits with respect to a general model of the problem, so that the results can be applied to various specific scenarios. Our information-theoretic analyses also provide general approaches to inferring global properties of a distributed information processing system from local properties of its components.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2017

Author dc:creator

• Xu, Aolin

Contributors dc:contributor

• Raginsky, Maxim
• Hajek, Bruce
• Milenkovic, Olgica
• Srikant, Rayadurgam
• Wu, Yihong

• distributed function computation
• distributed information processing
• fundamental limits
• information theory
• decentralized estimation
• Bayes risk
• small ball probability
• strong data processing inequality
• computation time
• network reduction
• diameter of network
• statistical learning
• adaptive composition
• stability of learning algorithms
• generalization error
• mutual information
• Gibbs algorithm
• adaptive data analytics

Statement dc:rights

• Copyright 2016 Aolin Xu

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/95358

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/95358