Shannon Theory
S.13.3
Lecture
Topics in Shannon Theory

Exact Expressions in Source and Channel Coding Problems Using Integral Representations

Igal Sason, Neri Merhav

Date & Time

01:00 am – 01:00 am

Abstract

We explore known integral representations of the logarithmic and power functions, and demonstrate their usefulness for information-theoretic analyses. We obtain compact, easily--computable exact formulas for several source and channel coding problems that involve expectations and higher moments of the logarithm of a positive random variable and the moment of order $\rho > 0$ of a non-negative random variable (or the sum of such i.i.d. random variables). These integral representations are used in a variety of applications, including the calculation of the degradation in mutual information between the channel input and output as a result of jamming, universal lossless data compression, Shannon and R\'{e}nyi entropy evaluations, and the ergodic capacity evaluation of the single-input, multiple--output (SIMO) Gaussian channel with random parameters (known to both transmitter and receiver). The integral representation of the logarithmic function and its variants are anticipated to serve as a rigorous alternative to the popular (but non--rigorous) replica method (at least in some situations).


Presenters

Igal Sason

Technion - Israel Institute of Technology

Neri Merhav

Technion - Israel Institute of Technology

Session Chair

Xiugang Wu

University of Delaware