Skip to main content

Meir Feder

Followers

47

Following

13

Co-authors

13

Public Views

Interests

Uploads

Papers by Meir Feder

Statistical Signal Processing Using a Class of Iterative Estimation Algorithms

Many Signal Processing problems may be posed as statistical parameter estimation problems. A desi... more Many Signal Processing problems may be posed as statistical parameter estimation problems. A desired solution for the statistical problem is obtained by maximizing the Likelihood (ML), the A-Posteriori probability (MAP) or by optimizing other criterion, depending on the a-priori knowledge. However, in many practical situations, the original signal processing problem may generate a complicated optimization problem e.g when the observed signals are noisy and "incomplete". A framework of iterative procedures for maximizing the likelihood, the EM algorithm, is widely used in statistics. In the EM algorithm, the observations are considered "incomplete" and the algorithm iterates between estimating the sufficient statistics of the "complete data" given the observations and a current estimate of the parameters (the E step) and maximizing the likelihood of the complete data, using the estimated sufficient statistics (the M step). When this algorithm is applied to signal processing problems it yield, in many cases, an intuitively appealing processing scheme. In the first part of the thesis we investigate and extend the EM framework. By changing the "complete data" in each step of the algorithm we achieve algorithms with better convergence properties. We suggest EIM type algorithms to optimize other (non ML) criteria. We also develop sequential and adaptive version of the EM algorithm. In the second part of the thesis we discuss some applications of this extended framework of algorithms. We consider, * Parameter estimation of composite signals, i.e signals that can be represented as a decomposition of simpler signals. This problem appear in e.g.-Multiple source location (or bearing) estimation-Multipath or multi-echo time delay estimation v Noise canceling in multiple microphone environment, for a speech enhancement problera.

Low-density lattice codes

by ofir shalvi and Meir Feder

Information Theory, IEEE …, Jan 1, 2008

Universal Compression of Memoryless Sources over Large Alphabets via Independent Component Analysis

by Amichai Painsky and Meir Feder

2015 Data Compression Conference, 2015

Source broadcasting to the masses: Separation has a bounded loss

by Uri Mendlovic and Meir Feder

2014 IEEE International Symposium on Information Theory, 2014

This work discusses the source broadcasting problem, i.e. transmitting a source to many receivers... more This work discusses the source broadcasting problem, i.e. transmitting a source to many receivers via a broadcast channel. The optimal rate-distortion region for this problem is unknown. The separation approach divides the problem into two complementary problems: source successive refinement and broadcast channel transmission. We provide bounds on the loss incorporated by applying time-sharing and separation in source broadcasting. If the broadcast channel is degraded, it turns out that separation-based time-sharing achieves at least a factor of the joint source-channel optimal rate, and this factor has a positive limit even if the number of receivers increases to infinity. For the AWGN broadcast channel a better bound is introduced, implying that all achievable joint source-channel schemes have a rate within one bit of the separation-based achievable rate region for two receivers, or within log 2 T bits for T receivers.

Finite-memory least squares universal prediction of individual continuous sequences

2011 IEEE International Symposium on Information Theory Proceedings, 2011

In this paper we consider the problem of universal prediction of individual continuous sequences ... more In this paper we consider the problem of universal prediction of individual continuous sequences with square-error loss, using a deterministic finite-state machine (FSM). The goal is to attain universally the performance of the best constant predictor tuned to the sequence, which predicts the empirical mean and incurs the empirical variance as the loss. The paper analyzes the tradeoff between the number of states of the universal FSM and the excess loss (regret). We first present the Exponential Decaying Memory (EDM) machine, used in the past for predicting binary sequences, and show bounds on its performance. Then we look explicitly for the optimal machine with a small number of states. We consider a class of machines denoted the Degenerated Tracking Memory (DTM) machines that outperform the EDM machine for a small number of states. Unfortunately, the regret of the DTM machines is bounded away from zero even with a large number of states making these machines suboptimal. Finally, we prove a lower bound on the achievable regret of any FSM that defines the best rate that the regret can vanish. We propose a new machine, the Enhanced Exponential Decaying Memory, which attains the bound and outperforms the EDM for any number of states.

Prediction of priors for communication over arbitrarily varying channels

2011 IEEE International Symposium on Information Theory Proceedings, 2011

Parallel bit interleaved coded modulation

2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2010

A new variant of bit interleaved coded modulation (BICM) is proposed. In the new scheme, called p... more A new variant of bit interleaved coded modulation (BICM) is proposed. In the new scheme, called parallel BICM, L identical binary codes are used in parallel using a mapper, a newly proposed finite-length interleaver and a binary dither signal. As opposed to previous approaches, the scheme does not rely on any assumptions of an ideal, infinite-length interleaver. Over a memoryless channel, the new scheme is proven to be equivalent to a binary memoryless channel. Therefore the scheme enables one to easily design coded modulation schemes using a simple binary code that was designed for that binary channel. The overall performance of the coded modulation scheme is analytically evaluated based on the performance of the binary code over the binary channel. The new scheme is analyzed from an information theoretic viewpoint, where the capacity, error exponent and channel dispersion are considered. The capacity of the scheme is identical to the BICM capacity. The error exponent of the scheme is numerically compared to a recently proposed mismatched-decoding exponent analysis of BICM.

Universal communication over modulo-additive individual noise sequence channels

2011 IEEE International Symposium on Information Theory Proceedings, 2011

Problem Statement Purpose Encoder Pr(y n |x n ) =? Decoder --Universal communication over an unkn... more Problem Statement Purpose Encoder Pr(y n |x n ) =? Decoder --Universal communication over an unknown channel with feedback Goal: target general channels (not memoryless / stationary) In this talk we 1. Define a competitive setting for universal communication 2. Present a positive result for the modulo-additive channel with an individual noise sequence Yuval Lomnitz, Meir Feder (TAU) Universal Communication www.eng.tau.ac.il/~yuvall 3 / 34 Problem Statement

A universal probability assignment for prediction of individual sequences

2013 IEEE International Symposium on Information Theory, 2013

The Posterior Matching Feedback Scheme for Joint Source-Channel Coding with Bandwidth Expansion

2009 Data Compression Conference, 2009

When transmitting a Gaussian source over an AWGN channel with an input power constraint and a qua... more When transmitting a Gaussian source over an AWGN channel with an input power constraint and a quadratic distortion measure, it is well known that op- timal performance can be obtained using an analog joint source-channel scalar scheme which merely scales the input and output signals. In the case of band- width expansion, such a joint source-channel analog scheme attaining optimal performance is no longer simple. However, when feedback is available a simple and sequential analog linear procedure based on the Schalkwijk-Kailath scheme for communication, is optimal. Recently, we have introduced a fundamental feedback communication scheme, termed posterior matching, which generalizes the Schalkwijk-Kailath scheme to arbitrary memoryless channels and input dis- tributions. In this paper, we show how the posterior matching scheme can be adapted to the joint source-channel coding setting with bandwidth expansion and a general distortion measure, when feedback is available.

Power adaptive feedback communication over an additive individual noise sequence channel

2009 IEEE International Symposium on Information Theory, 2009

We consider a real-valued additive channel with an individual unknown noise sequence. We present ... more We consider a real-valued additive channel with an individual unknown noise sequence. We present a simple sequential communication scheme based on the celebrated Schalkwijk-Kailath scheme, which varies the transmit power according to the power of the sequence, so that asymptotically the relation between the SNR and the rate matches the Gaussian channel capacity R ≈ 1 2 log(1 + SN R) for almost every noise sequence.

Capacity and error exponent analysis of multilevel coding with multistage decoding

2009 IEEE International Symposium on Information Theory, 2009

The capacity and random coding error exponent of multilevel coding (MLC) with multistage decoding... more The capacity and random coding error exponent of multilevel coding (MLC) with multistage decoding (MSD) are analyzed. General discrete memoryless channels with arbitrary input distributions are considered. The capacities of MLC with maximum likelihood decoding and with MSD are calculated, and it is shown that using MLC may result in loss in the achievable rate for reliable communication. Necessary and sufficient conditions for the rate loss to be zero are derived.

The posterior matching feedback scheme: Capacity achieving and error analysis

2008 IEEE International Symposium on Information Theory, 2008

Recently, we have introduced a sequential communication scheme for general memoryless channels wi... more Recently, we have introduced a sequential communication scheme for general memoryless channels with feedback based on the idea of posterior matching, providing a unified framework in which the known Horstein and Schalkwijk-Kailath schemes are special cases. In this paper, we show that the posterior matching scheme achieves the mutual information for a large family of channels and input distributions, and provide closed-form expressions for the attainable error probability over a range of rates. Moreover, we derive the achievable rates in a mismatched setting, where the scheme is designed according to the wrong channel model. In particular, our results hold for discrete memoryless channels, thereby confirming a longstanding conjecture that the Horstein scheme achieves capacity. The proof techniques employed utilize novel relations between information rates and convergence properties of iterated function systems.

Universal sequential learning and decision from individual data sequences

Proceedings of the fifth annual workshop on Computational learning theory - COLT '92, 1992

Sequential learning and decision algorithms are investigated, with various application areas, und... more Sequential learning and decision algorithms are investigated, with various application areas, under a family of additive loss functions for individual data sequences. Simple universal sequential schemes are known, under certain conditions, to approach optimality uniformly as fast as n-1logn, where n is the sample size. For the case of finite-alphabet observations, the class of schemes that can be implemented by finite-state machines (FSM&amp;#x27;s), is studied. It is shown that Markovian machines with sufficiently long memory exist that are asymptotically nearly as good as any given FSM (deterministic or randomized) for the purpose of sequential decision. For the continuous-valued observation case, a useful class of parametric schemes is discussed with special attention to the recursive least squares (RLS) algorithm.

Prediction of Individual Sequences using Universal Deterministic Finite State Machines

2006 IEEE International Symposium on Information Theory, 2006

We consider the problem of universal prediction of individual binary sequences where the universa... more We consider the problem of universal prediction of individual binary sequences where the universal predictor is a deterministic finite state machine with a fixed number of states. We examine the case of self-information loss, where the predictions are probability assignments. The performance of the predictors is measured by their redundancy w.r.t. the constant predictors class. We obtain an improved lower bound on the redundancy of any finite state (FS) predictor with K states. We construct a FS predictor based on the lower bound and compare the performance of the predictor to the lower bound. Numerical results show that the redundancy of the proposed FS predictor is close to that predicted by the lower bound.

Universal Communication—Part II: Channels With Memory

IEEE Transactions on Information Theory, 2014

ABSTRACT Consider communication over a channel whose probabilistic model is completely unknown ve... more ABSTRACT Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about the future. The existence of reliable feedback and common randomness is assumed. In a previous paper it was shown that the Shannon capacity cannot be attained, in general, if the channel is not known. An alternative notion of &quot;capacity&quot; was defined, as the maximum rate of reliable communication by any block-coding system used over consecutive blocks. This rate was shown to be achievable for the modulo-additive channel with an individual, unknown noise sequence, and not achievable for some channels with memory. In this paper this &quot;capacity&quot; is shown to be achievable for general channel models possibly including memory, as long as this memory fades with time. In other words, there exists a system with feedback and common randomness that, without knowledge of the channel, asymptotically performs as well as any block code, which may be designed knowing the channel. For non-fading memory channels a weaker type of &quot;capacity&quot; is shown to be achievable.

Communication with Feedback via Posterior Matching

2007 IEEE International Symposium on Information Theory, 2007

In this paper we describe a general algorithmic scheme for communication over any memoryless chan... more In this paper we describe a general algorithmic scheme for communication over any memoryless channel in the presence of noiseless feedback. The scheme is based on the idea of posterior matching, in which the information still missing at the receiver is extracted from the a-posteriori density function, and matched to any desirable input distribution. We analyze the error probability attained by this scheme for additive noise channels, and show that the well-known Schalkwijk-Kailath scheme for the AWGN channel with average power constraint and the Horstein scheme for the BSC, can be derived as special cases.

An achievable rate for the MIMO individual channel

2010 IEEE Information Theory Workshop, 2010

We consider the problem of communicating over a multiple-input multiple-output (MIMO) real valued... more We consider the problem of communicating over a multiple-input multiple-output (MIMO) real valued channel for which no mathematical model is specified, and achievable rates are given as a function of the channel input and output sequences known a-posteriori. This paper extends previous results regarding individual channels by presenting a rate function for the MIMO individual channel, and showing its achievability in a fixed transmission rate communication scenario.

Feedback communication over individual channels

2009 IEEE International Symposium on Information Theory, 2009

We consider the problem of communicating over a channel for which no mathematical model is specif... more We consider the problem of communicating over a channel for which no mathematical model is specified. We present achievable rates as a function of the channel input and output known a-posteriori for discrete and continuous channels, as well as a rate-adaptive scheme employing feedback which achieves these rates asymptotically without prior knowledge of the channel behavior.

Universal communication over unknown vector channels

2012 IEEE International Symposium on Information Theory Proceedings, 2012

Consider communication over a channel whose probabilistic model is completely unknown vector-wise... more Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about the future. The existence of reliable feedback and common randomness is assumed. In a previous paper it was shown that the Shannon capacity cannot be attained, in general, if the channel is not known. An alternative notion of "capacity" was defined, which is the maximum rate of reliable communication by any block-coding system used over consecutive blocks. This rate was shown to be achievable for the modulo-additive channel with an individual, unknown noise sequence, and not achievable for some channels with memory. In this paper this "capacity" is shown to be achievable for general channel models possibly including memory, as long as this memory fades with time. In other words, there exists a system with feedback and common randomness that without knowledge of the channel, asymptotically performs as well as any block-coding system, which may be designed knowing the channel. For non-fading memory channels a weaker type of "capacity" is shown to be achievable.

Statistical Signal Processing Using a Class of Iterative Estimation Algorithms

Many Signal Processing problems may be posed as statistical parameter estimation problems. A desi... more Many Signal Processing problems may be posed as statistical parameter estimation problems. A desired solution for the statistical problem is obtained by maximizing the Likelihood (ML), the A-Posteriori probability (MAP) or by optimizing other criterion, depending on the a-priori knowledge. However, in many practical situations, the original signal processing problem may generate a complicated optimization problem e.g when the observed signals are noisy and "incomplete". A framework of iterative procedures for maximizing the likelihood, the EM algorithm, is widely used in statistics. In the EM algorithm, the observations are considered "incomplete" and the algorithm iterates between estimating the sufficient statistics of the "complete data" given the observations and a current estimate of the parameters (the E step) and maximizing the likelihood of the complete data, using the estimated sufficient statistics (the M step). When this algorithm is applied to signal processing problems it yield, in many cases, an intuitively appealing processing scheme. In the first part of the thesis we investigate and extend the EM framework. By changing the "complete data" in each step of the algorithm we achieve algorithms with better convergence properties. We suggest EIM type algorithms to optimize other (non ML) criteria. We also develop sequential and adaptive version of the EM algorithm. In the second part of the thesis we discuss some applications of this extended framework of algorithms. We consider, * Parameter estimation of composite signals, i.e signals that can be represented as a decomposition of simpler signals. This problem appear in e.g.-Multiple source location (or bearing) estimation-Multipath or multi-echo time delay estimation v Noise canceling in multiple microphone environment, for a speech enhancement problera.

Low-density lattice codes

by ofir shalvi and Meir Feder

Information Theory, IEEE …, Jan 1, 2008

Universal Compression of Memoryless Sources over Large Alphabets via Independent Component Analysis

by Amichai Painsky and Meir Feder

2015 Data Compression Conference, 2015

Source broadcasting to the masses: Separation has a bounded loss

by Uri Mendlovic and Meir Feder

2014 IEEE International Symposium on Information Theory, 2014

This work discusses the source broadcasting problem, i.e. transmitting a source to many receivers... more This work discusses the source broadcasting problem, i.e. transmitting a source to many receivers via a broadcast channel. The optimal rate-distortion region for this problem is unknown. The separation approach divides the problem into two complementary problems: source successive refinement and broadcast channel transmission. We provide bounds on the loss incorporated by applying time-sharing and separation in source broadcasting. If the broadcast channel is degraded, it turns out that separation-based time-sharing achieves at least a factor of the joint source-channel optimal rate, and this factor has a positive limit even if the number of receivers increases to infinity. For the AWGN broadcast channel a better bound is introduced, implying that all achievable joint source-channel schemes have a rate within one bit of the separation-based achievable rate region for two receivers, or within log 2 T bits for T receivers.

Finite-memory least squares universal prediction of individual continuous sequences

2011 IEEE International Symposium on Information Theory Proceedings, 2011

In this paper we consider the problem of universal prediction of individual continuous sequences ... more In this paper we consider the problem of universal prediction of individual continuous sequences with square-error loss, using a deterministic finite-state machine (FSM). The goal is to attain universally the performance of the best constant predictor tuned to the sequence, which predicts the empirical mean and incurs the empirical variance as the loss. The paper analyzes the tradeoff between the number of states of the universal FSM and the excess loss (regret). We first present the Exponential Decaying Memory (EDM) machine, used in the past for predicting binary sequences, and show bounds on its performance. Then we look explicitly for the optimal machine with a small number of states. We consider a class of machines denoted the Degenerated Tracking Memory (DTM) machines that outperform the EDM machine for a small number of states. Unfortunately, the regret of the DTM machines is bounded away from zero even with a large number of states making these machines suboptimal. Finally, we prove a lower bound on the achievable regret of any FSM that defines the best rate that the regret can vanish. We propose a new machine, the Enhanced Exponential Decaying Memory, which attains the bound and outperforms the EDM for any number of states.

Prediction of priors for communication over arbitrarily varying channels

2011 IEEE International Symposium on Information Theory Proceedings, 2011

Parallel bit interleaved coded modulation

2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2010

A new variant of bit interleaved coded modulation (BICM) is proposed. In the new scheme, called p... more A new variant of bit interleaved coded modulation (BICM) is proposed. In the new scheme, called parallel BICM, L identical binary codes are used in parallel using a mapper, a newly proposed finite-length interleaver and a binary dither signal. As opposed to previous approaches, the scheme does not rely on any assumptions of an ideal, infinite-length interleaver. Over a memoryless channel, the new scheme is proven to be equivalent to a binary memoryless channel. Therefore the scheme enables one to easily design coded modulation schemes using a simple binary code that was designed for that binary channel. The overall performance of the coded modulation scheme is analytically evaluated based on the performance of the binary code over the binary channel. The new scheme is analyzed from an information theoretic viewpoint, where the capacity, error exponent and channel dispersion are considered. The capacity of the scheme is identical to the BICM capacity. The error exponent of the scheme is numerically compared to a recently proposed mismatched-decoding exponent analysis of BICM.

Universal communication over modulo-additive individual noise sequence channels

2011 IEEE International Symposium on Information Theory Proceedings, 2011

Problem Statement Purpose Encoder Pr(y n |x n ) =? Decoder --Universal communication over an unkn... more Problem Statement Purpose Encoder Pr(y n |x n ) =? Decoder --Universal communication over an unknown channel with feedback Goal: target general channels (not memoryless / stationary) In this talk we 1. Define a competitive setting for universal communication 2. Present a positive result for the modulo-additive channel with an individual noise sequence Yuval Lomnitz, Meir Feder (TAU) Universal Communication www.eng.tau.ac.il/~yuvall 3 / 34 Problem Statement

A universal probability assignment for prediction of individual sequences

2013 IEEE International Symposium on Information Theory, 2013

The Posterior Matching Feedback Scheme for Joint Source-Channel Coding with Bandwidth Expansion

2009 Data Compression Conference, 2009

When transmitting a Gaussian source over an AWGN channel with an input power constraint and a qua... more When transmitting a Gaussian source over an AWGN channel with an input power constraint and a quadratic distortion measure, it is well known that op- timal performance can be obtained using an analog joint source-channel scalar scheme which merely scales the input and output signals. In the case of band- width expansion, such a joint source-channel analog scheme attaining optimal performance is no longer simple. However, when feedback is available a simple and sequential analog linear procedure based on the Schalkwijk-Kailath scheme for communication, is optimal. Recently, we have introduced a fundamental feedback communication scheme, termed posterior matching, which generalizes the Schalkwijk-Kailath scheme to arbitrary memoryless channels and input dis- tributions. In this paper, we show how the posterior matching scheme can be adapted to the joint source-channel coding setting with bandwidth expansion and a general distortion measure, when feedback is available.

Power adaptive feedback communication over an additive individual noise sequence channel

2009 IEEE International Symposium on Information Theory, 2009

We consider a real-valued additive channel with an individual unknown noise sequence. We present ... more We consider a real-valued additive channel with an individual unknown noise sequence. We present a simple sequential communication scheme based on the celebrated Schalkwijk-Kailath scheme, which varies the transmit power according to the power of the sequence, so that asymptotically the relation between the SNR and the rate matches the Gaussian channel capacity R ≈ 1 2 log(1 + SN R) for almost every noise sequence.

Capacity and error exponent analysis of multilevel coding with multistage decoding

2009 IEEE International Symposium on Information Theory, 2009

The capacity and random coding error exponent of multilevel coding (MLC) with multistage decoding... more The capacity and random coding error exponent of multilevel coding (MLC) with multistage decoding (MSD) are analyzed. General discrete memoryless channels with arbitrary input distributions are considered. The capacities of MLC with maximum likelihood decoding and with MSD are calculated, and it is shown that using MLC may result in loss in the achievable rate for reliable communication. Necessary and sufficient conditions for the rate loss to be zero are derived.

The posterior matching feedback scheme: Capacity achieving and error analysis

2008 IEEE International Symposium on Information Theory, 2008

Recently, we have introduced a sequential communication scheme for general memoryless channels wi... more Recently, we have introduced a sequential communication scheme for general memoryless channels with feedback based on the idea of posterior matching, providing a unified framework in which the known Horstein and Schalkwijk-Kailath schemes are special cases. In this paper, we show that the posterior matching scheme achieves the mutual information for a large family of channels and input distributions, and provide closed-form expressions for the attainable error probability over a range of rates. Moreover, we derive the achievable rates in a mismatched setting, where the scheme is designed according to the wrong channel model. In particular, our results hold for discrete memoryless channels, thereby confirming a longstanding conjecture that the Horstein scheme achieves capacity. The proof techniques employed utilize novel relations between information rates and convergence properties of iterated function systems.

Universal sequential learning and decision from individual data sequences

Proceedings of the fifth annual workshop on Computational learning theory - COLT '92, 1992

Sequential learning and decision algorithms are investigated, with various application areas, und... more Sequential learning and decision algorithms are investigated, with various application areas, under a family of additive loss functions for individual data sequences. Simple universal sequential schemes are known, under certain conditions, to approach optimality uniformly as fast as n-1logn, where n is the sample size. For the case of finite-alphabet observations, the class of schemes that can be implemented by finite-state machines (FSM&amp;#x27;s), is studied. It is shown that Markovian machines with sufficiently long memory exist that are asymptotically nearly as good as any given FSM (deterministic or randomized) for the purpose of sequential decision. For the continuous-valued observation case, a useful class of parametric schemes is discussed with special attention to the recursive least squares (RLS) algorithm.

Prediction of Individual Sequences using Universal Deterministic Finite State Machines

2006 IEEE International Symposium on Information Theory, 2006

We consider the problem of universal prediction of individual binary sequences where the universa... more We consider the problem of universal prediction of individual binary sequences where the universal predictor is a deterministic finite state machine with a fixed number of states. We examine the case of self-information loss, where the predictions are probability assignments. The performance of the predictors is measured by their redundancy w.r.t. the constant predictors class. We obtain an improved lower bound on the redundancy of any finite state (FS) predictor with K states. We construct a FS predictor based on the lower bound and compare the performance of the predictor to the lower bound. Numerical results show that the redundancy of the proposed FS predictor is close to that predicted by the lower bound.

Universal Communication—Part II: Channels With Memory

IEEE Transactions on Information Theory, 2014

ABSTRACT Consider communication over a channel whose probabilistic model is completely unknown ve... more ABSTRACT Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about the future. The existence of reliable feedback and common randomness is assumed. In a previous paper it was shown that the Shannon capacity cannot be attained, in general, if the channel is not known. An alternative notion of &quot;capacity&quot; was defined, as the maximum rate of reliable communication by any block-coding system used over consecutive blocks. This rate was shown to be achievable for the modulo-additive channel with an individual, unknown noise sequence, and not achievable for some channels with memory. In this paper this &quot;capacity&quot; is shown to be achievable for general channel models possibly including memory, as long as this memory fades with time. In other words, there exists a system with feedback and common randomness that, without knowledge of the channel, asymptotically performs as well as any block code, which may be designed knowing the channel. For non-fading memory channels a weaker type of &quot;capacity&quot; is shown to be achievable.

Communication with Feedback via Posterior Matching

2007 IEEE International Symposium on Information Theory, 2007

In this paper we describe a general algorithmic scheme for communication over any memoryless chan... more In this paper we describe a general algorithmic scheme for communication over any memoryless channel in the presence of noiseless feedback. The scheme is based on the idea of posterior matching, in which the information still missing at the receiver is extracted from the a-posteriori density function, and matched to any desirable input distribution. We analyze the error probability attained by this scheme for additive noise channels, and show that the well-known Schalkwijk-Kailath scheme for the AWGN channel with average power constraint and the Horstein scheme for the BSC, can be derived as special cases.

An achievable rate for the MIMO individual channel

2010 IEEE Information Theory Workshop, 2010

We consider the problem of communicating over a multiple-input multiple-output (MIMO) real valued... more We consider the problem of communicating over a multiple-input multiple-output (MIMO) real valued channel for which no mathematical model is specified, and achievable rates are given as a function of the channel input and output sequences known a-posteriori. This paper extends previous results regarding individual channels by presenting a rate function for the MIMO individual channel, and showing its achievability in a fixed transmission rate communication scenario.

Feedback communication over individual channels

2009 IEEE International Symposium on Information Theory, 2009

We consider the problem of communicating over a channel for which no mathematical model is specif... more We consider the problem of communicating over a channel for which no mathematical model is specified. We present achievable rates as a function of the channel input and output known a-posteriori for discrete and continuous channels, as well as a rate-adaptive scheme employing feedback which achieves these rates asymptotically without prior knowledge of the channel behavior.

Universal communication over unknown vector channels

2012 IEEE International Symposium on Information Theory Proceedings, 2012

Consider communication over a channel whose probabilistic model is completely unknown vector-wise... more Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about the future. The existence of reliable feedback and common randomness is assumed. In a previous paper it was shown that the Shannon capacity cannot be attained, in general, if the channel is not known. An alternative notion of "capacity" was defined, which is the maximum rate of reliable communication by any block-coding system used over consecutive blocks. This rate was shown to be achievable for the modulo-additive channel with an individual, unknown noise sequence, and not achievable for some channels with memory. In this paper this "capacity" is shown to be achievable for general channel models possibly including memory, as long as this memory fades with time. In other words, there exists a system with feedback and common randomness that without knowledge of the channel, asymptotically performs as well as any block-coding system, which may be designed knowing the channel. For non-fading memory channels a weaker type of "capacity" is shown to be achievable.