The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Prev... more The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Previous work has concretized this connection by defining a signed measure µ on an abstract information spaceX, which is taken to represent the information that X contains. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information IpX; Y q " µpX XỸ q, the joint entropy HpX, Y q " µpX YỸ q, and the conditional entropy HpX|Y q " µpX zỸ q. We demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, while also being coherent with Yeung's I-measure [14]. We present the usability of our approach by reexamining the Gács-Körner common information from this new geometric perspective and characterising it in terms of our logarithmic atoms. We then highlight that our geometric refinement can account for an entire class of information quantities, which we call logarithmically decomposable quantities.
The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Prev... more The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Previous work has concretized this connection by defining a signed measure µ on an abstract information spaceX, which is taken to represent the information that X contains. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information IpX; Y q " µpX XỸ q, the joint entropy HpX, Y q " µpX YỸ q, and the conditional entropy HpX|Y q " µpX zỸ q. We demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, while also being coherent with Yeung's I-measure [14]. We present the usability of our approach by reexamining the Gács-Körner common information from this new geometric perspective and characterising it in terms of our logarithmic atoms. We then highlight that our geometric refinement can account for an entire class of information quantities, which we call logarithmically decomposable quantities.
Uploads
Papers by Pedro Mediano