Entropy Measures

Many information measures are suggested in the literature. Among these measures are the Shannon H(n, b) and the Awad A(n, b) entropy measures. In this work we suggest a new entropy measure, B(n, b, estimated b) , which is based on the maximum likelihood function. These three entropy measures were calculated from the gamma distribution and its normal approximation, the binomial and its Poisson approximation, and the Poisson and its normal approximation. The relative losses in these three entropy measures are used as a criterion for the appropriateness of the approximation.

License: CC BY-NC-SA 3.0 (https://creativecommons.org/licenses/by-nc-sa/3.0/)

A little collection of inequality measures and a Python script which implements some formulas from Amartya Sen's and James E. Foster's "On Economic Inequality" as well as some formulas which I developed. Also misunderstandings of the term "entropy" are addressed: Theil's measure is not an entropy. It is an redundancy (as defined for information theory in ISO/IEC DIS 2382-16.03.05.). --- (Sorry for the typos in the article. They won't affect the Python code however.)

When applying progressive Type-II right censoring in life-testing experiments, the selection of an optimal censoring scheme is an important issue. In the literature there are several optimality criteria. This paper concentrates on investigating the possibility of using entropy-information measures to design an optimality type-II progressive censoring scheme with an illustrative application to a simple form of Pareto distribution. According to the suggested criterion, a censoring scheme is called optimal if it maximizes an informational efficiency function that is defined as the ratio of available information in the censored sample to the available information in the complete sample. The paper provides mathematical formulas for the efficiency of progressive type-II censoring scheme based on sixteen entropy-information measures. It turns out that all six information measures under consideration do not help in picking an optimal scheme since their values are free of the censoring scheme vector. To select a sup-entropy measure that leads to an optimal scheme, we have designed a Mathematica-7 code that can compute the numerical value for each of the ten sup-entropy measures that are used in this paper. An illustrative numerical example showed that the optimal scheme is a one step censoring from left after observing first failure.

This paper deals with the loss of information due to summarizing data. This loss is measured by entropy measures. It suggests the concept of alpha-sufficient partition of the sample space of the random model together with the concept of efficiency of a random sample. These concepts are applied to simulated data from geometric and exponential distributions

Segmentation of color images is important for various image analysis problems and is somewhat more involved than the segmentation of gray scale images. Preserving the original colors in different segments of the original image is one of the main problems in color image segmentation. Here in this paper, presents an approach for color image segmentation, which almost preserves the colors in different segments of input color image. In this projected approach, threshold selection in each of the three component (RGB) images is done on the basis of different entropy measures such as the Shannon, Renyi, Havrda-Charvat, Kapur and Vajda entropy measures. Simulation results for all above mentioned entropy measures are also presented and it is observed that Havrda Charvat and Shannon entropy measures are better than other measures for color image segmentation problems.

This paper gives a review of the basic entropy measures, The centroid method is used to cluster  82 entropy measures according to their standardized values obtained from binomial distributions with m=5(5)30 and p=0.1(0.1)0.9. It is observed that there is a negligible effect of m and p on the obtained clusters. It is recommended to use only the cluster centers as representatives of those 82 entropy measures.

The relationship between cryptographic measures and chaotic dynamical complexity measures is studied in
this paper, including linear complexity and measure entropy, nonlinear complexity and source entropy.
Moreover, a method is presented to guarantee the complexity of chaos-based pseudorandom sequence due
to this relationship. This is important to the development of chaos-based cryptography.

License: CC BY-NC-SA 3.0 (https://creativecommons.org/licenses/by-nc-sa/3.0/)

„Common sense“ is a dangerous beast. But sometimes it is right. Common sense gives us the feeling, that the inequal distribution of ressources is fair, as some people contribute more to society and others less. However, „too much“ inequality is „unfair“, but also a perfectly equal distribution is „not natural“. So, where is the „right“ distribution – and why is it perceived to be right?

There has been research on this „common sense“, e.g. by Yoram Amiel in Thinking about Inequality: Personal Judgement and Income Distributions (2009). With his methods you can find a range of distributional equality (income inequality in that case) within which most people feel comfortable. (There are some cultural differences.) How could that „feeling“ come about? My assumption is, that  within this range the information gap between
•  an inequality, which could be „equalized“ by a consciously controlled redistribution and
•  an inequality, which converges towards equality by stochastic redistribution
reaches an maximum. This information gap may determine the degree of inequality perception, and then the intensity of inequality issuization. How can this gap be computed?

License: CC BY-NC-SA 3.0 (https://creativecommons.org/licenses/by-nc-sa/3.0/)

Computes inequality indices from quantile data. The data can be added in the URL or in a form. English example for data in the URL: http://www.poorcity.richcity.org/calculator/?quantiles=50,2.5|40,47.5|9,27|1,23 , German example: http://www.umverteilung.de/rechner/?quantiles=50,2.5|40,47.5|9,27|1,23). Python source code of algorithms used: https://www.academia.edu/9904452/Inequality_Measures