Maximum Entropy Principle

This work proposes a parametric probabilistic approach to model damage accumulation using the double linear damage rule (DLDR) considering the existence of limited experimental fatigue data. A probabilistic version of DLDR is developed in which the joint distribution of the knee-point coordinates is obtained as a function of the joint distribution of the DLDR model input parameters. Considering information extracted from experiments containing a limited number of data points, an uncertainty quantification framework based on the Maximum Entropy Principle and Monte Carlo simulations is proposed to determine the distribution of fatigue life. The proposed approach is validated using fatigue life experiments available in the literature.

This paper presents a framework that develops algorithms for solving combined locational and multihop routing optimization problems. The objective is to determine resource node locations in a multiagent network and to specify the multihop routes from each agent to a common destination through a network of resource nodes that minimize total communication cost. These problems are computationally complex (NP-hard) where

EEG time series were modeled as an output of the linear filter driven by white noise. Parameters describing the signal were determined in a way fulfilling the maximum entropy principle. Transfer function and the impulse response function were found. The solutions of the differential equations describing the system have the form of the damped oscillatory modes. The representation of the EEG time series as a superposition of the resonant modes with characteristic decay factors seems a valuable method of the analysis of the signal, since it offers high reduction of the data to the few parameters of a clear physiological meaning.

The two component extreme value (TCEV) distribution has recently been shown to account for most of the characteristics of the real flood experience. A new method of parameter estimation for this distribution is derived using the principle of maximum entropy (POME). This method of parameter estimation is suitable for application in both the site-specific and regional cases and appears simpler than the maximum likelihood estimation method. Statistical properties of the regionalized estimation were evaluated using a Monte Carlo approach and compared with those of the maximum likelihood regional estimators.

The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specif ic statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Plastino and Curado [Phys. Rev. E 72, 047103 (2005)] recently determined the equilibrium probability distribution for the canonical ensemble using only phenomenological thermodynamical laws as an alternative to the entropy maximization procedure of Jaynes. In the current paper we present another alternative derivation of the canonical equilibrium probability distribution, which is based on the definition of the Helmholtz free energy (and its being constant at the equilibrium) and the assumption of the uniqueness of the equilibrium probability distribution. Noting that this particular derivation is applicable for all trace-form entropies, we also apply it to the Tsallis entropy showing that the Tsallis entropy yields genuine inverse power laws.

In generalized statistical mechanics the second link is customarily relaxed. Of course, the generalized exponential function defining the probability distribution function after inversion, produces a generalized logarithm Λ(pi). But, in general, the mean value of −Λ(pi) is not the entropy of the system. Here we reconsider the question first posed in [Phys. Rev. E 66, 056125 (2002) and 72, 036108 ], if and how is it possible to select generalized statistical theories in which the above mentioned twofold link between entropy and the distribution function continues to hold, such as in the case of ordinary statistical mechanics.

We revisit the concavity property of the thermodynamic entropy in order to formulate a general proof of the minimum energy principle as well as of other equivalent extremum principles that are valid for thermodynamic potentials and corresponding Massieu functions under different constraints. The current derivation aims at providing a coherent formal framework for such principles which may be also pedagogically useful as it fully exploits and highlights the equivalence between different schemes. We also elucidate the consequences of the extremum principles for the general shape of thermodynamic potentials in relation to first-order phase transitions.

In transportation subnetwork-supernetwork analysis, it is well known that the origin-destination (O-D) flow table of a subnetwork is not only determined by trip generation and distribution, but also a result from traffic routing and diversion, due to the existence of internal-external, external-internal and external-external flows. This result indicates the variable nature of subnetwork O-D flows. This paper discusses an elastic O-D flow table estimation problem for subnetwork analysis. The underlying assumption is that each cell of the subnetwork O-D flow table contains an elastic demand function rather than a fixed demand rate and the demand function can capture all traffic diversion effect under various network changes. We propose a combined maximum entropy-least squares estimator, by which O-D flows are distributed over the subnetwork in terms of the maximum entropy principle, while demand function parameters are estimated for achieving the least sum of squared estimation errors. While the estimator is powered by the classic convex combination algorithm, computational difficulties emerge within the algorithm implementation until we incorporate partial optimality conditions and a column generation procedure into the algorithmic framework. Numerical results from applying the combined estimator to a couple of subnetwork examples show that an elastic O-D flow table, when used as input for subnetwork flow evaluations, reflects network flow changes significantly better than its fixed counterpart.

The focus of this paper is on the development of an origin–destination (O-D) demand estimation method for dynamic equilibrium traffic networks. It is hypothesized that the underlying equilibrium conditions in such networks are a compromise result of minimization of individual routing costs, minimization of traffic count matching errors, and maximization of O-D demand entropies. By adding an upper bound of travel demand and a dummy path with constant travel cost to each O-D pair, we formulated the dynamic O-D demand estimation problem as an excess-demand dynamic traffic assignment (DTA) problem defined for an expanded network with dummy paths. Such a formulation enables us to apply existing DTA solution methods and software tools for deriving the path flow pattern in the expanded network and thus simultaneously obtaining the O-D demand pattern in the original network. Following this problem transformation and network expansion strategy, an iterative solution procedure is accordingly proposed, which resorts to repeatedly solving the excess-demand DTA problem and adjusting the dummy path costs. An application of the proposed modeling and solution methods for an example cell-based network problem favorably illustrates great promise of the methodological advance and solution performance.

The present paper is concerned with the dynamic modeling and design of control laws for a small non-rigid multi-rotor airship constituted of an oblate-spheroid helium balloon coupled with an electric-powered hexa-rotor airframe. The vehicle is assumed to operate in windless and low-speed conditions. A six-degree-of-freedom nonlinear dynamic model is derived for it using the Newton-Euler approach and considering, among other efforts, a restoring torque due to the displacement of the balloon's center of buoyancy above the vehicle's center of mass and the added-mass effect resulting from the air-structure interaction. Using the derived model and assuming a timescale separation between the translational and rotational dynamics, the attitude and position control laws are designed separately from each other. Both laws are formulated using feedback linearization combined with control input saturation within appropriate parallelepipedal sets, which are carefully chosen to respect pre-defined bounds on the control torque, control force and maximum inclination angle. The effect of temperature and pressure fluctuations is taken into account through a parametric probabilistic approach, where Maximum Entropy Principle is used to construct a physically consistent stochastic model and Monte Carlo method is used as the stochastic solver to propagate the uncertainties through the system. Extensive simulation results show the effectiveness of the proposed control system and quantify the uncertainty of its performance over a wide range of local temperature and pressure.

"In this article, we have derived the probability density functions of the productand the quotient of two independent random variables having Gauss hypergeometricdistribution. These densities have been expressed in terms of Appell'srst hypergeometric function F1. Further, Renyi and Shannon entropies havealso been derived for the Gauss hypergeometric distribution"

Bu çalışmada en büyük entropi ilkesini kullanan yararlı yüke dayalı bir saldırı tespit sistemi, Me-PAYL önerilmiştir. Ağdan izlenmiş verileri kullanan en büyük entropi ve göreli entropiye dayalı bir ağ tabanlı tespit sistemi geliştirilmiştir. Daha fazla verim almak için en büyük entropi yaklaşımının avantajları PAYL [1] yöntemi ile birleştirilmiştir. Önerilen yöntem, Me-PAYL, yararlı yüklerin tamamını içeren kullanılabilir en büyük veri kümesi olan, DARPA 1999 Intrusion Detection Evaluation (IDEVAL) veri kümesi [2] ile test edilmiştir.

Maximum entropy principle does not seem to distinguish between the use of Tsallis and Renyi entropies as either of them may be used to derive similar power-law distributions. In this paper, we address the question whether the Renyi entropy is equally suitable to describe those systems with power-law behaviour, where the use of the Tsallis entropy is relevant. We discuss a specific class of dynamical systems, namely, one dimensional dissipative maps at chaos threshold and make our study from two aspects: i) power-law sensitivity to the initial conditions and the rate of entropy increase, ii) generalized bit cumulants. We present evidence that the Tsallis entropy is the more appropriate entropic form for such studies as opposed to Renyi form.

We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in those patterns. This constraint allows for a new statistical characterization of fractal objects and fractal dimension. PACS: 47.53.+n, 05.90.+m * Present address:

Uncertainty quantification (UQ) is a multidisciplinary area, that deals with quantitative characterization and reduction of uncertainties in applications. It is essential to certify the quality of numerical and experimental analyses of physical systems. The present manuscript aims to provide the reader with an introductory view about modeling and quantification of uncertainties in physical systems. In this sense, the text presents some fundamental concepts in UQ, a brief review of probability basics notions, discusses, through a simplistic example, the fundamental aspects of probabilistic modeling of uncertainties in a physical system, and explains what is the uncertainty propagation problem.

Consistent hydrodynamical models for electron transport in Si and GaAs semiconductors, free of any fitting parameter, have been formulated in (Cont. Mech. Thermodyn. 11 (1999) 307; Contemp. Mech. Thermodyn. 12 (1999) 31; Contemp. Mech. Thermodyn. 14 (2002) 405; COMPEL (to appear)) on the basis of the maximum entropy principle (MEP), by describing the valleys in the energy conduction band by means of the Kane dispersion relation. Explicit constitutive functions for fluxes and production terms appearing in the macroscopic balance equations of density, crystal momentum, energy and energy-flux have been obtained. Scatterings of electrons with polar (in the case of GaAs) and non-polar optical phonons, both for intervalley and intravalley interactions, and with acoustic phonons and impurities have been taken into account. Here we derive from the previous hydrodynamical models both low-and high-field mobilities. The results are compared with those given by the Caughey-Thomas formula and

Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden.

Analytical descriptions of the statistics of wireless channel models are desirable tools for communication systems engineering. When multiple antennas are available at the transmit and/or the receive side (the Multiple-Input Multiple-Output, or MIMO, case), the statistics of the matrix H representing the gains between the antennas of a transmit and a receive antenna array, and in particular the correlation between its coefficients, are known to be of paramount importance for the design of such systems. However these characteristics depend on the operating environment, since the electromagnetic propagation paths are dictated by the surroundings of the antenna arrays, and little knowledge about these is available at the time of system design.

Data mining algorithms produce huge sets of rules, practically impossible to analyze manually. It is thus important to develop methods for removing redundant rules from those sets. We present a solution to the problem using the Maximum Entropy approach. The problem of efficiency of Maximum Entropy computations is addressed by using closed form solutions for the most frequent cases. Analytical and experimental evaluation of the proposed technique indicates that it efficiently produces small sets of interesting association rules.

The quasi--equilibrium or maximum entropy approximation is applied in order to derive constitutive equations from kinetic models of polymer dynamics. It is shown in general and illustrated for an example how canonical distribution functions are obtained from the maximum entropy principle, how macroscopic and constitutive equations are derived therefrom and how these constitutive equations can be implemented numerically. In addition, a measure for the accuracy of the quasi--equilibrium approximation is proposed that can be evaluated while integrating the constitutive equations. In the example considered, it is confirmed that the accuracy of the approximation is increased by including more macroscopic variables. In steady elongational flow, it is found that more macroscopic variables need to be included above the coil--stretch transition to achieve the same accuracy as below.

At first sight, formation of self-assembled structures such as micelles, bilayers, DNA-cationic bilayers complexes, and the enzyme-substrate complexation in biological catalytic reactions look as totally different phenomena. However, common in all of them is the effective attractive interactions among the self-assembled molecular components that are required to occur. In general, the direct interactions among the components of an assembled structure are not sufficiently attractive to justify self-assembling by itself. Furthermore, in some cases these direct interactions are repulsive as in the case of like charged particles. In this paper, by means of theory and simulation, we consider the effective interactions of the following simple systems: 1) two likecharged parallel rods immersed in an electrolyte solution, 2) two like-charged parallel rods confined between two parallel plates, and 3) a sphere with an open cavity and a spherical macroparticle, both immersed in a hard sphere fluid. In the first two systems we show that effective attractive interaction among the components of a self-assembled structure may only occur when they are in high electrolyte volume fractions. In the lock-key system (3rd case), it is demonstrated that the perfect match between the spherical cavity and macroparticle is the most favorable condition for their self-assembly to occur. All these are examples of attraction driven by the need of the systems to optimize the available volume. That is, self-assembling is driven by the maximum entropy principle, or maximum missing information, as in information theory.

The approach of an ideal gas to equilibrium is simulated through a generalization of the Ehrenfest ball-and-box model. In the present model, the interior of each box is discretized, {\it i.e.}, balls/particles live in cells whose occupation can be either multiple or single. Moreover, particles occasionally undergo random, but elastic, collisions between each other and against the container walls. I show, both analitically and numerically, that the number and energy of particles in a given box eventually evolve to an equilibrium distribution $W$ which, depending on cell occupations, is binomial or hypergeometric in the particle number and beta-like in the energy. Furthermore, the long-run probability density of particle velocities is Maxwellian, whereas the Boltzmann entropy $\ln W$ exactly reproduces the ideal-gas entropy. Besides its own interest, this exercise is also relevant for pedagogical purposes since it provides, although in a simple case, an explicit probabilistic foundation for the ergodic hypothesis and for the maximum-entropy principle of thermodynamics. For this reason, its discussion can profitably be included in a graduate course on statistical mechanics.

Minimum output mutual information is regarded as a natural criterion for independent component analysis (ICA) and is used as the performance measure in many ICA algorithms. Two common approaches in information theoretic ICA algorithms are minimum mutual information and maximum output entropy approaches. In the former approach, we substitute some form of probability density function (pdf) estimate into the mutual information expression, and in the latter we incorporate the source pdf assumption in the algorithm through the use of nonlinearities matched to the corresponding cumulative density functions (cdf). Alternative solutions to ICA utilize higher order cumulant-based optimization criteria, which are related to either one of these approaches through truncated series approximations for densities. In this paper, we propose a new ICA algorithm motivated by the Maximum Entropy Principle (for estimating signal distributions). The optimality criterion is the minimum output mutual information, where the estimated pdfs are from the exponential family, and are approximate solutions to a constrained entropy maximization problem. This approach yields an upper bound for the actual mutual information of the output signals, hence the name Minimax Mutual Information ICA algorithm. In addition, we demonstrate that for a specific selection of the constraint functions in the maximum entropy density estimation procedure, the algorithm relates strongly to ICA methods using higher order cumulants.

Advances in computer processing power and emerging algorithms are allowing new ways of envisioning human computer interaction. This paper focuses on the development of a computing algorithm that uses audio and visual sensors to detect and track a user's affective state to aid computer decision making. Using our multi-stream fused hidden Markov model (MFHMM), we analyzed coupled audio and visual streams to detect 11 cognitive/emotive states. The MFHMM allows the building of an optimal connection among multiple streams according to the maximum entropy principle and the maximum mutual information criterion. Person-independent experimental results from 20 subjects in 660 sequences show that the MFHMM approach performs with an accuracy of 80.61% which outperforms face-only HMM, pitch-only HMM, energy-only HMM, and independent HMM fusion.

A finite difference scheme of Scharfetter-Gummel type is used to simulate a consistent energy-transport model for electron transport in semiconductors devices, free of any fitting parameters, formulated on the basis of the maximum entropy principle.

Le principal objet de cette communication est de faire une rétro perspective succincte de l'utilisation de l'entropie et du principe du maximum d'entropie dans le domaine du traitement du signal. Après un bref rappel de quelques définitions et du principe du maximum d'entropie, nous verrons successivement comment l'entropie est utilisée en séparation de sources, en modélisation de signaux, en analyse spectrale et pour la résolution des problèmes inverses linéaires.

In classification problems, lack of knowledge of the prior distribution may make the application of Bayes' rule inadequate. Uniform or arbitrary priors may often provide classification answers that, even in simple examples, may end up contradicting our common sense about the problem. Entropic priors, via application of the maximum entropy principle, seem to provide a much better answer and can be easily derived and applied to classification tasks when no more than the likelihood funtions are available. In this paper we present an application example in which the use of the entropic priors is compared to the results of the application of Dempster-Shafer theory.

The continuum limit of lattice models arising in two-dimensional turbulence is analyzed by means of the theory of large deviations. In particular, the Miller Robert continuum model of equilibrium states in an ideal fluid and a modification of that model due to Turkington are examined in a unified framework, and the maximum entropy principles that govern these models are rigorously derived by a new method. In this method, a doubly indexed, measure-valued random process is introduced to represent the coarse-grained vorticity field. The natural large deviation principle for this process is established and is then used to derive the equilibrium conditions satisfied by the most probable macrostates in the continuum models. The physical implications of these results are discussed, and some modeling issues of importance to the theory of long-lived, large-scale coherent vortices in turbulent flows are clarified.

Information valuable words are the strings with the significant deviation of real frequency from the expected one. The expected frequency is determined through the maximum entropy principle of the reconstructed (extended) frequency dictionary of strings composed from the shorter words. The information valuable words are found to be the complementary palindromes: they are read equally in opposite directions, if nucleotides are changed for the complementary ones ({A ⇋ T; C ⇋ G}) in one of them. Some properties of such symmetric words are discussed.

A general type of nonlinear Fokker-Planck equation is derived directly from a master equation, by introducing generalized transition rates. The H-theorem is demonstrated for systems that follow those classes of nonlinear Fokker-Planck equations, in the presence of an external potential. For that, a relation involving terms of Fokker-Planck equations and general entropic forms is proposed. It is shown that, at equilibrium, this relation is equivalent to the maximum-entropy principle. Families of Fokker-Planck equations may be related to a single type of entropy, and so, the correspondence between well-known entropic forms and their associated Fokker-Planck equations is explored. It is shown that the Boltzmann-Gibbs entropy, apart from its connection with the standard-linear Fokker-Planck equation-may be also related to a family of nonlinear Fokker-Planck equations.

The probabilistic distribution of wind speed is one of the important wind characteristics for the assessment of wind energy potential and for the performance of wind energy conversion systems, as well as for the structural and environmental design and analysis. In this study, an exponential family of distribution functions has been developed for the description of the probabilistic distribution of wind speed, and comparison with the wind speed data taken from different sources and measured at different geographical locations in the world has been made. This family of distributions is developed by introducing a pre-exponential term to the theoretical distribution derived from the maximum entropy principle (MEP). The statistical analysis parameter based on the wind power density is used as the suitability judgement for the distribution functions. It is shown that the MEPtype distributions not only agree better with a variety of the measured wind speed data than the conventionally used empirical Weibull distribution, but also can represent the wind power density much more accurately. Therefore, the MEP-type distributions are more suitable for the assessment of the wind energy potential and the performance of wind energy conversion systems.

In this paper the realized niche of the Snowy Plover (Charadrius alexandrinus), a 23 primarily resident Florida shorebird, is described as a function of the scenopoetic and 24 bionomic variables at the nest-, landscape-, and regional-scale. We identified some pos-25 sible geomorphological controls that influence nest-site selection and survival using data 26 collected along the Florida Gulf coast. In particular we focused on the effects of beach 27 replenishment interventions on the Snowy Plover (SP), and on the migratory Piping 28 Plover (PP) (Charadrius melodus) and Red Knot (RK) (Calidris canutus). Addition-29 ally we investigated the potential differences between the SP breeding and wintering 30 distributions using only regional-scale physiognomic variables and the recorded occur-31 rences. To quantify the relationship between past renourishment projects and shorebird 32 species we used a Monte Carlo procedure to sample from the posterior distribution of the 33 binomial probabilities that a region is not a nesting or a wintering ground conditional 34 on the occurrence of a beach replenishment intervention in the same and the previous 35 year. The results indicate that it was 2.3, 3.1, and 0.8 times more likely that a region 36 was not a wintering ground following a year with a renourishment intervention for the 37 SP, PP and RK respectively. For the SP it was 2.5. times more likely that a region 38 was not a breeding ground after a renourishment event. Through a maximum entropy 39 principle model we observed small differences in the habitat use of the SP during the 40 breeding and the wintering season. However the habitats where RK was observed ap-41 peared quite different. While ecological niche models at the macro-scale are useful for 42 determining habitat suitability ranges, the characterization of the species' local niche 43 is fundamentally important for adopting concrete multispecies management scenarios. 44 Maintaining and creating optimal suitable habitats for SP characterized by sparse low 45 vegetation in the foredunes areas, and uneven/low-slope beach surfaces, is the proposed 46 conservation scenario to convert anthropic beach restorations and SP populations into 47 a positive feedback without impacting other threatened shorebird species. 48 1 Introduction 49

We find the value of constants related to constraints in characterization of some known statistical distributions and then we proceed to use the idea behind maximum entropy principle to derive generalized version of this distributions using the Tsallis' and Renyi information measures instead of the well-known Bolztmann-Gibbs-Shannon. These generalized distributions will depend on q (real number) and in the limit (q=1) we obtain the "classical" ones. We found that apart from a constant, generalized versions of statistical distributions following Tsallis' or Renyi are undistinguishable.

Two-term Volterra series are often used to describe non-linear random processes in subject areas as diverse as communication theory and the dynamics of offshore structures. It is well known that the characteristic function, the moments and the cumulants of such processes can be calculated analytically and that the probability density function can be calculated accurately and efficiently. This paper considers the joint statistics between two such random processes by deriving: (i) an exact expression for the joint characteristic function; (ii) an efficient means for calculating the joint moments; (iii) an 'exact' numerical means for calculating the joint probability density function (jpdf). For the special case of a combined first-and second-order process and a pure first-order process it is shown that it is possible to derive analytical expressions for the characteristic function and to calculate the jpdf accurately and efficiently using saddle-point integration. In addition to the above, the maximum entropy principle is used to calculate the jpdf. q

We devise a maximum entropy technique to construct (approximate) time-dependent solutions to evolution equations endowed with source terms and, consequently, not preserving normalization. In some special cases the method yields exact solutions. It is shown that the present implementation of the maximum entropy prescription always (even in the case of approximate solutions) preserves the exact functional relationship between the time derivative of the entropy and the timedependent solutions of the evolution equation. Other properties of the maximum entropy solutions and some illustrative examples are also discussed. r 2006 Published by Elsevier B.V.

Maximum entropy principle does not seem to distinguish between the use of Tsallis and Renyi entropies as either of them may be used to derive similar kind of q-exponential distributions. In this paper, we address the question whether the Renyi entropy is equally suitable to describe those systems with q-exponential behaviour, where the use of the Tsallis entropy is relevant. We discuss a speciÿc class of dynamical systems, namely, one-dimensional dissipative maps at chaos threshold and make our study from two aspects: (i) power-law sensitivity to the initial conditions and the rate of entropy increase, (ii) generalized bit cumulants. We present evidence that the Tsallis entropy is more appropriate entropic form for such studies as compared to Renyi form.

Convex operational models (COMs) are considered as great extrapolations to larger settings of any statistical theory. In this article we generalize the maximum entropy principle (MaxEnt) of Jaynes' to any COM. After expressing Max-Ent in a geometrical and latttice theoretical setting, we are able to cast it for any COM. This scope-ampli?cation opens the door to a new systematization of the principle and sheds light into its geometrical structure.

The quasi--equilibrium approximation is employed as a systematic tool for solving the problem of deriving constitutive equations from kinetic models of liquid--crystalline polymers. It is demonstrated how kinetic models of liquid--crystalline polymers can be approximated in a systematic way, how canonical distribution functions can be derived from the maximum entropy principle and how constitutive equations are derived therefrom. The numerical implementation of the constitutive equations based on the intrinsic dual structure of the quasi--equilibrium manifold thus derived is developed and illustrated for particular examples. Finally, a measure of the accuracy of the quasi--equilibrium approximation is proposed that can be implemented into the numerical integration of the constitutive equation.