Entropy

We develop and apply several novel methods quantifying dynamic multi-agent team interactions. These interactions are detected information-theoretically and captured in two ways: via (i) directed networks (interaction diagrams) representing significant coupled dynamics between pairs of agents, and (ii) state-space plots (coherence diagrams) showing coherent structures in Shannon information dynamics. This model-free analysis relates, on the one hand, the information transfer to responsiveness of the agents and the team, and, on the other hand, the information storage within the team to the teamʼs rigidity and lack of tactical flexibility. The resultant interaction and coherence diagrams reveal implicit interactions, across teams, that may be spatially long-range. The analysis was verified with a statistically significant number of experiments (using simulated football games, produced during RoboCup 2D Simulation League matches), identifying the zones of the most intense competition, the extent and types of interactions, and the correlation between the strength of specific interactions and the results of the matches.

Ce texte a été publié en français en juin 2021 dans le quatrième numéro des Cahiers COSTECH : http://www.costech.utc.fr/CahiersCOSTECH/spip.php?article117.
Il constitue un travail préparatoire en vue de la publication d'un article en polonais dans la revue Er(r)go. Theory, Literature, Culture, N°44 1/2022 (https://journals.us.edu.pl/index.php/ERRGO).

Résumé : Cet article propose de considérer la crise écologique contemporaine comme un triple problème d'écologie environnementale, d'écologie mentale et d'écologie sociale : alors que les écosystèmes naturels se voient menacés par les infrastructures industrielles, qui exploitent les ressources minérales au service d'une économie consumériste globalisée, les écosystèmes psychiques et sociaux se voient menacés par les appareils connectés, qui exploitent les ressources libidinales, au service de l'économie de l'attention et des données. Nous proposons de penser cette triple crise écologique comme un processus d'augmentation de l'entropie à différents niveaux (thermodynamique, biologique et informationnel ou psycho-social), qui requiert une double transition énergétique, fondée sur l'économie et le soin des énergies physiques et minérales, et sur l'économie et le soin des énergies psychiques et libidinales. Nous nous appuyons sur les travaux de Nicholas Georgescu-Roegen et de Bernard Stiegler, qui envisagent cette transition comme une lutte contre l'augmentation de l'entropie, sur le plan physique et environnemental d'une part, et sur le plan psychique, technique et social d'autre part.

A total of 32 MCDM methods are included in the book. 10 of which are the methods used in determining the criteria weights and the rest are the MCDM methods used for the ranking of alternatives. The book also acts as a bridge from past to present. Kitapta toplam 32 ÇKKV yöntemine yer verilmiştir. Bunların 10’u kriter önem ağırlıklarının belirlenmesinde kullanılan yöntemler olup geriye kalanlar ise alternatiflerin sıralamaları amacıyla yararlanılan ÇKKV yöntemleridir. Kitap ayrıca geçmişten günümüze bir köprü görevi de görmektedir.

Contextuality and entanglement are valuable resources for quantum computing and quantum information. Bell inequalities are used to certify entanglement; thus, it is important to understand why and how they are violated. Quantum mechanics and behavioral sciences teach us that random variables measuring the same content (the answer to the same Yes or No question) may vary, if measured jointly with other random variables. Alice and Bob raw data confirm Einsteinian non-signaling, but setting dependent experimental protocols are used to create samples of coupled pairs of distant outcomes and to estimate correlations. Marginal expectations, estimated using these final samples, depend on distant settings. Therefore, a system of random variables measured in Bell tests is inconsistently connected and it should be analyzed using a Contextuality-by-Default approach, what is done for the first time in this paper. The violation of Bell inequalities and inconsistent connectedness may be explained...

Snoring, which may be decisive for many diseases, is an important indicator especially for sleep disorders. In recent years, many studies have been performed on the snore related sounds (SRSs) due to producing useful results for detection of sleep apnea/hypopnea syndrome (SAHS). The first important step of these studies is the detection of snore from SRSs by using different time and frequency domain features. The SRSs have a complex nature that is originated from several physiological and physical conditions. The nonlinear characteristics of SRSs can be examined with chaos theory methods which are widely used to evaluate the biomedical signals and systems, recently. The aim of this study is to classify the SRSs as snore/breathing/silence by using the largest Lyapunov exponent (LLE) and entropy with multiclass support vector machines (SVMs) and adaptive network fuzzy inference system (ANFIS). Two different experiments were performed for different training and test data sets. Experimental results show that the multiclass SVMs can produce the better classification results than ANFIS with used nonlinear quantities. Additionally, these nonlinear features are carrying meaningful information for classifying SRSs and are able to be used for diagnosis of sleep disorders such as SAHS.

In recent years, there are many great successes in using deep architectures for unsupervised feature learning from data, especially for images and speech. In this paper, we introduce recent advanced deep learning models to classify two emotional categories (positive and negative) from EEG data. We train a deep belief network (DBN) with differential entropy features extracted from multichannel EEG as input. A hidden markov model (HMM) is integrated to accurately capture a more reliable emotional stage switching. We also compare the performance of the deep models to KNN, SVM and Graph regularized Extreme Learning Machine (GELM). The average accuracies of DBN-HMM, DBN, GELM, SVM, and KNN in our experiments are 87.62%, 86.91%, 85.67%, 84.08%, and 69.66%, respectively. Our experimental results show that the DBN and DBN-HMM models improve the accuracy of EEGbased emotion classification in comparison with the state-ofthe-art methods.

In classical information theory, entropy rate and algorithmic complexity per symbol are related by a theorem of Brudno. In this paper, we prove a quantum version of this theorem, connecting the von Neumann entropy rate and two notions of quantum Kolmogorov complexity, both based on the shortest qubit descriptions of qubit strings that, run by a universal quantum Turing machine, reproduce them as outputs. *

This paper presents a numerical analysis of entropy generation in a two-dimensional rectangular channel where the inlet flow undergoes thermal decomposition resulting from a chemical reaction. The model considered viscosity and thermal conductivity to be dependent of temperature. Irreversibility due to mass transport was included in the entropy generation analysis. Relevant applications of this study are possible for the design of power generation systems and reactors. The effects of the Reynolds number, Schmidt number, and length of the heat source on thermal fluid dynamics, mass transfer, and irreversibility were also investigated. It was found that thermal decomposition increases at: a) low Reynolds numbers, b) low Schmidt numbers, and c) increased length of heat source. Additionally, overall entropy generation increased when Reynolds number and length of heat source were increased, although in all cases, overall irreversibility attains a minimum value at a specific Schmidt number.

Customer Relationship Management (CRM) is a fundamental tool in the hospitality industry nowadays, which can be seen as a big-data scenario due to the large amount of recordings which are annually handled by managers. Data quality is crucial for the success of these systems, and one of the main issues to be solved by businesses in general and by hospitality businesses in particular in this setting is the identification of duplicated customers, which has not received much attention in recent literature, probably and partly because it is not an easy-to-state problem in statistical terms. In the present work, we address the problem statement of duplicated customer identification as a large-scale data analysis, and we propose and benchmark a general-purpose solution for it. Our system consists of four basic elements: (a) A generic feature representation for the customer fields in a simple table-shape database; (b) An efficient distance for comparison among feature values, in terms of th...

In this study, we propose a novel method to investigate P300 variability over different trials. The method incorporates spatial correlation between EEG channels to form a cooperative coupled particle filtering method that tracks the P300 subcomponents, P3a and P3b, over trials. Using state space systems, the amplitude, latency, and width of each subcomponent are modeled as the main underlying parameters. With four electrodes, two coupled Rao-Blackwellised particle filter pairs are used to recursively estimate the system state over trials. A number of physiological constraints are also imposed to avoid generating invalid particles in the estimation process. Motivated by the bilateral symmetry of ERPs over the brain, the channels further share their estimates with their neighbors and combine the received information to obtain a more accurate and robust solution. The proposed algorithm is capable of estimating the P300 subcomponents in single trials and outperforms its non-cooperative counterpart.

In this chapter we present Excess Entropy Production for human aging system as the sum of their respective subsystems and electrophysiological status. Additionally, we support the hypothesis of human brain and central neural system quantumness and we strongly suggest the theoretical and philosophical status of human brain as one of the unknown natural Dirac magnetic monopoles placed in the center of a Riemann sphere.

The fundamental holographic principle is first proposed, then demonstrated in its validity and viability through a thought experiment and then finally derived. The Heisenberg uncertainty relations are shown to follow from this fundamental relation. The quantum blackhole entropy is then demonstrated using this holographic uncertainty relation along with the application of the Landauer’s principle for the thermodynamic erasure of a bit yielding a formula with a logarithmic correction. The blackhole entropy is found to be half the value normally delivered by any other method. So, it is proposed that there is a real relevant physical horizon at the twice the Schwarzschild radius dubbed the holographic information geometric horizon.

Lindsay in an article titled, "Entropy consumption and values in physical science," (Am. Sci. 1959, 47, 678-696) proposed a Thermodynamic Imperative similar to Kant's Ethical Categorical Imperative. In this paper, after describing the concept of ethical imperative as elaborated by Kant, we provide a brief discussion of the role of science and its relationship to the classical thermodynamics and the physical implications of the first and the second laws of thermodynamics. We finally attempt to extend and supplement Lindsay's Thermodynamic Imperative (TI), by another Imperative suggesting simplicity, conservation, and harmony.

Determining the strength of nonlinear, statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual information from limited samples is a challenging task. Since the mutual information is the difference of two entropies, the existing Bayesian estimators of entropy may be used to estimate information. This procedure, however, is still biased in the severely under-sampled regime. Here, we propose an alternative estimator that is applicable to those cases in which the marginal distribution of one of the two variables—the one with minimal entropy—is well sampled. The other variable, as well as the joint and conditional distributions, can be severely undersampled. We obtain a consistent estimator that presents very low bias, outperforming previous methods even when the sampled data contain few coincidences. As with other Bayesian estimators, our prop...

Content-based image retrieval focuses on intuitive and efficient methods for retrieving images from databases based on the content of the images. A new entropy function that serves as a measure of information content in an image termed as 'an information theoretic measure' is devised in this paper. Among the various query paradigms, query by example (QBE) is adopted to set a query image for retrieval from a large image database. In this paper, colour and texture features are extracted using the new entropy function and the dominant colour is considered as a visual feature for a particular set of images. Thus colour and texture features constitute the two-dimensional feature vector for indexing the images. The low dimensionality of the feature vector speeds up the atomic query. Indices in a large database system help retrieve the images relevant to the query image without looking at every image in the database. The entropy values of colour and texture and the dominant colour are considered for measuring the similarity. The utility of the proposed image retrieval system based on the information theoretic measures is demonstrated on a benchmark dataset.

Data compression at its base is concerned with how information is organized in data. Understanding this organization can lead to efficient ways of representing the information and hence data compression. In this paper we review the ways in which ideas and approaches fundamental to the theory and practice of data compression have been used in the area of bioinformatics. We look at how basic theoretical ideas from data compression, such as the notions of entropy, mutual information, and complexity have been used for analyzing biological sequences in order to discover hidden patterns, infer phylogenetic relationships between organisms and study viral populations. Finally, we look at how inferred grammars for biological sequences have been used to uncover structure in biological sequences.

A Mg/Fe hydrotalcite-like-compound (HTlc) was prepared and its affinity toward the removal of SeO 2− 3 from an aqueous medium was studied as a function of pH, time, temperature, particle dose, and SeO 2− 3 concentration. The fraction of SeO 2− 3 removal increases with decrease in both pH and temperature. The adsorption data are fitted to the Langmuir adsorption isotherm in the temperature range 303-333 K, and the thermodynamic parameters viz. standard Gibbs' free energy change (G •), enthalpy change (H •), and entropy change (S •) are calculated. The negative value of H • indicates that the adsorption process is exothermic. The apparent equilibrium constants (K a) are also calculated and found to decrease with increase in temperature.

We use empowerment, a recently introduced biologically inspired measure, to allow an AI player to assign utility values to potential future states within a previously un-encountered game without requiring explicit specification of goal states. We further introduce strategic affinity, a method of grouping action sequences together to form 'strategies', by examining the overlap in the sets of potential future states following each such action sequence. Secondly, we demonstrate an information-theoretic method of predicting future utility. Combining these methods, we extend empowerment to soft-horizon empowerment which enables the player to select a repertoire of action sequences that aim to maintain anticipated utility.

Image analysis is playing a very essential role in numerous research areas in the fields of science and technology, ranging from medical imaging to the computer science of automatic vision. Being involved in several applications, which are mainly based on a constant innovation of technologies, image analysis always requires different approaches and new algorithms, including continual upgrades and improvements of existing methods. Accordingly, the range of the analyses in which it is engaged can be considered as wide as the prospective future technologies. A challenge of image analysis is obtaining meaningful information by extracting specific features from images contained in large databases or during real-time acquisition. Among the problems requiring feature extraction are, to name a few, face detection and recognition, character recognition, and parametric determinations for augmented reality and other technologies. Another challenge is the secure encryption and decryption of multimedia data. These tasks demand highly sophisticated numerical and analytical methods. The contributions to this Special Issue provide a good overview of the most important demands and solutions concerning the abovementioned extraction, encryption, and decryption of data. In all the contributions, entropy plays a pivotal role. In the following, the reader can find subjects and problems according to the order of their publication.

The real-time adaptation between human and assistive devices can improve the quality of life for amputees, which, however, may be difficult to achieve since physical and mental states vary over time. This paper presents a co-adaptive humanmachine interface (HMI) that is developed to control virtual forearm prosthesis over a long period of operation. Direct physical performance measures for the requested tasks are calculated. Bioelectric signals are recorded using one pair of electrodes placed on the frontal face region of a user to extract the mental (affective) measures (the entropy of the alpha band of the forehead electroencephalography signals) while performing the tasks. By developing an effective algorithm, the proposed HMI can adapt itself to the mental states of a user, thus improving its usability. The quantitative results from 16 users (including an amputee) show that the proposed HMI achieved better physical performance measures in comparison with the traditional (nonadaptive) interface (p-value < 0:001). Furthermore, there is a high correlation (correlation coefficient < 0:9; p-value < :01) between the physical performance measures and self-report feedbacks based on the NASA TLX questionnaire. As a result, the proposed adaptive HMI outperformed a traditional HMI.

The concentration of people in densely populated urban areas, especially in developing countries, calls for the use of monitoring systems like remote sensing. Such systems along with spatial analysis techniques like digital image processing and geographical information system (GIS) can be used for the monitoring and planning purposes as these enable the reporting of overall sprawl at a detailed level.In the present work, urban sprawl of the Ajmer city (situated in Rajasthan State of India) has been studied at a mid scale level, over a period of 25 years (1977–2002), to extract the information related to sprawl, area of impervious surfaces and their spatial and temporal variability. Statistical classification approaches have been used for the classification of the remotely sensed images obtained from various sensors viz. Landsat MSS, TM, ETM+ and IRS LISS-III. Urban sprawl and its spatial and temporal characteristics have been derived from the classified satellite images. The Shannon's entropy and landscape metrics (patchiness and map density) have been computed in terms of spatial phenomenon, in order to quantify the urban form (impervious area). Further, multivariate statistical techniques have been used to establish the relationship between the urban sprawl and its causative factors. Results reveal that land development (160.8%) in Ajmer is more than three times the population growth (50.1%). Shannon's entropy and landscape metrics has revealed the spatial distribution of the urban sprawl over a period of last 25 years.

Aims: The present investigation deals with the development of thermotolerant mutant strain of yeast for studying enhanced productivity of ethanol from molasses in a fully controlled bioreactor. Methods and Results: The parental culture of Saccharomyces cerevisiae ATCC 26602 was mutated using UV treatment. A single thermotolerant mutant was isolated after extensive screening and optimization, and grown on molasses medium in liquid cultures. The mutant was 1AE45-fold improved than its wild parent with respect to ethanol productivity (7AE2 g l )1 h )1 ), product yield (0AE44 g ethanol g )1 substrate utilized) and specific ethanol yield (19AE0 g ethanol g )1 cells). The improved ethanol productivity was directly correlated with titres of intracellular and extracellular invertase activities. The mutant supported higher volumetric and product yield of ethanol, significantly (P £ 0AE05) higher than the parental and other strains. The mutated cultures produced 1AE8-and 2AE6-fold more extracellular and intracellular invertase productivity, respectively, than that produced by its wild parent at 40°C. Thermodynamic studies revealed that the cell system exerted protection against thermal inactivation during formation of products. Conclusions: A mutant derivative of Sacchromyces cerevisiae with improved productivity of ethanol and invertases has been obtained, which showed concomitant improvement in thermostability of endogenous metabolism for formation of both ethanol and invertases. Significance and Impact of the Study: The results of the present study are of commercial value as the mutant can be used for ethanol production in parts of Pakistan where the temperature may go up to 40°C in April. Ethanol product yield coefficient and volumetric productivity, revealed the hyper-productivity of ethanol from molasses at 40°C, which is not appropriate for wild organism.

Brain computer interface (BCI) technology as its name implies, relies upon decoding brain signals into operational commands. Aside from needing effective means of control, successful BCIs need to remain stable in varying physiological conditions. BCIs need to be developed with mechanisms to recognise and respond to physiological states (such as stress and fatigue) that can disrupt user capability. This paper compares a spectral analysis of EEG signals technique with a nonlinear method of sample entropy to detect changes in brain dynamics during moments of stress and fatigue. The results demonstrated few changes in the spectral frequency bands of the EEG during fatigue and stress conditions. However, when the EEG signals were analysed with the nonlinear technique of sample entropy the results indicated a reduction of complexity during moments of fatigue and stress and an increase in complexity during moments of engagement to the task.

A Self-Organizing Map (SOM) is typically trained in unsupervised mode, using a large batch of training data. If the data contain semantically related object groupings or classes, subsets of vectors belonging to such user-defined classes can be mapped on the SOM by finding the best matching unit for each vector in the set. The distribution of the data vectors over the map forms a two-dimensional discrete probability density. Even from the same data, qualitatively different distributions can be obtained by using different feature extraction techniques.

New entropy measures such as higher order fuzzy entropy and hybrid entropy of a set have been introduced. Various properties along with their proofs have been included. Applicability of these new measures to various problems has been highlighted.

To any discrete probability distribution P we can associate its entropy H(P ) =p i ln p i and its index of coincidence IC(P ) = p 2 i . The main result of the paper is the determination of the precise range of the map P (IC(P ), H(P )). The range looks much like that of the map P (P max , H(P )) where P max is the maximal point probability, cf. research from 1965 (Kovalevskij [18]) to 1994 (Feder and Merhav [7]). The earlier results, which actually focus on the probability of error 1 -P max rather than P max , can be conceived as limiting cases of results obtained by methods here presented. Ranges of maps as those indicated are called Information Diagrams. The main result gives rise to precise lower as well as upper bounds for the entropy function. Some of these bounds are essential for the exact solution of certain problems of universal coding and prediction for Bernoulli sources. Other applications concern Shannon theory (relations betweeen various measures of divergence), statistical decision theory and rate distortion theory. Two methods are developed. One is topological, another involves convex analysis and is based on a "lemma of replacement" which is of independent interest in relation to problems of optimization of mixed type (concave/convex optimization).

The main topic of this paper is a description of the generation of entropy at the end of the inflationary era. As a generalization of the present standard model of the Universe dominated by pressureless dust and a Lorentz invariant vacuum energy (LIVE), we first present a flat Friedmann universe model, where the dust is replaced with an ideal gas. It is shown that the pressure of the gas is inversely proportional to the fifth power of the scale factor and that the entropy in a comoving volume does not change during the expansion. We then review different measures of gravitational entropy related to the Weyl curvature conjecture and calculate the time evolution of two proposed measures of gravitational entropy in a LIVE-dominated Bianchi type I universe, and a Lemaitre-Bondi-Tolman universe with LIVE. Finally, we elaborate upon a model of energy transition from vacuum energy to radiation energy, that of Bonanno and Reuter, and calculate the time evolution of the entropies of vacuum energy and radiation energy. We also calculate the evolution of the maximal entropy according to some recipes and demonstrate how a gap between the maximal entropy and the actual entropy opens up at the end of the inflationary era.

We present a new approach for online handwritten signature classification and verification based on descriptors stemming from Information Theory. The proposal uses the Shannon Entropy, the Statistical Complexity, and the Fisher Information evaluated over the Bandt and Pompe symbolization of the horizontal and vertical coordinates of signatures. These six features are easy and fast to compute, and they are the input to an One-Class Support Vector Machine classifier. The results produced surpass state-of-the-art techniques that employ higher-dimensional feature spaces which often require specialized software and hardware. We assess the consistency of our proposal with respect to the size of the training sample, and we also use it to classify the signatures into meaningful groups.

RNA folding is a kinetic process governed by the competition of a large number of structures stabilized by the transient formation of base pairs that may induce complex folding pathways and the formation of misfolded structures. Despite of its importance in modern biophysics, the current understanding of RNA folding kinetics is limited by the complex interplay between the weak base-pair interactions that stabilize the native structure and the disordering effect of thermal forces. The possibility of mechanically pulling individual molecules offers a new perspective to understand the folding of nucleic acids. Here we investigate the folding and misfolding mechanism in RNA secondary structures pulled by mechanical forces. We introduce a model based on the identification of the minimal set of structures that reproduce the patterns of force-extension curves obtained in single molecule experiments. The model requires only two fitting parameters: the attempt frequency at the level of individual base pairs and a parameter associated to a free energy correction that accounts for the configurational entropy of an exponentially large number of neglected secondary structures. We apply the model to interpret results recently obtained in pulling experiments in the three-helix junction S15 RNA molecule (RNAS15). We show that RNAS15 undergoes force-induced misfolding where force favors the formation of a stable non-native hairpin. The model reproduces the pattern of unfolding and refolding force-extension curves, the distribution of breakage forces and the misfolding probability obtained in the experiments.

We determine all signals giving equality for the discrete Hirschman uncertainty principle. We single out the case where the entropies of the time signal and its Fourier transform are equal. These signals (up to scalar multiples) form an orthonormal basis giving an orthogonal transform that optimally packs a finite-duration discrete-time signal. The transform may be computed via a fast algorithm due to its relationship to the discrete Fourier transform.

Information-theoretic notion of entropy (in Shannon sense) is a versatile avenue in analyzing bioinformatic details as well as in mining data pertinent to molecular biology. Also, the informatics of bio-and medical-imaging can be comprehended via entropy considerations. Presented in this paper is a comprehensive review on salient methods, newer techniques and open-questions on unexplored efforts vis-à-vis the entropy principles cast on bioinformatic computations and bio-/medicalimage informatics. Supplementing energetics algorithms are indicated

A previously published two-dimensional discrete particle simulation model for radial mixing behavior of various slowly rotating drums for solid-state fermentation (SSF) has been extended to a three-dimensional model that also predicts axial mixing. Radial and axial mixing characteristics were predicted for three different drum designs: (1) without baf¯es; (2) with straight baf¯es; and (3) with curved baf¯es. The axial mixing behavior was studied experimentally with videoand image-analysis techniques. In the drum without baf¯es and with curved baf¯es the predicted mixing behavior matched the observed behavior adequately. The predicted axial mixing behavior in the drum with straight baf¯es was predicted less accurately, and it appeared to be strongly dependent on particle rotation, which was in contrast to the other drum designs. In the drum with curved baf¯es complete mixing in the radial and axial direction was achieved much faster than in the other designs; that is, it was already achieved after three to four rotations. This drum design may therefore be very well suited to SSF. It is concluded that discrete particle simulations provide valuable detailed knowledge about particle transport processes, and this may help to understand and optimize related heat and mass transfer processes in SSF.

We have developed a new software tool, LENSVIEW, for modelling resolved gravitational lens images. Based on the LENSMEM algorithm, the software finds the best fitting lens mass model and source brightness distribution using a maximum entropy constraint. The method can be used with any point spread function or lens model. We review the algorithm and introduce some significant improvements. We also investigate and discuss issues associated with the statistical uncertainties of models and model parameters and the issues of source plane size and source pixel size.

Insider attacks have the potential to inflict severe damage to an organizations reputation, intellectual property and financial assets. The primary difference between the external intrusions and the insider intrusions is that an insider wields power of knowledge about the information system resources, their environment, policies. We present an approach to detecting abnormal behavior of an insider by applying Dynamical System Theory to the insiders computer usage pattern. This is because abnormal system usage pattern is one of the necessary precursors to actual execution of an attack. A base profile of system usage pattern for an insider is created via applying dynamical system theory measures. A continuous monitoring of the insiders system usage and its comparison with this base profile is performed to identify considerable deviations. A sample system usage in terms of application system calls is collected, analyzed, and graphical results of the analysis are presented. Our results indicate that dynamical system theory has the potential of detecting suspicious insider behavior occurring prior to the actual attack execution.

The problem of maximizing the performance of the detection of ischemia episodes is a difficult pattern classification problem. The state space for this problem consists of regions that lie near class separation boundaries and require the construction of complex discriminants while for the rest regions the classification task is significantly simpler. The motivation for developing the supervising network self-organizing map (sNet-SOM) model is to exploit this fact for designing computationally effective solutions both for the particular ischemic detection problem and for other applications that share similar characteristics. Specifically, the sNet-SOM utilizes unsupervised learning for the "simple" regions and supervised for the "difficult" ones in a two stage learning process. The unsupervised learning approach extends and adapts the self-organizing map (SOM) algorithm of Kohonen. The basic SOM is modified with a dynamic expansion process controlled with an entropy based criterion that allows the adaptive formation of the proper SOM structure. This extension proceeds until the total number of training patterns that are mapped to neurons with high entropy (and therefore with ambiguous classification) reduces to a size manageable numerically with a capable supervised model. The second learning phase (the supervised training) has the objective of constructing better decision boundaries at the ambiguous regions. At this phase, a special supervised network is trained for the computationally reduced task of performing the classification at the ambiguous regions only. The utilization of sNet-SOM with supervised learning based on the radial basis functions and support vector machines has resulted in an improved accuracy of ischemia detection especially in the last case. The highly disciplined design of the generalization performance of the support vector machine allows designing the proper model for the number of patterns transferred to the supervised expert.

We present an exact Bayesian treatment of a simple, yet sufficiently general probability distribution model. We consider piecewise-constant distributions ( ) with uniform (second-order) prior over location of discontinuity points and assigned chances. The predictive distribution and the model complexity can be determined completely from the data in a computational time that is linear in the number of degrees of freedom and quadratic in the number of possible values of . Furthermore, exact values of the expectations of entropies and their variances can be computed with polynomial effort. The expectation of the mutual information becomes thus available, too, and a strict upper bound on its variance. The resulting algorithm is particularly useful in experimental research areas where the number of available samples is severely limited (e.g., neurophysiology). Estimates on a simulated data set provide more accurate results than using a previously proposed method.