Foundations of Probability

In 1935 Einstein, Podolsky, and Rosen (EPR) published an important paper in which they claimed that the whole formalism of quantum mechanics together with what they called ``Reality Criterion'' imply that quantum mechanics cannot be complete. That is, there must exist some elements of reality that are not described by quantum mechanics. There must be, they concluded, a more complete description of physical reality behind quantum mechanics. There must be a state, a hidden variable, characterizing the state of affairs in the world in more details than the quantum mechanical state, something that also reflects the missing elements of reality. Under some further but quite plausible assumptions, this conclusion implies that in some spin-correlation experiments the measured quantum mechanical probabilities should satisfy particular inequalities (Bell-type inequalities). The paradox consists in the fact that quantum probabilities do not satisfy these inequalities. And this paradoxi...

At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the process of inference, which make it clear that these are inferential theories. That is, rather than being a description of the behavior of the universe, these theories describe how observers can make optimal predictions about the universe. In such a picture, information plays a critical role. What is more is that little clues, such as the fact that black holes have entropy, continue to suggest that information is fundamental to physics in general.  In the last decade, our fundamental understanding of probability theory has led to a Bayesian revolution. In addition, we have come to recognize that the foundations go far deeper and that Cox’s approach of generalizing a Boolean algebra to a probability calculus is the first specific example of the more fundamental idea of assigning valuations to partially ordered sets. By considering this as a natural way to introduce quantification to the more fundamental notion of ordering, one obtains an entirely new way of deriving physical laws. I will introduce this new way of thinking by demonstrating how one can quantify partially-ordered sets and, in the process, derive physical laws. The implication is that physical law does not reflect the order in the universe, instead it is derived from the order imposed by our description of the universe. Information physics, which is based on understanding the ways in which we both quantify and process information about the world around us, is a fundamentally new approach to science.

The logical interpretation of probability, or “objective Bayesianism” – the theory that (some) probabilities are strictly logical degrees of partial implication – is defended.
The main argument against it is that it requires the assignment of prior probabilities, and that any attempt to determine them by symmetry via a “principle of insufficient reason” inevitably leads to paradox. Three replies are advanced: that priors are imprecise or of little weight, so that disagreement about them does not matter, within limits; that it is possible to distinguish reasonable from unreasonable priors on logical grounds; and that in real cases disagreement about priors can usually be explained by differences in the background information. It is argued also that proponents of alternative conceptions of probability, such as frequentists, Bayesians and Popperians, are unable to avoid committing themselves to the basic principles of logical probability.

The last century saw the application of Boolean algebra to the construction of com- puting machines, which work by applying logical transformations to information contained in their memory. The development of information theory and the gen- eralization of Boolean algebra to Bayesian inference have enabled these computing machines, in the last quarter of the twentieth century, to be endowed with the abil- ity to learn by making inferences from data. This revolution is just beginning as new computational techniques continue to make di± cult problems more accessible. Recent advances in our understanding of the foundations of probability theory have revealed implications for areas other than logic. Of relevance to intelligent machines, we recently identied the algebra of questions as the free distributive algebra, which will now allow us to work with questions in a way analogous to that which Boolean algebra enables us to work with logical statements. In this paper, we examine the found...

In accordance with an idea in [8], in this paper we sketch a method to design expert systems, probabilistic in nature. Indeed, we assume that the probability that an individual satisfies a property is the percentage of similar individuals satisfying such a property. In turn, we call "similar" two individual satisfying the same observable properties. Such an approach is extended to the case of vague properties. We adopt a formalism very close to the one of formal concept analysis.

In his classic book Savage (1954, 1972) develops a formal system of rational decision making. It is based on (i) a set of possible states of the world, (ii) a set of consequences, (iii) a set of acts, which are functions from states to consequences, and (iv) a preference relation over the acts, which represents the preferences of an idealized rational agent. The goal and the culmination of the enterprise is a representation theorem: Any preference relation that satisfies certain arguably acceptable postulates determines a (finitely additive) probability distribution over the states and a utility assignment to the consequences, such that the preferences among acts are determined by their expected utilities. Additional problematic assumptions are however required in Savage's proofs. First, there is a Boolean algebra of events (sets of states) which determines the richness of the set of acts. The probabilities are assigned to members of this algebra. Savage's proof requires that this be a σ-algebra (i.e., closed under infinite countable unions and intersections), which makes for an extremely rich preference relation. On Savage's view we should not require subjective probabilities to be σ-additive. He therefore finds the insistence on a σ-algebra peculiar and is unhappy with it. But he sees no way of avoiding it. Second, the assignment of utilities requires the constant act assumption: for every consequence there is a constant act, which produces that consequence in every state. This assumption is known to be highly counterintuitive. The present work contains two mathematical results. The first, and the more difficult one, shows that the σ-algebra assumption can be dropped. The second states that, as long as utilities are assigned to finite gambles only, the constant act assumption can be replaced by the more plausible and much weaker assumption that there are at least two non-equivalent constant acts. The second result also employs a novel way of deriving utilities in Savage-style systems – without appealing to von Neumann-Morgenstern lotteries. The paper discusses the notion of " idealized agent " that underlies Savage's approach, and argues that the simplified system, which is adequate for all the actual purposes for which the system is designed, involves a more realistic notion of an idealized agent.

This article offers a new argument for the claim that there can be nontrivial objective chance in a deterministic world. Using a formal model of the relationship between different levels of description of a system, the article shows how objective chance at a higher level can coexist with its absence at a lower level. Unlike previous arguments for the level-specificity of chance, the present argument shows, in a precise sense, that higher-level chance does not collapse into epistemic probability, despite higher-level properties supervening on lower-level ones. The article demonstrates that the distinction between objective chance and epistemic probability can be drawn, and operationalized, at every level of description. There is, therefore, not a single distinction between objective and epistemic probability but a family of such distinctions.
Preprint: http://eprints.lse.ac.uk/61063/

We present a simple and clear foundation for finite inference that unites and significantly extends the approaches of Kolmogorov and Cox. Our approach is based on quantifying lattices of logical statements in a way that satisfies general lattice symmetries. With other applications such as measure theory in mind, our derivations assume minimal symmetries, relying on neither negation nor continuity nor differentiability. Each relevant symmetry corresponds to an axiom of quantification, and these axioms are used to derive a unique set of quantifying rules that form the familiar probability calculus. We also derive a unique quantification of divergence, entropy and information.

Non-bivalent languages (languages containing sentences that can
be true, false or neither) are given a probabilitistic interpretation in terms of betting quotients. Necessary and sufficient conditions for avoiding Dutch books—the laws of non-bivalent probability—in such a setting are provided.

We solve two fundamental problems of probabilistic reasoning: given finitely many conditional probability assessments, how to determine whether the assessments are mutually consistent, and how to determine what they imply about the conditional probabilities of other events? These problems were posed in 1854 by George Boole, who gave a partial solution using algebraic methods. The two problems are fundamental in applications of the Bayesian theory of probability; Bruno de Finetti solved the second problem for the special case of unconditional probability assessments in what he called ‘the fundamental theorem of probability’. We give examples to show that previous attempts to solve the two problems, using probabilistic logic and similar methods, can produce incorrect answers. Using ideas from the theory of imprecise probability, we show that the general problems have simple, direct solutions which can be implemented using linear programming algorithms. Unlike earlier proposals, our methods are formulated directly in terms of the assessments, without introducing unknown probabilities. Our methods work when any of the conditioning events may have probability zero, and they work when the assessments include imprecise (upper and lower) probabilities or previsions. The main methodological contribution of the paper is to provide general algorithms for making inferences from any finite collection of (possibly imprecise) conditional probabilities.

The effect of Richard T. Cox's contribution to probability theory was to generalize Boolean implication among logical statements to degrees of implication, which are manipulated using rules derived from consistency with Boolean algebra. These rules are known as the sum rule, the product rule and Bayes' Theorem, and the measure resulting from this generalization is probability. In this paper, I will describe how Cox's technique can be further generalized to include other algebras and hence other problems in science and mathematics. The result is a methodology that can be used to generalize an algebra to a calculus by relying on consistency with order theory to derive the laws of the calculus. My goals are to clear up the mysteries as to why the same basic structure found in probability theory appears in other contexts, to better understand the foundations of probability theory, and to extend these ideas to other areas by developing new mathematics and new physics. The relevance of this methodology will be demonstrated using examples from probability theory, number theory, geometry, information theory, and quantum mechanics. 'set intersection' for 'logical conjunction', 'set union' for 'logical disjunction', 'set complementation' for 'logical complementation', and 'is a subset of' for 'implies' and you have the same algebra. We exchanged quite a few things above and its useful to break them down further. We exchanged the objects we are studying: 'sets' for 'logical statements'. Then we exchanged the operations we can use to combine them, such as 'set intersection' for 'logical conjunction'. Finally, we exchanged the means by which we order these objects: 'is a subset of' for 'implies'. The obvious implication of this is that Cox's results hold equally well for defining measures on sets. That is we can assign real numbers to describe the degree to which one set is a subset of another set. The algebra allows us to have a sum rule, a product rule, and a Bayes' Theorem analog-just like in probability theory! It has been recognized for some time that there exist relationships between other theories and probability theory, but the underlying reasons for these relationships have not been well understood. The most obvious example is quantum mechanics, which has much in common with probability theory. However, it is clearly not probability theory since quantum amplitudes are complex numbers rather than real numbers. Another example is the analogy recognized by Carlos Rodríguez between the cross-ratio in projective geometry and Bayes' Theorem [5]. In this paper, I will describe how Cox's technique can be further generalized to include other algebras and hence other problems in science and mathematics. I hope to clear up some mysteries as to why the same basic structure found in probability theory appears in other contexts. I expect that these ideas can be taken much further, and my hope is that they will eventually lead to new mathematics and new physics. Last, scattered throughout this paper are many observations and connections that I hope will help lead us to different ways of thinking about these topics. LATTICES AND ALGEBRAS Basic Ideas The first step is to generalize the basic idea behind the elements described in our Boolean algebra example: objects, operations, and ordering relations. We start with a set of objects, and we select a way to compare two objects by deciding whether one object is 'greater than' another object. This means of comparison allows us to order the objects. A set of elements together with a binary ordering relation is called a partially ordered set, or a poset. It is called a partial order to allow for the possibility that some elements in the set cannot be directly compared. In our example with logical implication, the ordering relation was 'implies', so that if 'a implies b', written a → b, then b is in some sense 'greater than' a, or equivalently, a is in some sense 'included in' b. An ordering relation is generally written as a ≤ b, and read as 'b includes a' or 'a is contained in b'. When a ≤ b, but a = b then we write a < b, and read it as 'a is properly contained in b'. Last, if a < b, but there does not exist any element x in the set P such that a < x < b, then we say that 'b covers a', and write a ≺ b. In this case, b is an immediate successor to a in the hierarchy imposed by the ordering relation. It should be stressed that this notation is

These notes comment on Williams' fundamental essay Notes on Conditional Previsions, written as a research report in 1975 and published in the present issue. Basic aspects of that work are discussed, including historical background and relevance to the foundations of probability; examples are supplied to help understanding.

Both Popper and Good have noted that a deterministic microscopic physical approach to probability requires subjective assumptions about the statistical distribution of initial conditions. However, they did not use such a fact for defining an a priori probability, but rather recurred to the standard observation of repetitive events. This observational probability may be hard to assess for real-life decision problems under uncertainty that very often are - strictly speaking - non-repetitive, one-time events. This may be a reason for the popularity of subjective probability in decision models. Unfortunately, such subjective probabilities often merely reflect attitudes towards risk, and not the underlying physical processes. In order to get ‘as objective as possible’ a definition of probability for one-time events, this paper identifies the origin of randomness in individual chance processes. By focusing on the dynamics of the process, rather than on the (static) device, it is found that any process contains two components: observer-independent (= ‘objective’) and observer-dependent (= ‘subjective’). Randomness, if present, arises from the subjective definition of the ‘rules of the game’, and is not - as in Popper's propensity - a physical property of the chance device. In this way, the classical definition of probability is no longer a primitive notion based upon equally possible cases, but is derived from the underlying microscopic processes, plus a subjective, clearly identified, estimate of the branching ratios in an event tree. That is, equipossibility is not an intrinsic property of the system object/subject but is forced upon the system via the rules of the game/measurement. Also, the typically undefined concept of symmetry in games of chance is broken down into objective and subjective components. It is found that macroscopic symmetry may hold under microscopic asymmetry. A similar analysis of urn drawings shows no conceptual difference with other games of chance (contrary to Allais' opinion). Finally, the randomness in Lande's knife problem is not due to ‘objective fortuity’ (as in Popper's view) but to the rules of the game (the theoretical difficulties arise from intermingling microscopic trajectories and macroscopic events).

Reasoning within such domains as engineering, science, management, or medicine is traditionally based on formal methods employing probabilistic treatment of uncertainty. It seems natural to base articial reasoning systems in these domains on the normative foundations of probability theory. Two usual objections to this approach are (1) probabilistic inference is computationally intractable in the worst case, and (2) probability theory is incomprehensible for humans and, hence, probabilistic systems may be hardly usable. The rst objection has been addressed eectively in the last decade by a variety of ecient exact and approximate schemes for probabilistic reasoning, applied in several practical systems. In this paper, I review the state of the art with respect to the second objection. First I argue that the observed discrepancies between human and probabilistic reasoning and the anticipated diculties in building user interfaces are not a good reason for rejecting probability theory. On the contrary | they provide motivation for a normative treatment of uncertainty. I point out that probability theory rests on qualitative foundations that capture essential properties of a domain along with such concepts such as relevance and conicting evidence. In addition, graphical probabilistic models, as opposed to rule-based systems, integrate numerical and structural properties of a domain and provide a natural representation of causality. Finally, availability of a full quantitative specication of a model allows for manipulating the level of precision for both reasoning and explanation.

The problem of inferring probability comparisons between events from an initial set of comparisons arises in several contexts, ranging from decision theory to artificial intelligence to formal semantics. In this paper, we treat the problem as follows: beginning with a binary relation ≥ on events that does not preclude a probabilistic interpretation, in the sense that ≥ has extensions that are probabilistically representable, we characterize the extension ≥+ of ≥ that is exactly the intersection of all probabilistically representable extensions of ≥. This extension ≥+ gives us all the additional comparisons that we are entitled to infer from ≥, based on the assumption that there is some probability measure of which ≥ gives us partial qualitative information. We pay special attention to the problem of extending an order on states to an order on events. In addition to the probabilistic interpretation, this problem has a more general interpretation involving measurement of any additive quantity: e.g., given comparisons between the weights of individual objects, what comparisons between the weights of groups of objects can we infer?

We continue our studies of the foundation of probability theory using elementary category theory. We propose a classification scheme of probability domains in terms of cogenerators and their algebraic and topological properties and use the scheme to describe the transition from classical to fuzzy probability. We show that Łukasiewicz tribes form a category of natural probability domains in which σ -fields of sets are "minimal" and measurable [0, 1]-valued functions are "maximal" objects. The maximal objects form an epireflective subcategory in which both the classical and fuzzy probability can be modelled. This leads to a better understanding of the transition.

I describe a realist, ontologically objective interpretation of probability, "far-flung frequency (FFF) mechanistic probability". FFF mechanistic probability is defined in terms of facts about the causal structure of devices and certain sets of frequencies in the actual world. Though defined partly in terms of frequencies, FFF mechanistic probability avoids many drawbacks of well-known frequency theories and helps causally explain stable frequencies, which will usually be close to the values of mechanistic probabilities. I also argue that it's a virtue rather than a failing of FFF mechanistic probability that it does not define single-case chances, and compare some aspects of my interpretation to a recent interpretation proposed by Strevens.

We present a simple and clear foundation for finite inference that unites and significantly extends the approaches of Kolmogorov and Cox. Our approach is based on quantifying lattices of logical statements in a way that satisfies general lattice symmetries. With other applications such as measure theory in mind, our derivations assume minimal symmetries, relying on neither negation nor continuity nor differentiability. Each relevant symmetry corresponds to an axiom of quantification, and these axioms are used to derive a unique set of quantifying rules that form the familiar probability calculus. We also derive a unique quantification of divergence, entropy and information.

At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the process of inference, which make it clear that these are inferential theories. That is, rather than being a description of the behavior of the universe, these theories describe how observers can make optimal predictions about the universe. In such a picture, information plays a critical role. What is more is that little clues, such as the fact that black holes have entropy, continue to suggest that information is fundamental to physics in general. In the last decade, our fundamental understanding of probability theory has led to a Bayesian revolution. In addition, we have come to recognize that the foundations go far deeper and that Cox's approach of generalizing a Boolean algebra to a probability calculus is the first specific example of the more fundamental idea of assigning valuations to partially-ordered sets. By considering this as a natural way to introduce quantification to the more fundamental notion of ordering, one obtains an entirely new way of deriving physical laws. I will introduce this new way of thinking by demonstrating how one can quantify partially-ordered sets and, in the process, derive physical laws. The implication is that physical law does not reflect the order in the universe, instead it is derived from the order imposed by our description of the universe. Information physics, which is based on understanding the ways in which we both quantify and process information about the world around us, is a fundamentally new approach to science.

In 1935 Einstein, Podolsky, and Rosen (EPR) published an important paper [17] in which they claimed that the whole formalism of quantum mechanics together with what they called “Reality Criterion” imply that quantum me-chanics cannot be complete. That is, there must exist ...

This conference was devoted to fundamental questions raised by quantum mechanics, especially in quantum information theory. As has become customary in our series of conference in Vxj, we were glad to welcome a fruitful assembly of theoretical physicists, experimentalists, mathematicians and even philosophers interested in the foundations of probability and physics. This conference belongs to the series of Vxj conferences in foundations of quantum mechanics (especially probabilistic foundations) ...

The effect of Richard T. Cox’s contribution to probability theory was to generalize Boolean implication among logical statements to degrees of implication, which are manipulated using rules derived from consistency with Boolean algebra. These rules are known as the sum rule, the product rule and Bayes’ Theorem, and the measure resulting from this generalization is probability. In this paper, I will describe how Cox’s technique can be further generalized to include other algebras and hence other problems in science and mathematics. The result is a methodology that can be used to generalize an algebra to a calculus by relying on consistency with order theory
to derive the laws of the calculus. My goals are to clear up the mysteries as to why the same basic structure found in probability theory appears in other contexts, to better understand the foundations of probability theory, and to extend these ideas to other areas by developing new mathematics and new physics. The relevance of this methodology will be demonstrated using examples from
probability theory, number theory, geometry, information theory, and quantum mechanics.

This paper applies ideas introduced in my "Mechanistic Probability" to probabilities in the social sciences. Among other things, I argue that sensitive dependence of individual outcomes on vagaries of individual circumstances are the rule rather than the exception. I argue that even though such idiosyncratic sequences of events appear peripheral to much work in the social sciences, their commonality is in fact what allows that work to proceed as it does. More specifically, sensitive dependence of individual outcomes on circumstances helps produce enough stability in social phenomena that they can be studied at all, allowing claims about probabilities of social phenomena to succeed, and making sense of policy interventions.