Arguments against Hidden Variables in Quantum Systems

Arguments Against the Existence of Hidden Variables in Quantum Systems K. Lee Lerner scholar.harvard.edu/kleelerner [email protected] This is a DRAFT COPY of an article subsequently published in the RUSA-award-winning Science and Its Times: Understanding the Social Significance of Scientific Discovery, edited by Neil Schlager and Josh Lauer, and published in eight volumes by Thomson Gale (now Cengage Gale) from 1999 to 2001. In 2018 Amazon added Science and Its Times to its "Best of History Books" collection. The standard model of quantum physics offers a theoretically and mathematically sound model of particle behavior that serves as an empirically validated middle-ground between the need for undiscovered hidden variables that determine particle behavior, and a mystical anthropocentric universe where it is the observations of humans that determine reality. Although the implications of the latter can be easily dismissed as New Age-like metaphysical nonsense, the debate over the existence of hidden variables in quantum theory remained a subject of serious scientific debate during the 20th century. Based upon our everyday experience, well explained by the deterministic concepts of classical physics, it is intuitive that there be hidden variables to determine quantum states. Nature is not, however, obliged to act in accord with what is convenient or easy to understand. Although the existence and understanding of heretofore hidden variables might seemingly explain Albert Einstein’s “spooky” forces, the existence of such variables would simply provide the need to determine whether they, too, included their own hidden variables. Quantum theory breaks this never-ending chain of causality by asserting (with substantial empirical evidence) that there are no hidden variables. Moreover, quantum theory replaces the need for a deterministic evaluation of natural phenomena with an understanding of particles and particle behavior based upon statistical probabilities. Although some philosophers and metaphysicists would like to keep the hidden variable argument alive, the experimental evidence is persuasive, compelling, and conclusive that such hidden variables do not exist. The classic 1935 paper written by Einstein, Boris Podolsky, and Nathan Rosen (EPR) and titled, "Can Quantum-Mechanical Description of Physical Reality Be Considered Complete?" presented a gedanken-experimente (German for "thought experiment") that seemingly mandates hidden variables. What eventually became known as the EPR paradox struck at the ability of particles to remain correlated in entangled states even though those particles might be separated by a great distance. The quantum wave represents all states and all potentialities. The properties of matter can be described in terms of waves and particles. De Broglie waves describe the wave properties of matter related to momentum. Waves can also be described as a function of probability density. The differential equation for quantum waves is the Schrödinger equation (also termed the quantum wave function) that treats time, energy, and position. In quantum theory, not all possible states, attributes such as position, velocity, spin, etc., of matter, have equal probabilities. Although states are undetermined until measured, some are more likely than others. Quantum theory allows predictions of states based upon the probabilities represented in the quantum wave function. Quantum entanglement is a concept of quantum theory that relies on the superposition of possible states for particles. In a two-particles entangled system, the act of measuring one of the entangled particles causes that particle’s quantum wave to collapse to a definite state (e.g., a defined velocity or position). With regard to superposition, if one of two particles with opposite spins, that in a combined state would have zero spin, is measured and determined to be spinning in a particular direction, the spin of the other particle must be equal in magnitude but in the opposite direction. Superposition allows a particle to exist in all possible states of spin simultaneously and the spin of a particle is not determined until measured. Simultaneous with the collapse of the first particle’s wave state, the quantum wave of the second particle also collapses to a definite state. Such correlations must be instantaneous, and EPR argued that if there were any distance between the particles, any force acting between the particles would have to exceed the speed of light. Einstein termed these forces as “spooky actions at a distance.” EPR specifically identified three main problems with the standard interpretations of quantum mechanics that did not allow for the existence of hidden variables. Because of the limitations of special relativity, EPR argued that there could be no transacting force that instantaneously determines the state of the second particle in a two-particle system where the particles were separated and moving in opposite directions. EPR also challenged the uncertainty limitations found in quantum systems wherein the measurement of one state (e.g., velocity) makes impossible the exact determination of a second state (e.g., position). Most importantly, the EPR paper challenged the quantum view of nature as, at the quantum level, a universe explained only by probability rather than classical deterministic predictability where known causes produce known results. Einstein, in particular, objected to the inherent fundamental randomness of quantum theory (explaining his often quoted, “God does not play dice!”) challenge to Neils Bohr and other quantum theorists) and argued that for all it empirical usefulness in predicting line spectra and other physical phenomena, quantum theory was incomplete and that the discovery of hidden variables would eventually force modifications to the theory that would bring it into accord with relativity theory (especially concerning the absolute limitation of the speed of light). Quantum theory, highly dependent on mathematical descriptions, depicts the wave nature of matter with a wave function (quantum waves). The wave function is used to calculate probabilities associated with finding a particle in a given state (e.g., position or velocity). When an observer interacts with a particle by attempting to measure a particular state, the wave particle collapses, and the particle takes on a determinable state that can be measured with a high degree of accuracy. If a fundamental particle such as an electron is depicted as a quantum wave, then it has a certain probability of being at any two points at the same time. If, however, an observer attempts to determine the location of the particles and determines it to be at a certain point, then the wave function has collapsed in that the probability of finding the electron at any other location is, in this measured state, zero. The EPR paradox seemingly demands that for the wave function to collapse at the second point, some signal must be, in violation of special relativity, instantaneously transmitted from the point of measurement (i.e., the point of interaction between the observer and the particle) to the any other point, no matter how far away that point may be, so that at that point the wave function collapses to zero. David Bohm’s subsequent support of EPR through a reconciliation of quantum theory with relativity theory was based upon the existence local hidden variables. Bohm’s hypothesis, however, suffering from a lack of empirical validation, smoldered on the back burners of theoretical physics until John Bell’s inequalities provided a mechanism to empirically test the hidden variable hypothesis versus the stand interpretation of quantum mechanics. Bell’s theorem (a set of inequalities) and work dispelled the idea that there are undiscovered hidden variables in quantum theory that determine particle states. Bell’s inequalities, verified by subsequent studies of photon behavior, predicted testable differences between entangled photon pairs that were in superposition and entangled photons whose subsequent states were determined by local hidden variables. Most importantly, Bell provided a very specific mechanism, based upon the polarization of photons, to test Bohm’s local hidden variable hypothesis. Polarized photons are created by passing photons through optical filters or prisms that allow the transmission of light polarized in one direction (a particular orientation of the planes of the perpendicular electromagnetic wave) while blocking differently oriented photons. Most useful to tests of the EPR assertions are polarized photons produced by atomic cascades. Such photons are produced as electron decay from higher energy orbitals toward their ground state via a series of quantum jumps form one allowable orbital level to another. The law of the conservation of energy dictates that as electrons instantaneously transition from one orbital level to another, they must give off a photon of light with exactly the same amount of energy as the difference between the two orbitals. An electron moving to toward the ground state that makes that transition through two discreet orbital jumps (e.g., from the 4th orbital to the third and then from the 3rd to the first) must produce two photons with energy (frequency and wavelength differences) directly related to the differences in potential energy of the various orbitals). Of particular interest to EPR studies, however, is the fact that in cascades where there is no net rotational motion, the photons produced are quantum-entangled photons with regard to the fact that they must have specifically correlated polarizations. If the polarization of one photon can be determined, the other can be exactly known without any need for measurement. Although the details of the measurement process, based upon the angles of various filters and measurement of arrival times of polarized photon pairs taking different paths, are beyond the scope of this article, the most critical aspect is that the outcomes predicted by standard quantum theory are different than the outcomes predicted by if hidden variables exist. This difference in predicted outcomes makes it possible to test Bell’s inequalities and, in fact, a number experiments have been performed to exactly test for these differences. In every experiment to date, the results are consistent with the predictions made by the standard interpretation of quantum mechanics and inconsistent for the existence of any local hidden variables as proposed by Bohm. In 1982, the French physicist Alain Aspect, along with others, performed a series of experiments that demonstrated that between photons separated by short distances there was “action at a distance.” In 1997, Nicolas Gisin and colleagues at the University of Geneva extended the distances between entangled photons to a few kilometers. Measurements of particle states at the two laboratory sites showed that the photons adopted the correct state faster than light could have possibly traveled between the two laboratories. In modern physics, Einstein’s “spooky” actions underpin the concept of non-locality. Local, in this context means forces that operate within the photons. Although Bell’s inequality does not rule out the existence of non-local hidden variables that could act instantaneously over even great distances, such non-local hidden variables or forces would have a seemingly impossible theoretical and empirical barrier to surmount. If such non-local hidden variables exist, they must act or move faster than the speed of light and this, of course, would violate on of the fundamental assertions of special relativity. Just as quantum theory is well supported by empirical evidence, so too is relativity theory. Accordingly, for hidden variables to exist, both quantum and relativity theories would need to be rewritten. Granting that quantum and relativity theories are incompatible and that both may become components of a unified theory at some future date, this is certainly not tantamount to evidence for hidden variables. The only hope for hidden variable proponents is if the hidden variables can act nonlocally, or if particles have a way to predict their future state and make the needed transformations as appropriate. Such transactional interpretations of quantum theory use a reverse-causality argument to allow the existence of hidden variables that does not violate Bell’s inequality. Other “many worlds” interpretations transform the act of measurement into the selection of a physical reality among a myriad of possibilities. Not only is there no empirical evidence to support this hypothesis, but also it severely strains Ockham’s razor (the idea that given equal alternative explanations, the simpler is usually correct). In common, hidden variable proponents essentially argue that particles are of unknown rather than undefined state when in apparent superposition. Although the hidden variable, transactional, or “many worlds” interpretations of quantum theory would make the quantum world more understandable in terms of conventional experience and philosophical understanding, there is simply no experimental evidence that such an interpretations of quantum theory have any basis or validity. The mere possibility that any argument may be true does not in any way provide evidence that a particular argument is true. In contrast to the EPR paradox, it is a mistake to assume that quantum theory demands or postulates faster-than-light forces or signals (superluminal signals). Both quantum theory and relativity theory preclude the possibility of superluminal transmission, and to this extent, quantum theory is normalized with relativity theory. For example, the instantaneous transformation of electrons from one allowed orbital (energy state) to another are most properly understood in terms of wave collapse rather than through some fast-than-light travel. The proper mathematical interpretation of the wave collapse completely explains quantum leaps, without any need for fast-than-light forces or signal transmission. Instead of a physical form or independent reality, the waveform is best understood as the state of an observer’s knowledge about the state of a particle or system. Most importantly, although current quantum theory does not completely rule out the existence of hidden variables under every set of conceivable circumstances, the mere possibility that hidden variables might exist under such special circumstances is in no way proof that hidden variables do exist. There is simply no empirical evidence that such hidden variables exist. More importantly, quantum theory makes no claim to impart any form of knowing or consciousness on the behavior of particles. Although it is trendy to borrow selected concepts from quantum theory to prop up many New Age interpretations of nature, quantum theory does not provide for and mystical mechanisms. The fact that quantum theory make accurate depictions and predictions of particle behavior does not mean that the mathematical constructs of quantum theory depict the actual physical reality of the quantum wave. Simply put, there is no demand that the universe present us with easy-to-understand mechanisms of action. Further Reading: Bell, J., “On the Einstein Podolsky Rosen Paradox.” Physics. v1, n3, 194-201. Bohr, N., "Quantum Mechanics and Physical Reality." Nature. 136:1024-1026. 1935. Cushing, J. T., and E. Mc Mullin. Philosophical Consequences of Quantum Theory. University of Notre Dame Press, 1989. Einstein, E., Podolski, B., and Rosen, N. "Can Quantum Mechanical Description of Physical Reality be Considered Complete?" Physical Review 47:776-780. 1935. Heisenberg, W. (trans. Eckart, C, and Hoyt, F.C.) The Physical Principles of the Quantum Theory. New York: Dover. 1930. Popper, K. Quantum Theory and the Schism in Physics. Hutchinson: London, 1982. Schrödinger, E. "Discussion of Probability Relations Between Separated Systems" Proceedings of the Cambridge Philosophical Society. 31:555-562, 1935a. Von Neumann, J. (trans. Geyer, R.). Mathematical Foundations of Quantum Mechanics Princeton. Princeton University Press. 1955. _______________________ "Recognized for his use of language, accuracy, and balanced presentation, K. Lee Lerner's portfolio covering science and global issues has garnered respected writing, book and media awards. His dossier spans every continent, includes two global circumnavigations, and features coverage from areas suffering civil war, violent protests, drought, famine, and disease outbreaks. That experience, built on a scholarly foundation in science, allows his evidence-based writing to bring clarity to chaotic and complex issues. Contributing editor of more than 40 academic books and writer and/or producer for more than two dozen major media projects, for more than three decades — across print, broadcast media, and digital platforms -- Lerner's 'Taking Bearings,' essays have ranged across the human intellectual enterprise. He has served on the board of advisors for the venerable American Men and Women of Science since 2003 and his Academia site (https://harvard.academia.edu/kleelerner)consistently ranks among those most frequently accessed by students, scholars, and decision makers from around the world." — National Press Club biography. Additional information is available at scholar.harvard.edu/kleelerner CC BY-NC-ND otherwise ©LMG All Rights Reserved. Permission to use excerpts from this DRAFT COPY, with appropriate acknowledgments, is granted for academic use. Commercial use is strictly prohibited.