Few of Albert Einstein’s sayings have been as widely quoted as his remark that God does not play dice with the universe. People have naturally taken his quip as proof that he was dogmatically opposed to quantum mechanics, which views randomness as a built-in feature of the physical world. When a radioactive nucleus decays, it does so spontaneously; no rule will tell you when or why. When a particle of light strikes a half-silvered mirror, it either reflects off it or passes through; the outcome is open until the moment it occurs. You do not need to visit a laboratory to see these processes: lots of Web sites display streams of random digits generated by Geiger counters or quantum optics. Being unpredictable even in principle, such numbers are ideal for cryptography, statistics and online poker.

Einstein, so the standard tale goes, refused to accept that some things are indeterministic—that they just happen, and there is not a darned thing anyone can do to figure out why. Almost alone among his peers, he clung to the clockwork universe of classical physics, ticking mechanistically, each moment dictating the next. The dice-playing line became emblematic of the B side of his life: the tragedy of a revolutionary turned reactionary who upended physics with relativity theory but was, as Niels Bohr put it, “out to lunch” on quantum theory.

Over the years, though, many historians, philosophers and physicists have challenged this story line. Diving into what Einstein actually said, they have found that his thinking about indeterminism was far more radical and nuanced than is commonly portrayed. “It becomes a kind of a mission to get the story right,” says Don A. Howard, a historian at the University of Notre Dame. “It’s amazing when you dig into the archives and see the disparity from the common narrative.” As he and others have shown, Einstein accepted that quantum mechanics was indeterministic—as well he might, because he was the one who had discovered its indeterminism. What he did not accept was that this indeterminism was fundamental to nature. It gave every indication of arising from a deeper level of reality that the theory was failing to capture. His critique was not mystical but focused on specific scientific problems that remain unsolved to this day.

The question of whether the universe is a clockwork or a craps table strikes at the heart of what we suppose physics to be: a search for simple rules that underlie the wondrous diversity of nature. If some things happen for no reason, they mark the limits of rational inquiry. “Fundamental indeterminism would mean an end to science,” worries Andrew S. Friedman, a cosmologist at the Massachusetts Institute of Technology. And yet philosophers throughout history have supposed that indeterminism is a prerequisite for human free will. Either we are all gears in the clockwork, so that everything we do is preordained, or we are the agents of our own destiny, in which case the universe must not be deterministic after all. This dichotomy has had very real consequences for how society holds people responsible for their actions. Assumptions about free will suffuse our legal system; to be culpable, an offender must have acted with intent. The courts continually wrestle with whether people are innocent by reason of insanity, adolescent impulsiveness or detrimental social background.

Whenever people talk about a dichotomy, though, they usually aim to expose it as false. Indeed, many philosophers think it is meaningless to say whether the universe is deterministic or indeterministic. It can be either, depending on how big or complex your object of study is: particles, atoms, molecules, cells, organisms, minds, communities. “The distinction between determinism and indeterminism is a level-specific distinction,” says Christian List, a philosopher at the London School of Economics and Political Science. “If you have determinism at one particular level, it is fully compatible with indeterminism, both at higher levels and at lower levels.” The atoms in our brain can behave in a completely deterministic way while still giving us freedom of action because atoms and agency operate on different levels. Likewise, Einstein sought a deterministic subquantum level without denying that the quantum level was probabilistic.

What Einstein Objected To

How Einstein ever got tagged as antiquantum is almost as big a mystery as quantum mechanics itself. The very notion of quanta—of discrete units of energy—was his brainchild in 1905, and for a decade and a half he stood practically alone in its defense. Einstein came up with most of what physicists now recognize as the essential features of quantum physics, such as light’s peculiar ability to act as both particle and wave, and it was his thinking about wave physics that Erwin Schrödinger built on to develop the most widely used formulation of quantum theory in the 1920s. Nor was Einstein against randomness. In 1916 he showed that when atoms emit photons, the timing and direction of emission are random. “This goes against the popular image of Einstein as an adversary to probability,” says philosopher Jan von Plato of the University of Helsinki.

But Einstein and his contemporaries faced a serious problem. Quantum phenomena are random, but quantum theory is not. The Schrödinger equation is 100 percent deterministic. It describes a particle or system of particles using a so-called wave function, which expresses particles’ wave nature and accounts for the undulating patterns that collections of particles can form. The equation predicts what happens to the wave function at every moment with complete certainty. In many ways, the equation is more deterministic than Newton’s laws of motion: it does not lead to muddles such as singularities (where quantities become infinite and thus indescribable) or chaos (where motion becomes unpredictable).

The tricky part is that the determinism of the Schrödinger equation is the determinism of the wave function, and the wave function is not directly observable, as the positions and velocities of particles are. Instead the wave function specifies the quantities that can be observed and the likelihood of each eventuality. The theory leaves open what exactly the wave function is and whether it should be taken literally as a real wave out there in the world. Thus, it also leaves open whether observed randomness is intrinsic to nature or just a facade. “People say that quantum mechanics is indeterministic, but that’s too quick,” says philosopher Christian Wüthrich of the University of Geneva in Switzerland.

Werner Heisenberg, another early pioneer of quantum theory, envisioned the wave function as a haze of potential existence. If it fails to pinpoint unequivocally where a particle is located, that is because the particle is not, in fact, located anywhere. Only when you observe the particle does it materialize somewhere. The wave function might have been spread out over a huge region of space, but at the instant the observation is made, it abruptly collapses to a narrow spike at a single position, and the particle pops up there. When you so much as look at a particle—bam!—it stops behaving deterministically and leaps to an end result like a kid grabbing a seat in musical chairs. No law governs collapse. There is no equation for it. It just happens.

Collapse became a core ingredient of the Copenhagen interpretation, the view of quantum mechanics named for the city where Bohr had his institute and Heisenberg did much of his early work. (Ironically, Bohr himself never accepted wave function collapse.) Copenhagen takes the observed randomness of quantum physics at face value, with no possibility of further explanation. Most physicists accepted it, if only because of a psychological anchoring effect: it was a good enough story, and it was the first.

Although Einstein was not antiquantum, he was definitely anti–Copenhagen interpretation. He recoiled from the idea that the act of measurement should cause a break in the continuous evolution of a physical system, and that was the context in which he began to complain about divine dice rolling. “It’s that, specifically, that Einstein is lamenting in 1926 and not a blanket metaphysical assertion of determinism as an absolutely necessary condition,” Howard says. “He’s specifically in the thick of these arguments about whether or not wave function collapse introduces discontinuities.”

Collapse could not be a real process, Einstein reasoned. It would require instantaneous action at a distance—a mysterious mechanism ensuring that, say, the left side and right side of a wave function both collapse to the same narrow spike even when no force is coordinating them. Not just Einstein but every physicist of his day thought such a process impossible; it would operate faster than light, in apparent violation of relativity theory. In effect, quantum mechanics does not just give you dice to play with. It gives you pairs of dice that always come up doubles, even if you roll one in Vegas and the other on Vega. For Einstein, it seemed obvious that the dice must be loaded—possessing hidden attributes that fix their outcome in advance. But Copenhagen denied any such thing, implying the dice really do affect each other instantly across the vastness of space.

Einstein was further troubled by the power that Copenhagen accorded to measurement. What is a measurement, anyway? Is it something that only conscious beings or tenured professors can do? Heisenberg and other Copenhagenists failed to elaborate. Some suggested that we create reality in the act of observing it—an idea that sounds poetic, perhaps a little too poetic. Einstein also thought it took a lot of chutzpah for Copenhagenists to claim that quantum mechanics was complete, a final theory never to be superseded. He regarded all theories, including his own, as stepping-stones to something greater.

In fact, Howard argues that Einstein would have been happy to entertain indeterminism as long as his concerns were addressed—if, for example, someone could spell out what a measurement was and how particles could stay in sync without acting at a distance. As a sign that Einstein considered indeterminism a secondary concern, he made the same demands of deterministic alternatives to Copenhagen and rejected them, too. Another historian, Arthur Fine of the University of Washington, thinks Howard overstates Einstein’s receptiveness to indeterminism but agrees that the man’s thinking was more solidly grounded than the dice-playing sound bite has led generations of physicists to assume.

Random Thoughts

If you tug on Copenhagen’s loose ends, Einstein thought, you should find that quantum randomness is like every other type of randomness in physics: the product of deeper goings-on. The dancing of a dust mote in a shaft of sunlight betrays the complex motions of unseen air molecules, and the emission of a photon or radioactive decay of a nucleus is analogous, Einstein figured. In his estimation, quantum mechanics is a broad-brush theory that expresses the overall behavior of nature’s building blocks but lacks the resolution to capture individual cases. A deeper, more complete theory would explain the motion in full without any mysterious jumps.

In this view, the wave function is a collective description, like saying that a fair die, repeatedly tossed, will land roughly the same number of times on each side. Wave function collapse is not a physical process but the acquisition of knowledge. If you roll a six-sided die and it lands on, say, four, the range of one to six “collapses” to the actual outcome of four. A godlike demon able to track all the atomic details affecting the die—the exact way your hand sends the cube tumbling across the table—would never speak of collapse.

Credit: Jen Christiansen; Source: “Emergent Chance,” by Christian List and Marcus Pivato, in The Philosophical Review, Vol. 124, No. 1; January 2015

Einstein’s intuitions were backed up by his early work on the collective effects of molecular motion—studied in the branch of physics known as statistical mechanics—in which he had demonstrated that physics could be probabilistic even when the underlying reality was deterministic. In 1935 Einstein wrote to philosopher Karl Popper, “I do not believe that you are right in your thesis that it is impossible to derive statistical conclusions from a deterministic theory. Only think of classical statistical mechanics (gas theory, or the theory of Brownian movement).”

The probabilities in Einstein’s way of thinking were just as objective as those in the Copenhagen interpretation. Although they did not appear in the fundamental laws of motion, they expressed other features of the world; they were not merely artifacts of human ignorance. Einstein gave Popper the example of a particle that moves around a circle at steady speed; the chance of finding the particle in a given arc of the circle reflects the symmetry of its path. Similarly, a die has a one-sixth chance of landing on a given side because it has six equal sides. “He did understand better than most at that time that there was significant physical content in the details of statistical-mechanical probabilities,” Howard says.

Another lesson of statistical mechanics was that the quantities we observe do not necessarily exist on a deeper level. For instance, a gas has a temperature, but a single gas molecule does not. By analogy, Einstein came to believe that a subquantum theory needed to mark a radical break from quantum mechanics. In 1936 he wrote, “There is no doubt that quantum mechanics has seized hold of a beautiful element of truth.... However, I do not believe that quantum mechanics will be the starting point in the search for this basis, just as, vice versa, one could not go from thermodynamics (resp. statistical mechanics) to the foundations of mechanics.” To fill in that deeper level, Einstein sought a unified field theory, in which particles derive from structures that look nothing like particles. In short, conventional wisdom is wrong that Einstein repudiated the randomness of quantum physics. He was trying to explain the randomness, not to explain it away.

Do Your Level Best

Although Einstein’s overall project failed, his basic intuition about randomness still holds: indeterminism can emerge from determinism. The quantum and subquantum levels—or any other pair of levels in the hierarchy of nature—consist of distinct types of structures, so they abide by different types of laws. The laws governing one level can leave a genuine element of randomness even if the laws underneath it are completely regimented. “A deterministic microphysics does not induce a deterministic macrophysics,” says philosopher Jeremy Butterfield of the University of Cambridge.

Think of a die at the atomic level. It can be constructed from zillions of atomic configurations that look utterly indistinguishable to the eye. If you track any one of these configurations as the cube is rolled, it will lead to a specific outcome—deterministically. In some configurations, the die ends up showing one dot; in others, two; and so on. Therefore, a single macroscopic condition (being rolled) can lead to multiple possible macroscopic outcomes (showing one of six faces) [see box graphic above]. “If we describe the die at a macrolevel, we can think of it as a stochastic system, which admits objective chance,” says List, who has studied the meshing of levels with Marcus Pivato, a mathematician at CY Cergy Paris University in France.

Although the higher level builds (in the jargon, “supervenes”) on the lower one, it is autonomous. To describe dice, you need to work at a level where dice exist, and when you do so, you cannot help but neglect atoms and their dynamics. If you cross one level with another, you commit the fallacy of a category mistake, which is like asking about the political affiliations of a tuna sandwich (to use an example from philosopher David Z. Albert of Columbia University). “When we have phenomena that can be described at multiple levels, we have to be conceptually very careful in not mixing levels,” List says.

For this reason, the die roll is not merely apparently random, as people sometimes say. It is truly random. A godlike demon might brag that it knows exactly what will happen, but it knows only what will happen to the atoms. It does not even know what a die is, because that is higher-level information. The demon never sees a forest, only trees. It is like the protagonist of Argentine writer Jorge Luis Borges’s short story “Funes, the Memorious,” a man who remembers everything and grasps nothing. “To think is to forget a difference, to generalize, to abstract,” Borges wrote. For the demon to know which side the die lands on, you have to tell it what to look for. “The demon would only be able to infer the higher-level history if the demon was given our specification of how we partition the physical level,” List says. Indeed, the demon might well come to envy our mortal perspective.

The level logic works the other way, too. Indeterministic microphysics can lead to deterministic macrophysics. A baseball can be made of particles behaving randomly, yet its flight is entirely predictable; the quantum randomness averages out. Likewise gases consist of molecules executing enormously complicated—and effectively indeterministic—movements, yet their temperature and other properties follow laws that are dead simple. More speculatively, some physicists such as Robert Laughlin of Stanford University suggest that the lower level is utterly irrelevant. The building blocks could be anything and still produce the same collective behavior. After all, systems as diverse as water molecules, stars in a galaxy and cars on a highway obey the same laws of fluid flow.

Free at Last

When you think in terms of levels, the worry that indeterminism might mark the end of science evaporates. There is no big wall around us, cordoning off a law-abiding chunk of the universe from the anarchic and inexplicable beyond. Instead the world is a layer cake of determinism and indeterminism. Earth’s climate, for example, supervenes on Newton’s deterministic laws of motion, but weather reports are probabilistic, whereas seasonal and longer-term climate trends are again predictable. Biology, too, supervenes on deterministic physics, but organisms and ecosystems require different modes of description, such as Darwinian evolution. “Determinism doesn’t explain everything,” says Tufts University philosopher Daniel C. Dennett. “Why are there giraffes? Because it was ‘determined’ that there would be?”

Human beings are embedded within this layer cake. We have the powerful sense of free will. We often do the unpredictable, and in most of life’s decisions, we feel we were capable of doing otherwise (and often wish we had). For millennia, so-called philosophical libertarians—not to be confused with the political kind—have argued that human freedom requires particle freedom. Something must break the deterministic flow of events, such as quantum randomness or the “swerves” that some ancient philosophers thought atoms can undergo.

The trouble with this line of thought is that it would free the particles but leave us enslaved. Whether your decision was preordained at the big bang or made by a mutinous particle, it is not your decision. To be free, we need indeterminism not at the particle level but at the human level. And that is possible because the human and particle levels are autonomous. Even if everything you do can be traced to earlier events, you can be the author of your actions because neither you nor the actions exist at the level of matter, only at the macrolevel of mind. “This macroindeterminism riding on microdeterminism may secure free will,” Butterfield says. Macroindeterminism is not the cause of your decision. It is your decision.

Some might complain that you are still a puppet of the laws of nature, that your freedom is an illusion. But the word “illusion” itself conjures up desert mirages and ladies sawed in half: things that are unreal. Macroindeterminism is not like that. It is quite real, just not fundamental. It is comparable to life. Individual atoms are completely inanimate, yet enormous masses of them can live and breathe. “Anything to do with agents, their intentional states, their decisions and choices: none of this features in the conceptual repertoire of fundamental physics, but that doesn’t mean those phenomena aren’t real,” List observes. “It just means that those are very much higher-level phenomena.”

It would be a category mistake, not to mention completely unenlightening, to describe human decisions as the mechanics of atoms in your brain. Instead you need to use all the concepts of psychology: desire, possibility, intention. Why did I choose water over wine? Because I wanted to. My desire explains my action. Most of the times that we ask “Why?” we are seeking someone’s motivations rather than the physics backstory. Psychological explanations presume the kind of indeterminism that List is talking about. For example, game theorists model human decisions by laying out the range of options and showing which one you will select if you are acting rationally. Your freedom to choose a certain option steers your choice even if you never plump for that option.

To be sure, List’s arguments do not explain free will fully. The hierarchy of levels opens up space for free will by separating psychology from physics and giving us the opportunity to do the unexpected. But we have to seize the opportunity. If, for example, we made every decision on a coin toss, that would still count as macroindeterminism but would hardly qualify as free will in any meaningful sense. Some people’s decision-making may be so debilitated that they cannot be said to act freely.

This way of thinking about determinism also makes sense of an interpretation of quantum theory that was developed in the years after Einstein’s death in 1955: the many-worlds interpretation. Advocates argue that quantum mechanics describes a collection of parallel universes—a multiverse—that behaves deterministically in the large but looks indeterministic to us because we are able to see only a single universe. For instance, an atom might emit a photon to the left or to the right; quantum theory leaves the outcome open. According to the many-worlds interpretation, that is because the same situation arises in a zillion parallel universes; in some, the photon goes deterministically left, and in others, it goes right. Not being able to tell which of those universes we reside in, we cannot predict what will happen, so the situation from the inside looks inexplicable. “There is no true randomness in the cosmos, but things can appear random in the eye of the beholder,” says cosmologist Max Tegmark of the Massachusetts Institute of Technology, a prominent proponent of this view. “The randomness reflects your inability to self-locate.”

That is very similar to saying that a die or brain could be constructed from any one of countless atomic configurations. The configurations might be individually deterministic, but because we cannot know which one corresponds to our die or our brain, we have to think of the outcome as indeterministic. Thus, parallel universes are not some exotic idea out there in the cosmos. Our body and brain are little multiverses, and it is the multiplicity of possibility that endows us with freedom.