The Nature of Reality
This is partly a review of 'QUANTUM: Einstein, Bohr and the Great Debate About the Nature of Reality' by Manjit Kumar. See the New York Times review here.
The Great Debate
Quantum theory is the most successful scientific theory in history. Since its formulation in the heady days of the late 1920s it has been tested innumerable times and has been found to be in complete accord with experimental results. Yet its implications for reality and science are radical. To quote the Times reviewer,
"For Bohr, physics was not about finding out what nature is, but about what can be said about it. Quantum mechanics was a complete theory of the behavior of matter and light, and we just have to come to terms with the limitations it places on what can be known, for example as illustrated by the Heisenberg uncertainty principle. Einstein was having none of it. He believed that there is an objective world out there and that it is the job of scientists to describe it."
In particular, Quantum theory says that when the momentum of a particle is known, its location has no physical reality. Einstein was so troubled by this that in 1935 he co-wrote a paper with Podolsky and Rosen, commonly referred to as the EPR paper. (Check it out, the math is trivial). They define reality as follows.
"If, without in any way disturbing a system, we can predict with certainty the value of a physical quantity, then there exists an element of physical reality corresponding to this physical quantity."
They declare a physical theory to be complete if "every element of physical reality has a counterpart in physical theory". To prove the incompleteness of the theory they set up a thought experiment with a pair of maximally entangled particles. This means that both particles are described by one wave function in such a way that measuring the property of one fixes the property of the other, thereby making it an element of reality. One can measure one particles' velocity and thus predict the other particle's with certainty. But this would violate the Heisenberg Uncertainty Principle since we can always measure the other particle's position with certainty. Thus, we are forced to require that two or more physical quantities can be regarded as simultaneous elements of reality only when they can be simultaneously measured or predicted. Since this contradicts the aforementioned definition of reality, they conclude that quantum theory is incomplete.
Bohr and the other orthodox quantum theorists subscribed to the Copenhagen interpretation. They effectively denied the existence of external reality, claiming that a theory is good to go as long as it describes the outcomes of all possible measurements, and that any talk of what-actually-exists-out-there goes beyond the realm of science.
To everyone's surprise there turned out to be an empirical test that could distinguish between these views of the world.
The Bell Tolls
In 1964, John S. Bell published a paper titled On the EPR Paradox. He proved that local realism implied that a certain inequality, now called Bell's inequality be satisfied. This inequality was then found to be violated thus proving that no local hidden variable theory could ever replace quantum mechanics. For some, this settles the question decidedly in favour of the Copenhagen interpretation. Quantum theorists are happy to give up reality itself but many thinkers hope to find a non-local hidden variable theory. I have harboured similar hopes myself because I think there are strong epistemological reasons not to deny the existence of an external reality.
Metaphysical Issues
Not too long ago I visited a close friend of mine who is a philosopher of science. He showed me a very interesting paper by Bas C. Van Fraassen titled The Charybdis of Realism: Epistemological Implications of Bell's Inequality. Fraassen reformulates Bell's Inequality in a manner that strips the issue at hand of all paraphernalia, reducing it to the core epistemological conundrum. I am going to quote shamelessly without quotes from this paper in the rest of this section.
A fundamental metaphysical question is: How is reasonable expectation about future events possible? To which a realist like me would answer: Reasonable expectation of future events is possible only on the basis of some understanding of (or, reasonable certainty about) causal mechanisms that produce those events.
Suppose there is a correlation between two (sorts of) events, such as lung cancer and heavy smoking. That is a correlation in the simultaneous presence of two factors: having lung cancer now and being a heavy smoker now. An explanation that has at least the form to satisfy us traces both back to a common cause (in this case, a history of smoking which both produced the smoking habit and irritated the lungs). Characteristic of such a common cause is that, relative to it, the two events are independent. Thus present smoking is a good indication of lung cancer in the population as a whole; but it carries no information of that sort for people whose past smoking history is already known.
Now let L and R be perfectly anti-correlated phenomena that have a given common cause C. Moreover, suppose that L given C is independent of R, and R given C in independent of L. That is, we have in terms of probability distributions
P(L + R = 0) = one
P(L | C & R) = P(L | C) = zero or one
P(R | C & L) = P(R | C) = zero or one
These innocuous assumptions along with basic probability theory imply Bell's inequality. But we know that it is violated in experiments. Thus we find that a casual theory of the world is at odds with the way the world is.
Remarks in Lieu of a Conclusion
To say that I am uneasy with the implications of the above is an understatement. Before reading the Van Fraassen paper, I had convinced myself that there will be a non-local hidden variable theory that would eventually supplant quantum mechanics. Now its clear that not just local realism, but causation itself is not really kosher. In other words, the realist's answer to the fundamental metaphysical question is flawed and is, at best, incomplete.
How, then, are reasonable expectations about future events possible at all?
Without causality, in what sense is nature still intelligible? The very idea of science as natural philosophy is shaken. As Peter Dear put it in The Intelligibility of Nature, "The hallmark of natural philosophy is its stress on intelligibility: it takes natural phenomena and tries to account for them in ways that not only hold together logically but also rest on ideas and assumptions that seem right, that make sense; ideas that seem natural."
Science is reduced to mere instrumentality and natural philosophy declared dead. This result belongs in the same league as Gödel's incompleteness theorems, Arrow's Impossibility theorem, and apparent incompatibility of determinism and free will. For a great survey of these dismal topics read 'Impossibility:The limits of science and the science of limits' by John D. Barrow.
If the universe gets any more depressing, I am never getting out of bed again.
[Update: The philosopher who showed me the paper has pointed out that a non-local theory is entirely possible. The possibility is not in doubt and I stand corrected for suggesting such a thing. While that may be true as an existence result, we don't even have a causal theory to comprehend something that has been known for a century. By shunning the question, quantum physicists have so far failed to construct a covariant quantum field theory, even as philosophers have failed to construct a credible zoo of non-local causal structures. We fail completely when it comes to such questions, the entire miserable lot.]