Our Modular Minds

I believe that ultimately human consciousness can be described by a program. Now this doesn’t mean we’re all in the Matrix, simply that our mind is a giant seething logical machine with values that are manipulated by rules. There is no strange new science in the sense of new specialties that must be discovered in order for the mind to be understood, but a progression in a new kind of science as Wolfram dubbed it – the study of how complexity arises.

A List of Rules

When I first heard that you could program a computer as a child I was amazed. A strange wonder that I could only spend a few shared minutes with at school, something that could draw, add, and write far faster than I could ever dream of – and I could tell it what to do? I wasn’t quite sure how to inform it to bend to my every wish, so I started with Turtle (actually called LOGO I later found) upon my teacher’s recommendation. I fed Turtle long lists of instructions – move forward, draw a line, turn left, repeated in all and any ways I could think of. He would draw glowing green shapes across my screen, and never tired.

The Need for Modularity

The only problem was that while the Turtle seemed infallible, I certainly could not say the same. I was making the classic beginner’s mistake – I would write one giant chunk of code that was supposed to cause my turtle to dance in precisely the way I wanted. Any little mistake would send it widely off course and I would end up with a mess that barely looked like the original design at the best of times. I later learned that using a programming concept called “modules” could help me isolate these errors and make code more efficient and reusable. Just like a company could have a manufacturing and engineering division which could communicate with standardized blueprints, a program could have different modules that would exchange data in a standardized manner. A modular program is more stable since mistakes are typically limited in influence to the module they’re contained in, and each module can be modified by separate influences with only the understanding that they are supposed to behave and communicate in a certain manner.

Damage as Evidence

So is our mind modular? Well, if it wasn’t, we could assume that a brain injury affecting a certain part of the brain would have a consistent and general impact across all of our consciousness. The only problem is that we generally only see a nonspecific mental decline like this from a nonspecific trauma, say impact blows to the head over a long period of time. Injuries in specific areas seem to be correlated with deficits in certain mental abilities – while leaving others totally intact.

A stroke can basically be thought of as an incident where blood flow is drastically affected in a certain specific area of the brain. This subregion of the brain is unable to function due to lack of blood flow, and very strange things can occur.

Howard Engel is a Canadian novelist who had a stroke. Upon waking one morning, he found that the morning paper seemed to be written in some strange script, an alphabet he could not understand. Everything else appeared normal, except his visual cortex had been damaged in a specific area which prevented him from visually parsing letters and words. As a writer, he despaired – it seems that his livelihood had been lost. Soon he realized a critical distinction which gave him hope – he may be unable to read visually, but could he write? Howard sat down and traced these strange looking symbols, his pen gliding over the bizarre shapes over and over. And eventually, the concepts came back to him. In a strange sense, he could now read again. Years and years of writing had associated certain movements of his hand with letters and concepts. Instead of words in his head put to paper by hand movements and a pen, he had to move concepts in the opposite direction – moving his hand over shapes written previously by others, the concepts echoed back up his motor cortex.

And it worked. There was irreparable damage to his visual cortex, a critical module malfunctioning. So he hacked his brain, redistributing resources from his motor cortex which had been trained to recognized these same symbols and concepts necessary for reading by his constant writing. Howard now traces the shapes he sees on the inside of his front teeth with his tongue. His speed has steadily increased, and he says he can now read about half of the subtitles in a foreign film before they flash off the screen.

It doesn’t seem too strange to suggest that there are different localized modules in the brain for motor control, visual interpretation, and other concepts easily identifiable with different aspects of the physical world. But are there modules with finer distinctions, working on different parts of our mental experience rather than different parts of the physical world?

The Wason Selection Task

The Wason selection task is a very interesting experiment in the field of psychological reasoning. Before I spoil it for you by talking too much, let’s just do it right now. Look at the following cards.

Assume the cards have a number on one side, and a color on the other. What cards need to be flipped over to make sure that all even numbers have red backs? Make sure you’ve picked a card or cards.

Got it? Now a bit of unsettling but ego-salvaging news. When this experiment is done with undergraduates, only 10 to 20 percent get it right. The correct answer is to flip over two cards: the number 4 to make sure it has a red back, and the blue card to make sure that it doesn’t have an even number on the other side. Most people suggest flipping over the 4 and the red card – this is wrong, as it doesn’t matter if the red card has an odd number on the other side.

Now let’s mix it up a bit. Instead of numbers and colors, let’s try people and social activities. Assume the cards have a drink on one side, and a person of a certain age on the other. What cards need to be flipped over to make sure everyone drinking beer is old enough to do so?

Now the answer flows quickly and easily, and almost everyone gets it correct. We need to flip over the beer card to make sure that the person on the other side is old enough, and we need to flip over the card showing the underage drinker to make sure they’re playing by the rules.

The weird thing is that both examples are logically equivalent. Instead of numbers and colors, we’ve just used people and drinks. But something very important has occurred, and it happens time and time again as these tests are administered. It seems that people are fast and accurate at solving this task only if it is described as a test of social obligations. They both can be described identically with logic – but that doesn’t appear to matter to our mind. We appear to have a module dedicated to social reasoning and conflict, and can only solve these problems quickly if it involves determining if someone is cheating or breaking social conventions. This ancient module would hold significant survival value – a general logic verification module not quite so much.

Modules Upon Modules

There appears to be significant evidence for a modular mind, not just in terms of divisions between senses such as sight or hearing and other actions like movement but also more abstract modules that deal with concepts such as social rules. Stroke victims can literally rewire their brains, passing concepts upward into their consciousness through paths never intended to be used in such a strange manner, duplicating the work of other modules lost to injury. These modules live in a strange world of physical interaction and abstract mental space, a huge interconnected mass with no clear outline behind it. The big question now becomes: is a sufficiently complex system able to understand itself, and are we that system?

Color and Reality

When I was a kid, I used to wonder if everyone saw the world in the same way. We can all look at the same grass, but maybe the color I called green showed up in my brain as the color my friend called blue. Maybe all of our colors were shifted around to the point where all the colors were accounted for, but how we perceived them was shuffled up. I thought it would be remarkably exciting, and hoped that I could see the world through someone else’s brain to see if, in fact, this was true.


My eight year old self would be bitterly disappointed technology today has not progressed far enough to make that wish a reality. At the time, we had to settle the debate by another manner – asking an adult, a source of concrete and immutable knowledge. The answer I was given was that everyone sees the same colors of course (although why this was so obvious was never really clear) and if they didn’t it wouldn’t matter much since we couldn’t tell. Color was “real” – bits of light had a color (later I found out we could call it the wavelength of a photon), it hit our eyes, and our brains converted it to a beautiful image.

The only problem is that this is wrong.

Color as Wavelength

Well, alright. Before you get upset, it isn’t completely wrong. We were all taught about Sir Isaac Newton who discovered that a glass prism can split white light apart into its constituent colors.


While we consider this rather trivial today, at the time you’d be laughed out of the room if you suggested this somehow illustrated a fundamental property of light and color. The popular theory of the day was that color was a mixture of light and dark, and that prisms simply colored light. Color went from bright red (white light with the smallest amount of “dark” added) to dark blue (white light with the most amount of “dark” added before it turned black).

Newton showed this to be incorrect. We now know that light is made up of tiny particles called photons, and these photons have something called “wavelength” that seems to correspond to color. Visible light is made up of a spectrum, a huge number of photons each with a different wavelength our eyes can see. When combined, we see it as white light.


So this appears to resolve my childhood debate. Light of a single wavelength (like that produced by a laser) corresponds to a single “real” color. The brain just translates wavelengths into colors somehow, and that is that. There’s just one problem.

We’re missing a color!

Color as Experience

To find out just what we’re missing, we have to consider how we can combine colors. For instance, you learned some basic color mixing rules as a kid. In this case, let’s use additive color mixing since we’re mixing light.


Let’s find two colors on the spectrum line, and then we can estimate the final color they’ll produce when you mix them by finding the midpoint.

Red and green make yellow.


Green and blue make turquoise.


Red and blue make…


Green? What? That doesn’t seem to make any sense! Red and violet make pink! But where is pink in our spectrum? It’s not violet, it’s not red – it seems like it should be simultaneously above and below our spectrum. But it’s not on the spectrum at all!

So we’re forced to realize a very interesting conclusion. The wavelength of a photon certainly reflects a color – but we cannot produce every color the human eye sees by a single photon of a specific wavelength. There is no such thing as a pink laser – two lasers must be mixed to produce that color. There are “real” colors (we call them pure spectral or monochromatic colors) and “unreal” colors that only exist in the brain.

A Color Map

So what are the rules for creating these “unreal” colors from the very real photons that hit your eye? Well, in the 1920s W. David Wright and John Guild both conducted experiments designed to map how the brain mixed monochomatic light into the millions of colors we experience everyday. They set up a split screen – on one side, they projected a “test” color. On the other side, the subject could mix together three primary colors produced by projectors to match the test color. After a lot of test subjects and a lot of test colors, eventually the CIE 1931 color space was produced.


I consider this to be a map of the abstractions of the human brain. On the curved border we can see numbers, which correspond to the wavelengths in the spectrum we saw earlier. We can imagine the spectrum bent around the outside of this map – representing “real” colors. The inside represents all the colors our brain produces by mixing – the “unreal” colors.

So let’s try this again – with a map of the brain instead of a map of photon wavelengths. Red and green make yellow.


Green and blue make turquoise.


Blue and red make…


Pink! Finally! Note that pink is not on the curved line representing monochromatic colors. It is purely a construction of your brain – not reflective of the wavelength of any one photon.

Is Color Real?

So is color real? Well, photons with specific wavelengths seem to correspond to specific colors. But the interior of the CIE 1931 color space is a representation of the a most ridiculously abstract concept, labels that aren’t even labels, something our brain experiences and calculates from averaged photon wavelengths. It is an example of what philosophers call qualia – a subjective quality of consciousness.

I later learned that my childhood argument was a version of the inverted spectrum argument first proposed by John Locke, and that the “adult” perspective of everyone seeing the same colors (and it not really mattering if they didn’t) was argued by the philosopher Daniel Dennett.

I have come no closer to resolving my question from long ago of “individual spectrums” – but for the future, I vow to pay more attention to the idle questions of children.

The Mystics and Realists of Quantum Physics

It is said that when the 20th century is long gone, it will be remembered for two revolutionary theories – those of relativity and quantum physics. While both have led to a deeper understanding of our world, quantum physics stands alone in its profound weirdness – the ability to accurately predict totally counter-intuitive aspects of the physical world. From the simple indisputable oddity of the double slit experiment to the philosophical implications of Schrodinger’s cat, it becomes clear that we still understand very little of the actual mechanics of our world.

When explanations are lacking, the mystical is often brought up to fill the void. This has degenerated today into complete pop-psychology crap such as The Secret or What the Bleep Do We Know, but the role that human consciousness plays as an “observer”, if any, was considered very early by the founders of these theories. These arguments brought forth by some of the finest thinkers of our time echo to this day.

Niels Bohr


Winner of the Nobel Prize in Physics in 1922, employed by the Manhattan Project, and father of the Bohr model familiar to every high school student, Niels Bohr was first accused by Einstein of introducing “mystic” elements in his explanation of quantum physics – mystic elements which in Einstein’s view had no place in science.

This was part of the famous Bohr-Einstein debates, and was perhaps not a fair criticism. Bohr appeared to not worry excessively about the “reality” underpinning the equations of quantum theory, and was simply more concerned about the equations of quantum theory rather than their implications. He rejected the hypothesis that the wave function collapse requires a conscious observer, insisting that “It still makes no difference whether the observer is a man, an animal, or a piece of apparatus”.

His view is perhaps best summarized in the following quote recalled by Heisenberg:

This argument looks highly convincing at first sight. We can admittedly find nothing in physics or chemistry that has even a remote bearing on consciousness. Yet all of us know that there is such a thing as consciousness, simply because we have it ourselves. Hence consciousness must be part of nature, or, more generally, of reality, which means that, quite apart from the laws of physics and chemistry, as laid down in quantum theory, we must also consider laws of quite a different kind. But even here I do not really know whether we need greater freedom than we already enjoy thanks to the concept of complementarity.

In short, if the numbers work out, don’t worry too much.

Wolfgang Pauli


But some did worry. Pauli was a skeptic’s skeptic – a man so dedicated to rationality it led him down a strange path. In 1927 the Solvay Conference was busy reaching consensus that Bohr’s approach was the best way to regard quantum physics (the Cophenhagen Interpretation), but Pauli was equally confident in a different interpretation. He tried to trace out just what part of consciousness it is that seems to prevent an in-depth, rational understanding. Deeply influenced by Schopenhauer’s The World as Will and Representation, Pauli appropriated his concept of a will “which breaks through space and time”.

He viewed that the acquisition of knowledge in mathematics or quantum physics “gives rise, however, to a situation transcending natural science” that can even acquire a “religious function” in human experience. This is not a belief in the religions of old, but as Pauli states “I do not believe in the possible future of mysticism in the old form. However, I do believe that the natural sciences will out of themselves bring forth a counter pole in their adherents, which connects to the old mystic elements.”

Perhaps the most interesting viewpoint on Pauli was that of Heisenberg, who viewed Pauli’s paradigm as even more rational than Bohr’s equation-focused approach because only he acknowledged that the scientific evidence pointed to something pre-rational or ‘mystical’. Pauli claimed that consciousness was presented philosophically by mystics and studied scientifically by psychologists. With the advent of quantum mechanics, physicists should then be able to unify both approaches. Unfortunately, we continue to wait.

Albert Einstein

Einstein 1921

Einstein was a scientific superstar, with fame not equalled to this day. One day, a quote was making the round in British newspapers that Einstein subscribed to the theory that “the outer world is a derivative of consciousness”. His response was swift and critical.

No physicist believes that. Otherwise he wouldn’t be a physicist. Neither do [Eddington and Jeans]. . . . These men are genuine scientists and their literary formulations must not be taken as expressive of their scientific convictions. Why should anybody go to the trouble of gazing at the stars if he did not believe that the stars were really there?

Einstein’s opposition to Bohr’s view or more “mystical” approaches is often cast as the great divide between the philisophies of idealism and those philosophies based on realism. Pauli often referred to Einstein’s “philosophical prejudice” assuming that reality is independent of any mind. In fact, his approach and objections were far more subtle. His major concern was the incredibly subjective aspect of consciousness introducing an unmeasurable “geist”, and this clash with the precise and well defined philosophical principles of physics such as locality or determinism.

This led to Einstein’s famous attempt at “breaking” quantum physics, the EPR paradox. At first a thought experiment which appeared to demonstrate quantum physics violating the seemingly well established principle of locality, later experiments showed that quantum physics instead proved locality false.

Violations of locality and determinism seemed to bother Einstein greatly, and this can be seen in his famous quote objecting to the randomness involved in wave function collapse under Bohr’s interpretation, that “God does not play dice”.

Bohr, summing up the debate perfectly, replied “Einstein, stop telling God what to do with his dice.”

John von Neumann


The “last of the great mathematicians”, von Neumann solved one of the great problems of quantum theory. While the theory itself was established and experimentally verified, it lacked a “deep” mathematical understanding based on an axiomatic approach. He treated the world as a Hilbert space, an infinite dimensional structure.

While classical mechanics approached the world as a collection of points with six different characteristics (position and momentum along the x, y, and z axis), von Neumann’s approach considers a quantum system as a point in infinite dimensional space, corresponding to the infinite amount of possible outcomes. This led to very interesting implications in terms of “measurement”. While the “measurement” of a classical system simply involved finding one or more of those six values, the “measurement” of a quantum system involved mathematical operators acting on an infinite amount of values to produce a finite measurement.

The interesting conclusion arises when we consider the “real” interpretation of these mathematical operators. While we may say that an scientific instrument has caused wave function collapse, we run into the problem that no physical system (and a scientific instrument is a physical system completely described by quantum mechanics) can cause wave function collapse. We can describe the entire ensemble perfectly as a Hilbert space. But we do not experience this Hilbert space – we measure and experience only finite values.

The conclusion von Neumann reached is that consciousness, whatever it is, appears to be the only thing in physics that can ultimately cause this collapse or observation. This does not mean that consciousness is “required” for the universe to work, but that wave function collapse appears to be caused by consciousness and we observe only a tiny slice. We are therefore an “abstract ego” acting as a measurement device on the infinite values of true reality.


Today, the argument has largely died down, a combination of practicality and lack of any suitably shocking experimental results. The majority of physicists today take the approach of “it works”, namely that quantum physics produces accurate predictions of the real world and that the mathematical formalism is just that – a mathematical formalism that produces accurate results.

It may not reflect the true reality of the world (whatever it is), but it is suitably accurate to any level of precision that we are physically able to obtain. One may stay awake at night wondering “why”, but one will not get much work done with this approach. Perhaps more clarity lies in the future, but in the meantime – we will have to tolerate crap that tells us we can “will” our way to riches with quantum mechanics (and coincidentally make the authors rather rich, will indeed) instead of a rational approach dedicated to the pursuit of truth.