Gelernter Wrapup

A few more remarks about David Gelernter's essay in Technology Review, which I hope won't run as long as the ones I made yesterday but probably will.

First, a little terminology, for anyone who hasn't read or has only skimmed the essay. Gelernter uses the term "consciousness" to denote the possession of what philosophers call qualia. He's not talking about the differences between the brain states of waking and sleeping animals, and he's not talking about self-consciousness -- an animal's ability to recognize itself in a mirror, or to use the states of its own body (including its brain) as subjects for further cognition.

Qualia are (purportedly -- I'd like to think this post casts doubt on the very intelligibility of the idea) the felt character of experience. When my thermostat registers a certain drop in temperature, it throws on the heat. Similarly, when I register a certain drop in temperature, I throw on a sweater. But unlike the thermostat (the story goes), I feel cold. This feeling is not reducible to either the average kinetic energy of the air molecules around me or the physical act of putting on clothing: it's its own thing. On this picture, every human perception or sensation has an associated quale (the singular of qualia): the painfulness of pain, the redness of red things, the coldness of cold. To be conscious, in Gelernter's sense, is to have qualia.

Gelernter divides artificial-intelligence theorists into two camps: cognitivists and anticognitivists. Cognitivists believe that, if human beings have qualia (an important if!), then a robot that behaves exactly like a human being (even if its body is vinyl and its "brain" is a huge Rube Goldberg machine made of tinker toys) does, too.

Okay, so armed with these distinctions, let's take a look at a couple of Gelernter's initial claims:

(1) "This subjectivity of mind has an important consequence: there is no objective way to tell whether some entity is conscious."

(2) "we know our fellow humans are conscious"

Amazingly, these claims occur in immediate succession. How are we to reconcile them? Are human beings not "entities"? Let's assume they are. It follows that Gelernter is defending some form of "knowledge" that stands in no need of -- indeed, does not even admit of the possibility of -- objective justification.

What are we to do with claims to such knowledge? Are we under any obligation to take them seriously? Do they even require rebuttal? If they aren't anchored in any objective criteria at all, how could they be rebutted? Indeed, they can't. They can simply be denied.

And this is the position in which cognitivists and anticognitivists find themselves: simply denying each other's unfounded knowledge claims. The anticognitivist says, "We know our fellow humans are conscious." And the cognitivist says, "No we don't -- at least, not in any way that we don't also know that a perfect behavioral simulacrum of a human is conscious."

Gelernter refuses to acknowledge, however, that he and his disputants have reached such an impasse. He insists that the consciousness of his fellows is something he deduces. "We know our fellow humans are conscious," Gelernter says,

but how?...You know the person next to you is conscious because he is human. You're human, and you're conscious--which moreover seems fundamental to your humanness. Since your neighbor is also human, he must be conscious too.

If there is an argument here, however, it is entirely circular: the sole criterion for ascribing consciousness to our fellow humans is -- they're human!

Gelernter then moves on to the Chinese room, which I discussed yesterday. After rehearsing Searle's argument, however, he adds that

we don't need complex thought experiments to conclude that a conscious computer is ridiculously unlikely. We just need to tackle this question: What is it like to be a computer running a complex AI program?

Well, what does a computer do? It executes "machine instructions"--low-level operations like arithmetic (add two numbers), comparisons (which number is larger?), "branches" (if an addition yields zero, continue at instruction 200), data movement (transfer a number from one place to another in memory), and so on. Everything computers accomplish is built out of these primitive instructions."

The obvious cognitivist rejoinder, as I mentioned yesterday, is that neurons just relay electrical signals, faster or slower, and emit higher concentrations of this or that neurotransmitter. Everything brains accomplish is built out of these primitive operations. If consciousness can emerge from the accumulation of mechanistic neural processes, why can't it similarly emerge from the accumulation of mechanistic computational processes? Again, Gelernter responds by simply identifying consciousness and humanness, without any argumentative support:

The fact is that the conscious mind emerges when we've collected many neurons together, not many doughnuts or low-level computer instructions.

I.e., the sole criterion for ascribing consciousness to collections of neurons, rather than collections of logic gates, is -- they're neurons! QED.

If Gelernter were to read these posts and conclude that, in fact, his essay consisted entirely of non sequiturs and circular arguments, neither of which I think is likely, I would nonetheless expect him to maintain his anticognitivist stance. While cognitivist arguments can, I believe, show that anticognitivist arguments prove nothing, neither do they prove anything themselves. But as a Wittgensteinian pragmatist, I take this to show that the distinction between cognitivism and anticognitivism is meaningless. I agree with Gelernter's assertion that "there is no objective way to tell whether some entity is conscious", whether, ultimately, he himself does or not. And I think that the upshot is that the very idea of consciousness -- in his sense, consciousness as the possession of qualia -- is one on which we can get no intellectual purchase.

Uplift the bytecode!

MIT's Technology Review magazine has published a long essay by Yale computer scientist David Gelernter that addresses some of the best-trodden arguments in the philosophy of mind with somewhat less aplomb than you might expect from a bright 11-year-old. This is mildly distressing to me, both because the central topic of the essay -- the possibility of conscious machines -- is one to which I've devoted a lot of time and energy and because in my day job, I'm a copy editor at Technology Review. So what follows may be treasonous. On the other hand, I've read the essay carefully, several times, so I'm intimately acquainted with all its flaws. (I should add that Gelernter appears to have been delightful to work with, and that for all I know, he's a brilliant computer scientist. But if he is, then his susceptibility to circular argument and non sequitur suggests that there may be more to the notion of philosophical training than we Wittgensteinians/Rortians tend to think there is.)

Gelernter appears to swallow whole what I'll call the Original Statement of John Searle's "Chinese room" thought experiment. The Original Statement should be distinguished from succeeding restatements because it, unlike them, is transparently fallacious. (I think that the restatements also fail to make the point Searle and others hope they will, but I agree with Rorty that their proponents and opponents beg all questions against each other. Or almost all.)

In the Original Statement, Searle asks us to imagine that someone has devised a computer program that can pass the "Turing test" in Chinese. That is, a native Chinese speaker typing questions and remarks into a computer and receiving replies generated by the program would be unable to tell whether or not she was actually instant-messaging another person. Now suppose that, instead of executing the program on a computer, Searle executes it by hand. He's locked in a room -- the Chinese room -- and sets of Chinese symbols are slid to him under the door. According to instructions in a thick manual, he correlates the symbols he receives with another set of Chinese symbols, which he slides back under the door -- the program's output.

Searle doesn't understand a word of Chinese; he's just lining up symbols with symbols (a process that may require a few pencil-and-paper calculations). And from this he concludes that the room doesn't understand Chinese either.

Now, I would have thought that the fallacy of that conclusion was obvious, but history has shown that it isn't. Who cares whether Searle can understand Chinese? He's just a small and not very important part of the system -- what Dan Dennett has called a "meat servo" -- and it's the system that understands Chinese.

Searle's role is analogous to that of the read/write head in the magnetic-tape memory of an old computer -- or perhaps the laser diode in the CD tray of a modern-day Dell. His job is just to fetch data and shuttle it where he's told to. Saying that the Chinese room can't understand Chinese because Searle can't is like saying that my computer can't play chess because the diode in the CD tray can't.

In the paper in which he proposed the Chinese-room thought experiment, Searle actually anticipated this objection (which might make you wonder why he bothered with the Original Statement at all), which he sensibly called the "systems response". I don't find his rejoinder to the systems response convincing, but for present purposes, that's irrelevant. Because Gelernter doesn't even get that far.

After declaring, "I believe that Searle's argument is absolutely right", Gelernter goes on to propose a thought experiment of his own, one that runs, in part, as follows:

Of course, we can't know literally what it's like to be a computer executing a long sequence of instructions. But we know what it's like to be a human doing the same. Imagine holding a deck of cards. You sort the deck; then you shuffle it and sort it again. Repeat the procedure, ad infinitum. You are doing comparisons (which card comes first?), data movement (slip one card in front of another), and so on. To know what it's like to be a computer running a sophisticated AI application, sit down and sort cards all afternoon. That's what it's like.

Well, no, Dave, that's not what it's like. Again, that's what it's like to be the CPU. But the CPU, like Searle in the Chinese room, is just a small part of the system.

Gelernter's argument is analogous to saying, "The corpus callosum shuttles electrical signals between hemispheres of the brain. You want to know what it's like to be a corpus callosum? Well, imagine standing next to a computer with a USB thumb drive plugged into it. When the computer sounds an alert, you take the USB drive out and stick it in another computer. When that computer sounds an alert, you stick the USB drive back in the first computer. That's what it's like to be a corpus callosum. Therefore humans can never be conscious."

Notice that I am not here making the standard argument that neurons and neuronal processes, taken in isolation, are every bit as mechanistic as logic gates and binary operations. (I'll take that one up tomorrow.) Instead, I'm reproducing what we might call the synecdochal fallacy, common to both Searle and Gelernter, of substituting the part for the whole.

I'm sure that at this point I've taxed the patience of anyone who's not as much of a phil o' mind nerd as I am, so I'll stop for now. But tomorrow I'll address a couple of Gelernter's fallacious arguments that are all his own.

AMENDMENT (6/28/07, 5:20 p.m. ET):

A correspondent (who shall remain nameless) objects to the following line:

"He's just a small and not very important part of the system -- what Dan Dennett has called a 'meat servo' -- and it's the system that understands Chinese."

The objection is this:

"It's no good saying, 'The system understands,' because that's what's at issue."

It's a good point and may suggest that philosophy, which demands an incredibly high level of linguistic precision, should not be undertaken in blogs. But I plan on ignoring that suggestion, in the hope that my readers will read me with charity.

What I should have said, instead of "it's the system that understands Chinese", is

"It's the system's ability to understand Chinese that's in question."

The point was just that the Chinese-room thought experiment falls prey to the synecdochal fallacy. I didn't mean to imply that the refutation of the Chinese-room argument proves the possibility of conscious machines.