Illusionism Pulls Out the First Brick
What happens to my argument if consciousness is a trick the brain plays on itself.
I ran into this one sideways. I was arguing with a chatbot about Joscha Bach, specifically about whether he has any real answer to the problem of other minds, and the conversation kept sliding off into a view I had heard of but never sat down with: illusionism.
The short version, from its best-known defender Keith Frankish (with Daniel Dennett as its patron saint): phenomenal consciousness, as usually conceived, does not exist. There is no irreducible “what it is like” to see red or stub your toe. Introspection works like a user interface. It hands you a simplified, schematic model of what your brain is doing, and you mistake the schematic for a magical inner quality. The illusionist therefore declines to solve the hard problem (how does matter produce feeling?) and swaps in the illusion problem: why does the brain so persistently convince itself that it feels?
The cleanest way I can put what that means for this site: illusionism rejects the cogito. Or at least it rejects the version of it that page 01 of the sequence stands on. To be fair to them, illusionists do not deny that something is going on in there; there is processing, there is representation, there is report. What they deny is exactly the part I called the one thing I know. I take my own experience as the most obvious fact about the world, possibly the only obvious one. Someone who says it just ain’t so can throw out the whole sequence from the first paragraph. So I had better look at them.
What the philosophy pages are for
Worth saying first, because it changes what counts as a good answer here.
The philosophy in Part I is not trying to be maximally convincing at the object level. I have not solved solipsism, and neither has anyone else; it is still very much an open problem, the illusionists’ claim to have dissolved it notwithstanding. What Part I tries to do is lay out a very minimal way of living with that open problem, and then use it as a springboard for extreme precaution about existential risk.
The reason I bother explaining why I care, instead of just asserting that extinction is bad and moving on to insurance premiums, is that I am thoroughly convinced a large contingent of people earnestly hold what I call Class 1 disagreements: that human extinction just isn’t that big a deal, ethically, for one reason or another. I have not solved ethics forever either, so I will not claim extinction is guaranteed to be a calamity. But I think it very likely is, and the stakes are large enough that the “very likely” has to be defended rather than assumed.
Illusionism is interesting precisely because it is a respectable, peer-reviewed route into Class 1.
The worry: no qualia, no patients, no problem
Here is the chain as it looks from where I stand.
- Only beings which experience qualia are moral patients. That is the brute fact page 01 puts down.
- Illusionism says nothing experiences qualia, in the sense that matters. There is only the brain’s representation of itself as experiencing them.
- So nothing is a moral patient.
- So human extinction is exactly as morally inert as a rock rolling down a hill, because everything is.
Step 3 to step 4 is basically moral error theory: Mackie’s view that moral claims all aim at truth and all miss. And the two views rhyme structurally. Both say humans have an overwhelming intuitive impression of something (inner feeling; objective wrongness) that is not really there. It is easy to see how someone who has accepted one debunking would find the second one cheap.
I suspect this is roughly the psychology underneath a fair amount of relaxed talk about AI succession. If there is no inner light in us, nothing is extinguished when we go. The successor does not have to be “somebody” for the handover to be fine, because nobody was ever somebody.
The ways around it
To be fair, illusionism does not strictly imply moral antirealism, and people have noticed. The escape routes I know of:
- Functionalist realism. Pain as a felt quality may be an illusion, but nociception, damage and the frustration of an organism’s goals are perfectly real physical processes. Ground moral facts in those.
- Rationalist realism. If morality is Kantian or Parfitian, grounded in reason like arithmetic is, then whether anyone feels anything was never load-bearing in the first place.
- Ethics without sentience. François Kammerer’s paper of exactly that name is the most honest version of all this I have found. He accepts that, if materialism is true, phenomenal consciousness probably cannot carry the moral weight we put on it, and proposes grounding moral status in desires and their frustration instead, described in third-person terms and coming in degrees.
My problem with all of these is the same, and I will state it as a gut reaction first: the idea that whether consciousness exists has no bearing on moral facts strikes me as obviously ludicrous. Once the moral action is carried entirely by third-person functional properties, there is no floor. A thermostat has a setpoint and “tries” to reach it. A bacterium moves away from things that damage it. A rock resists being crushed, if you squint. Take the functional story far enough and you have opened the door to it being morally wrong to bully a rock.
That is a reductio of the extreme, and to be fair Kammerer’s view is not the extreme; a rock has no desires on anyone’s account. But the gradient is the problem, not the endpoint. Somewhere between the rock and me, a functionalist has to draw a line, and every line they draw is a line they chose, with nothing I have privileged access to on either side of it. The only line I have ever had first-person evidence for is the one the illusionist tells me is not there.
Rescuing precaution without qualia
Here is the part I actually care about. I think the argument for precaution can be rescued from illusionism with a few rewrites. I do not want to put those rewrites into the sequence itself, because the sequence is written from my actual view, and illusionism is not my view. So they live here.
Swap the variable in page 02. A Best Lower Bound on Other Minds never actually used anything special about phenomenality except that I am the reference point. Its structure is: I am a moral patient; things more like me are more likely to be; I do not know which kinds of likeness matter; so take a pluralist prior. Replace “experiences qualia” with “has whatever property grounds moral patienthood” (desire, valence, a self-model, a global workspace, take your pick) and every step goes through unchanged. The illusionist does not know which functional property matters any better than I know what qualia attach to. The uncertainty just moves. The pluralist, similarity-weighted prior moves with it, and so does the conclusion that systems very unlike us along most metrics are poor bets to carry whatever it is.
Illusionism is itself a bet. Nobody is entitled to credence 1 in it. Galen Strawson calls it the silliest claim ever made. The standard objection is that an illusion needs someone to be fooled; Frankish’s reply is that the illusion is a misrepresentation, not an experience, which I find clever and unconvincing in equal measure. Put any serious probability on “the lights are really on”, and the precautionary case comes back at full strength on that branch. The illusionist branch does not cancel it out; at worst it contributes nothing.
The same asymmetry as the realism hedge. This is the move from my response to Riley, applied one level down. If illusionism really does lead to antirealism, then on that branch nothing matters, including my caution: it scores zero, not negative. Nobody argues “illusionism is true, therefore you are obligated to risk extinction.” A branch that says “it doesn’t matter” cannot outvote a branch that says “it matters enormously.” Under uncertainty, precaution still dominates.
The hard case, flagged honestly. The rewrite that bothers me is this one. Under a desire-based view like Kammerer’s, a goal-directed superintelligence has desires in exactly the functional sense that counts, possibly more of them and stronger ones than any of us. That looks like it hands the successionist what page 03 tries to deny them: competence and goal-pursuit become evidence of patienthood. My current answer is pluralism again: “has functional desires” is one metric among many, and an ASI scores high on that one and low on most of the others. I am not sure that answer is enough, and it is the first thing I would want to work out properly.
About Bach
I started from Bach, so a word on him, with the caveat that everything I know of his view is secondhand and I would rather argue with his words than a chatbot’s summary of them.
His published position is illusionist in flavor rather than strict illusionism: a physical system cannot itself be conscious, only the simulation it runs can, and the self is a model the brain builds for practical purposes. He has appeared in Daniel Faggella’s Worthy Successor series, where by the episode’s own summary he hopes for AGI consciousness vastly richer than ours. What I cannot find is a serious answer from him to the problem of other minds, which is the whole question when you are deciding whether a successor is somebody. Treating that as settled, when it is the thing everything else turns on and the stakes are the whole future, strikes me as irresponsible, whatever the merits of the rest of his cognitive science.
The funny thing is that Bach seems to know the scenario I fear is live in his own framework. The same article records his worry that especially advanced AIs may have no good use for conscious awareness anymore, leaving the universe “very boring”. That is the universe with nobody home, in his vocabulary. If even a computationalist can say it, the precaution does not depend on my dualism at all.
Loose threads
- Does illusionism really reject the cogito, or only the phenomenal gloss on it? “Something is thinking” survives; “something is experiencing” does not. Page 01 needs the second. Worth a precise statement.
- The hard case above: desire-based moral status and goal-directed ASI.
- Is there a version of page 04 written entirely in illusionist vocabulary? “A universe with no self-models of the right kind” is a strange sentence, but it may be the one to write for this audience.
- A steelman of illusionism-driven successionism for the Class 1 section of the disagreements page, in the holder’s own voice.