Are the Builders to Blame?
If building the frontier ought to be a crime, what are the people building it today?
A question I have been putting off.
This site argues that developing frontier AI risks human extinction, and that it should be made illegal. But it isn’t illegal. Thousands of people go to work every morning and do it, openly, for good salaries, with the blessing of their governments. Should we hold them morally responsible for what they are doing?
The legal answer is easy. You cannot punish someone for an act that was lawful when they did it, and any law I would want should say so plainly. The moral question does not come with that exit.
Here is the case for yes. Most of these people are not naive. The risk has been discussed in public for over a decade, often by their own employers and colleagues. “It was legal” has never been much of a moral defence on its own, and “if I don’t, someone else will” describes nearly every harm done by a crowd. If you believe, as many of them say they do, that there is a real chance this ends everyone, then continuing to build it looks like a choice you own.
Here is the case for no, or for not much. Moral blame usually tracks what a reasonable person could have known, and reasonable, informed people disagree sharply about the risk. Someone who honestly thinks the danger is overstated is not in the same position as someone who thinks it is serious and goes in anyway. Most of the work is also divided so finely that no one engineer’s contribution is decisive. And a movement that brands a whole profession as wrongdoers mostly teaches that profession to stop listening.
There is a practical side too. A regime like the one the gist describes works best when the people inside the labs see the exit as open and the law as fair. Condemning them in advance makes that harder.
So I have not settled it. My tentative view is that responsibility is real but uneven: heaviest on those who set direction and believe the risk is serious, lightest on those who do not and have no say. That is still a claim about roles, not about anyone by name; the non-targeting policy holds here as everywhere else.
If you work at a frontier lab and have thought about this, I would like to hear how you answer it.