Charles Stross: Individuality, Obedience, and Atrocity

I’d like to start by saying that Ken MacLeod is very sorry he couldn’t be here to accept the award for the novel he didn’t write this year. So he sent me instead, just to continue the tradition of Scottish socialist excellence in libertarian science fiction. Having said that, I was slightly surprised to win the award. Having seen the long list of nominees I am very surprised to win this award, because there is some excellent fiction on the list.

I think though in retrospect I can see why Glasshouse was of interest to the Libertarian Futurist Society, because while the politics in it is implicit rather than explicit, the focus of the novel is very much on issues of individual responsibility and identity, and how to behave in a coercive society.

One of the things that has fascinated me for some time is the study of human psychology, and in particular human obedience. What makes people behave in social contexts and carry out atrocities, or actions of great evil. For a long time we tended to believe, historically, that you can explain evil acts by assuming that the perpetrators of the acts were themselves evil. However, the first cracks in the wall, in the belief in absolute evil and the evil of evil-doers—I’m sounding like George Bush here—began probably in the 1940s, when there was an increasing recognition that many of the people who had been complicit in the worst crimes of the Nazis, for example, were in fact perfectly ordinary people who had somehow become trapped in a spiral of very extra-ordinary violence.

I’d like to mention the work of a pioneering experimental psychologist, Professor Stanley Milgram, who in the late 1950s and early 1960s conducted some experiments, which have been verified since then; they have been repeated but can no longer actually be carried out ethically. Stanley Milgram wanted to investigate whether or not you can make perfectly ordinary people commit atrocities. He came up with an interesting mechanism for testing this. The way Milgram’s experiment worked, if I can just give a brief run-down of it, was you take people off the street and tell them, “We’re going to pay you $25 to participate in a psychology experiment.” They’re introduced to the experimenter who’s wearing a white coat and carrying a clipboard, and is very much an authority figure. They’re ushered into a room where there’s an imposing piece of machinery with a button and dial, and there’s a screen with a room behind it.

The experimenter explains to the subject, “In that room there’s a fellow we’re conducting an experiment on. We’re going to ask him a series of questions, and when he gets them wrong, we want you to push this button which will administer an electric shock. We’re experimenting on human motivation. As we turn this dial up, the severity of the shock will increase.”

Now, what was actually happening here, is that a situation has been set up. The person on the other side of the screen is not being given an electric shock but is in fact an actor who is feigning pain. They’re being asked some questions and they're getting some of them wrong. Each time they get one wrong, the voltage on the apparatus, the alleged voltage on the apparatus, is increased. Their screams and protests increase as well. The purpose of the experiment was to find out at what point these ordinary people, these men off the street, who have come in to do the job, will start to protest and complain to the experimenter: “You know, you’re hurting this man. You can’t do this.”

The scariest finding of all, was that perhaps 35 to 40 percent of the people they ushered in off the street would continue to administer electric shocks in accordance with the instructions of the experimenter, to a point where they were told that it was dangerous or possibly fatal to the victim, the actor. Even more interestingly, the only correlation they found in this, was that the more educated the experimental subjects, Harvard undergraduates, were more likely to continue obeying the instructor, than ordinary working class men.

So, the first thing, the very important lesson that Milgram deduced, was that people will tend to obey instructions from authority figures, even when these instructions are quite manifestly wrong; making them do something that is quite bad; inflicting pain on another person, for no obviously apparent gain.

But, while Stanley Milgram’s experiment goes some way to explaining how you can co-opt ordinary people into behaving in evil ways, there are some even more powerful and scary social psychology experiments. One of the other ones I’ve been quite fascinated with, which fed into the writing of the book, Glasshouse, was conducted by Professor Philip Zimbardo, of Stanford University. He’s an eminent psychology professor, and I believe he was lately called in to testify as an expert witness in the Abu Ghraib prisoner abuse trials, in the States. Now, he conducted an experiment in the early 1970s, which was cancelled two days in—it was terminated—of a five-day run. More recently, a British television documentary attempted to replicate the experiment. Their ethics oversight, again, had to cancel it after just two days. It’s a very disturbing experiment, because what it indicates is not only that people will obey instructions to commit bestial acts, but certain types of social organizations are actually toxic. That people placed in certain situations will behave increasingly abusively. Not necessarily because they are evil, but because their understanding of the situation they’re in leads them to believe in certain ways, from which abuse emerges.

Now the Stanford prison study was an interesting one. Zimbardo was interested in investigating how conditions of abuse arise in prisons. So he set up a small prison with video cameras monitoring what was going on. They then picked up a bunch of volunteers, students who’d volunteered to spend two weeks living in a psychology experiment. His volunteers were randomized into two groups. One group were rounded up by police, arrested, given a prison uniform and then placed into cells. Another group—and I stress, these men were assigned to these group purely by random—were taken into the police station, issued with mirrored sunglasses and uniforms, and told they were prison guards. The striking and frightening thing about this particular experiment, where the guards were set to guard the prisoners, is that some of the worst patterns of abuse of prisoners by prison guards that are observed in real world prisons, began to emerge within 24 hours in a psychology experiment.

The experiment was terminated when Zimbardo’s wife, who was another faculty member took one look at what was going on and said, “This is illegal. You could be charged with assault for this.” There were some questions of serious psychological damage to the prisoners. It turns out that certain types of social organization, particularly the prisoner one, where you have the paradigm of guards with authority and prisoners who have been deprived of individuality—they were identified only by numbers, then had been deprived of privacy—this sort of power-relationship imposes behavioral constraints on human beings that drive them very strongly to behave in certain ways.

Now I began looking at this and thinking, to what extent is our behavior actually a product, not of our intentions or desires, but merely of the society we find ourselves in. So that was one of the germs of the novel, Glasshouse. Another—I think I can confess at this point to being a science fiction fan, would anyone be surprised? In particular I am a fan of the American author, John Varley. I’ve been waiting with some interest for a novel of his, Steel Town Blues, which had been pre-announced sort of seven years earlier to come out. And, in 2003, much to my disappointment, his newest science fiction novel, Red Thunder, emerged. Rather than being the novel I had been hoping for and expecting for seven years, it was a Heinlein juvenile. Now it was admittedly a good Heinlein juvenile, but it wasn’t a John Varley novel.

So I decided, “Damn it, I’m going to write a John Varley novel.” One of his tropes is societies in which biological engineering and technology are so advanced that people’s physical morphologies change. You can decide to go and have a sex-change over the weekend because you got bored being a man or a woman and wanted to try out the other side for a bit. Or, two arms? Not enough. I need another two arms. Go to the body shop and buy some extra arms. His works, the Eight World novels, are set in a universe sufficiently far enough away from ours, a few hundred years in the future, that this is unexceptional. That the sense of physical identity and consistency that we have—you know, we are born into a given body and we live through it and we age and we die, but we don’t change sex or skin color, or number of arms in which this fluidity of identity has been normalized in Varley’s work. And I decided, if you have a universe like this in which people are what they want to be, rather than what they are born as, what happens if you start imposing constraints along the lines of Zimbardo’s prison study?

In fact, the original crude idea for what I was going to write in Glasshouse, was: take the Stanford prison study protocol, and apply it to gender roles, to relationships between the sexes. You take a bunch of post-humans and randomize their identity as men and women in a bizarre parody of 20th century middle America. As one friend of mine put it, “you invented Desperate Housewives meets The Prisoner in deep space.”

Along the way, various other things began to occur to me. I’d sort of inverted the Stanford prison study protocol, and the idea of determinism in social relationships and how our societies affect our behavior. I’d come up with a society in which identity is somewhat fluid, and this society was emerging from a very bloody civil war. Because once you have fluidity of shape and of memory and identity, you come up with the potential for entirely new types of tyranny.

Human history so far has one very consistent constant, which is that tyranny stops at your nose. A dictator or tyrant or king can make people say things in public and do things in public but they can’t affect what their subjects think about them. One of the things in Glasshouse is that if you contemplate the possibility of editing people’s memory, you bring about the potential for a cognitive dictatorship, one in which people’s memories can be forcibly edited by the government. And that has a lot of very, very unpleasant side effects.

In Glasshouse the polity that has emerged after a very bloody civil war, they know they have won the war, but they’re not sure why the war was fought in the first place. All they know is that somebody really wanted to censor something from their history so that nobody knew what it was. And to that extent it appears to have been successful. Our protagonist has been injected into effectively a rehabilitation center or prison camp for war criminals, and ultimately it’s a hall of mirrors. We are not sure whether our protagonist, Robin, is actually a war criminal, or an investigator looking into what is happening inside the rehabilitation center. All we know is that Robin committed terrible acts before the novel begins, and at the end becomes somewhat reconciled to them. Because another side effect of being able to edit our morphology and our memories, is we can ultimately regain some aspect of control of our role in society.

If society dictates to some extent what we do, and if our position in society constrains our behavior, if you then add the ability to change our physical form and our identity, you can also change the niche into which you slot. So there is to some extent a trade-off here, and I was trying to, I guess, get to grips with the interplay between identity and responsibility in a social context in this book. At this point I’m beginning to run out of things to say about it, except thank you very much indeed for the Prometheus Award, and I’m, well, deeply flattered and somewhat taken aback.

Thank you very much indeed.