Long the domain of science fiction, researchers are now working to create software that perfectly models human and animal brains. With an approach known as whole brain emulation (WBE), the idea is that if we can perfectly copy the functional structure of the brain, we will create software perfectly analogous to one. The upshot here is simple yet mind-boggling. Scientists hope to create software that could theoretically experience everything we experience: emotion, addiction, ambition, consciousness, and suffering.

“Right now in computer science, we make computer simulations of neural networks to figure out how the brain works,” Anders Sandberg, a computational neuroscientist and research fellow at the Future of Humanity Institute at Oxford University, told Ars. “It seems possible that in a few decades we will take entire brains, scan them, turn them into computer code, and make simulations of everything going on in our brain.”

Everything. Of course, a perfect copy does not necessarily mean equivalent. Software is so… different. It’s a tool that performs because we tell it to perform. It’s difficult to imagine that we could imbue it with those same abilities that we believe make us human. To imagine our computers loving, hungering, and suffering probably feels a bit ridiculous. And some scientists would agree.

But there are others—scientists, futurists, the director of engineering at Google—who are working very seriously to make this happen.

For now, let’s set aside all the questions of if or when. Pretend that our understanding of the brain has expanded so much and our technology has become so great that this is our new reality: we, humans, have created conscious software. The question then becomes how to deal with it.

And while success in this endeavor of fantasy turning fact is by no means guaranteed, there has been quite a bit of debate among those who think about these things whether WBEs will mean immortality for humans or the end of us. There is far less discussion about how, exactly, we should react to this kind of artificial intelligence should it appear. Will we show a WBE human kindness or human cruelty—and does that even matter?

The ethics of pulling the plug on an AI

In a recent article in the Journal of Experimental and Theoretical Artificial Intelligence, Sandberg dives into some of the ethical questions that would (or at least should) arise from successful whole brain emulation. The focus of his paper, he explained, is “What are we allowed to do to these simulated brains?” If we create a WBE that perfectly models a brain, can it suffer? Should we care?

Again, discounting if and when, it’s likely that an early successful software brain will mirror an animal’s. Animal brains are simply much smaller, less complex, and more available. So would a computer program that perfectly models an animal receive the same consideration an actual animal would? In practice, this might not be an issue. If a software animal brain emulates a worm or insect, for instance, there will be little worry about the software’s legal and moral status. After all, even the strictest laboratory standards today place few restrictions on what researchers do with invertebrates. When wrapping our minds around the ethics of how to treat an AI, the real question is what happens when we program a mammal?

“If you imagine that I am in a lab, I reach into a cage and pinch the tail of a little lab rat, the rat is going to squeal, it is going to run off in pain, and it’s not going to be a very happy rat. And actually, the regulations for animal research take a very stern view of that kind of behavior,” Sandberg says. “Then what if I go into the computer lab, put on virtual reality gloves, and reach into my simulated cage where I have a little rat simulation and pinch its tail? Is this as bad as doing this to a real rat?”

As Sandberg alluded to, there are ethical codes for the treatment of mammals, and animals are protected by laws designed to reduce suffering. Would digital lab animals be protected under the same rules? Well, according to Sandberg, one of the purposes of developing this software is to avoid the many ethical problems with using carbon-based animals.

To get at these issues, Sandberg’s article takes the reader on a tour of how philosophers define animal morality and our relationships with animals as sentient beings. These are not easy ideas to summarize. “Philosophers have been bickering about these issues for decades,” Sandberg says. “I think they will continue to bicker until we upload a philosopher into a computer and ask him how he feels.”

While many people might choose to respond, “Oh, it’s just software,” this seems much too simplistic for Sandberg. “We have no experience with not being flesh and blood, so the fact that we have no experience of software suffering, that might just be that we haven’t had a chance to experience it. Maybe there is something like suffering, or something even worse than suffering software could experience,” he says.

Ultimately, Sandberg argues that it’s better to be safe than sorry. He concludes a cautious approach would be best, that WBEs “should be treated as the corresponding animal system absent countervailing evidence.” When asked what this evidence would look like—that is, software designed to model an animal brain without the consciousness of one—he considered that, too. “A simple case would be when the internal electrical activity did not look like what happens in the real animal. That would suggest the simulation is not close at all. If there is the counterpart of an epileptic seizure, then we might also conclude there is likely no consciousness, but now we are getting closer to something that might be worrisome,” he says.

So the evidence that the software animal’s brain is not conscious looks…exactly like evidence that a biological animal’s brain is not conscious.

Virtual pain

Despite his pleas for caution, Sandberg doesn’t advocate eliminating emulation experimentation entirely. He thinks that if we stop and think about it, compassion for digital test animals could arise relatively easy. After all, if we know enough to create a digital brain capable of suffering, we should know enough to bypass its pain centers. “It might be possible to run virtual painkillers which are way better than real painkillers,” he says. “You literally leave out the signals that would correspond to pain. And while I’m not worried about any simulation right now… in a few years I think that is going to change.”

This, of course, assumes that animals’ only source of suffering is pain. In that regard, to worry whether a software animal may suffer in the future probably seems pointless when we accept so much suffering in biological animals today. If you find a rat in your house, you are free to dispose of it how you see fit. We kill animals for food and fashion. Why worry about a software rat?

One answer—beyond basic compassion—is that we’ll need the practice. If we can successfully emulate the brains of other mammals, then emulating a human is inevitable. And the ethics of hosting human-like consciousness becomes much more complicated.

Beyond pain and suffering, Sandberg considers a long list of possible ethical issues in this scenario: a blindingly monotonous environment, damaged or disabled emulations, perpetual hibernation, the tricky subject of copies, communications between beings who think at vastly different speeds (software brains could easily run a million times faster than ours), privacy, and matters of self-ownership and intellectual property.

All of these may be sticky issues, Sandberg predicts, but if we can resolve them, human brain emulations could achieve some remarkable feats. They are ideally suited for extreme tasks like space exploration, where we could potentially beam them through the cosmos. And if it came down to it, the digital versions of ourselves might be the only survivors in a biological die-off.

What makes us special?

Of course all of this is moot if we never get to this point with the technology. So let’s get back to the questions of if and when.

For many, the idea of successful brain emulation is so strange, so far beyond our normal experience, that it feels best to dismiss it. While Sandberg sees his work as an effort to prevent suffering at a future time, others might ask why he wastes intellectual capital worrying about an event that will never occur. Software cannot experience emotion or become conscious, critics say, or it will at least need more than just a digital copy of the brain to do so. There is *something* else in there that makes us special, right?

Many scientists say no.

“Nearly every honest scientist in the neuroscience community will, upon closer scrutiny, admit that they do believe it is ‘in principle’ possible to emulate a brain’s functions,” says Randal Koene, neuroscientist and neuroengineer. “To claim otherwise would be to claim that there is some kind of non-physical magic at work in living brains, and scientists who tend toward that sort of thinking are exceedingly rare these days.”

Koene is one of a group of people who came up with the term “whole brain emulation.” He can envision a day in which we have software brains as capable and conscious as our current biological ones. And Koene agrees with Sandberg that one day software will have the capability to suffer. “If you have a software implementation of a whole brain emulation, then I’m pretty confident that the mind produced by that software is able to suffer just as you or I could. I think it’s quite possible that other types of software will also be able to suffer, some that are based on animal or artificial intelligence designs,” he says.

Kenneth Hayworth, a neuroscientist and president of the Brain Preservation Foundation, also sees no reason why a digital brain should be somehow less than a real one. And to those who would argue that these digital uploads of a person’s mind are merely copies, Hayworth suggests considering what type of thing a person is to start. “We have discovered through cognitive science and neuroscience that we are like a program; we are like a data file on a computer in the sense that the information that makes us unique is the only thing that is truly us.”

(It should be noted that Hayworth isn’t too worried yet about the legal and moral status of conscious simulations. “This will not happen tomorrow,” he says. “I would hope that in 50 to 100 years that we would have gotten our act together such that [emulations] would have full rights.”)

A notable voice of dissent on the prospect of WBEs is Duke Neuroscience Professor Miguel Nicolelis. Nicolelis has made headlines for his lab’s work with brain-machine interfaces and primate neuroprosthetics. In 2013, he was quoted in MIT’s Technology Review as saying, “Downloads will never happen… There are a lot of people selling the idea that you can mimic the brain with a computer… You could have all the computer chips ever in the world and you won’t create a consciousness.”

When asked for a comment today, Nicolelis simply responded, “I stand by my previous comments: simulating the work of a human brain in a digital machine is impossible.”

Even if Nicolelis is rare in his belief that we won’t be able to mimic a brain on a computer, there are more widespread concerns that the prospect of WBE has been oversold. The Human Brain Project, for example, is an attempt to create a brain-like system of computer chips, funded by the European Union. But that project has come under fire due to allegations of mismanagement and lack of realistic goals. But the problems may be much deeper, as some scientists say the concept was premature from the start and was sold to government agencies that were ill-equipped to evaluate it.

Let’s pause for a moment to remember that we still do not have a full understanding of our own human consciousness. At exactly what point do humans pass from living to dead? We don’t know. Where does our feeling of consciousness originate? It’s under debate. Why do some people wake up from anesthesia during surgery, and how does the brain orchestrate this awakening? Surgeons wish they knew.

Unlike the other scientists quoted, Alice Parker does not self-identify as a futurist or trans-humanist. But the University of Southern California professor is known for her opinions on the feasibility of WBEs, and she leads a project to reverse engineer the brain using analog computations. Parker is cautious about the idea—“We are a long way away,” she insists—because she believes our understanding of the brain is still so rudimentary. “There are mechanisms that seem to be critically important that we are just discovering… We who are building electronic circuits are following neuroscientists and mimicking whatever we can with the sense that things are emerging every day and you never know when there will be a complete paradigm shift as a result of new information,” she says.

Parker softens these statements by noting that neural networks designed to perform specific functions, such as “recognizing cats in an image,” are coming along nicely. But broader abstract reasoning by a computer, Parker estimates, will take decades. Unlike Nicolelis, she still believes it is possible. “If we fully understood the neural mechanisms, biological mechanisms, it is possible to have cognitive processing that appears to be able to do abstract reasoning,” she says. “I think when we are at that point when there seems to be awareness, then we need to have that discussion about the way these organisms should be treated.”

On the issue of whether consciousness would emerge from a WBE, Parker says that no one really knows, and opinions differ among even her most highly prestigious colleagues. To her, even Hayworth’s assertion that the brain is like a program feels too light. “I think it’s so much more complicated and it’s so much more multidimensional, and there’s so many more subtleties,” she says. “[To say a brain is like a program] is too simplified. It’s breathtakingly complicated.”

Starting small

Today we are still quite far from conscious WBEs, even for the simplest animals. For all we know about the world around us—how distant planets move and change, how chemicals combine and separate, how continents break apart and rise up—it’s surprising how little we know about the human body and, within our vast ignorance, the human brain.

But we are learning. These efforts are motivated by plentiful funding through sources like the BRAIN Initiative in the US, the Human Brain Project in the EU, and prize money offered by private organizations. Beyond monetary incentives, these researchers are motivated by a mix of both intense scientific curiosity and the drive for both ephemeral and true immortality. “I want to be part of it. I want to make it happen. Even if I don’t get there,” Hayworth says.

One of the biggest obstacles is our lack of knowledge about the “connectome”—the brain’s wiring diagram. Scientists still do not have a comprehensive map of the brain’s complex network, and they still don’t understand how and why the parts connect. Even here they are making some amazing progress, and the brain initiative being promoted by the Obama administration promises to push that progress further.

If we want to simulate an entire organism now, however, some suggest starting small. In theory, pick an organism where the connectome is possible to map and we can work up. Open Worm is an open source distributed labor effort to model C. elegans, one of the most studied organisms on earth. C. elegans is a nematode with exactly 302 neurons. The model mirrors the worm on the cellular level, and it has grown in complexity as its creators added components and their connections (recently the team announced some success in mimicking realistic muscle movements). In terms of creating an accurate emulator, Open Worm also faces competition from other teams hoping to digitalize this model organism of choice.

Others are starting at the top and working down. Hayworth leads an effort to digitize the brain by preserving it in a hardened polymer and removing slices of it layer by impossibly thin layer. Each new surface is imaged by an electron microscope and digitally added to the slice prior to creating a 3D rendering with 5x5x5 nm resolution, “which really allows us to see everything.” Hayworth says. “You want to count synaptic vesicles, you want to see in great detail. You can see all of that.”

Unfortunately, work like this is “very very slow,” according to Hayworth. His team is currently working on the brain of a fruit fly. Imaging and recording the whole thing, he says, takes about a year. Even with the best technology, this high-resolution approach completes about a cubic millimeter a year. Other labs are hard at work imaging the brains of mice and humans on longer time tables (5-10 years for a completed image), lower resolutions, or smaller pieces. Still, Hayworth remains optimistic. “It would be hard for me not to believe that we will be doing a whole human brain this century and that we will be able to understand it at the circuitry level,” he says.

The researcher argues that if we can do that, we can read off the circuitry that encodes memories, likes and dislikes, behaviors, and personality. He believes that if we can preserve the brain’s circuitry at this level of detail, we can preserve the essence of the person. Even if we don’t figure out how to create WBEs for another 100 years, Hayworth wants us to preserve our brains (through cryonics or chemical preservation) until we do.

If this vision of how to understand an individual’s brain is wrong, it’s still possible that we could eventually manage to achieve the dream of many futurists and perform the equivalent of a download of an individual’s thoughts, feelings, and memories. To do anything with that, however, we’d also have to upload it—and we’d upload it into a WBE.

So perhaps the answer to why we should care whether the earliest emulations suffer is that we might eventually become emulations ourselves. The idea here is that concern for suffering software may stem from pure compassion, but there is a selfish element to these questions. It would be the worst kind of irony to be immortalized as an emulation only to have our own cruelties thrust upon us. Essentially, we may need to worry about suffering in software because the software may become us.

Human-emulation interactions

Unless we endure a great die-off and stagnant fertility, there is likely to be a period of human emulations and humans coexisting. What will become of these humans? Emulations, once sufficiently advanced, have the potential to think, reproduce, travel, and adapt, faster and with fewer resources. They will be in a sense super-human, and we humans could be lesser, animal-like creatures.

How they treat us will likely depend on how we’ve developed the emulations in the first place. If we show unkindness, if we withhold pity and disregard pain, the software may mirror these lessons back to us. Children who are treated with cruelty often become cruel. But if we strive to reduce the suffering in this new form of consciousness, if we instead demonstrate empathy, these are more likely the lessons that will be learned. There is no guarantee, of course. Kindly treated children sometimes become coldhearted, but it makes sense to try to stack the deck in our favor. It’s new characters in the same old story: our actions affect the outcomes of our descendants.

Contemplating such scenarios is the specialty of Robin Hanson, an economics professor at George Mason University who is working on a book on WBE. Hanson uses his background in physics and economics to imagine the consequences of future technologies. He believes the rise of emulations that exceed the capacity of human brains is inevitable, and the social consequences that follow will be par with the beginning of human agriculture.