In November 2019, Leon Kowalski found himself in the offices of a large corporation in Los Angeles, answering some odd questions. “You’re in a desert. You look down and you see a tortoise…” When the questioner moved on to ask about his mother, things didn’t end well.
You might remember seeing this – but not in our world. It is the opening scene of Ridley Scott’s Blade Runner, released in 1982 and now no longer set in the future. Leon is a replicant, and the questioner, from the Tyrell Corporation that created him, is using the Voight-Kampff Test to distinguish him from a real human. It’s a fictional modification of the Turing Test, first proposed (as the Imitation Game) by British mathematician Alan Turing in 1950 to study the question of whether machines can “think”. The best way, Turing suggested, would be to interrogate the machine as if it were human and observe how it behaved.
“Machine behaviour” is the next frontier in artificial intelligence (AI), bringing together computer scientists with neuroscientists, developmental psychologists and social scientists. It implicitly recognises that we’re already not sure what kind of reasoning our current AI uses, or how it relates to our own. The issue is no longer if a machine thinks, but how.
A better understanding of the “machine mind” might be essential for making trustworthy AI: medical systems less likely to misdiagnose patients, or self-driving cars less prone to lethal errors of judgement. It might let us anticipate how our own interactions with such computer systems will evolve. And it might even give us insights into our own minds.
It’s a stretch to attribute “mind” in the human sense to the AI systems we have now. Even the most sophisticated have no more self-awareness than your satnav or Amazon’s recommender systems; they are just algorithms designed to turn input signals into outputs by data-processing in silicon circuitry. The AI that beats the best human players at games like Go and chess, or that makes medical diagnoses based on physiological data, uses the technique called deep learning.
This is crudely modelled on the human brain, in that it uses networks of silicon-based “neurons” or nodes – logic devices that output electrical signals in response to inputs – to make decisions.
The strength of the connections between nodes is adjusted during a “training” phase until a particular type of input can reliably produce the right output. This is the “learning”. For example, the neural network could be trained to respond to digital images of cats with the output “cat”, while images of anything else will deliver the output “not cat”.
Neural networks (then called perceptrons) were first proposed in the 1960s, but only in the past few years has their performance become reliable enough to be of much value. That improvement required the inclusion of more “layers” of nodes: this is what makes the learning “deep”. It’s thanks to deep learning that AlphaGo, an AI algorithm created by DeepMind (now a subsidiary of Google’s parent company Alphabet Ltd), has been trouncing grandmasters of Go since 2015, and that algorithms such as Google Translate now generally deliver trustworthy text rather than comical gibberish.
Yet we don’t really know what rules underlie the machinations of the network. Sometimes they seem bafflingly perverse. Occasionally, for example, image-recognition AI will make errors, such as mistaking a banana for a toaster, that even a toddler would find silly. Worse, it is possible to fool the systems with “adversarial” images – to tweak the pixels of a puppy, say, so that while to the human eye the image remains unchanged, the AI will classify it as an ostrich. Or we can construct seemingly random pixel patterns that will be confidently identified as ducks.
“AI is kind of dumb in a lot of ways,” says David Cox, director of the Massachusetts Institute of Technology (MIT)-IBM Watson AI Lab in Cambridge, Massachusetts. “It’s not hard to find the gaps.”
Whatever representation of “duck” the AI is learning in this example, it is not like ours. Some classifications might have nothing to do with the main object – for example, the system might decide that all images with blurry backgrounds are animals, simply because that was the case in most wildlife photos used to train it.
It’s not the end of the world if an AI mistakes a dog for an ostrich. But it’s a lot more serious if a medical diagnostic AI such as IBM’s Watson Health mistakes an image of a tumour for a healthy organ. As ever more decisions and diagnoses are handed over to AI, we need to know how it is thinking.
Machines like me: in the opening scene of Blade Runner a non-human replicant is exposed by a test. Credit: TCD/Prod.DB / Alamy
The way AI acquires its skills today – through training and learning – is remarkably close to the way a child learns. Alan Turing predicted this: “Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child’s?” He added: “If this were then subjected to an appropriate course of education one would obtain the adult brain.”
During the early years of AI, few people thought that this was the way to do it. Most researchers focused instead on so-called symbolic AI, in which clearly defined decision-making rules were built into the system by design. That, however, tends to be too rigid. To make a cat-recognising AI, you could spend a lot of time and effort programming in the concept of (say) pointy ears, only to have the algorithm fail the moment a cat is shown from a perspective where the ears aren’t obvious. Deep learning gains in versatility, but at the cost of relinquishing knowledge of how it reasons.
If we can figure out what “cat” really means to an image-classifying AI, we might be able to persuade it that a field of random pixels really isn’t remotely cat-like. The problem is that we still don’t fully understand what “cat” means to a four-year-old child either – even though she can reliably identify as such the family pet, Tom from Tom and Jerry, and the silhouettes of Thomas O’Malley and Duchess on the Paris skyline. What’s more, a child doesn’t need to be trained with a thousand images of cats of all shapes and sizes before she can do this.
It’s not that we humans need less data, says Cox, but that we use different data, and more smartly. “There’s an almost frightening amount of information we bring to the table even when we solve a very narrow, specific task,” he says. “We have all this unspoken, base-level common-sense knowledge that we learn through experience, curiosity and being embodied in the environment.” But, he adds, “A lot of common sense is unspoken and unwritten – we take it for granted.”
The US defence department’s research agency Darpa recently launched a project called “machine common sense”, which states that “the absence of common sense prevents intelligent systems from understanding their world, behaving reasonably in unforeseen situations… Its absence is considered the most significant barrier between the narrowly focused AI applications of today and the more general, human-like AI systems hoped for in the future.” But it’s going to be hard to give AI common sense when we don’t really know what that is.
Although the military has many, perhaps troubling reasons to want better AI – for face recognition or more accurate drone strikes, say – it’s not unusual for Darpa to invest in basic research, and this is just one component of a $2bn AI initiative. One striking aspect of the Darpa programme requires every team funded by the scheme to include child psychologists. Yet even they don’t claim to know how children learn. “There is no one dogma about it,” says Harvard psychologist Tomer Ullman. “We all look at a dog and we know it’s a dog but we don’t know how we know it’s a dog.”
Ullman, a cognitive scientist who researches how reasoning develops in children and adults, exemplifies the growing trend to study artificial intelligence behaviourally, as though the machine were some kind of exotic new creature with a mind of its own. “The questions people are asking in child development are very much those that people are asking in AI,” he says – questions such as “What is the state of knowledge that we start with, how do we get more, what’s the learning algorithm?”
Children don’t simply learn a set of rules from one experience which they then blindly apply to the next. To navigate the world, we need to be able to intuit some general principles about how things and people behave. Children develop expectations, for example, about what happens when things are dropped on the floor: some go splat, some bounce or shatter. In other words, they develop an intuitive physics – which is far from infallible, but better than guessing from scratch how events might unfold.
And they develop, too, an intuitive psychology: expectations about what people do. We associate actions with motivations and sympathies, even in very abstract scenarios. Show a ten-month-old, pre-verbal infant two inanimate objects moving as if they are “behaving” in a particular way – a sphere seeming to either help or hinder a cube’s progress up a slope, say – and she will very probably then show a preference to play with a “helpful” object.
It stands to reason that, for our own survival, we’d have minds attuned to distinguishing such behaviours. But what program in the infant’s mind supplies the criterion for “helpful” behaviour? How do we even know that she interprets the behaviour in terms of “helping”? And if we figure that out, can we build it into a machine?
An ability to make distinctions like this could be crucial to truly smart AI, particularly in situations where it needs to interpret and anticipate human behaviour. We drive differently when we see a grown man striding along the pavement than when we see a small child running. But it’s rarely as simple as programming a self-driving car to go slower if it detects a child-sized being nearby (even assuming it could do that).
It’s less easy than you might think to program responsible or ethical behaviour into AI, because our decisions so often depend on context and the need to adapt to circumstances. Isaac Asimov’s seminal collection of stories, I, Robot, made precisely that point. Published in 1950, these tales explored how three laws designed to keep robots safe – to avoid harming, to obey humans, and to preserve themselves – could get undermined by the messy complexity of real life. In AI just as in bureaucracy, rules can go badly awry if applied by rote rather than with human understanding and empathy.
One approach is to make the governing principles more detailed: to program every eventuality explicitly into the machine. But that’s an impossible task, because in our everyday experience virtually every scenario is unique. Many AI researchers now believe that to improve the performance of today’s deep-learning systems we will need to imbue them with “representations” of the world: models that describe how things and people tend to behave. Only then will they be able to understand and respond to the kinds of demands we routinely make of other people – so that, for example, if you ask a robot to give you your cup of coffee, it knows you want it right-side up and handled with considerably more care than a pen or a magazine.
What are the rules of our own intuitive psychology? This is what Blade Runner’s Voight-Kampff Test – introduced in Philip K Dick’s 1968 novel Do Androids Dream of Electric Sheep? on which the film was based – was really getting at. Does the replicant have a good enough internal model of a human mind to be able to emulate it? (Turing himself never meant his Imitation Game to be a practical proposal for distinguishing “machine” from human. Rather, it was meant to challenge our preconceptions: if a machine were perfect at playing the Imitation Game, on what grounds could we deny that it “thinks”?)
Tomer Ullman and his colleague John McCoy have devised a stripped-down version of the Turing Test designed to reveal the kinds of mind that we think we and AIs have. And it’s as brutal as Blade Runner. You and the AI (which might look just like a human) face a panel of human judges, who have been told that one of the two of you is a machine. The panel must decide which is the robot – and destroy it. The catch is that you are allowed to say just one word each to try to prove that you are the real human.
Which word do you choose?
The aim is not to put us in some fiendish predicament and see if we can “beat the machine” in order to escape it. Rather, Ullman’s test is a way of examining our assumptions about the cognitive algorithms that distinguish machine from human. There is no “right” word to choose, but some are more likely than others to fit with what the judges think a human and a robot would choose. “In order to pass this test, you need to have something like an intuitive theory of the robot mind,” says Ullman.
He and McCoy tested people to see which words they would, as judges, consider most indicative of humans. The best words were obscenities, although fast foods were good too: here’s one situation where pizza and burgers might save your life.
***
Our abstract notions about machines’ minds are likely to influence the ways we use and interact with them. This applies in situations ranging from trading “bots” operating in financial markets to dating services and robots used for the care of the elderly and children. Worrying whether your kids ought to be polite to Alexa is just the tip of the iceberg.
In April a group of researchers in fields ranging from computing science to ecology and psychology published a paper in Nature stressing the importance, even urgency, of recognising the emerging discipline of machine behaviour. It’s far from ideal, they said, if this issue is explored only by those who build the machines – computer scientists and robotics engineers who typically have no training in understanding behaviour or social sciences.
The paper’s primary author was Iyad Rahwan, a Syrian scientist who has just departed MIT’s prestigious Media Lab to head up the Center for Humans and Machines at the Max Planck Institute for Human Development in Berlin. He calls the paper “a call to arms”. “To be able to live in the world with these increasingly sophisticated AIs,” he says, “we need a behavioural science of machines as much as an engineering science of machines.”
Rahwan explains how easily AI systems designed for seemingly innocuous purposes could end up perpetrating unethical behaviour. “Let’s say you ask the machine to sell more IVF [reproductive] services. The machines could learn all kinds of unexpected, potentially unethical strategies to achieve those goals – like, let’s encourage people to delay having children because then maybe they’ll need IVF in the future. Maybe we should give them more deals on holidays? It’s not far-fetched for an algorithm tasked with that goal to learn this strategy. It has to be a very villainous human to think of something like this. But the people who are building these systems wouldn’t even know. So unless we build a science of the behaviour of the machine, we’re not going to be able to keep these things under control.”
AI pioneer: the computer scientist Alan Turing. Credit: Fine Art Images/Heritage Images/Getty
In short, we humans need to get to know our machines – and perhaps we need to design them in a way that allows us to understand them. “People are going to have to change the way they work to incorporate AI,” says Cox. “For that to happen, the technology has to meet them halfway – they have to be able to understand why the system is telling them to do things.” Doctors, for example, should demand from an AI diagnosis not just a recommendation for intervention but a justification based on cause-and-effect reasoning. “It’s absolutely imperative that we have that explainability,” says Cox.
Without such transparent logic, he says, the real danger of AI is not that it will develop megalomaniac tendencies like Skynet in the Terminator films, but that it will be used in inappropriate ways that ignore its limitations. “We’re going to see lots of Wild West applications,” he says, pointing to a court in Florida that used AI to make decisions about parole. For all the talk of objectivity, the system was just learning from the past data, with all its inherent biases. In 2016, Microsoft’s Twitter chatbot Tay began emulating racist and sexist trolls within hours of its release and had to be shut down. Today’s deep-learning AI “is guaranteed to take the past and give it back to you”, says Cox.
How machines behave might influence how we do, too. Rahwan and his colleagues recently looked at how cooperation can develop between humans and machines, using an algorithm that could signal its intentions and goals by using threats or offers. It was able to achieve cooperation on a par with what humans alone attain – but only by using more threatening strategies.
“If machines are more vindictive, I may have to develop new norms to cope,” says Rahwan. “Will this then impact the way I interact with other humans? Is there some kind of behavioural contagion? We have no clue.” But there are already some indications, he adds, that “children who interact with chatbots such as Alexa start using more imperative language with other children – ordering them rather than asking them politely”.
We will also have to ask whether AIs in human form – the Leon Kowalskis of the world – should be compelled to disclose their identity to us or not. What if tricking us into thinking that an AI is human actually leads to better interactions, Rahwan asks. He and his colleagues recently tested how people and AI bots perform together in the “Prisoner’s Dilemma”, a classic scenario in game theory that can create cooperation between two players. Here the players are offered the choice of cooperating with one another or not – with various pay-offs linked to those choices. If just one player chooses cooperation, the other player gets the biggest reward. If both cooperate, they each get a moderate pay-off; if they both refuse they get nothing. People are more inclined to cooperate, the researchers found, if they think they’re playing another human rather than a bot. So should we forgo that potentially beneficial outcome for the sake of transparency?
The dilemmas of machine behaviour represent one of the most striking examples of Marshall McLuhan’s dictum that “We shape our technologies, and then our technologies shape us.” While we lack any grasp of the type of reasoning at work in AI, we’re likely to indulge our habit of anthropomorphising, of projecting human-like cognition into places it doesn’t exist – witness already how we curse our computers and satnavs for their perversity and uncooperativeness.
But it’s one thing to attribute mind to dumb matter. It’s another, perhaps more dangerous thing to attribute the wrong kind of mind to systems that, in some sense, genuinely possess it. Our machines don’t think like us – so we’d better get to know them.
Philip Ball’s most recent book is “How To Build A Human” (William Collins)
Content from our partners
Related
