The advert for Apple’s latest iPhone is one of the most disturbing things I’ve seen in a long time. Superficially it’s a cheery little scene: a family gathers around the grave of a recently deceased goldfish, which the father makes a clumsy attempt to eulogise. He gets its name wrong and struggles to explain what made the fish special. His young daughter looks on sadly, but her older sibling – played by Bella Ramsey of Game of Thrones and The Last of Us – opens the Apple Intelligence app and types “Kristy with her fish, sad vibes”. Cutting their father off, they put the phone on top of the animal’s grave and a slideshow of pictures of the young girl and the fish, set to a slushy rock song, begins to play on the phone’s screen.
A real father would want to help his daughter grieve. He would tell his oldest child that it is crass to interrupt the funeral of a beloved pet with their phone. But no: he is shown quietly mouthing his thanks to Bella and the machine intelligence they have summoned. No one has to care what the little girl feels as she settles down by the grave, hypnotised by the screen. The awkward business of feeling something has been handed off. Her emotions have been fed to the iPhone 16 Pro (available now, from £999).
Will the phone keep her entertained and distracted through every difficult scenario? Will she, as an adult, peacefully settle in front of a screen into which a former lover has typed “dump Kristy, sad feels, make me look good”? When her inattentive parents finally die, will the iPhone 57 Pro seek to persuade her that they too were just on-screen content?
Yes, probably, because the generative AI revolution will not stop at fish. There are already companies that offer a machine that mimics the communications of dead people – these models are chillingly referred to as “griefbots” – in order to engineer away the problem of having loved someone. The Evening Standard announced this week that it would publish a column by a digital zombie of Brian Sewell, simultaneously desecrating the great critic’s corpse while preventing writers of the future from following in his trade.
Why? For money, obviously. Most people don’t realise how much is being gambled on imposing this technology on the public. Last year, almost all of the wealth gained by the richest people in the world (96 per cent of the gains recorded by the Bloomberg Billionaires Index) came from the boom in certain AI-related stocks, such as Nvidia and Microsoft. Generative AI is not necessarily useless or evil, but it is being implemented as fast as possible, with little regard for the consumer, in order to please financial markets that are in thrall to its promises.
If you buy the new iPhone, or Google’s new Pixel phone, you are taking part in an experiment into what happens when you outsource thinking and feeling to generative AI. You don’t know how it will affect your child’s emotional development if you let a machine take over the job of teaching them how to react to life’s more difficult moments. You don’t know what it will cost you if you accept that a machine will always know the right thing to say.
This is particularly dangerous because we seem predisposed to trust machines. In a recent study, radiologists in a hospital in Germany were shown mammograms accompanied by “AI-generated” diagnoses (the diagnoses had actually been faked by the researchers, and some of them were deliberately wrong). These trained specialists, some of whom had been practising radiology for more than a decade, spent more than 40 minutes on average looking at the scans. But the “AI” diagnosis often persuaded them to go against their training: they regularly agreed with wrong diagnoses, led by “automation bias”. Having agreed to hand over some of their confidence and intelligence to the machine, they skewed towards trusting it.
Generative AI is currently repeating the pattern followed by social media platforms from the mid-Noughties. It wasn’t clear how much value, if any, the technology would create. Everyone knew it would have downsides. But it was unstoppable, because if the market decides you’re going to take over the world, you probably will. The founders and early investors in social media companies became very, very rich, long before anyone else could decide if they really wanted what they were selling. The outcome for Mark Zuckerberg was a net worth of nearly $200bn. As we are belatedly realising, the outcome for the wider population was a tidal wave of fraud, bullying, anxiety and narcissism.
Generative AI has much of the same momentum behind it today as did social media. But it offers to replace still more of what makes us human – not just our connections to other people, but the emotions and principles from which those bonds are made. Now is a good time to exercise some consumer choice about whether those are things you want to hold on to.
[See also: My addiction to vertical video]
Content from our partners
Related
