First, a disclaimer: this piece isn’t grounded in any data analysis whatsoever. Sure, I’ve done my research, but this is mostly about the unquantifiable – the way many of us seem to feel.
When Nate Silver relaunched an expanded version of his “data-driven” reporting website FiveThirtyEight last week, he did so with a bit of a bang – a “manifesto”, to be precise, one that was likely drafted (per his suggestion) in the middle of the night with the aid of a lot of Red Bull. It’s cogent enough, if a bit rambling: Silver lays out the ills of the state of modern journalism, with its hunches and false trends and big, blustery ideas, stuff that’s rarely backed up by hard facts. He posits his own analytical methods – the ones that predicted the winner of the 2012 US presidential election correctly in every single state – as the antidote:
Narrative accounts of individual news events can be informative and pleasurable to read, and they can have a lot of intrinsic value whether or not they reveal some larger truth. But it can be extraordinarily hard to make generalisations about news events unless you stop to classify their most essential details according to some numbering or ordering system, turning anecdote into data.
Silver has to rely on the dreaded anecdote to illustrate the pitfalls of baseless punditry, notably Peggy Noonan’s assertion that “all the vibrations” and all the lawn signs in her neighborhood were pointing to a Romney victory in 2012. In an interview with New York Magazine on the eve of the site’s launch, Silver had a go at op-ed sections more broadly, especially the New York Times, his former employer. (“They don’t permit a lot of complexity in their thinking. They pull threads together from very weak evidence and draw grand conclusions based on them.”)
If our choice is Silver or the New York Times op-ed pages, well, that doesn’t even seem like a fair fight. So many of us are weary of opinions dressed up as reporting, or of the media’s habit of using statistical outliers to, in Silver’s words, erroneously “indicat[e] some broader trend”. The New York Times itself is famous for its “trend pieces”; just this weekend their own public editor poked fun at the paper for a recent style article that suggested that the monocle is on its way back.
We like Nate Silver, and maybe we even need Nate Silver – so why did everyone spend the past week hating on Nate Silver? Well, not everyone: the hundred-plus comments on the manifesto were gushingly positive. (A sample: “Bye bye, rest of the internet, I probably won’t miss you.”) But every day last week I was bombarded with articles that pecked away at FiveThirtyEight, scrutinising the methods of collection or the presentation of data in individual pieces, or, more often, knocking at the core premise of his project, of the “intersection of data and journalism”. There are only a handful of articles to critique, but people are declaring FiveThirtyEight dead on arrival. Paul Krugman weighed in twice! First to take issue with an individual argument, and later to declare: “So far it looks like something between a disappointment and a disaster.”
The offerings of the first week were admittedly not fantastic – I wasn’t personally impressed with much, and was actually a bit angered by the one that hugged closest to my current academic discipline (the digital humanities), a textual analysis of Shakespeare’s plays. (Because while there are plenty of interesting ways to use literary text analysis, this is not one of them – the data she looks at is fine, but the way she connects this information to either the plays themselves or their historical context is essentially useless.) But one would think that a new venture would be afforded a little room to find its bearings. The good will that Silver earned with his rock-solid political predictions seems to have slipped away. From the big name journalists, the pundits on the defensive, there has to be at least a little Schadenfreude here – or rather, the niggling desire to see this project fail, and delight when one lame data-driven piece after another is published.
But then, Silver didn’t just malign the pundits: he suggested most, if not all, journalists were lacking in quantitative analytical skills: “Students who enter college with the intent to major in journalism or communications have above-average test scores in reading and writing, but below-average scores in mathematics.” (For my own self-preservation, let me go on the record and say that my maths score was – to everyone’s surprise – ten points higher than the verbal. Ten less than the writing, though.) Setting aside the fact that most people I know working in the media came from all sorts of disciplines, there’s certainly truth here. I have to veer into the realm of the anecdotal to say that many writers I know seem to lack a fondness for numbers, and some even harbor a bit of distrust for technology. And I think a good majority of them would place themselves in Silver’s exception-to-the-rule camp, that their work is “rigorous without being quantitative”, “Robert Caro to Richard Ben Cramer.”
(Or, say, Janet Malcolm, or some other “rigorous” female journalist. Let’s not forget that there was another major thread of dissent running through the launch of FiveThirtyEight: Emily Bell’s assertion that Silver, along with new ventures from Ezra Klein and Glenn Greenwald, are failing to be as revolutionary they claim: “It’s impossible not to notice that in the Bitcoin rush to revolutionise journalism, the protagonists are almost exclusively – and increasingly – male and white.” Silver responded that, “about 85 per cent of our applications come in from men. That worries us,” which suggests that while he’s complicit, he’s not responsible. But perhaps the gender and race issues here are too thorny for the space allotted.)
Is it possible for a data-driven journalist to tell a good story? If you’re calling an election or breaking down a sporting match, good, rigorous data analysis can shine. But tackling the rest of the news? Or for that matter, culture? Traditional journalists might be uniformly bad at maths, but does maths matter when crafting a narrative? In the backlash against FiveThirtyEight, a few themes have emerged. There’s the idea that all the data analysis in the world can’t always tell a good story – sometimes the numbers don’t offer any space for strong conclusions. There’s the idea that data can never tell the whole story – that the reader needs more to grab onto than the aggregate. Silver speaks of the difficulty in telling stories from “generalisations” that aren’t steeped in data; I’d argue it’s an enormous challenge to tell any story with only generalisations.
And then there’s the idea, tapping into traditional journalists’ indignation, that there isn’t much space in the art of the written word for number-crunching. It’s hard to take this at face value, when you see numbers baldly manipulated or simply misunderstood in news articles every day. But it’s harder to swallow Silver’s antidote when journalism-by-the-numbers is so often such an unsatisfying read. The prose regularly (and occasionally smugly) sits back on the phrase “we looked at the numbers” or throws down a chart or a graph, as though the act of providing these numbers speaks for itself. I see this a lot here in our relatively new field of the digital humanities, too: research in history and English and other fields somehow quantified, but nothing substantive beyond the data, just people sitting proudly on the fact that they crunched any numbers at all. I saw a tweet a few weeks back that drove me crazy, something along the lines of, “Metadata is the metaphor of our times: more than the bare facts, but less than the full story.” This is fundamental misunderstanding of metadata, but if we substitute data analysis, it might ring a bit truer.
Perhaps there needs to be more collaboration between traditional journalists and statisticians, rather than each team going it alone or a push for the next crop of journalists to try to master both skills. Right now, there’s just something lacking in the questions a lot of data-driven journalists are asking – and, for now, at least, there’s still something lacking in the way they present the answers. I’m reminded of Choire Sicha’s fantastic line to the tech bros whose start-up manifestos clogged Medium in its early months: “Words are like code. You have to put them in the right order for things to work.” Numbers can’t be treated like words, but there’s clearly a need for both of them in journalism. We just need to find a better way for them to live side by side.
Content from our partners
Related
