The census may have a nugget, but Tesco already has the gold

The government has nothing on supermarkets when it comes to keeping to tabs on people.

You won’t find the next generation of gold prospectors getting their hands dirty. Nope, you’ll see them behind a computer mining data.

In the 21st century, data is the new oil, and all you need to do is drill into the right databases to find out what you need.

I point this out having just trawled through my census form and realised what a pointless, expensive waste of time it is. And that’s before I decide whether my religion is Jedi or not.

I predict now that this is the last one we’ll do, certainly on paper, this way. Especially given that ministers could simply call up Tesco to get pretty well all the answers they need within hours.

Here is why: Tesco has an astonishing databank, built up through its Clubcard reward scheme. The data from card swipes is analysed by a company it owns called Dunnhumby.

Dunnhumby’s website says: “We have access to the shopping behaviour of 13 million households. This helps manufacturers to understand the purchase decisions and habits of customers better than anyone else.”

The raw data alone has no value; it’s how you crunch it. And this lot are so good at it that they can sell it on to other firms.

What this means is that Tesco knows exactly where its stores need to be located and doesn’t fill its shelves with stuff it can’t sell. That’s the secret of its profits. It also segregates communities. The really poor areas are never going to get sun-dried tomatoes. Cigarette companies work in a similar way to get round the ban on advertising.

Another company, Experian, has financial and location data sewn up. This company offers a free credit rating service and even knows about our web viewing habits.

So deep is Experian’s reach that it was able to map exactly how and where the spending cuts would hit and even the health of the nation.

But you don’t need to drill deep to find out about people. Wired organised a great stunt recently in which it did a basic trawl of personal details openly available and made some shocking discoveries.

The government is playing catch-up with its own site, partly driven by campaigns to open access to official data, but it’s way behind.

It’s a frustration among ministers past and present. Labour’s former Cabinet Office minister Liam Byrne has highlighted that, on pressing issues, the departmental advice and evidence didn’t go far enough. That’s why there are so many consultation documents out of Whitehall.

At the local level, health, police and other agencies will tell you about the “hidden” people who show up to use services but don’t exist officially. The new slum landlords won’t say they’ve got ten people in a house. Westminster and other councils have warned for years about how unreliable the census figures are. Yet the government will still plan services around them.

Follow it though, and there are three conclusions. The government could contract out the census or mine its own databases properly.

Or, to get a real understanding of its citizens, the government could actually talk to citizens and communities. Get into a deep dialogue with people at local level; a form of crowdsourcing. But governments won’t go there while they are focused on focus groups.

Finally, Tesco has enough data to know who’s buying bad food and aspirin. It could save the NHS a fortune by identifying unhealthy people right now. The bigger stores have a pharmacy. Tesco could take over the health service.

It has the buying power, data, national presence and supply chain. And with deficit reduction, every little helps . . .