Cash registers with scanners are great. Especially for getting to grip with the nitty-gritty of a store, company or holding. That’s the theory, at least. Real life is, as always, different. The request for an analysis and reporting tool for the marketing department of a supermarket with more than 2,000 shops was a welcomed change, since we are normally involved in what is called robot journalism. But data are data.

So we started to investigate how our client was currently dealing with the millions of data sent from around 9,000 cash registers across Germany. It became one of the most complicated investigations of my life, even though I had worked for many years as a freelance journalist in the Middle East. That was easy stuff compared with a German supermarket chain.

“The data are pretty much coming in in real time” – so far, so good. A clear-cut statement. But it was the only thing to be clear in the next two weeks. There were project teams working on the data, joined by external data analysts, highly paid big data experts, teams composed of university students, guys from marketing, IT, accounting and finance. Then there were the executive assistants, tax consultants, controllers and, of course, the highly important and very expensive consultancies. No one group ever talked about the same data. Instead of being on the same page, everyone was off on their own page. Each interested party, each expert used their own analysis and, of course, the reports to the board, coming in at a delay of two to three weeks, were crap. Did anyone bother to read what the customers themselves were doing in the supermarkets? Probably not. They were busy reading opinions regarding the analysis of opinions on pre-selected data.

We hit the ground running. First, we did the most important thing for the project: we explained to everybody working on the data that we were not the enemy. That we wanted to help everyone work easier and more efficiently by sharing with them the enormous amount of customer experience we had collected. We jettisoned the over-paid consultancies, but then this was something to be expected.

To make a long story short: after four weeks we presented the first analysis based on the daily deluge of data, distributed them to the different levels of management. The marketing board was close to tears when the first top-level summary was delivered directly to their smartphones, 80 lines, easy to read, with the “action required” points leading the way, all at a delay of 40 seconds to real time.

We know it’s still taking too long but we’re working on it.