Throughout my BI, Analytics, Research and Insight career, I’ve noticed an interesting phenomenon.
Those of us who are involved in creating Insight sometimes forget who and what it’s for:
Its for other people to make a decision or take action. Period.

In my view to solve this problem, Insight creators need to do more of three things; We should stop creating Insight we love, and instead create Insight other people will use. It’s essential to limit the amount of Insight produced without a full understanding of the decision or action the work informs. And we should be ready to prove ourselves, but know the detail rarely truly matters.

I’ll delve into these points, but first, I’ll explain why forgetting who and what insight is for can be so troublesome. And then what we can do to improve if we find ourselves suffering from this problem.


Washed away by the Data deluge

For context, when I say ‘Insight’, I mean Data, Reporting, BI, MI, Research, Analytics, Data Science, and related outputs. Essentially any outputs which inform people and organisations. You probably know, or indeed are an insight creator such as Data Manager, Reporting Analyst, Researcher or that most heralded of recent job title, Data Scientist. Those responsible for providing business metrics for data driven companies.

There’s little debate about the importance of Insight in business, public policy and life in general. It’s a simple fact that the amount of Insight generated continues to grow at an astonishing and terrifying rate, at least for those of us whose job is to process it!

Why then does the PwC annual CEO survey show leaders say there’s a gap between what they need to make effective decisions, and what they receive from Insight?

When considering Insight on the preferences of their customers, only 15% of CEO’s say the data they receive is adequate, despite 95% of them saying it’s critical. Incredibly, this sentiment has barely improved in the last decade!

At face value, this doesn’t make sense. We know that Insight is a fundamental organisational commodity. We know that we have more of it than ever, so much so that I have anecdotally often heard statements including ‘too much’ or ‘overwhelming’.

But let me ask you this, how many other fundamental organisational commodities like revenue, talent or demand would we ever say there is ‘too much’? They all need applied effort to process to gain the maximum organisational benefit, just like Insight. Yet I’m willing to wager you’ve rarely heard an executive bemoan having ‘too much’ revenue coming in, talent at her disposal or demand for her product!

Why then, is this happening with Insight? Why are some organisations still finding it difficult to utilise this essential commodity? And why is it so hard to prove Insights value?


 Whose fault is it?

Does the fault lie with Insight Creators? Are Finance, HR and Sales professionals better at managing these fundamental commodities? I doubt it; I’ve been lucky to directly work with some of the most talented individuals I’ve ever met during my career as an Insight professional. I’m sure those of you who work in this space or are consumers of insights can also attest to that, so ability surely isn’t the problem.

So then, are the fundamental commodities Finance, HR and Sales professionals govern easier to manage? Again, having provided insights for professionals in these spaces, and knowing the effort they put into their professions, I’d have to say not.

So what is it? In some organisations, its unquestionably true that a lack of analytical talent, insufficient technological maturity and poor data governance all play a role, but strides in these spaces have been monumental in the last decade, so I’d like to posit that an often overlooked factor is;

The ‘output’ that Insights creators produce often requires a high degree of “post-processing” conducted by non-experts and some Insight creators sometimes crucially forget this fact.


Made by us, not for us

Let me explain “post-processing” first. Take the high-level output of an HR professional, the processes and procedures for the management of people which results in capable teams. The consumers who “post-process” or use this output you’d expect are competent and experienced at managing people. Be it integrating a recruit or conducting a disciplinary procedure in a proper and considerate way, the people “post-processing” should to some degree be ‘people experts’.

The same is true for the high-level output of finance professionals, the movement and management of money. The consumers who “post-process” or use this output are organisational leaders whom we should assume are experienced and expert money managers. After all, data shows that up to a quarter of CEO’s are MBA’s and a fifth are qualified accountants by trade.

So, HR and finance professionals mostly produce work for those experienced and competent in the use of the very outputs they create. The same is simply not as accurate for Insight creators, and we would do well to remember that fact.

Of course, Insight in one form or another has been around since the very beginning of structured organisations. Still, Insight in its modern guise is a relatively new and continually evolving area of expertise. And that evolution is happening at a frightening pace.

When ‘post-processing’ financial outputs, most leaders will be tasked with interpreting data produced using methods with which they are likely familiar. That’s not to say that this area is not evolving, of course, it is, but when compared to ‘insight’ that pace of change is likely to be less dramatic. Therefore the ability of that leader to understand and apply that financial information to a business decision should be high.

Remember that’s what its for…for other people to use.


Constantly Evolving Techniques

Finance professionals don’t need to consider building for business use because their methods of creation and usage are already deeply ingrained in organisational understanding.

In contrast, sometimes, when a leader was learning to develop Insight at the beginning of her career, the methods she used are worlds away from the techniques used today. I’d bet that for you, the methods used to create what Insight creators are putting on your desk, will sometimes be unrecognisable to anything you’ve previously produced.

From personal experience, some of my Junior Intelligence analysts have built machine learning models I’m virtually incapable of understanding without significant effort. I knew only how to apply them for the benefit of my organisation. Thank goodness!

Is it any wonder then, that the people we build insights for are not as experienced at “post-processing” our outputs? That sometimes Insight Value gets lost if we don’t consider the possibility of a gap in understanding?

I’ve seen two common mistakes in this regard. Firstly, Insight creators getting carried away with a beautiful piece of work which impresses themselves or their peers but is unusable by the Insight consumer.

And Insight creators delivering great work which provides accurately on its specific requirements, but built without finding out its ultimate purpose. Frustratingly, this means the work does not enable that particular outcome, making it “unusable” from the consumers perspective.


So what?

Early in my career, Often my team produced elegant analysis only to be met by blank faces and statements of “so what?” at the end of our presentation.

I’ve had to deal with low morale from team members who believed that their roles were pointless, because “no one does anything with my analysis”. We put massive amounts of effort into explaining methodologies and producing vast amounts of data to validate our findings. And we did this because we wanted to prove our worth, and prove the validity of what we were saying.

But how useful is it to consumers of our insights, the people who need to make a decision or take action, if we do not focus on that person and what they need to do? If we overwhelm them with a deluge of Insight which dilutes the point when the only point is ever decision and action.

My advice is to have this consistently at the forefront of our minds:


Don’t create Insight you love, create Insight other people will use

Insight Creators love to provide context, trying to take insight consumers “through a journey”, but I’ve found that sometimes this is for the creators benefit more so than the consumers.

The only “proof” that ultimately matters is the result gained from the decision or action taken from your Insight. That’s it.

HR’s does not prove its worth by a thorough explanation of the procedures they use and copious amounts of data. An increasingly talented and high-morale organisation proves it. Just as a beautifully designed dashboard does not determine Insight’s value, its effective decision making and the actions of the broader organisation that shows its worth.

By focusing on the person who needs to use your Insight, you’ll limit the amount of ‘post-processing’, and help them deliver the organisational benefit they need.


Limit the amount of Insight you produce without a full understanding of the decision or action the work intends to inform.

Your requirements gathering process should be built around not what a person wants to know, but primarily, what do they ultimately wish to achieve for the organisation. Only then can we deliver fully rounded Insight for people to use. Is the marketing director looking for an evaluation, the actual state of things, or are they looking for proof of an existing hypothesis they hope to utilise for a marketing strategy? Nuances like this matter for us to provide an insight which genuinely delivers.


Be ready to prove yourself, but know the detail rarely matters.

Not for a second am I saying that good Insight doesn’t require incredible amounts of processing and data to produce. I’m merely saying that for the people it is actually for, the detail rarely matters. If they want detail, they’ll ask, but when it comes to delivering insight…get to the point.

Give them the number, make the recommendation, flag the problem and what to do about it. If the context is needed, have it ready, but make sure that the person who needs your Insight can use it to make the decision or take action.