John Dunford: How online metrics can distract from objectives

Charities need insights backed by data, not decisions driven by the wrong metrics

John Dunford
John Dunford

Everything you do should help to serve your mission. However, the structure of the online world can in fact mean that more data results in less focus for charities.

This is where charities can get lost in the gap between how we interpret data (analytics) and what we can actually learn from those numbers (insights).

Safety in numbers? Not always

For charities, there are three systemic reasons why simple online data might actually undermine your work.

First, platforms such as Facebook and Twitter profit from organisational behaviour in the same way they do from personal accounts. That simply means driving revenue by using all the tools of persuasive technology to get as many eyeballs on ads as possible. The data they supply to organisations is geared towards generating content that serves this goal.

Second, the bulk of technology our sector runs on – from email systems to analytics tools and payment processors – has been designed and built for commercial purposes. The business needs of the tech provider are fulfilled by providing the best service possible, which means optimising for income generation. This is of course crucial for many charities but not by any means the reason they exist.

Finally, the negative side of "what gets measured gets done" has huge implications for charities. The explanation comes from behavioural economics and how displaying goals and metrics for certain outcomes can make otherwise rational actors behave in seemingly irrational ways, such as chasing "vanity metrics", as Mob Lab so brilliantly described it. This will be familiar if you’ve ever dealt with a colleague who was interested in the number of likes a post received when this was in no way connected to your work.

All this means that the structures and incentives that have powered the growth of online business might in fact be pushing you to give precious time and focus to things that don’t actually matter for you.

Measure what matters

Simply put, the data we are being served up is just not optimised for people trying to change the world ahead of making a profit. We need to interrogate this data at a level beyond corporate activity, and for charities this means going beyond the validity of the numbers to how the data helps to increase your impact.

With a complex change goal, metrics can serve only as a proxy. Until you’re clear on how that data is contributing in a causal way towards what you’re ultimately trying to achieve, approach targets that are easily measured with caution. 

Charity reports offer a great window into this exact issue. While they come in different forms, the communication aim is always to generate deep engagement from a relatively small pool of people, whether they are journalists, board members, political elites, or change-makers. A report can of course be the foundation of a mass public campaign — as with Oxfam’s widely reported annually updated statistics on global inequality — but their very form as reports means they are not designed or intended for mass consumption, however great that would be.

In this case, applying the standard metrics that are offered for a media or sales funnel completely misses the point. Total page views or shares on Facebook, which speak to how far something spreads online, will tell you little about the performance of a report aimed at getting influential figures to go deep. However, measures such as how long people spend on the page and the source of that traffic will give you information you can use to truly assess and improve your work. The key lies in using data to assess what you’re trying to do and shutting out the rest of the noise.

The charity sector needs insights backed by data, not decisions driven by the wrong metrics.

Insight over analytics

Not all data is created equal and, while it should provide the signposts that help guide your decision-making, when your goals are complex and long term you need to follow the right directions. Interrogate your numbers and your own reasons for following them and you will work towards your mission guided by intelligent insights that inform, as opposed to being driven to a result you weren’t aiming for.

John Dunford is campaigns lead at The Developer Society and one of Third Sector's Fundraisers: the New Generation 2018 winners

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in