Is it better to prevent or correct?

This link gives you access to all the articles and archives for </Hello CDO>

“Prevention or Correction” is something society has wrestled with for a very long time. This article focuses on why our experience of “prevention or correction” ideas frames the CDO’s responsibilities and explores a linkage to a company’s approach to data.

Almost irrespective of where you reside, we live with police, penal, and political systems that cannot fully agree on preventing or correcting. It is not that we disagree with the “why”; is it the “how” that divides us! I am a child of an age when lefthandedness was still seen as something to correct, so we have made some progress.

A fundamental issue is that prevention is the best thinking; but if you prevent it, it does not occur. We are then left with the dilemma, “did you prevent it, or was it not going to occur anyway?” The finance team now ask, “Have we wasted resources on something that would not have happened?”

When something we don’t like does occur, we jump into action as we can correct it. We prioritise, allocate resources and measure them. We (humanity) appear to prefer a problem we can solve (touch, feel, see) and can measure rather than prevent. A proportion of crime is driven by the poor economic situation of a population with no choice. Yet, we keep that same population in the same economic conditions as we have limited resources committed to correction (control.) It is way more messy and complex, and we need both, but prevention is not an easy route. Just think about our current global pandemic, climate change or sustainability. Prevention was a possibility in each case, but we kick into action now correction is needed.

In our new digital and data world, we are facing the same issues of prevention vs correction. Should we prevent data issues or correct data issues, and who owns the problem?

In the data industry right now, we correct and just like the criminal services, we have allocated all our budget to correction, so we don’t have time or resources for prevention. I would be rich if I had a dollar for every time I hear, “We need results, Fish, not an x*x*x*x data philosophy.”

For anyone who follows my views and beliefs, I advocate data quality founded on lineage and provenance (prevention). I am not a fan of the massive industry that has been built up to correct data (correction). I see it as a waste on many levels, but FUD sells to a naive buyer. I am a supporter of having a presence at the source of data to guarantee attestation and assign rights. I cannot get my head around the massive budgets set aside to correct “wrong” data. We believe that data quality is knowing the data is wrong and thinking we can correct it! We have no idea if the correction is correct, and the updated data still lacks attestation. We measure this madness and assign KPI to improve our corrective powers. In this case, because it is data and not humans, prevention is the prefered solution as neither a cure nor correction work at any economic level that can be justified other than your a provider of the service or one of the capital providers. But as already mentioned, prevention is too hard to justify. The same analogy is a recurring theme in the buy (outsource) vs build debate for the new platform, but that is another post.

Unpacking to expose and explore some motivations.

We know that the CDO job emerged in 2002, and some 20 years on, it is still in its infancy. Tenures are still short (less than three years), and there is a lack of skills and experience because this data and digital thing are all new compared to marketing, finance, or accounting. As I have written before, our job descriptions for the CDO remain poorly defined, with mandates too broad and allocated resources too limited. All many articles at </Hello CDO> I focus on the conflicts CDO’s have because of the mandates we are given.

Whilst mandates are too broad, thankfully, security and privacy are becoming new roles with their own mandates. The CDO is still being allocated a data transformation mandate but being asked to correct the data whilst doing it. Not surprisingly, most data projects fail as the board remains fixated that the data we have is of value, and we should allocate all our resources correcting and securing it. A bit like the house built on sand, we endlessly commit resources to unpin a house without foundations because it is there, rather than moving to the more secure ground and building the right house. Prevention or correction?

All CDO’s face the classic problem of being asked to solve the world’s debt famine crisis with no budget, no resources and yesterday would be good. Solve the data quality problem by correcting the waste at the end. Because of this commitment to correct data, we find we are trapped. “We should just finish the job as we have gone so far,” is what the finance team says, “prevention is like starting all over again”; it will be too expensive, wreck budgets and we will miss our bonus hurdle as it means a total re-start. The budget is too big so let’s just put in some more foundations into the house built on sand.

It is one of my favourite saying “the reason it is called a shortcut, it because it is a short cut and it misses something out” I often use it when we boil a decision down to an ROI number, believing that all the information needed can be presented in a simple single dimensional number. The correction ideals will always win when we use shortcuts, especially ROI.

Prevention is a hard sell. Correction is a simple sell.

Who benefits from an endless correction policy? Probably the CDO! We get to look super busy, it is easy to get the commitment, and no one will argue at the board/ senior leadership team to do anything else. It might take three years to see the transformation has not happened, and there is plenty of demand for CDO’s? Why would a CDO want to support prevention? Why is the CDO role so in question?

Prevention takes time, is complex, and you may not see results during your tenure. I often reflect on what will be my CDO legacy? Correction is instant, looks busy and gets results that you can be measured on. Correction means there is always work to be done. Prevention means you will eventually be out of a job. Since prevention cannot be measured and is hard to justify with an ROI calculation, maybe the CEO needs to focus on measuring the success of analytics?

Note to the CEO

Data is complex, and like all expert discipline areas, you will seek advice, opinion and counsel from various sources to help form a balanced view. Data quality is a thorny one as most of those around you will inform you of the benefits of correction over prevention. Correction wins, and it is unlikely any balanced view arguing for prevention.

Perhaps it is worth looking at the CDO job description, focusing on what the KPIs are for and how you shift a focus to the outcomes of the analysis. To improve analysis and outcomes demands better data quality, which correction can only get you so far. You get prevention by the back door.