71point4 > Blog > Data analytics > Rules Based Analytics

Rules Based Analytics

Posted by: Illana Melzer
Category: Data analytics

Sturgeon’s Law: 90% of everything is crap

This is the primary, dominant and most significant rule when you work with data. Crap finds its way into very many statistics that we rely on to frame our understanding of the world. The next time you use any number with authority stop and ask yourself: “Could Sturgeon apply?”. Chances are it does.

Twyman’s Law: Any figure that looks interesting or different is usually wrong

A sibling heuristic of Sturgeon, and one that is particularly important for analysts wishing to impress managers or clients with dramatic new findings or groundbreaking analysis.

Segal’s Law: If you have one clock, you know the time. If you have two, you are not sure

Because of Sturgeon, it is always good to find triangulating data. More often this additional data will not triangulate. You have been Segal’ed. This leaves you with a dilemma as an analyst. Should you pick one dataset, hide the other and build a convincing slide deck? Or should you present both data sets to the client, own the unknown and risk undermining the very premise on which they have hired you, namely that spending money on analysis is worthwhile? I have always followed the latter approach, admittedly to my detriment professionally. I blame my parents.

Goodhart’s Law: When a measure becomes a target, it ceases to be a good measure

Goodhart’s law explains Sturgeon in some contexts. For instance, when you are working with KPIs produced by a bureaucrat whose incentives (financial or otherwise) are linked to that set of KPIs you should be very wary. To overcome Goodhart you should always, always try to find corroborating evidence although you should be aware that Segal may kick in. If your client is a bureaucrat, pray that it doesn’t or be prepared to lose your scruples.

Box’s Law: All models are wrong, but some are useful

This law is true and it makes us feel better, so it is a personal favourite. We need our models to be useful even when we know the underlying data is Sturgeon and the assumptions are made up. At the end of the day, we must make a decision, and we cannot bring ourselves to flip a coin. Plus, some decisions are not binary so coin flipping doesn’t help.

What makes Box true is the process of building the model – identifying the parameters, looking for the data and most importantly, running the scenarios and sensitivities. For clients, Box does not hold if all they see is the model output, even if summarised beautifully in waterfall charts. Sadly, this is often the outcome in consulting projects. We must strive to avoid this.

Not quite Socrates’ Law: I am the wisest man alive, for I know one thing, and that is that I know nothing

Because I am not a man I have modified this slightly; “I am the wisest woman alive, for I know one thing, and that is that he knows nothing”.

Both the law and its modification are very useful to men and women. You know nothing. So does everyone around you.

The way to deal with this is to argue. We want many views because if we have one view when everyone knows nothing, we should be very afraid.

Arguments may make you feel unsafe, and you may get a bit flustered and angry. It happens. Because of this, it is best to have these arguments on Zoom or Teams.  When you feel unsafe, mute yourself and if your camera is on, turn it off. You can now totally lose it. Scream at your counterpart. Tell him; “you are a shmuck” or whatever. You will feel better. You can then calmly and constructively rejoin the meeting. Helpful hint: If your camera is on and you are not on mute, you may be terminated.

Author: Illana Melzer

Leave a Reply