The Myth of Repeatability

Warren Church

Seth Godin famously said that “all marketers are liars.” His insight was that “the lie the audience really wants to believe” is useful inspiration for creating motivating content - in his case, stories. He didn’t mean to have it taken literally. Yet nearly everywhere today - from Google click-bait to famous management guru best-sellers - you’ll see it used as content rather than inspiration. The prototypical lie is that “the secret to being successful lies in doing the 1, 3, 5 or 7 things that successful people or companies have done.” And lots and lots of people want to believe it.

These siren songs all get their power from what I call the Myth of Repeatability. Success in anything can be achieved merely by following a recipe - or the newest business or marketing best-seller. Just check the airport bookstores to see what three things you can do to make you or your company more innovative, more disruptive, more powerful or more competitive in the time United gives you when they delay your next flight. Simple.

The ‘true fact’ is that human behavior is anything but simple. It’s a complex system where outcomes are dependent on interactions with each other and with the environment in individually unpredictable ways. Think of the last meeting you attended. Imagine writing down everything everyone did or said in that meeting - make a recipe. Now staff a new meeting with the same number of people, give them the recipe, and see what happens. Even if somehow (a random probability) they reach the same general conclusion, it won’t be by the same route. 

What science knows is that context determines behavior - and recreating the exact same context is why they build things like the Large Hadron Collider. Random trials and repeatability are the basis of scientific experimentation and peer-reviewed evidence. Not so the prototypical business case studies, where the stories are made up only after a successful result was produced. Then the factors that “created” the success are reverse-mined from the data. The millions of unsuccessful results are never mentioned. What science also knows is that these case studies run afoul of at least two well-known experimental biases: selection bias and confirmation bias.*  What will work with your people in your business in your situation at a specific point in time is unique. Your time is better spent understanding the situational realities than a case study or the latest management fad. 

There is no harm in reading case studies or the latest best-selling business book, just be self-aware that it is appealing because you want to believe it - not because it will work. The probability of getting the same results by the same route with human beings is about the same as being able to precisely predict the weather outside your house next Monday. As Climate research scientists say, climate is what you expect, and weather is what you get. Results will vary.

* For the interested, and therefore interesting, reader: 

In psychology and cognitive science, confirmation bias (or confirmatory bias) is a tendency to look for or interpret information in a way that confirms one's preconceptions, leading to statistical errors. Selection bias is the selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved, thereby ensuring that the sample obtained is not representative of the population intended to be analyzed.