It feels good to have notable British statistician, George Box, on my side when building market models for clients.  Box famously said, “All models are wrong, but some are useful.”  It is the one area of my consulting practice in which I embrace being wrong.  

My market sizing models – wrong.
My ROI calculations – wrong.
My forecasts– wrong again. 

There are endless ways my models are wrong…

• Assumptions about competitive pricing models

• Assumption about our own pricing decisions in the future

• Assumptions about consumer behaviour

• Assumptions about segmentation variables, size of segments and segment dynamics

• Assumptions about the macro-economic forces at play

…to list a few. 

So, why am I not more self-abashed and how do I strive to generate models that fall into the not just Wong, but Wrong-But-Useful category?

First, I lean into inaccuracy and embrace directionality.  The analytical tools at our disposal are precise, often more precise than the business question being asked.  I work with clients to understand what business decisions are going to be driven by the model and what we want the model to do for us.  Then, in many cases, we agree that a simple, directional approach to modelling is more suitable to the task at hand.  Clients often already know if they are entering a market or making a product investment decision but just need the model to a) validate the decision or b) highlight the risks and barriers that need to be addressed or side-stepped.  An overly precise model then, doesn’t serve its purpose.  However, something that showcases the proportionality of key decisions would be deemed a Wrong-But-Useful model.  

A Wrong-But-Useful model also has clearly defined assumptions and testable variables. Much like Frank L. Baum, I allow a tornado (chart) to drive the narrative structure of my strategic recommendations.  Tornado charts identify factors that not only have a greater impact but also have higher uncertainty.  While the absolute impact of these variables may be incorrect, the ranking of them is massively valuable.  

Those most sensitive variables should fall under the greatest scrutiny and investment in additional research and risk mitigation strategies should follow. 

In my experience, the only person who hasn’t been suitably impressed by the utility of a tornado was the Wicked Witch of the East and her sister.

A Case Study: Modelling to Determine a Free-Upgrade Policy

A client engaged my services to help the organisation consider the financial impact of providing free upgrades to their current install base; a move that on the surface would negatively impact operating margin.  The model was able to show them the degree of impact this decision would have on their bottom line, but the tornado chart was the star of the show in demonstrating the single biggest variable in overcoming the cost of this strategic move: the possibility of increased market share as a result of this corporate policy.  

This work, completed in Australia, was meant to drive the decisions for the rest of the regions.  Instead of simply sharing these results and providing a recommendation to focus on competitive messaging to drive market share, we developed simple sensitivity models for each country GM to input their own country dynamics and simulate their own result.  In this way, the GMs didn’t have to trust our analysis but they proved it to themselves in simple GM-proof models that led us all down the same analytical yellow-brick road. 

Were these models accurate? No.  But like a map not drawn to scale, it got the job done and guided a pivotal strategic decision.

Next time the opportunity presents itself to use a model to answer a tricky strategic question, know that your modelling will be inaccurate.  Just ask George Box.  The more useful exercise is understanding the uncertainties and your business’s tolerance for them.  Strategy is easy with the right insights and analytics even if those analytics are…well…wrong!