blog
Experimentation - It Takes Guts, Pays Glory
by Mike Ma
There are two ways to get fired from Harrah's: stealing from the company, or failing to include a proper control group in your business experiment.
- Gary Loveman, CEO, Harrah's Entertainment, MIT Sloan Management Review, August 2009
Think about that statement for a second. Two quick conclusions can be drawn. First, Loveman assumes that everyone should be conducting experiments in their businesses. Second, if you experiment poorly you are committing an act equivalent to stealing - a pretty powerful statement.
We are in the throes of business planning for 2010. Now is the time to learn from the problems of growing budgets by an arbitrary 10% in fat times and cutting across the board by 10% or more in lean times. Both methods have failed the industry, and we are left to figure out what works and what doesn't. When we arrive at this kind of reckoning, we typically pull our managers around a table and make gut calls about "what we see on the frontlines" or "what will bring us inline with our competitors."
I propose that we start supplementing these important, qualitative discussions with data from experiments conducted inside a firm's four walls. It takes guts to do this, since it requires:
1. A Control Group - As Loveman states, we have to choose to not do something to a segment of our clients or workforce, and that is admittedly hard. It is understandable to think that if something can give us or our team an edge, it should be applied to everyone. But without a control group, you can't know what was received and what the value of the investment was.
2. The Possibility of Being Wrong - In essence, you need to be making bets, and bets are scary, especially if they don't pan out in this economy.
In my talks with executives over the past few weeks, I have started to hear some good ideas on how to run these experiments. The best examples seem to involve starting small in order to gain familiarity with the process and its benefits. By experimenting with different coverage models, value added programs, and e-Mail messaging this year, we can begin to decide what adds real value to our businesses and what can be redlined in good faith.
We need experiments starting now, folks. Otherwise, we are just guessing.