Q&A: The Experimenter

Gary Loveman, the CEO of Caesars Entertainment, says there are three ways to get fired from the hotel and casino company: theft, sexual harassment, and running an experiment without a control group.

Loveman, who has a PhD in economics from MIT and was a professor at Harvard Business School, has impressed the importance of data analysis on his employees, who are expected to quickly scale small tests into company-wide initiatives. For example, they might test which is likelier to get customers to spend more: a free meal or a free night in a hotel.

Michael Schrage, a research fellow at the MIT Sloan School Center for Digital Business, recently discussed the company’s policy with Loveman for Technology Review.

TR: What’s the most important thing about Caesars’ culture of experimentation?

Loveman: We need to overcome hunch and intuition with empirical evidence. We’ve set up a process and a discipline for evaluating our intuitions and improving our understanding of what our customers prefer. We can start with a hunch or strong belief, but we act on it through experiment. We want evidence. We’ve gone from the introduction of experimentation as a technique to a culture of experimentation as a business discipline. We hire people predisposed to do this by temperament and by background. Organizationally, we’re committed—and I’m committed—to making sure we have the discipline to have the decisions we make informed by this evidence. 

What’s been the biggest source of resistance to this?

Impatience and risk aversion. Let’s say that one of our properties had lower revenues than they’d like, and they think they know the reason why. Instead of running an experiment to test that reason, they don’t use a control group and pollute the entire process. This impatience and hubris breaks the discipline I want us to have. A well-designed experiment is the better way of testing that reason and learning what matters.

I’m most interested in experiments around pricing. We have a lot of perishable inventory—hotel rooms and the like—and I want to make sure we’re getting the best prices. I’m always paying particular attention to the way we price our bundles and packages and offerings to loyal customers and new customers. We want to minimize complexity and confusion, but there are a lot of ways we can experiment with prices, and I pay particular attention to them.

One set of experiments we’re doing focuses on bringing people to Atlantic City who have other gaming choices closer to home now. Customers receive a lot of incentives from us, but we want to be more efficient and relevant in targeting them. How much richer do our offers have to be? Should we include food and entertainment offers?  We’re doing a lot of highly targeted and segmented experiments to see what mixes work best with people from these geographies.

What makes so many executives prefer to rely on their experience and analysis over simple experiments?

There’s a romantic appreciation for instinct and, frankly, an absence of rigor for the application of more scientific approaches. What I found in our industry was that the institutionalization of instinct was a source of many of its problems. 

When you got your economics PhD from MIT in 1989, subdisciplines like behavioral economics and experimental economics had a mixed reputation. Now—a couple of Nobel Prizes in the field later—they seem to be cornerstones of how many businesses and industries try to innovate.

My impression is that when I got my PhD, we were really manipulating mathematics for our own amusement, and we weren’t producing all that much to help real people make real decisions. That was dissatisfying to me and, frankly, frustrating. The notion that we could do experiments based on the central tenets of economics and have that make a real-world difference was exciting. Of course, with Freakonomics and Predictably Irrational these themes have become more popularized and accessible. It’s a very heartening development, and it’s increased my enthusiasm for my own discipline enormously. 

What do you like to tell your academic colleagues about the challenges of real-world experimentation and innovation?

Honestly, my only surprise is that it is easier than I would have thought. I remember back in school how difficult it was to find rich data sets to work on. In our world, where we measure virtually everything we do, what has struck me is how easy it is to do this. I’m a little surprised more people don’t do this.