Even Amazon needs economists
by Miklós Koren
Suppose you’re Jeff Bezos contemplating a radical new thing for Amazon customers. How would you decide whether it’s worth it? Your engineers might put together a working prototype, roll it out to a bunch of randomly selected customers and do some A/B testing. After your data science team has crunched the numbers, you have your answer: the average customer will spend $X more on Amazon. X>Y, so you decide to scale up.
Not so fast. Without a model, no amount of A/B testing (or, for the science community, randomized control trials) and no amount of observational data are going to tell you how your customers will behave when they encounter something radically new. Models are useful for thinking about changes that have not happened before. A model might be a simple mathematical relationship summarized in a few formulas, a statistical model making assumptions about probabilities and distributions, or a theory of consumer behavior based on insights from economics and psychology.
How would economics, in particular, help you make a better decision? It brings two fundamental ideas to the table. First, that people respond to incentives. When you change the monetary, psychological or social incentives of your customers, they’re going to change their behavior. When shipping is free and fast, I may become a more impulsive shopper. Second, it offers a useful systems view of the world. Surely, when drones zigzag the sky with Amazon shipments, the entire shipping and retail industries will change, with potentially profound implications for manufacturing, as well. The way the shipping container transformed the world could not have been foreseen by A/B testing a prototype.
None of this is new. The Nobel Laureate Robert Lucas said as much in 1976, when macroeconomists were getting overly excited about predicting the future. Back in those days, large-scale forecasting models worked with around 50 variables, and it was easy to get carried away by the newfound computing power. Lucas cautioned that such atheoretical forecasts would not be useful for economic policy analysis.
Using big data will sometimes mean forgoing the quest for why in return for knowing what. (The Rise of Big Data)
Amidst all the enthusiasm for big data, Lucas’s warnings are once again relevant. And yes, even Amazon needs economists.
This blog post is based on my presentation at the Budapest BI Forum 2014.