Recommendation Nation
Learning to love customers like you.
I love books, I like music, and I don’t mind the news. When I’m sent a link to something a friend thinks I should read, hear, or view, I take it seriously. Recommendations are essential to my quality of life.
It’s a good thing I feel this way, because recommendations are everywhere on the Internet. Wherever I shop online, some sliver of my screen is prompting me with a come-hither like “Customers who bought this item also … .” Pop-ups and context-sensitive advertisements have been supplemented by this low, seductive whisper of automated suggestion. The truth is that I now get more good recommendations about more things, more often, from Bayesian algorithms than from my best friends. Perhaps this should make me wistful, but it doesn’t. Better technology doesn’t mean worse friends.
Unlike human recommenders, Apple.com, Amazon.com, and Google.com never insult me by implying that I spend my time, money, or attention on the wrong things. They’re simply making relevant–and occasionally novel–recommendations based on my past choices and the things I attend to in real time. The focus of digital personalization has shifted from what I am interested in now to what I might be interested in next. All the choices I make in the moment are absorbed into a sphere of suggestion where, after they have been statistically weighted, they are reborn as offers and advice.
Increasingly, I find myself as curious about a site’s recommendations as about what it sells. That a site is trying to sell me something else seldom annoys me. On the contrary, I like it that Internet companies have dedicated such ingenuity, memory, and processing power to offering me good suggestions. But “good” needs to get much better if recommendations are to expand beyond telling me what I might like right now.
Consider Amazon, whose site displays some of the irksome limitations of current recommendation engines. The company has been a pioneer in this technology since shortly after its launch in 1995. Greg Linden, who is now with Microsoft, helped write Amazon’s first recommendation engine, Instant Recommendations, which succeeded where an older system called BookMatcher had failed. The engine evolved incrementally. “We learned what worked and what didn’t by seeing how changes in the recommendations helped people find new books,” Linden says. “We enjoyed helping people discover books they probably would not have found on their own. It was never about marketing–just matching people to books they would love. But it turns out people do buy more when you help them find what they need.”
Today, Amazon makes recommendations on the basis of a customer’s browsing and buying history, other items viewed or purchased by customers who’ve bought the product being viewed, and items that seem related to that product. On Amazon, reviews, recommendations, and rankings become an essential part of browsing and shopping. For example, while I was checking out Predictably Irrational, Daniel Ariely’s new book about apparently dysfunctional decision making, the “Customers Who Bought This Item Also Bought …” strip tipped me off to a forthcoming title I had never heard of: Nudge, by the University of Chicago behavioral economist Richard Thaler and the University of Chicago law professor Cass Sunstein. Click, and I’m there. It’s precisely the sort of real-time connection that makes Amazon shopping superior to both in-person and online alternatives.
Click-throughs are the currency of the recommendation nation. The more choices you make (or decline to make), the more finely tuned the recommendations become. The more your peers interact with Amazon, the better Amazon’s engines can infer which recommendations will make the most sense for you and the most dollars for them. The result is that recommendations can become breathtakingly profitable examples of what economists call “network effects,” where a network’s value is proportional to the number of its participants.
But as useful as these algorithms can be, they’re also subject to sudden bouts of apparent blindness. It ticks me off, for example, that Amazon’s recommendation engines do not intelligently distinguish the books I browse or buy for me from the ones I browse or buy as gifts. Yes, I can click a little box when I buy something as a gift. Additionally, if I visit “My Amazon,” there is a tab that offers to “improve my recommendations”: on the long scroll of everything I have bought, I can click a box that says “This was bought as a gift” and another box that says “Don’t use for recommendations.” But these features are far from obvious (I discovered them only in writing this review, and I use Amazon a lot). Nor do Amazon’s engines use my history of gift buying to suggest presents for particular friends. Would such suggestions bug me? No. In fact, I’d like Amazon to make it easy for me to switch back and forth between browsing for myself and browsing for others. I’d cheerfully choose to be a recommendation beta user if such an option were offered–much as I’d be happy to have a “personal shopper” help me out come birthdays and holidays. Just ask nice.
Different issues emerge with the “Just for You” recommendation engine at Apple’s iTunes, which was introduced in 2005. I can forgive the fact that my purchase of Bohemian Rhapsody prompted “Just for You” to recommend The Best of Foreigner Live, but not that buying Van Halen’s “Dance the Night Away” provoked a recommendation for Rush. While I accept that recommendation engines have their own quantitative quirks and eccentricities, those suggestions are just terrible. Apple’s engine appears to give more weight to era than it does to genre, tempo, or style. (An Apple spokesperson whom I contacted declined to be more specific about how its recommendation engines work.)
Apple’s recommendation software is worse than Amazon’s in other ways, too. When I buy a song or two from one band, why does the engine ask if I want to buy an entire album from another? I should get individual song recommendations before I get album suggestions. Apple’s iTunes pushes albums and songs: it feels like a hard sell. I want to be sonically seduced, not commercially assaulted. Get me to sample–for free, of course–another song before asking if I own or want to own the entire album. If I like the song, I’ll buy it. Honest!
The “Just for You” interface looks pretty enough. But as an interactive experience, it’s displeasing. Unlike Amazon, the site feels more like a record shop that wants to move product than the den of a friend with great taste in music. Recommendation engines should liberate retailers from bad online store design, but the iTunes site reminds me of what I like least about shopping. Where is Jonathan Ive, Apple’s legendary industrial designer, when we need him?
These complaints notwithstanding, my bet is that recommendation algorithms and interfaces will rapidly branch out. In the future, perhaps Google’s Gmail will tell you whom you should forward that urgent e-mail to, or remind you to keep in touch with a friend you’ve inadvertently ignored. Marrying Gmail’s context-sensitive advertising to a decent recommendation engine would boost the value of both. What’s more, it’s easy to imagine Facebook suggesting what information should be shared with whom–or who should be sharing more with you.
The rise of the social graph (an abstract representation of the social connections between users of digital networks; see “Between Friends,” March/April 2008) should enable different companies’ recommendation engines to work together, offering financial advice, travel options, and more. Wouldn’t it be intriguing to see what stocks and funds people like you bought? Perhaps these technologies will ultimately go meta, with some startup offering recommendation engines that let you pick the best recommendation engines for you. Advice about advice might be a great business.
For all my excitement about the future of recommendation services, I can’t help feeling the way I felt about search in 2001. Existing recommendation engines have a lot of value, but they’re still primitive. Distinctions between browsing and comparison (that is, between looking at products and choosing between them) are poorly understood. We’ve yet to see how user-generated tags make product and service descriptions more precise and useful. The more specific, explicit, and time-sensitive the tag, the better the potential recommendations will be.
Smart people all over the world are working on these problems. Billions of dollars are at stake. Netflix is offering a million dollars to anyone who can improve the efficacy of its (exceptionally successful) recommendation engine. That’s a small price to pay for a company whose future depends on its ability to compete with Blockbuster and the digital video delivery companies of the future. It is an interesting and important problem, because it’s not only individuals who watch the movies, but couples, families, and friends. Perhaps the winning algorithm will be optimized for the preferences of groups.
When I get good recommendations, I spend my time and money differently. Even better recommendations will dramatically increase the value of that time and money. That’s a digital future I crave and expect. I hope Internet innovators take my recommendations as seriously as I take theirs.
Michael Schrage is a consultant on innovation, a researcher at MIT’s Sloan School, and the author of Serious Play: How the World’s Best Companies Simulate to Innovate.
