Measuring the Business Value of Recommender Systems
Recommender Systems are nowadays successfully used by all major web sites — from e-commerce to social media — to filter content and to make suggestions in a personalized way. Academic research largely focuses on the value of recommenders for consumers, e.g., in terms of reduced information overload. To what extent and in which ways recommender systems create business value is, however, much less clear, and the literature on the topic is scattered.
In this talk, we review existing publications on field tests of recommender systems and report which business-related performance measures were used in such real-world deployments. We summarize common challenges of measuring the business value in practice and critically discuss the value of algorithmic improvements and offline experiments as commonly done in academic environments. Overall, our review indicates that various open questions remain both regarding the realistic quantification of the business effects of recommenders and the performance assessment of recommendation algorithms in academia.
Dietmar Jannach is a full professor of Information Systems at AAU Klagenfurt, Austria. Before joining AAU in 2017, he was a professor of Computer Science at TU Dortmund, Germany. In his research, he focuses on the application of intelligent system technology to practical problems and the development of methods for building knowledge-intensive software applications. In the last years, Dietmar Jannach worked on various practical aspects of recommender systems. He is the main author of the first text book on the topic published by Cambridge University Press in 2010 and was the co-founder of a tech startup that created an award-winning product for interactive advisory solutions.