A Method for Estimating Users’ Preferences for Websites
A site’s recommendation system relies on understanding its users’ preferences in order to offer relevant recommendations. These preferences are based on the attributes that make up the items and content shown on the site, and they are estimated from the data of users’ interactions with the site. However, there is another important aspect of users’ preferences that is often overlooked – their preferences for the site itself over other sites. This shows the users’ base level propensities to engage with the site.
Estimating these preferences for the site faces significant obstacles. Firstly, the focal site usually has no data on its users’ interactions with other sites, making these interactions their unobserved behaviors for the focal site. Secondly, the Machine Learning literature in recommendation does not provide a model for this particular situation. Even if a model is developed, the problem of lacking ground truth evaluation data still remains.
In this article, we present a method to estimate individual users’ preferences for a focal site using only the data from that site. By computing the focal site’s share of a user’s online engagements, we can personalize recommendations to individual users. We introduce a Hierarchical Bayes Method and demonstrate two different ways of estimation – Markov Chain Monte Carlo and Stochastic Gradient with Langevin Dynamics.
We also propose an evaluation framework for the model using only the focal site’s data. This allows the site to test the model and assess its effectiveness. Our results show strong support for this approach to computing personalized share of engagement and its evaluation.
Abstract:A site’s recommendation system relies on knowledge of its users’ preferences to offer relevant recommendations to them. These preferences are for attributes that comprise items and content shown on the site, and are estimated from the data of users’ interactions with the site. Another form of users’ preferences is material too, namely, users’ preferences for the site over other sites, since that shows users’ base level propensities to engage with the site. Estimating users’ preferences for the site, however, faces major obstacles because (a) the focal site usually has no data of its users’ interactions with other sites; these interactions are users’ unobserved behaviors for the focal site; and (b) the Machine Learning literature in recommendation does not offer a model of this situation. Even if (b) is resolved, the problem in (a) persists since without access to data of its users’ interactions with other sites, there is no ground truth for evaluation. Moreover, it is most useful when (c) users’ preferences for the site can be estimated at the individual level, since the site can then personalize recommendations to individual users. We offer a method to estimate individual user’s preference for a focal site, under this premise. In particular, we compute the focal site’s share of a user’s online engagements without any data from other sites. We show an evaluation framework for the model using only the focal site’s data, allowing the site to test the model. We rely upon a Hierarchical Bayes Method and perform estimation in two different ways – Markov Chain Monte Carlo and Stochastic Gradient with Langevin Dynamics. Our results find good support for the approach to computing personalized share of engagement and for its evaluation.