Movie Recommendation System using Collaborative filtering on Netflix Prize Dataset (2006)
The following model is based on the blog by Simon Funk
Netflix Prize was a competition started by Netflix in 2006 to improve upon their existing recommender system, Cinematch. The goal of the competition was to improve upon the RMSE performance of Cinematch by 10% (RMSE ~0.855). Netflix released the dataset for the competition which consisted of actual ratings by different users. The dataset contined about 100M ratings by about 500K users and for 17.7K movies. The dataset guaranteed that each user has rated atleast one movie and that each movie has been rated by atleast one user. In addition to the train dataset, Netflix released a probe dataset which consisted of around 1.4M ratings. The probe dataset is a subset of train dataset. Other than the train and probe dataset, Netflix released the quiz data, the output for which was never released. The quiz data was used to calculate the final leaderboard position. Netflix also released a separate movie metadata file, that consisted of movie names and year of release.
A simple intuitive approach for recommendation system would be to use the average ratings
of movies and users to predict the rating of unrated movies. Let bi represent the average rating of ith movie and bj represent the rating offset from average movie rating for jth user. One would think to simply average over the ratings available for a movie to calculate the average rating for movie. But consider a movie that has been rated only by a single user. The average of such a movie is highly biased. Thus, we use Bayesian Average to calculate average. If Ra and Va are the mean and variance of all movies' average ratings (which defines prior expectation for a new movie's average rating before you have observed any actual rating) and Vb is the average variance of individual movie ratings (which tells how indicative each new observation is of the true mean), then,
K is a hyperparameter and can be experimented with different values. In my model, I use K=25. Using the above Bayesian average, we can compute a better estimate of the mean of individual movie rating and individual user rating offset. Thus, we can write bi :
And similarily for bj :
The baseline method though quite simple has obvious drawbacks, the major one being that every user is recommended same set of movies. Thus, it makes sense to add some complexity to the model, that would adapt the system to suggest movies based on individual preferences and not only on average movie ratings.
Consider the ratings as a large matrix, R, of shape (num_users, num_movies). We can factorize this matrix into 2 matrices, U and M, using SVD:
where, shape of U is (num_users, num_features) and M is (num_movies, num_features). These num_features represent the amount of distinct features in movie matrix, and preference of users for each feature in user matrix.
Note that simple SVD breaks a matrix into 3 sub-matrices:
But can be fused into the adjacent matrices.
The SVD looks great as a approach, but there is only one issue, we don't have the complete matrix to perform SVD. There are about 8.5B total cells in the matrix out of which we only know values for ~100M cells (quite sparse). To tackle this problem, we can use Gradient Descent to compute the matrices U and M, using MSE (square of RMSE) as the loss function.
Other than adapting better to individual user preference, SVD has another advantage in terms of model size, where instead of 8.5B parameters, it only uses 20M paramters.
Combining the Bayesian and SVD approach, one can improve on the RMSE. The equations for this looks like:
One could try to experiment with fine-tuning bi and bj while training, or keep them fixed. In the experiments I ran, keeping bi and bj fixed performed better on probe set, because it prevented overfitting for users and movies with small number of ratings available. The results for this experiments match with those mentioned by Simon Funk in his blog (Probe RMSE: ~0.932). Simon Funk suggested adding L2 regularization to feature vectors. I did not use regularization but instead used early stopping to prevent overfitting. However, L2 regularization could give a few decimal of improvement over the probe dataset.
SVD is in some sense a linear model and linear models could be limiting. One way to deal with this could be to add a non-linear function to the final prediction. I used sigmoid as a non-linear function, but you could try experimenting with different non-linear functions.
As suggested earlier, one could experiment with using L2 regularization on the feature vectors to improve RMSE on probe data. Another addition to the existing model could be to incorporate movie release date into the model. This can be done by adding a temporal paramter, . The idea is that ratings of movies change with time, movies in certain era are more liked by people, taste of people change with time. One fun thing to try would be to pass the feature vectors for user and movie into a neural network. The neural network here would serve as the non-linear function.
1- Clone the repository and head inside the repository folder
2- Download the Netflix Prize Dataset. If you have kaggle api installed and setup, simply run
sh run.sh data
If you do not have kaggle api set up, head to https://www.kaggle.com/netflix-inc/netflix-prize-data,
and download and extract the data in the data folder. If you follow this step, manually create,
output and logs directory in the main folder.
3- Train the data
sh run.sh train [v1/v2/v3/v4]
4- Test the data
sh run.sh test [v1/v2/v3/v4]
The results are saved in output/[v1/v2/v3/v4]/test_result.csv
5- Get recommendations for user
sh run.sh recommend [v1/v2/v3/v4] $user_id
where user_id is the id of user for which recommendation is required,
The results are saved in output/[v1/v2/v3/v4]/recommend_result.csv