Caution
This repository contains the final project. The notes for the course are in a separate repository.
Project for Algorithms for Massive Datasets course at Università degli Studi di Milano - Master degree in Computer Science (a.y. 2025/26).
The implementation can be found in the main.ipynb notebook, while the report in its own folder (written in Typst, which is better than LaTeX).
One-click run with Google Colab:
Compiled Report PDF: report.pdf
The project implements a content-based recommendation system for news articles using the New York Times Articles & Comments (2020) dataset. User interests are inferred from their commenting behavior, and both users and articles are represented as feature vectors based on article metadata (newsdesk, section, keywords) and comment history. To efficiently identify similar user-article pairs from billions of potential comparisons, the system uses Local Sensitive Hashing (LSH) with random hyperplane sketches, followed by cosine similarity calculation. The implementation leverages Apache Spark for distributed processing, enabling scalability to massive datasets. Results show the approach generates high-quality recommendations for focused user profiles, with poorer performance for noisy user profiles.