Amazon Movies and TV Information Retrieval System

Authors: Manikanth Goud, Michael Kmetiuk

Overview

This project builds and evaluates an end to end information retrieval (IR) and Retrieval Augmented Generation (RAG) pipeline for the Amazon Reviews 2023 Movies & TV dataset. It combines BM25 and semantic retrieval into a hybrid retriever, feeds the retrieved passages to an LLM (Llama 3.3 70b via Groq) to generate final answers, and makes it all available through an interactive Streamlit application.

The corpus spans 21,861 unique products, 983,972 Amazon reviews, and 1,619,968 searchable chunks.

Live App

Environment	Purpose	URL
Production	Public view	View Production Version

---

RAG Pipeline Workflow

Steps:

1. The user query is sent simultaneously to BM25 (keyword) and FAISS (semantic) retrievers.
2. Reciprocal Rank Fusion (RRF) merges the two ranked lists into a single top-k result.
3. Retrieved passages are assembled into a structured prompt context.
4. Llama 3.3 70b (Groq) generates the final answer; if reviews lack the information it calls the Tavily web search tool as a fallback.

---

Project Structure

.
├── README.md
├── environment.yml              # Conda environment specification
├── requirements.txt             # Pip dependencies (Python 3.11)
├── requirements_aws.txt         # Pip dependencies for AWS/Python 3.9
├── .gitignore
│
├── data/
│   ├── raw/                     # Downloaded/converted Parquet files (git-ignored)
│   │   ├── reviews_raw.parquet
│   │   ├── meta_raw.parquet
│   │   └── merged.parquet
│   └── processed/               # Built indices and document store
│       ├── bm25_index.pkl
│       ├── faiss_index/
│       ├── movies_and_tv_documents.pkl
│       └── feedback.csv         # User feedback from the app (auto-created)
│
├── notebook/
│   ├── milestone1_exploration.ipynb   # EDA, data download, index building, metric evaluation
│   ├── milestone2_rag.ipynb           # RAG pipeline exploration and qualitative evaluation
│   └── LLM_experiment.ipynb           # LLM comparison: Llama 3.3 70b vs Llama 3.1 8b
│
├── src/
│   ├── bm25.py                  # BM25  retrieval
│   ├── semantic.py              # Semantic retrieval with sentence embeddings + FAISS
│   ├── hybrid.py                # Hybrid retriever (EnsembleRetriever + RRF)
│   ├── rag_pipeline.py          # Full RAG pipeline: hybrid retriever + Groq LLM + tools
│   ├── web_search_tool.py       # Tavily web search tool for the RAG agent
│   ├── retrieval_metrics.py     # RR, Precision@K evaluation
│   ├── utils.py                 # Document building / serialization helpers
│   └── build.py                 # Index build script
│
├── results/
│   ├── milestone1_discussion.md
│   ├── milestone2_discussion.md
│   └── final_discussion.md
│
└── app/
    └── app.py                   # Streamlit app (Search + RAG modes)

---

Setup

Prerequisites

• Python 3.11 (local) or Python 3.9 (AWS EC2)
• Conda (recommended for local) or pip + virtualenv
• ~4 GB free disk space for data and model weights
• Internet access for the initial data download and model download
• Groq API key (free at console.groq.com) - required for RAG mode
• Tavily API key (free at tavily.com) - optional, enables web search fallback

---

Step 1 - Clone the repository

git clone https://github.com/UBC-MDS/DSCI_575_project_goudmani_mkmetiuk
cd DSCI_575_project_goudmani_mkmetiuk

---

Step 2 - Create the environment

Option A - Conda (recommended, exact package pins):

conda env create -f environment.yml
conda activate rag_project

Option B - pip + virtualenv (local, Python 3.11):

python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Option C - pip + virtualenv (AWS EC2, Python 3.9):

python3 -m venv rag_env
source rag_env/bin/activate
pip install -r requirements_aws.txt

Note: Installation may take several minutes.

---

Step 3 - Set API keys

Create a .env file in the repo root (or export the variables in your shell):

GROQ_API_KEY=your_groq_key_here
TAVILY_API_KEY=your_tavily_key_here   # optional

The app loads these automatically via python-dotenv. RAG mode is disabled if GROQ_API_KEY is not set; web search is disabled if TAVILY_API_KEY is not set.

---

Building the Indices

The app requires pre-built BM25 and FAISS indices. You can build them either by running the build script (recommended) or by running the notebook.

Option A - Build script (recommended)

From the repo root, run:

python src/build.py

This downloads the raw data, merges it, builds the document store, and writes all index files. No Jupyter required. The script will take several minutes on first run due to the data download and embedding step.

Option B - Run the notebook

If you also want to explore the data and evaluate retrieval metrics, run the notebooks:

jupyter lab

Notebook	What it does
notebook/milestone1_exploration.ipynb	Downloads data, builds BM25 + FAISS indices, evaluates retrieval metrics
notebook/milestone2_rag.ipynb	Explores the RAG pipeline: loads the hybrid retriever, runs example queries through the Groq LLM, shows retrieved passages and generated answers
notebook/LLM_experiment.ipynb	Compares Llama 3.3 70b vs Llama 3.1 8b on 5 identical queries using the same hybrid retriever

Verify the outputs

After running either option, confirm the following files exist:

data/processed/
├── bm25_index.pkl
├── faiss_index/
│   ├── index.faiss
│   └── index.pkl
└── movies_and_tv_documents.pkl

---

Running the Streamlit App

The app requires the indices built by the notebook. Make sure the environment is activated and API keys are set before running.

Step 1 - Start the app

# From the repo root:
streamlit run app/app.py

Streamlit will launch automatically or will print a local URL. In case of the later, open it in your browser.

Step 2 - Use the app

The app has two tabs:

Search Only tab

The sidebar lets you choose:

Control	Description
Retrieval Mode	BM25 (keyword), Semantic (vector), or Hybrid (Reciprocal Rank Fusion)
Top-K results	Number of results to return (3–20)
Hybrid α	Weight between BM25 (α = 0) and Semantic (α = 1); only shown in Hybrid mode

Type a query and press Search or hit Enter. Each result card shows the product title, a review snippet, the star rating, and a relevance score. Use the 👍 / 👎 buttons to leave relevance feedback - saved locally to data/processed/feedback.csv.

RAG Mode tab

Type a question and press Ask RAG or hit Enter. The agent:

1. Searches Amazon reviews using the hybrid retriever
2. Optionally falls back to Tavily web search for current information
3. Displays the generated answer, the tools called, and the retrieved source passages

RAG mode requires GROQ_API_KEY to be set. Web search requires TAVILY_API_KEY.

Troubleshooting

Symptom	Fix
"Indices could not be loaded" on startup	Run python src/build.py (or milestone1_exploration.ipynb) to build index files
"RAG agent unavailable: GROQ_API_KEY not set"	Add GROQ_API_KEY to your .env file and restart the app
ModuleNotFoundError for bm25 or semantic	Make sure you launched Streamlit with the conda/venv environment activated
Slow first load	FAISS index and sentence-transformer model are loaded once at startup; subsequent queries are fast
Port already in use	Run streamlit run app/app.py --server.port 8502

---

Source Modules

File	Description
src/bm25.py	build_bm25_search, load_bm25, execute_bm25_search
src/semantic.py	build_semantic_search, load_semantic, execute_semantic_search using sentence-transformers + FAISS
src/hybrid.py	create_hybrid_retriever - combines BM25 + FAISS via LangChain EnsembleRetriever
src/rag_pipeline.py	load_rag_agent, run_rag_agent - LangGraph ReAct agent with hybrid retrieval + Groq LLM
src/web_search_tool.py	web_search - Tavily-backed web search tool for the RAG agent
src/retrieval_metrics.py	evaluate_retrieval - computes nDCG@K, MAP@K, MRR, Precision@K
src/utils.py	build_documents, chunk_documents, save_documents, load_documents

---

Key Dependencies

Library	Purpose
rank-bm25	BM25 lexical retrieval
sentence-transformers	Dense passage/query encoding (all-MiniLM-L6-v2)
faiss-cpu	Approximate nearest-neighbour index
langchain-community	BM25Retriever, FAISS vectorstore
langchain-classic	EnsembleRetriever for hybrid search
langchain-groq	Groq LLM integration
langgraph	ReAct agent framework
tavily-python	Web search tool (optional)
streamlit	Interactive web app
pytrec-eval-terrier	Standard IR metric computation
python-dotenv	.env file loading
pandas / numpy	Data manipulation

---

Results

Milestone	Discussion
Milestone 1	results/milestone1_discussion.md - BM25 vs semantic retrieval metrics
Milestone 2	results/milestone2_discussion.md - Model rationale, prompt experiments, qualitative RAG evaluation
Final	results/final_discussion.md - LLM comparison, tool integration, code quality, cloud deployment plan

---

Milestones

• Milestone 1 - Data exploration, BM25 baseline, semantic retrieval baseline, metric comparison (nDCG, MAP, MRR, P@K)
• Milestone 2 - Hybrid retriever (RRF), RAG pipeline (Groq LLM + tool calling), Tavily web search integration, updated Streamlit app
• Final - LLM comparison (Llama 3.3 70b vs Llama 3.1 8b), tool integration documentation, code quality improvements, cloud deployment plan, app deployed at informationbm25semantic.streamlit.app