PaperLens AI is a LLM-powered Python tool for summarizing and applying topic modeling on a large number of research paper PDFs completely for free using the Groq API. It was developed primarily for doing quick literature reviews during my internship at Stockholm School of Economics, but can also be used for basic Retrieval-Augmented Generation (RAG). Using the default settings it can process ca 4 standard-length papers/minute using the Llama3 8B model.
- Install the script by running one of below commands in a terminal depending on your OS:
Windows PowerShell:
iwr https://raw.githubusercontent.com/crabfeather/paperlens/main/PaperLensAI.py -O PaperLensAI.py
Linux/MacOS:
curl https://raw.githubusercontent.com/crabfeather/paperlens/main/PaperLensAI.py -o PaperLensAI.py
- Verify the latest version of Python3 and PIP are installed by running:
python --version
pip --version
- Install the required libraries:
pip install PyPDF2
pip install groq
- Edit the script to include a free Groq API key generated from https://console.groq.com/keys
- Modify the prompts and AI query parameters in the script to your liking. If needed edit the rate limits and chunk size accordingly.
- Run the script with Python for your folder of PDFs:
python PaperLensAI.py <input-folder>