Indexing_And_Searching_Using_pyserini

This repository demonstrates indexing and searching the TREC datasets using Pyserini.

The process involves three steps:

1. Converting HTML documents to JSONL format;
2. Indexing the JSONL data with Lucene;
3. Performing BM25 searches.

The project utilizes Python, Pyserini, a compatible JDK, and several Python libraries (BeautifulSoup, tqdm, PyTorch, TorchVision, Transformers). Version compatibility among these components is crucial for successful execution.