- AI Search Assistant with Local Knowledge Base
- Quick Start
- Use Different LLM Endpoints
- Usage Examples
- Main Components
- Community
LeetTools is an AI search assistant that can perform highly customizable search workflows and save the search results and generated outputs to local knowledge bases. With an automated document pipeline that handles data ingestion, indexing, and storage, we can easily run complext search workflows that query, extract and generate content from the web or local knowledge bases.
LeetTools can run with minimal resource requirements on the command line with a DuckDB-backend and configurable LLM settings. It can be easily integrated with other applications need AI search and knowledge base support.
Here is an illustration of the LeetTools digest flow where it can search the web (or local KB) and generate a digest article from the search results:
And here is an example output article generated by the digest flow for the query How does Ollama work?.
Currently LeetTools provides the following workflow:
- answer : Answer the query directly with source references (similar to Perplexity). 📖
- digest : Generate a multi-section digest article from search results (similar to Google Deep Research). 📖
- search : Search for top segements that match the query. 📖
- news : Generate a list of news items for the specified topic. 📖
- extract : Extract and store structured data for given schema. 📖
- opinions: Generate sentiment analysis and facts from the search results. 📖
We can use any OpenAI-compatible LLM endpoint, such as local Ollama service or public provider such as Gemini or DeepSeek. We can switch the service easily by defining environment variables or switching .env files.
Run with pip
If you are using an OpenAI compatible LLM endpoint, you can install and run LeetTools with pip as follows:
% conda create -y -n leettools python=3.11
% conda activate leettools
% pip install leettools
% export EDS_LLM_API_KEY=<your_api_key>
% leet flow -t answer -q "How does GraphRAG work?" -k graphrag -l infoThe above flow -t answer command will run the answer flow with the query "How does
GraphRAG work?" and save the scraped web page to the knowledge base graphrag. The
-l info option will show the essential log messages.
By default the data is saved under ${HOME}/leettools, you can set a different LeetHome environment variable to change the location:
% export LEET_HOME=<your_leet_home>
% mkdir -p ${LEET_HOME}The default API endpoint is set to the OpenAI API endpoint, which you can modify by
changing the EDS_DEFAULT_LLM_BASE_URL environment variable:
% export EDS_DEFAULT_LLM_BASE_URL=https://api.openai.com/v1Run with source code
% git clone https://github.com/leettools-dev/leettools.git
% cd leettools
% conda create -y -n leettools python=3.11
% conda activate leettools
% pip install -r requirements.txt
% pip install -e .
# add the script path to the path
% export PATH=`pwd`/scripts:${PATH}
% export EDS_LLM_API_KEY=<your_api_key>
% leet flow -t answer -q "How does GraphRAG work?" -k graphrag -l infoSample Output
Here is an example output of the answer flow:
# How Does Graphrag Work?
GraphRAG operates by constructing a knowledge graph from a set of documents, which
involves several key steps. Initially, it ingests textual data and utilizes a large
language model (LLM) to extract entities (such as people, places, and concepts) and
their relationships, mapping these as nodes and edges in a graph structure[1].
The process begins with pre-processing and indexing, where the text is segmented into
manageable units, and entities and relationships are identified. These entities are
then organized into hierarchical "communities," which are clusters of related topics
that allow for a more structured understanding of the data[2][3].
When a query is made, GraphRAG employs two types of searches: Global Search, which
looks across the entire knowledge graph for broad connections, and Local Search, which
focuses on specific subgraphs for detailed information[3]. This dual approach enables
GraphRAG to provide comprehensive answers that consider both high-level themes and
specific details, allowing it to handle complex queries effectively[3][4].
In summary, GraphRAG enhances traditional retrieval-augmented generation (RAG) by
leveraging a structured knowledge graph, enabling it to provide nuanced responses that
reflect the interconnected nature of the information it processes[1][2].
## References
[1] [https://www.falkordb.com/blog/what-is-graphrag/](https://www.falkordb.com/blog/what-is-graphrag/)
[2] [https://medium.com/@zilliz_learn/graphrag-explained-enhancing-rag-with-knowledge-graphs-3312065f99e1](https://medium.com/@zilliz_learn/graphrag-explained-enhancing-rag-with-knowledge-graphs-3312065f99e1)
[3] [https://medium.com/data-science-in-your-pocket/how-graphrag-works-8d89503b480d](https://medium.com/data-science-in-your-pocket/how-graphrag-works-8d89503b480d)
[4] [https://github.com/microsoft/graphrag/discussions/511](https://github.com/microsoft/graphrag/discussions/511)We can run LeetTools with different env files to use different LLM providers and other related settings.
# you may need to pull the models first
% ollama pull llama3.2
% ollama pull nomic-embed-text
% ollama serve
% cat > .env.ollama <<EOF
EDS_DEFAULT_LLM_BASE_URL=http://localhost:11434/v1
EDS_LLM_API_KEY=dummy-llm-api-key
EDS_DEFAULT_INFERENCE_MODEL=llama3.2
EDS_DEFAULT_EMBEDDING_MODEL=nomic-embed-text
EDS_EMBEDDING_MODEL_DIMENSION=768
EOF
# Then run the command with the -e option to specify the .env file to use
% leet flow -e .env.ollama -t answer -q "How does GraphRAG work?" -k graphrag.ollama -l infoFor another example, since DeepSeek does not provide an embedding endpoint yet, we can
use the "EDS_DEFAULT_DENSE_EMBEDDER" setting to specify a local embedder with a default
all-MiniLM-L6-v2 model:
### to you can put the settings in the .env.deepseek file
% cat > .env.deepseek <<EOF
LEET_HOME=</Users/myhome/leettools>
EDS_DEFAULT_LLM_BASE_URL=https://api.deepseek.com/v1
EDS_LLM_API_KEY=<sk-0d8-mykey>
EDS_DEFAULT_INFERENCE_MODEL=deepseek-chat
EDS_DEFAULT_DENSE_EMBEDDER=dense_embedder_local_mem
EOF
# Then run the command with the -e option to specify the .env file to use
% leet flow -e .env.deepseek -t answer -q "How does GraphRAG work?" -k graphrag -l infoIf you want to use another API provider (OpenAI compatible) for embedding, say a local Ollama embedder, you can set the embedding endpoint URL and API key separately as follows:
% cat > .env.deepseek <<EOF
EDS_DEFAULT_LLM_BASE_URL=https://api.deepseek.com/v1
EDS_LLM_API_KEY=<sk-0d8-mykey>
EDS_DEFAULT_INFERENCE_MODEL=deepseek-chat
# this specifies to use an OpenAI compatible embedding endpoint
EDS_DEFAULT_DENSE_EMBEDDER=dense_embedder_openai
# the following specifies the embedding endpoint URL and model to use
EDS_DEFAULT_EMBEDDING_BASE_URL=http://localhost:11434/v1
EDS_DEFAULT_EMBEDDING_MODEL=nomic-embed-text
EDS_EMBEDDING_MODEL_DIMENSION=768
EOFWe can build a local knowledge base with PDFs from the web. Suppose we have set up the local Ollama service as described above, now we can use the following commands to build a local knowledge base with PDFs from the web:
# create a KB with a URL
# the book downloded here is "Foundations of Large Language Models"
# it has 231 pages and take some time to process
% leet kb add-url -e .env.ollama -k llmbook -r "https://arxiv.org/pdf/2501.09223"
# now you can query the KB with any topic you want to explore
% leet kb flow -e .env.ollama -t answer -k llmbook -l info \
-q "How does LLM Finetuning process work?" We have a more detailed example to show how to use the local Ollama service with the DeepSeek-r1:1.5B model to build a local knowledge base.
We can create a knowledge base with a list of URLs or a search query, and then generate a list of news items from the KB. Here is an example:
# create a KB with a google search
# -d 1 means to search for news from the last day
# -m 30 means to scrape the top 30 search results
% leet kb add-search -k genai -q "LLM GenAI Startups" -d 1 -m 30
# you can add single url to the KB
% leet kb add-url -k genai -r "https://www.techcrunch.com"
# you can also add a list of urls, example in [docs/sample_urls.txt](docs/sample_urls.txt)
% leet kb add-url-list -k genai -f <file_with_list_of_urls>
# generate a news list from the KB
% leet flow -t news -q "LLM GenAI Startups" -k genai -l info -o llm_genai_news.md
# Next time you want to refresh the KB and generate the news list
# this command will re-ingest all the docsources specified above
% leet kb ingest -k genai
# run the news flow again with parameter you need
% leet flow -t news --info
====================================================================================================
news: Generating a list of news items from the KB.
This flow generates a list of news items from the updated items in the KB:
1. check the KB for recently updated documents and find news items in them.
2. combine all the similar items into one.
3. remove items that have been reported before.
4. rank the items by the number of sources.
5. generate a list of news items with references.
====================================================================================================
Use -p name=value to specify options for news:
article_style : The style of the output article such as analytical research reports, humorous
news articles, or technical blog posts. [default: analytical research reports]
[FLOW: news]
days_limit : Number of days to limit the search results. 0 or empty means no limit. In
local KB, filters by the import time. [FLOW: news]
news_include_old : Include all news items in the result, even if it has been reported
before.Default is False. [default: False] [FLOW: news]
news_source_min : Number of sources a news item has to have to be included in the result.Default
is 2. Depends on the nature of the knowledge base. [default: 2] [FLOW: news]
output_language : Output the result in the language. [FLOW: news]
word_count : The number of words in the output section. Empty means automatics.
[FLOW: news]
Note: scheduler support and UI view are coming soon.
The main components of the backend include:
- 🚀 Automated document pipeline to ingest, convert, chunk, embed, and index documents.
- 🗂️ Knowledge base to manage and serve the indexed documents.
- 🔍 Search and retrieval library to fetch documents from the web or local KB.
- 🤖 Workflow engine to implement search-based AI workflows.
- ⚙ Configuration system to support dynamic configurations used for every component.
- 📝 Query history system to manage the history and the context of the queries.
- 💻 Scheduler for automatic execution of the pipeline tasks.
- 🧩 Accounting system to track the usage of the LLM APIs.
The architecture of the document pipeline is shown below:
See the Documentation for more details.
Acknowledgements
Right now we are using the following open source libraries and tools (not limited to):
We plan to add more plugins for different components to support different workloads.
Get help and support
Please feel free to connect with us using the discussion section.
Contributing
Please read Contributing to LeetTools for details.
License
LeetTools is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.