Skip to content

Commit d28d42e

Browse files
sumana-2705sumana-2705
authored
Update README with Setup Instructions and Architecture Overview (#146)
* Update README.md Signed-off-by: Sumana Sree Angajala <110307215+sumana-2705@users.noreply.github.com> * Update README.md Signed-off-by: Sumana Sree Angajala <110307215+sumana-2705@users.noreply.github.com> * added snowflake architecture image Signed-off-by: Sumana Sree Angajala <110307215+sumana-2705@users.noreply.github.com> * added snowflake-architecture image to images folder Signed-off-by: Angajala Sumana Sree <sumanasree2705@gmail.com> * Edited README for snowflake branch Signed-off-by: Sumana Sree Angajala <110307215+sumana-2705@users.noreply.github.com> --------- Signed-off-by: Sumana Sree Angajala <110307215+sumana-2705@users.noreply.github.com> Signed-off-by: Angajala Sumana Sree <sumanasree2705@gmail.com>
1 parent e39d1a5 commit d28d42e

File tree

2 files changed

+121
-119
lines changed

2 files changed

+121
-119
lines changed

‎README.md‎

Lines changed: 121 additions & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -1,164 +1,166 @@
1-
# DEV Branch: Hyperledger Labs AIFAQ prototype
1+
# Snowflake Branch: Hyperledger Labs AIFAQ prototype
22

3-
The scope of this Hyperledger Labs project is to support the users (users, developer, etc.) to their work, avoiding to wade through oceans of documents to find information they are looking for. We are implementing an open source conversational AI tool which replies to the questions related to specific context. This is a prototype which allows to create a chatbot running a RESTful API which requires GPU. Here the official Wiki pages: [Hyperledger Labs aifaq](https://labs.hyperledger.org/labs/aifaq.html) and [Hyperledger Labs wiki](https://wiki.hyperledger.org/display/labs/AI+FAQ). Please, read also the [Antitrust Policy and the Code of Conduct](https://wiki.hyperledger.org/pages/viewpage.action?pageId=41587043). Every Monday we have a public meeting and the invitation is on the Hyperledger Labs calendar: [[Hyperledger Labs] FAQ AI Lab calls](https://wiki.hyperledger.org/display/HYP/Calendar+of+Public+Meetings).
3+
![Hyperledger Labs](https://img.shields.io/badge/Hyperledger-Labs-blue?logo=hyperledger)
4+
![Apache License 2.0](https://img.shields.io/badge/license-Apache%202.0-green.svg)
5+
![Contributions Welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg)
6+
![Python](https://img.shields.io/badge/python-3.9+-blue.svg?logo=python)
47

5-
## Background
8+
[![GitHub Stars](https://img.shields.io/github/stars/hyperledger-labs/aifaq?style=social)](https://github.com/hyperledger-labs/aifaq/stargazers)
9+
[![GitHub Forks](https://img.shields.io/github/forks/hyperledger-labs/aifaq?style=social)](https://github.com/hyperledger-labs/aifaq/network/members)
10+
[![Language Stats](https://img.shields.io/github/languages/top/hyperledger-labs/aifaq)](https://github.com/hyperledger-labs/aifaq)
11+
[![Issues](https://img.shields.io/github/issues/hyperledger-labs/aifaq)](https://github.com/hyperledger-labs/aifaq/issues)
12+
[![Pull Requests](https://img.shields.io/github/issues-pr/hyperledger-labs/aifaq)](https://github.com/hyperledger-labs/aifaq/pulls)
613

7-
The system is an open source python project which implements an AI chatbot that replies to HTTP requests. The idea is to implement an open source framework/template, as example, for other communities/organizations/companies. Last results in open LLMs allow to have good performance using common HW resources.\
8-
Below the application architecture:
14+
![Language Stats](https://img.shields.io/github/languages/count/hyperledger-labs/aifaq)
15+
![Python](https://img.shields.io/badge/Python-85%25-blue?logo=python)
16+
![HTML](https://img.shields.io/badge/HTML-10%25-orange?logo=html5)
17+
![Other](https://img.shields.io/badge/Others-5%25-lightgrey?logo=github)
918

10-
<img src="./images/prototype_schema_v1.drawio.png" alt="LLM chatbot schema" width="750"/>
19+
---
20+
## 🚀 Overview
1121

12-
We use RAG (Retrieval Augmented Generation [arxiv.org](https://arxiv.org/abs/2312.10997)) for question answering use case. That technique aims to improve LLM answers by incorporating knowledge from external database (e.g. vector database).
22+
The **Hyperledger Labs AIFAQ Prototype** is an open-source conversational AI tool designed to answer questions from technical documentation, FAQs, and internal knowledge bases with high accuracy and context awareness. This implementation of AIFAQ integrates deeply with **Snowflake**, providing secure multi-user support, persistent chat history, and access to powerful LLMs like OpenAI, Anthropic, and Snowflake Cortex.
1323

14-
The image depicts two workflow:
24+
👉 Official Wiki Pages:
1525

16-
1. The data ingestion workflow
17-
2. The chat workflow
26+
- [Hyperledger Labs Wiki](https://lf-hyperledger.atlassian.net/wiki/spaces/labs/pages/20290949/AI+FAQ+2025)
1827

19-
During the ingestion phase, the system loads context documents and creates a vector database. For example, the document sources can be:
28+
👉 Weekly Community Calls:
2029

21-
- An online software guide (readthedocs template)
22-
- The GitHub issues and pull requests
30+
- Every Monday (public) — join via [Hyperledger Labs Calendar](https://wiki.hyperledger.org/display/HYP/Calendar+of+Public+Meetings).
2331

24-
In our case, they are the readthedocs guide and a wiki page.\
25-
After the first phase, the system is ready to reply to user questions.
32+
---
33+
## Features
2634

27-
Currently, we use the open source [HuggingFace Zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta), and in the future we want to investigate other open source models.\
28-
The user can query the system using HTTP requests, but we want to supply UI samples, as external module.
35+
- User Authentication: Secure, multi-user access with isolated document and chat histories
36+
- LLM Integration: Seamless access to Cortex, OpenAI, and Anthropic models via Snowflake external functions
37+
- Multi-Document Support: Upload and query multiple documents per session
38+
- Persistent Chat History: Retrieve and continue conversations across sessions
39+
- Streamlit UI: Intuitive document upload and chat interface
2940

30-
## Open Source Version
41+
---
42+
## 🛠️ Architecture
3143

32-
The software is under Apache 2.0 License (please check **LICENSE** and **NOTICE** files included). We use some 3rd party libraries: here is the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html) and here is the information for ([Assembling LICENSE and NOTICE files](https://infra.apache.org/licensing-howto.html#mod-notice)).
44+
![Snowflake integration architecture](./images/snowflake-architecture.png)
3345

34-
## Installation
46+
1. Flexible Document Ingestion: AIFAQ supports various source formats (PDFs, HTML, YouTube transcripts, etc.) ingested into Snowflake via external tables, raw storage, and pipelines using tools like Snowpipe and Lambda-based metadata extractors.
47+
2. Preprocessing & Embedding: Documents are chunked using Snowpark UDFs and embedded using LLM-based models. Embedding vectors are stored in Snowflake, forming the searchable knowledge base alongside metadata.
48+
3. Access Control & Governance: Fine-grained access is enforced through Snowflake's role-based permissions, row-level security, and data masking policies to protect sensitive content.
49+
4. LLM Query Augmentation & Retrieval: User queries are augmented with context by retrieving relevant chunks from the vector database (via Cortex Vector Search or SQL API), then sent to external LLMs (OpenAI, Anthropic) for response generation.
50+
5. Automation & Monitoring: Updates to documents automatically re-trigger embedding pipelines using Snowflake Streams and Tasks, while monitoring tools like Snoopy and event notifications ensure system observability and orchestration.
3551

36-
<u>**This document does not contain commercial advertisement: all the tools/products/books/materials are generic and you have to consider those as examples!**</u>\
37-
This software needs GPU for the execution: if you do not have a local GPU you could use a Cloud GPU. There are several solutions to use Cloud GPU:
52+
---
53+
## 📝 Setup Instructions (Snowflake Branch)
54+
Follow these steps to configure your Snowflake environment using the provided `setup.sql` script.
3855

39-
1. Cloud Provider (AWS, GCP, ...)
40-
2. On-Demand GPU Cloud (vast.ai, RunPod, ...)
41-
3. Cloud GPU IDE
42-
43-
Currently, I use a Cloud GPU IDE ([Lightning Studio](https://lightning.ai/studios)), after signup/login, create new Studio (project):
44-
45-
![New Studio Button](/images/new_studio.png)
46-
47-
select the left solution:
48-
49-
![Select Studio Code](/images/studio_code.png)
50-
51-
click on the **Start** button, and rename the new Studio:
52-
53-
![Rename Studio](/images/rename_studio.png)
54-
55-
Then, and copy-paste the github api repo code:
56-
57-
![Copy Paste Code](/images/copy_paste_code.png)
58-
59-
and create two folders:
60-
61-
1. chromadb (it will contains vector database files)
62-
2. rtdocs (it will contains the ReadTheDocs documentation)
63-
64-
That version works with Hyperledger fabric documents (Wiki and ReadTheDocs).
65-
66-
## Usage
67-
68-
### Download ReadTheDocs documentation
69-
70-
Open a new terminal:
71-
72-
![Open Terminal](/images/open_terminal.png)
73-
74-
and download the documentation executing the command below:
75-
76-
```console
77-
wget -r -A.html -P rtdocs https://hyperledger-fabric.readthedocs.io/en/release-2.5/
56+
1. Set up a role for the chatbot and grant access to required resources:
7857

7958
```
59+
CREATE OR REPLACE ROLE chatbot_user;
8060
81-
actually, after a minute we can interrupt (CTRL + C) because it starts to download previous versions:
82-
83-
![Wget Command](/images/wget_rtdocs.png)
84-
85-
Now, we can move into rtdocs folder and move the **release-2.5** content to **rtdocs**. We need to compress the content of the folder, moving there and use that command:
61+
GRANT USAGE ON WAREHOUSE compute_wh TO ROLE chatbot_user;
62+
GRANT USAGE ON DATABASE llm_chatbot TO ROLE chatbot_user;
8663
87-
![Compress files](/images/compress_files.png)
88-
89-
and move the readthedocs.tar.gz to the parent directory (../):
90-
91-
```console
92-
- mv readthedocs.tar.gz ..
93-
- cd ..
9464
```
65+
2. Initialize the database and schema for storing documents and chat data:
9566

96-
repeating the two commands until we are into rtdocs folder:
97-
98-
![Move Command](/images/move_command.png)
99-
100-
now remove hyperledger… folder and the content:
101-
102-
![Compress files](/images/remove_command.png)
103-
104-
uncompress the file here and remove compress file:
105-
106-
```console
107-
- tar -xzvf rtdocs.tar.gz
108-
- rm rtdocs.tar.gz
10967
```
68+
CREATE OR REPLACE DATABASE llm_chatbot;
69+
CREATE OR REPLACE SCHEMA chatbot;
70+
USE SCHEMA llm_chatbot.chatbot;
11071
111-
### Install requirements
112-
113-
Move to the parent folder and execute the command below:
114-
115-
```console
116-
pip install -r requirements.txt
11772
```
73+
3. Create two core tables, one for document chunks and another for chat history:
11874

119-
### Activate GPU
75+
```
76+
CREATE OR REPLACE TABLE documents (
77+
user_id STRING,
78+
doc_id STRING,
79+
doc_name STRING,
80+
chunk_id STRING,
81+
chunk_text STRING,
82+
embedding VECTOR(FLOAT, 1536)
83+
);
84+
85+
CREATE OR REPLACE TABLE chat_history (
86+
user_id STRING,
87+
session_id STRING,
88+
doc_id STRING,
89+
turn INT,
90+
user_input STRING,
91+
bot_response STRING,
92+
timestamp TIMESTAMP
93+
);
94+
```
95+
4. External Function – OpenAI: Create an external function to call OpenAI's API:
12096

121-
After the requirements installation we can switch to GPU before to execute the ingestion script:
97+
```
98+
CREATE OR REPLACE EXTERNAL FUNCTION openai_complete(prompt STRING)
99+
RETURNS STRING
100+
API_INTEGRATION = my_api_integration
101+
HEADERS = (
102+
"Authorization" = 'Bearer <OPENAI_API_KEY>',
103+
"Content-Type" = 'application/json'
104+
)
105+
URL = 'https://api.openai.com/v1/completions'
106+
POST_BODY = '{
107+
"model": "gpt-3.5-turbo-instruct",
108+
"prompt": "' || prompt || '",
109+
"max_tokens": 200
110+
}';
122111
123-
![Activate GPU](/images/activate_gpu.png)
112+
```
113+
> Replace <OPENAI_API_KEY> with your actual OpenAI API key.
124114
125-
then select the L4 solution:
115+
5. External Function – Anthropic: Similarly, set up a function to call Anthropic's Claude model:
126116

127-
![Select L4](/images/select_L4.png)
117+
```
118+
CREATE OR REPLACE EXTERNAL FUNCTION anthropic_complete(prompt STRING)
119+
RETURNS STRING
120+
API_INTEGRATION = my_api_integration
121+
HEADERS = (
122+
"x-api-key" = '<ANTHROPIC_API_KEY>',
123+
"Content-Type" = 'application/json'
124+
)
125+
URL = 'https://api.anthropic.com/v1/complete'
126+
POST_BODY = '{
127+
"model": "claude-3-opus-20240229",
128+
"prompt": "Human: ' || prompt || '\nAssistant:",
129+
"max_tokens": 200
130+
}';
128131
129-
and confirm (it takes some minutes).
132+
```
133+
> Replace <ANTHROPIC_API_KEY> with your actual key.
130134
131-
### Ingest step
135+
6. Deploy the chatbot interface using the Streamlit app stored in your project:
132136

133-
Run the ingest.py script:
137+
```
138+
CREATE OR REPLACE STREAMLIT chatbot_ui
139+
FROM '/chatbot_app'
140+
MAIN_FILE = '/app.py';
141+
```
134142

135-
![Run Ingest](/images/run_ingest.png)
143+
---
136144

137-
it will create content in chromadb folder.
145+
## 🌐 Open Source License
138146

139-
### Run API
147+
- **License:** Apache 2.0 (see [`LICENSE`](./LICENSE) and [`NOTICE`](./docs/NOTICE))
148+
- **3rd Party Libraries:** [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html)
149+
- **License Assembly:** [Assembling LICENSE and NOTICE](https://infra.apache.org/licensing-howto.html#mod-notice)
140150

141-
Now, we can run the API and test it. So, run api.py script:
142151

143-
![Run API](/images/run_api.png)
152+
## 🤝 Contributing
144153

145-
and test it:
154+
We welcome contributions! Please check our [CONTRIBUTING](./docs/CONTRIBUTING.md) guidelines and [Antitrust Policy and Code of Conduct](https://lf-hyperledger.atlassian.net/wiki/spaces/HIRC/pages/19169404/Anti-trust+Policy+Notice+Code+of+Conduct).
146155

147-
```console
148-
curl --header "Content-Type: application/json" --request POST --data '{"text": "How to install Hyperledger fabric?"}' http://127.0.0.1:8080/query
149-
```
150156

151-
below the result:
157+
## 📆 Join Us!
152158

153-
![Show results](/images/curl_results.png)
159+
Join our weekly public calls every Monday! See the [Hyperledger Labs Calendar](https://wiki.hyperledger.org/display/HYP/Calendar+of+Public+Meetings) for details.
154160

155-
## Current version notes
156161

157-
That is a proof-of-concept: a list of future improvement below:
162+
## 📢 Stay Connected
158163

159-
1. This is the first version of the prototype and it will be installed on a GPU Cloud Server
160-
2. At the same time, we'd like to pass to the next step: the Hyperledger Incubation Stage
161-
3. We will investigate other open source models
162-
4. Evaluation of the system using standard metrics
163-
5. We would like to improve the system, some ideas are: fine-tuning, Advanced RAG, Decomposed LoRA
164-
6. Add "guardrails" which are a specific ways of controlling the output of a LLM, such as talking avoid specific topics, responding in a particular way to specific user requests, etc.
164+
- [Slack Discussions](https://join.slack.com/t/aifaqworkspace/shared_invite/zt-337k74jsl-tvH_4ct3zLj99dvZaf9nZw)
165+
- [Hyperledger Labs Community](https://lf-hyperledger.atlassian.net/wiki/spaces/labs/pages/20290949/AI+FAQ+2025)
166+
- Official Website: [aifaq.pro](https://aifaq.pro)

‎images/snowflake-architecture.png‎

193 KB
Loading

0 commit comments

Comments
 (0)