feat: Integrate Flock Extension#6
Conversation
brandonzhou2002
changed the title
| @@ -0,0 +1,111 @@ | |||
| # QuackIR: Dense Retrieval with Flock + Ollama | |||
|
|
|||
| This short guide shows how to run a minimal dense-retrieval pipeline with the Flock extension and a local Ollama embedding model. | |||
There was a problem hiding this comment.
could you please add a brief description of what the Flock extension is? maybe with a link? same for ollama. let's assume the reader has no background knowledge except for the previous guides
There was a problem hiding this comment.
for sure. added
|
|
||
| The `ollama serve &` command starts the Ollama server in the background. | ||
| The environment variables control concurrency and memory usage. | ||
| If you encounter `server busy, please try again. maximum pending requests exceeded` error while running the script, consider increasing the `OLLAMA_MAX_QUEUE` value, or reducing the `batch_size` parameter in the `options_json` field when registering the model. |
There was a problem hiding this comment.
what do you mean by "registering the model"? perhaps link to a dedicated ollama guide for ollama specific setup?
| + Pull an embedding model: | ||
|
|
||
| ```bash | ||
| ollama pull embeddinggemma |
There was a problem hiding this comment.
embeddinggemma is a specific model you're using for the guide right? please clarify that so readers know they can also play around with different models following the same instructions. add a brief description/link to the model?
| top_k = 10 | ||
|
|
||
| """ | ||
| 2) Initialize Flock + register model alias |
There was a problem hiding this comment.
oh is this what "register model" refer to? maybe clarify that in the earlier text or move the message about parameters here?
| The `ollama serve &` command starts the Ollama server in the background. | ||
| The environment variables control concurrency and memory usage. | ||
| If you encounter `server busy, please try again. maximum pending requests exceeded` error while running the script, consider increasing the `OLLAMA_MAX_QUEUE` value, or reducing the `batch_size` parameter in the `options_json` field when registering the model. | ||
| Tune these settings based on your hardware capacity and workload. |
There was a problem hiding this comment.
if relevant, ie, has significant requirements for hardware, could you provide what the minimum requirements are? or something along the lines of, on this hardware setup, this is how long you would expect this process to take
| embedding_dim = 768 | ||
| model_alias = "Embedder" | ||
| queries_file = "collections/nfcorpus/queries.tsv" | ||
| output_path = Path("runs/run.quackir.duckdb.dense.flock.nfcorpus.txt") |
There was a problem hiding this comment.
please include embedding model name in run file name so it's easy to find
|
|
||
| class SecretProvider(Enum): | ||
| OLLAMA = 'OLLAMA' | ||
| OPENAI = 'OPENAI' |
There was a problem hiding this comment.
do we support openai and azure llm right now? if so, mention that in the documentation? eg add brief sections on if you wanted to do this with openai or azure, here's what steps would be different
There was a problem hiding this comment.
Yes we do support azure and openAI but I haven't tested with them yet. I will still add a section to show how to do it!
| ) | ||
| except Exception as e: | ||
| msg = str(e) | ||
| if skip_if_exists and ("Duplicate key" in msg or "already exists" in msg): |
There was a problem hiding this comment.
add comment for what this does?
| table_name = "corpus_dense" | ||
| corpus_file = "collections/nfcorpus/quackir_corpus.jsonl" | ||
| embedding_dim = 768 | ||
| model_alias = "Embedder" |
There was a problem hiding this comment.
what does this do? add comment?
| searcher.close() | ||
| ``` | ||
|
|
||
| ## Evaluate with trec_eval |
brandonzhou2002
left a comment
brandonzhou2002
left a comment
There was a problem hiding this comment.
Thanks for reviewing @lilyjge! I've resolved the comments and lemme know if there are any questions.
| @@ -0,0 +1,111 @@ | |||
| # QuackIR: Dense Retrieval with Flock + Ollama | |||
|
|
|||
| This short guide shows how to run a minimal dense-retrieval pipeline with the Flock extension and a local Ollama embedding model. | |||
There was a problem hiding this comment.
for sure. added
|
|
||
| class SecretProvider(Enum): | ||
| OLLAMA = 'OLLAMA' | ||
| OPENAI = 'OPENAI' |
There was a problem hiding this comment.
Yes we do support azure and openAI but I haven't tested with them yet. I will still add a section to show how to do it!
2 participants
This PR integrates Flock into the repo for dense retrieval, and adds a walkthrough markdown file to run the dense retrieval with Flock using Ollama local server.
Running process and time
