Replies: 2 comments
-
Thank you for bringing this up. To address your query, building connectors is definitely within the scope of llm-app. However, it's worth noting that the connectors in pathway are architecturally and conceptually distinct from those you mentioned in danswer and khoj. To better understand our approach and to create a connector for pathway, I recommend going through our tutorial here: Custom Python Connectors. This should provide a clear overview and guide on our unique take on connectors. The idea of a recursive web crawling feature sounds intriguing, we'll certainly consider it. |
Beta Was this translation helpful? Give feedback.
-
Thanks @mdmalhou ! I'm delighted to hear that it is in scope |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
A common theme, and arguably table stakes, among RAG software like
llm-app
seems to be the concept of connectors. Here are some examples:If building connectors are in scope for this project, I'd like to put in a request for recursive web crawling similar to Langchain's recursive url loader or this one
Beta Was this translation helpful? Give feedback.
All reactions