You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* started refactor
* wip add local cache
* working local logic for get_element
* format with black
* update sync sdk generation
* Refactor element retrieval logic to utilize a unified Config class for managing configurations across async and sync APIs. Updated get_element and related functions to support cached selectors and improved error handling. Removed deprecated code and streamlined selector handling in both async and sync contexts. Added new CachedSelectorDTO for better cache management.
* Add CachedExtractDTO and update extract logic for caching scripts
- Introduced CachedExtractDTO for managing cached script data.
- Updated get_script function to accept URL instead of domain.
- Modified extract logic to include caching behavior for scripts.
- Removed unused cache flags from ExtractDTO and adjusted related logic.
* refactor extract agent
* add test implemenation of setup_auth
* add local storagestate storage
* refactor to flatten project structure
* update to test multiple cached selectors and scripts
* add logger configuration and remove unused cache utility functions
* remove old stuff in cli
* Update README.md
---------
Co-authored-by: Arian Hanifi <[email protected]>
Let us know if you're interested in contributing! We're working on integrating the core logic for getting elements and extraction into the sdk!
11
-
</p>
12
-
</div>
7
+
> **Notice:** The Dendrite SDK is not under active development anymore. However, the project will remain fully open source so that you and others can learn from it. Feel free to fork, study, or adapt this code for your own projects as you wish – reach out to us on Discord if you have questions! We love chatting about web AI agents. 🤖
13
8
14
9
## What is Dendrite?
15
10
@@ -24,33 +19,60 @@
24
19
25
20
#### A simple outlook integration
26
21
27
-
With Dendrite, it's easy to create web interaction tools for your agent.
22
+
With Dendrite it's easy to create web interaction tools for your agent.
asyncio.run(send_email("[email protected]", "Hello", "This is a test email"))
53
+
51
54
```
52
55
53
-
To authenticate you'll need to use our Chrome Extension **Dendrite Vault**, you can download it [here](https://chromewebstore.google.com/detail/dendrite-vault/faflkoombjlhkgieldilpijjnblgabnn). Read more about authentication [in our docs](https://docs.dendrite.systems/examples/authentication-instagram).
56
+
You'll need to add your own Anthropic key or [configure which LLMs to use yourself](https://docs.dendrite.systems/concepts/config).
57
+
58
+
59
+
```.env
60
+
ANTHROPIC_API_KEY=sk-...
61
+
```
62
+
63
+
To **authenticate** on any web service with Dendrite, follow these steps:
64
+
65
+
1. Run the authentication command
66
+
67
+
```bash
68
+
dendrite auth --url outlook.live.com
69
+
```
70
+
71
+
2. This command will open a browser that you'll be able to login with.
72
+
73
+
3. After you've logged in, press enter in your terminal. This will save your cookies locally so that they can be used in your code.
74
+
75
+
Read more about authentication [in our docs](https://docs.dendrite.systems/examples/authentication).
Initialize the Dendrite client and start doing web interactions without boilerplate.
64
-
65
-
[Get your API key here](https://dendrite.systems/app)
66
-
67
85
```python
68
-
from dendrite import Dendrite
86
+
from dendrite import AsyncDendrite
87
+
88
+
asyncdefmain():
89
+
client = AsyncDendrite()
69
90
70
-
client = Dendrite(dendrite_api_key="sk...")
91
+
await client.goto("https://google.com")
92
+
await client.fill("Search field", "Hello world")
93
+
await client.press("Enter")
71
94
72
-
client.goto("https://google.com")
73
-
client.fill("Search field", "Hello world")
74
-
client.press("Enter")
95
+
if__name__=="__main__":
96
+
import asyncio
97
+
asyncio.run(main())
75
98
```
76
99
77
100
In the example above, we simply go to Google, populate the search field with "Hello world" and simulate a keypress on Enter. It's a simple example that starts to explore the endless possibilities with Dendrite. Now you can create tools for your agents that have access to the full web without depending on APIs.
78
101
79
-
## More powerful examples
102
+
## More Examples
80
103
81
-
Now, let's have some fun. Earlier we showed you a simple send_email example. And sending emails is cool, but if that's all our agent can do it kind of sucks. So let's create two cooler examples.
104
+
### Get any page as markdown
82
105
83
-
### Download Bank Transactions
106
+
This is a simple example of how to get any page as markdown, great for feeding to an LLM.
107
+
108
+
```python
109
+
from dendrite import AsyncDendrite
110
+
from dotenv import load_dotenv
111
+
112
+
asyncdefmain():
113
+
browser = AsyncDendrite()
114
+
115
+
await browser.goto("https://dendrite.systems")
116
+
await browser.wait_for("the page to load")
117
+
118
+
# Get the entire page as markdown
119
+
md =await browser.markdown()
120
+
print(md)
121
+
print("="*200)
122
+
123
+
# Only get a certain part of the page as markdown
124
+
data_extraction_md =await browser.markdown("the part about data extraction")
125
+
print(data_extraction_md)
126
+
127
+
if__name__=="__main__":
128
+
import asyncio
129
+
asyncio.run(main())
130
+
```
84
131
85
-
First up, a tool that allows our AI agent to download our bank's monthly transactions so that they can be analyzed and compiled into a report that can be sent to stakeholders with `send_email`.
132
+
### Get Company Data from Y Combinator
133
+
134
+
The classic web data extraction test, made easy:
86
135
87
136
```python
88
-
from dendrite import Dendrite
137
+
from dendrite import AsyncDendrite
138
+
import pprint
139
+
import asyncio
89
140
90
-
defget_transactions() -> str:
91
-
client = Dendrite(auth="mercury.com")
92
141
93
-
# Navigate and wait for loading
94
-
client.goto(
95
-
"https://app.mercury.com/transactions",
96
-
expected_page="Dashboard with transactions"
97
-
)
98
-
client.wait_for("The transactions to finish loading")
Finally, it would be cool if we could add the amount of monthly visitors from Google Analytics to our report. We can do that by using the `extract` function:
184
+
Here's how to get the amount of monthly visitors from Google Analytics using the `extract` function:
When you want to scale up your AI agents, we support using browsers hosted by Browserbase. This way you can run many agents in parallel without having to worry about the infrastructure.
147
247
148
-
To start using Browserbase just swap out the `Dendrite` class with `DendriteRemoteBrowser` and add your Browserbase API key and project id, either in the code or in a `.env` file like this:
248
+
To start using Browserbase just swap out the `AsyncDendrite` class with `AsyncDendriteRemoteBrowser` and add your Browserbase API key and project id, either in the code or in a `.env` file like this:
149
249
150
250
```bash
151
251
# ... previous keys
@@ -154,17 +254,19 @@ BROWSERBASE_PROJECT_ID=
154
254
```
155
255
156
256
```python
157
-
# from dendrite import Dendrite
158
-
from dendrite importDendriteRemoteBrowser
159
-
160
-
...
161
-
162
-
#client = Dendrite(...)
163
-
client = DendriteRemoteBrowser(
164
-
# Use interchangeably with the Dendrite class
165
-
browserbase_api_key="...", # or specify the browsebase keys in the .env file
166
-
browserbase_project_id="..."
167
-
)
257
+
# from dendrite import AsyncDendrite
258
+
from dendrite importAsyncDendriteRemoteBrowser
259
+
260
+
asyncdefmain():
261
+
# client = AsyncDendrite(...)
262
+
client =AsyncDendriteRemoteBrowser(
263
+
# Use interchangeably with the AsyncDendrite class
264
+
browserbase_api_key="...", # or specify the browsebase keys in the .env file
0 commit comments