Open
Description
crawl4ai version
0.6.2
Expected Behavior
The instance variable proxy_config of the BrowserConfig class supports the Dict type. The example is as follows
However, when proxy_config passes in Dict type data, AttributeError: 'dict' object has no attribute 'server' will be raised when ProxySettings is initialized.
When proxy_config passes in Dict type data, the ProxySettings class initialization should not throw an exception
Current Behavior
When proxy_config passes in Dict type data, AttributeError: 'dict' object has no attribute 'server' will be raised when ProxySettings is initialized.
Is this reproducible?
Yes
Inputs Causing the Bug
The proxy_config instance variable of the BrowserConfig class passes in Dict data for initialization. Since no data processing is performed on it, self.proxy_config is of Dict type.
see https://github.com/unclecode/crawl4ai/blob/main/crawl4ai/async_configs.py#L442
The self.config used to initialize the ProxySettings class is an instance of the BrowserConfig class. As can be seen above, self.config.proxy_config is of Dict type, so when executing self.config.proxy_config.server, AttributeError: 'dict' object has no attribute 'server' is thrown.
see https://github.com/unclecode/crawl4ai/blob/main/crawl4ai/browser_manager.py#L641
Steps to Reproduce
Run the following code to reproduce the exception
Code snippets
# pass a dict proxy_config raises AttributeError: 'dict' object has no attribute 'server'
import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig
proxy_config = {
"server": "your_server",
"username": "your_username",
"password": "your_password"
}
async def test_dict_proxy():
browser_config = BrowserConfig(proxy_config=proxy_config)
async with AsyncWebCrawler(config=browser_config) as crawler:
result = await crawler.arun(url="https://httpbin.org/ip")
print(result)
if __name__ == "__main__":
asyncio.run(test_dict_proxy())
OS
WSL2
Python version
3.10.13
Browser
No response
Browser version
No response
Error logs & Screenshots (if applicable)
No response