Skip to content

[Bug]: Setting dict type proxy_config in BrowserConfig throws exception #1057

Open
@Aaron2516

Description

@Aaron2516

crawl4ai version

0.6.2

Expected Behavior

The instance variable proxy_config of the BrowserConfig class supports the Dict type. The example is as follows
Image

However, when proxy_config passes in Dict type data, AttributeError: 'dict' object has no attribute 'server' will be raised when ProxySettings is initialized.

When proxy_config passes in Dict type data, the ProxySettings class initialization should not throw an exception

Current Behavior

When proxy_config passes in Dict type data, AttributeError: 'dict' object has no attribute 'server' will be raised when ProxySettings is initialized.
Image

Is this reproducible?

Yes

Inputs Causing the Bug

The proxy_config instance variable of the BrowserConfig class passes in Dict data for initialization. Since no data processing is performed on it, self.proxy_config is of Dict type.

see https://github.com/unclecode/crawl4ai/blob/main/crawl4ai/async_configs.py#L442

The self.config used to initialize the ProxySettings class is an instance of the BrowserConfig class. As can be seen above, self.config.proxy_config is of Dict type, so when executing self.config.proxy_config.server, AttributeError: 'dict' object has no attribute 'server' is thrown.

see https://github.com/unclecode/crawl4ai/blob/main/crawl4ai/browser_manager.py#L641

Steps to Reproduce

Run the following code to reproduce the exception

Code snippets

# pass a dict proxy_config raises AttributeError: 'dict' object has no attribute 'server'

import asyncio

from crawl4ai import AsyncWebCrawler, BrowserConfig


proxy_config = {
    "server": "your_server",
    "username": "your_username",
    "password": "your_password"
}


async def test_dict_proxy():
    browser_config = BrowserConfig(proxy_config=proxy_config)
    async with AsyncWebCrawler(config=browser_config) as crawler:
        result = await crawler.arun(url="https://httpbin.org/ip")
        print(result)

if __name__ == "__main__":
    asyncio.run(test_dict_proxy())

OS

WSL2

Python version

3.10.13

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    🐞 BugSomething isn't working📌 Root causedidentified the root cause of bug

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions