Skip to content

Performance optimization: Only register request and response loggers when needed #335

@jl-martins

Description

@jl-martins

First of all, thanks for this plugin. I've been actively using it and it's really great!

Recently, I ran cProfile on a Scrapy spider that I'm developing and the results suggest that the request and response loggers are each consuming approximately 9% of the execution time. I did the same for the BooksSpider from the scrapy-playwright examples and got the same results:

Image

Image

Is there some reason to register these handlers when the log level of the scrapy-playwright logger is higher than DEBUG?

If not, I can open a PR to only register these handlers when the log level is less than or equal to DEBUG.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions