Skip to content

Add Jobs Comparison Monitor#2

Open
ogiaquino wants to merge 1 commit intoscrapy-plugins:mainfrom
ogiaquino:main
Open

Add Jobs Comparison Monitor#2
ogiaquino wants to merge 1 commit intoscrapy-plugins:mainfrom
ogiaquino:main

Conversation

@ogiaquino
Copy link
Collaborator

No description provided.

# run: >-
# python -m pip install -e .
#
# - name: Publish distribution 📦 to Private PyPI

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a ChatGPT comment?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be alright IMHO.

@ogiaquino please add Adrian and Andrey to help us reviewing this PR

@ogiaquino ogiaquino requested review from Gallaecio and wRAR January 8, 2026 12:39
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intentional that there are two conftest files?

)

__all__ = [
SPIDERMON_JOBS_COMPARISON,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this really work and pass ruff? __all__ is a list of strings

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I guess it works because those constants are strings with the same values as names...

You can also filter which jobs to compare based on their close reason using the
``SPIDERMON_JOBS_COMPARISON_CLOSE_REASONS`` setting. The default value is ``()``,
which doesn't filter any job based on close_reason. To only consider successfully finished jobs,
use ``("finished", ) instead.``
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
use ``("finished", ) instead.``
use ``("finished", )`` instead.

The job that will have all the desired arguments will be processed.
Example {"debug_url": "https://www.google.com"} or {"is_full_crawl": True}
You can enable this filter by setting SPIDERMON_JOBS_COMPARISON_ARGUMENTS_ENABLED as True in the settings.
Otherwise, this filter will not be applied
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Otherwise, this filter will not be applied
Otherwise, this filter will not be applied.

@@ -0,0 +1,13 @@
[tox]
envlist = min,py39,py310,py311,py312
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no min env.

Comment on lines +23 to +28
- python-version: "3.9"
tox: min
- python-version: "3.10"
- python-version: "3.11"
- python-version: "3.12"
- python-version: "3.13"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't match the version range declared as supported in ‎README.rst and setup.py, and there is no min env.

Comment on lines +11 to +12
quote-style = "double"
indent-style = "space"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the default values.

@@ -0,0 +1,13 @@
[tool.ruff]
line-length = 88
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default value.

@@ -0,0 +1,2 @@
[bdist_wheel]
universal = 1
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is specific to Python 2.

long_description_content_type="text/x-rst",
license="BSD",
packages=find_packages(),
package_data={"zyte_spidermon": ["VERSION"]},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this file exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants