Releases: ScrapeOps/scrapeops-scrapy-sdk
Scrapy Deprecation Fix
scrapeops_scrapy.middleware.retry.RetryMiddleware no longer emits ScrapyDeprecationWarning on Scrapy 2.15+.
Scrapy 2.15 warns whenever a
downloader middleware requires a spider argument on process_response /
process_exception, as that argument will stop being passed in a future
Scrapy release.
The middleware now stores the crawler (via
from_crawler) and accesses the running spider through
self.crawler.spider; the spider parameter is kept with a default of
None so existing and future Scrapy versions are both supported.
Behaviour is unchanged — the same ScrapeOps signals are emitted with the
Dependency Updates - v0.6.0
v0.6.0
Dependency and Python-support modernisation; no runtime behaviour changes.
Dependencies
- Bumped floors:
tld>=0.13.2,requests>=2.32.3,itemadapter>=0.11.0. - Removed
json5frominstall_requires(unused). - Removed the direct
urllib3pin — a compatibleurllib3is now resolved transitively viarequests. The old pin was only needed as a workaround forrequests 2.29.0(psf/requests#6432), which is no longer relevant. - No upper bounds on any runtime dep.
Python support
- Dropped Python 3.8 (EOL October 2024).
python_requiresis now>=3.9. - Added classifiers for Python 3.12 and 3.13.
Housekeeping
setup.pyandscrapeops_scrapy/__init__.pyare now aligned on the same version string.
Upgrading
Users on Python 3.9+ can upgrade with pip install -U scrapeops-scrapy; no config or code changes needed. Users on Python 3.8 should upgrade their interpreter or pin scrapeops-scrapy<0.6.
Fixes Release - 0.5.7
Fixed silent failure in scrapeops-scrapy when settings Contain Non-Serializable objects
0.5.6
Version: 0.5.4 Latest
Beta Version: 0.3.5
Changes
- Bug fix with exception logging when middleware not enabled
Beta Version: 0.3.3
Changes
- Added support for Scrapy stats parsing
- Changed API auth from query parameters to headers
- Fixed issue when API key invalid
0.3.1
Beta Version: 0.3
ScrapeOps SDK Beta Version:
- Job Monitoring (Pages, Items, Runtimes, Status Codes, Errors, Latencies)
- Item Validation
- Health Checks and Alerts via ScrapeOps
- Scrapyd Integration via ScrapeOps