RFC: Scrapy integration — optional Scrapling-style / adaptive parsing on responses

[vigneshwarrvenkat]

Feature description

Allow Scrapy users to opt into Scrapling’s parsing/adaptive selector behavior on Response objects (e.g. middleware, decorator, or adapter), so they can keep Scrapy’s crawl pipeline while benefiting from Scrapling’s selection model when desired.

Motivation

• Lowers migration cost for Scrapy-heavy teams.
• Surfaces Scrapling’s differentiators (adaptive selection) in the ecosystem where many users already live.

Proposal (high level)

• Define an explicit, documented integration point (no “magic” global monkeypatch).
• Clarify compatibility targets (Scrapy versions) and what happens to response.selector / response.css / response.xpath.

Non-goals (for v1)

• Replacing Scrapy’s scheduler/engine or duplicating spider templates.

Related

• ROADMAP.md: “Create a Scrapy plugin/decorator…”

Open questions

• Preferred API: middleware vs subclass vs helper factory?
• How should callbacks type-check across Scrapy + Scrapling?

(I searched existing feature requests for this topic.)

[D4Vinci]

Thanks for the suggestion, this is already in my mind

[vigneshwarrvenkat]

@D4Vinci Thanks for the feedback. I will start working on it.

[vigneshwarrvenkat]

@D4Vinci Hello Sir
Created this PR. Please do take a look.

[vigneshwarrvenkat]

@D4Vinci Can you please provide your feedback on the feature PR.