Skip to content

Commit 15ea452

Browse files
committed
initial changes
1 parent 9fa5002 commit 15ea452

78 files changed

Lines changed: 12 additions & 5639 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 1 addition & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -59,27 +59,7 @@ if element:
5959

6060
## Usage
6161

62-
If you're using Scrapy, you can use the `from_response` function to create a `TypedSoup` object from a Scrapy response:
63-
64-
```python
65-
from typed_soup import from_response
66-
from scrapy.http.response.html import HtmlResponse
67-
68-
# Assume 'response' is an HtmlResponse object
69-
soup = from_response(response)
70-
71-
# Find an element
72-
element = soup.find("div", class_="example")
73-
if element:
74-
print(element.get_text())
75-
76-
# Find all elements
77-
elements = soup("p")
78-
for elem in elements:
79-
print(elem.get_text())
80-
```
81-
82-
Or, without Scrapy, you can explicity wrap a `BeautifulSoup` object in `TypedSoup`:
62+
Wrap a `BeautifulSoup` object in `TypedSoup` to add type safety:
8363

8464
```python
8565
from typed_soup import TypedSoup
@@ -107,7 +87,6 @@ I'm adding functions as I need them. If you have a request, please open an issue
10787

10888
And then these help create a `TypedSoup` object:
10989

110-
- `from_response`
11190
- `TypedSoup`
11291

11392
## Type Safety Benefits

poetry.lock

Lines changed: 6 additions & 982 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ authors = [{ name = "Robert Shecter", email = "robert@public.law" }]
66
license = { text = "MIT" }
77
readme = "README.md"
88
requires-python = ">=3.10"
9-
dependencies = ["beautifulsoup4", "scrapy", "toolz"]
9+
dependencies = ["beautifulsoup4", "toolz"]
1010
classifiers = [
1111
"Development Status :: 4 - Beta",
1212
"Intended Audience :: Developers",
@@ -27,7 +27,6 @@ keywords = [
2727
"html",
2828
"parsing",
2929
"type-safe",
30-
"scrapy",
3130
"type-hints",
3231
"static-typing",
3332
"mypy",

tests/test_typed_soup.py

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
import unittest
22
from bs4 import BeautifulSoup
3-
from scrapy.http.response.html import HtmlResponse
4-
from typed_soup import TypedSoup, from_response
3+
from typed_soup import TypedSoup
54

65

76
class TestTypedSoup(unittest.TestCase):
@@ -32,16 +31,5 @@ def test_implicit_find_all(self):
3231
self.assertEqual(elements[0].get_text(), "First")
3332
self.assertEqual(elements[1].get_text(), "Second")
3433

35-
def test_from_response(self):
36-
# Create a mock HtmlResponse
37-
response = HtmlResponse(url="http://example.com",
38-
body="<div>Test</div>", encoding="utf-8")
39-
typed_soup = from_response(response)
40-
element = typed_soup.find("div")
41-
self.assertIsNotNone(element)
42-
if element:
43-
self.assertEqual(element.get_text(), "Test")
44-
45-
4634
if __name__ == "__main__":
4735
unittest.main()

typed_soup/__init__.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
from .typed_soup import TypedSoup, from_response
1+
from .typed_soup import TypedSoup
22

3-
__all__ = ["TypedSoup", "from_response"]
3+
__all__ = ["TypedSoup"]

typed_soup/typed_soup.py

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
1-
from typing import TypeVar, Optional, List, Any, Dict, cast
1+
from typing import TypeVar, Optional, List, Any, Dict
22
from bs4 import BeautifulSoup, Tag
3-
from scrapy.http.response.html import HtmlResponse
43

54
T = TypeVar('T')
65

@@ -100,17 +99,3 @@ def get_content_after_element(self) -> str:
10099
def string(self) -> str | None:
101100
"""Get the string content of the element (similar to BeautifulSoup's .string)."""
102101
return self._element.string
103-
104-
105-
def make_soup(response: HtmlResponse) -> BeautifulSoup:
106-
"""
107-
Create a BeautifulSoup object from the Response body.
108-
"""
109-
return BeautifulSoup(cast(str, response.body), "html.parser")
110-
111-
112-
def from_response(response: HtmlResponse) -> TypedSoup:
113-
"""
114-
Create a type-safe BeautifulSoup object from an HTML response.
115-
"""
116-
return TypedSoup(make_soup(response))

typings/scrapy/__init__.pyi

Lines changed: 0 additions & 22 deletions
This file was deleted.

typings/scrapy/addons.pyi

Lines changed: 0 additions & 30 deletions
This file was deleted.

typings/scrapy/core/downloader/__init__.pyi

Lines changed: 0 additions & 62 deletions
This file was deleted.

typings/scrapy/core/downloader/handlers/__init__.pyi

Lines changed: 0 additions & 28 deletions
This file was deleted.

0 commit comments

Comments
 (0)