Skip to content

Commit 8e7e86f

Browse files
pelmepelme
committed
Async iteration
1 parent fa05ac0 commit 8e7e86f

File tree

11 files changed

+449
-46
lines changed

11 files changed

+449
-46
lines changed

‎docs/assets/starlette.webm‎

151 KB
Binary file not shown.

‎docs/changelog.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Changelog
22

3+
## NEXT
4+
- Add async streaming support. See [the Streaming docs for more information](https://htpy.dev/streaming/). [PR #38](https://github.com/pelme/htpy/pull/38).
5+
36
## 25.4.1 - 2025-04-12
47
- Add the `Renderable` protocol, a consistent API to render an `htpy` object as HTML or to iterate over it. `Element`, `Fragment`, `ContextProvider`, and `ContextConsumer` are all `Renderable`. [PR #92](https://github.com/pelme/htpy/pull/92). Thanks to [Stein Magnus Jodal (@jodal)](https://github.com/jodal) and [Dave Peck (@davepeck)](https://github.com/davepeck).
58
- Deprecate `render_node()` and `iter_node()` and direct iteration over elements. Call `Renderable.__str__()` or `Renderable.iter_chunks()` instead. [Read the Usage docs for more details](usage.md#renderable).

‎docs/streaming.md‎

Lines changed: 124 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -14,43 +14,40 @@ client while the page is being generated.
1414
streaming will be the easiest way to get going. Streaming can give you
1515
improved user experience from faster pages/rendering.
1616

17-
This video shows what it looks like in the browser to generate a HTML table with [Django StreamingHttpResponse](https://docs.djangoproject.com/en/5.0/ref/request-response/#django.http.StreamingHttpResponse) ([source code](https://github.com/pelme/htpy/blob/main/examples/djangoproject/stream/views.py)):
18-
<video width="500" controls loop >
1917

20-
<source src="/assets/stream.webm" type="video/webm">
21-
</video>
18+
## Example
19+
This video shows what it looks like in the browser to generate a HTML table with
20+
[Django
21+
StreamingHttpResponse](https://docs.djangoproject.com/en/5.0/ref/request-response/#django.http.StreamingHttpResponse)
22+
([source
23+
code](https://github.com/pelme/htpy/blob/main/examples/djangoproject/stream/views.py)):
2224

23-
This example simulates a (very) slow fetch of data and shows the power of
25+
This example simulates a (very) slow data source and shows the power of
2426
streaming: The browser loads CSS and gradually shows the contents. By loading
2527
CSS files in the `<head>` tag before dynamic content, the browser can start
2628
working on loading the CSS and styling the page while the server keeps
2729
generating the rest of the page.
30+
<video width="500" controls loop >
31+
<source src="/assets/stream.webm" type="video/webm">
32+
</video>
2833

29-
## Using Generators and Callables as Children
3034

31-
Django's querysets are [lazily
32-
evaluated](https://docs.djangoproject.com/en/5.0/topics/db/queries/#querysets-are-lazy).
33-
They will not execute a database query before their value is actually needed.
35+
## Synchronous streaming
3436

35-
This example shows how this property of Django querysets can be used to create a
36-
page that streams objects:
37+
Instead of calling `str()` of an element, you may iterate/loop over it. You will then
38+
get "chunks" of the element as htpy renders the result, as soon as they are ready.
3739

38-
```python
39-
from django.http import StreamingHttpResponse
40-
from htpy import ul, li
40+
To delay the calculation and allow htpy to incrementally render elements, there
41+
are two types of lazy constructs that can be used:
4142

42-
from myapp.models import Article
43+
- Callables/lambdas without any arguments
44+
- Generators
4345

44-
def article_list(request):
45-
return StreamingHttpResponse(ul[
46-
(li[article.title] for article in Article.objects.all())
47-
])
48-
```
46+
These will be evaluated lazily and
4947

50-
## Using Callables to Delay Evalutation
48+
### Callables/lambda
5149

52-
Pass a callable that does not accept any arguements as child to delay the
53-
evaluation.
50+
Pass a callable that does not accept any arguments as child. When htpy renders the children, it will call the function to retrieve the result.
5451

5552
This example shows how the page starts rendering and outputs the `<h1>` tag and
5653
then calls `calculate_magic_number`.
@@ -70,6 +67,7 @@ element = div[
7067
calculate_magic_number,
7168
]
7269

70+
# Iterate over the element to get the content incrementally
7371
for chunk in element:
7472
print(chunk)
7573
```
@@ -105,9 +103,111 @@ print(
105103
div[
106104
h1["Fibonacci!"],
107105
"fib(20)=",
108-
lambda: str(fib(20)),
106+
lambda: fib(20),
109107
]
110108
)
111109
# output: <div><h1>Fibonacci!</h1>fib(12)=6765</div>
112110

113111
```
112+
113+
### Generators
114+
115+
Generators can also be used to gradually retrieve output. You may create a
116+
generator function (a function that uses the `yield` keyword) or an generator
117+
comprehension/expression.
118+
119+
```py
120+
import time
121+
from collections.abc import Iterator
122+
123+
from htpy import Element, li, ul
124+
125+
126+
def numbers() -> Iterator[Element]:
127+
yield li[1]
128+
time.sleep(1)
129+
yield li[2]
130+
131+
132+
def component() -> Element:
133+
return ul[numbers]
134+
135+
136+
for chunk in component():
137+
print(chunk)
138+
```
139+
140+
Output:
141+
142+
```html
143+
<ul>
144+
<li>
145+
1
146+
</li>
147+
<li> <|- Appears after 1 second
148+
2 <|
149+
</li> <|
150+
</ul>
151+
```
152+
153+
154+
## Asynchronous streaming
155+
156+
htpy can be used in fully async mode.
157+
158+
This intended to be used with ASGI/async web frameworks/servers such as
159+
Starlette, Sanic, FastAPI and Django.
160+
161+
Combined with an ORM, database adapter or reading backing data from an async
162+
source, all parts of the stack will be fully async and the client will get the data incrementally.
163+
164+
htpy will `await` any awaitables and iterate over async iterators. Use async iteration on a htpy element or use `aiter_node()` to render any `Node`.
165+
166+
167+
### Starlette, ASGI and uvicorn example
168+
169+
```python
170+
title="starlette_demo.py"
171+
import asyncio
172+
from collections.abc import AsyncIterator
173+
174+
from starlette.applications import Starlette
175+
from starlette.requests import Request
176+
from starlette.responses import StreamingResponse
177+
178+
from htpy import Element, div, h1, li, p, ul
179+
180+
app = Starlette(debug=True)
181+
182+
183+
@app.route("/")
184+
async def index(request: Request) -> StreamingResponse:
185+
return StreamingResponse(await index_page(), media_type="text/html")
186+
187+
188+
async def index_page() -> Element:
189+
return div[
190+
h1["Starlette Async example"],
191+
p["This page is generated asynchronously using Starlette and ASGI."],
192+
ul[(li[str(num)] async for num in slow_numbers(1, 10))],
193+
]
194+
195+
196+
async def slow_numbers(minimum: int, maximum: int) -> AsyncIterator[int]:
197+
for number in range(minimum, maximum + 1):
198+
yield number
199+
await asyncio.sleep(0.5)
200+
201+
```
202+
203+
Run with [uvicorn](https://www.uvicorn.org/):
204+
205+
206+
```
207+
$ uvicorn starlette_demo:app
208+
```
209+
210+
In the browser, it looks like this:
211+
<video width="500" controls loop >
212+
<source src="/assets/starlette.webm" type="video/webm">
213+
</video>

‎examples/async_example.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
import asyncio
2+
import random
3+
4+
from htpy import Element, b, div, h1
5+
6+
7+
async def magic_number() -> Element:
8+
await asyncio.sleep(2)
9+
return b[f"The Magic Number is: {random.randint(1, 100)}"]
10+
11+
12+
async def my_component() -> Element:
13+
return div[
14+
h1["The Magic Number"],
15+
magic_number(),
16+
]
17+
18+
19+
async def main() -> None:
20+
async for chunk in await my_component():
21+
print(chunk)
22+
23+
24+
asyncio.run(main())

‎examples/starlette_app.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import asyncio
2+
from collections.abc import AsyncIterator
3+
4+
from starlette.applications import Starlette
5+
from starlette.requests import Request
6+
from starlette.responses import StreamingResponse
7+
from starlette.routing import Route
8+
9+
from htpy import Element, div, h1, li, p, ul
10+
11+
12+
async def index(request: Request) -> StreamingResponse:
13+
return StreamingResponse(await index_page(), media_type="text/html")
14+
15+
16+
async def index_page() -> Element:
17+
return div[
18+
h1["Starlette Async example"],
19+
p["This page is generated asynchronously using Starlette and ASGI."],
20+
ul[(li[str(num)] async for num in slow_numbers(1, 10))],
21+
]
22+
23+
24+
async def slow_numbers(minimum: int, maximum: int) -> AsyncIterator[int]:
25+
for number in range(minimum, maximum + 1):
26+
yield number
27+
await asyncio.sleep(0.5)
28+
29+
30+
app = Starlette(
31+
debug=True,
32+
routes=[
33+
Route("/", index),
34+
],
35+
)

‎examples/stream_generator.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
import time
2+
from collections.abc import Iterator
3+
4+
from htpy import Element, li, ul
5+
6+
7+
def numbers() -> Iterator[Element]:
8+
yield li[1]
9+
time.sleep(1)
10+
yield li[2]
11+
12+
13+
def component() -> Element:
14+
return ul[numbers]
15+
16+
17+
for chunk in component():
18+
print(chunk)

0 commit comments

Comments
 (0)