Async streaming

[pelme]

This PR adds the possibility to use async awaitables/iterators/generators to generate a response. A sample Starlette example is added too. Thoughts? Ideas?

[raisjn]

just a drive by comment: this is cool - would love to use this in a Starlette app. i will build some prototypes off this branch to get a feel for it, but i think asyncio support would be very useful for me.

[pelme]

@raisjn cool, thanks for dropping the comment!

• I feel like everyone should use streaming (sync or async) for improved user experience and loading times with minimal effort. It is truly a underused thing in current web development that could give a nice boost in web performance. I am also thinking of adding a Starlette HtpyResponse/Django HtpyResponse/Flask HtpyResponse classes etc which is based on the respective StreamingResponse classes. Just to lower the friction and make it as straightforward as possible to use streaming. Now "streaming" is kind of "opt-in" if you use the StreamingResponse classes etc, I would like to make it easier+make the main path to using htpy be streaming by default.

• I think this PR is pretty much in a good shape of being merged. I have re-written the implementation/tests in multiple times and think it is pretty solid/well tested at this point. Any review/feedback on the implementation would be very welcome though! I have not merged it yet because a) I am not using async myself in my day-to-day project and b) there have not been any visible feedback/interest in it yet.

• @raisjn if you would build some prototype and play with it and get back with feedback that would be very valuable and we could get this going! 🙂

[raisjn]

i'm still playing with htpy (cool!) and async iteration. the code looks fine to me, but will report back after more days (weeks?) of testing and trying to build the below functionality:

comments on async delivery:

from what i can tell, it resolves the work in sequential order. what i am aiming for is pipelined rendering (also known as partial pre-rendering in next.js). I believe pipelined delivery can be built on top of (or alongside) htpy's async rendering by using some JS + an async queue.

it would look something like this:

div[
  Placeholder(do_some_work()),
  Placeholder(do_some_work())
]

Placeholder would emit a placeholder div that will be later filled with the actual async work when it finishes. this seems like it is straightforward for me to implement with the streaming functionality you've built (so thank you!)

[pelme]

Interesting! I think just sending CSS/JS before the full page is rendered gives a good boost for free for anyone. Streaming multiple parts of the page and then recombining it with javascript sounds interesting! It feels like there should be a small lib for that. It feels simple in theory! 😆

[raisjn]

I think just sending CSS/JS before the full page is rendered gives a good boost for free for anyone

i agree - this is a great boost! i'm not 100% sure using async iter is a good answer for this functionality as it would end up being a hidden implementation detail instead of an explicit consideration by devs: consider adding a special function like flush().

html[
head[
  link[...]
  script[...]
],
flush(),
body[
  div[foobar]
],
flush(),
some_extra_work()

the exact API doesn't matter as long as it is explicit, it could also be a special tag (early_flush[some_html_code]).

[pelme]

why would a explicit flush function/tag be useful? currently with this PR, everything will be "flushed" as soon as it is ready anyways.

[raisjn]

coming back to this: it turns out i don't need async iterators in htpy, have architected an app with async delivery that allows parallel pipelining instead of sequential and uses htpy without async. in my first reading of this PR, i had assumed async iter was in parallel. (collect all async in one pass, then await them in parallel, then do next set, etc).

re: why use explicit flush: i think it would help for people who aren't aware of the mechanics of rendering and using async generator vs sync generator.

[wbadart]

I started writing _aiter_node_context on my fork before searching for existing issues. Glad I didn't go too far down that rabbit hole before stopping to check! Your take on streaming w/in the web ecosystem as a whole really resonates with me.

I have a local fork with this branch merged with master (wbadart/htpy@1ad78ff), and just about everything has been going smoothly. The one thing I wanted to point out was a possible bug in the interaction with fragment (which it seems was added later; and please let me know if this is actually the intended behavior):

# ipython for top-level `async for`

async def my_component():
    return p["hello!"]

my_div = div[my_component()]
async for chunk in my_div:
    print(chunk)

# <div>
# <p>
# Hello!
# </p>
# </div>


my_fragment = fragment[my_component()]
async for chunk in my_fragment:
    print(chunk)

# TypeError: 'async for' requires an object with __aiter__ method, got Fragment
# ^this one makes sense


my_fragment_in_div = div[fragment[my_component()]]
async for chunk in my_fragment_in_div:
    print(chunk)

# <div>
# Traceback (most recent call last):
# ...
# ValueError: <coroutine object f at 0x1042f3110> is not a valid child element. Use async iteration to retrieve element content: https://htpy.dev/streaming/

full traceback

Traceback (most recent call last):
  File "", line 2, in main
  File "/.../htpy/htpy/__init__.py", line 353, in _aiter_context
    async for x in _aiter_node_context(self._children, context):
  File "/.../htpy/htpy/__init__.py", line 267, in _aiter_node_context
    yield str(_escape(x))
              ^^^^^^^^^^
  File "/.../.venv/lib/python3.12/site-packages/markupsafe/__init__.py", line 43, in escape
    return Markup(s.__html__())
                  ^^^^^^^^^^^^
  File "/.../htpy/htpy/__init__.py", line 451, in __str__
    return render_node(self)  <--- this is in Fragment.__str__
           ^^^^^^^^^^^^^^^^^
  File "/.../htpy/htpy/__init__.py", line 467, in render_node
    return _Markup("".join(iter_node(node)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../htpy/htpy/__init__.py", line 208, in _iter_node_context
    yield from _iter_node_context(x._node, context_dict)  # pyright: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../htpy/htpy/__init__.py", line 217, in _iter_node_context
    raise ValueError(
ValueError:  is not a valid child element. Use async iteration to retrieve element content: https://htpy.dev/streaming/

If you look at that stack trace, it looks as though the rendering method on Fragment switches us back into a sync context, meaning that fragments can't have async children anywhere in their subtree. (I've gotten around this so far by simply switching my fragments to divs.)

Thank you for your great work on this very enjoyable library!

[pelme]

cool, thanks for the feedback and picking up the work on this! have not been pursuing async iteration since I do not use it myself no-one showed (enough) intereste in using it yet!

there probably needs to be an explicit check for elif isinstance(x, Fragment): ... in _aiter_node_context to deal with the Fragment this PR has not been updated after the fragment was merged. when running the tests, I would assume that pretty much all tests in test_fragment.py would currently fail, right? the render fixture should test both the sync and async version.

Any feedback on the docs/explaination about async and streaming (both sync and async) would be appreciated.

I would like to merge #92 first and then I can fix the last bits here. Btw, #92 changes the chunk iteration from plain __iter__ (for x in elem:...) to for x in elem.stream_chunks(): ... to make it more explicit. I am happy with having an explicit method for it but not with the naming. What do you think about .iter_chunks() and .aiter_chunks() for sync and async versions? There are more details in #92 but basically render_node, iter_node (and aiter_node) would not be a thing, you would always call the methods on the elements/fragments/context objects directly.

[wbadart]

Personally I'm very interested in async iteration, and would encourage you to release async support once it's rectified with the new iteration protocol. The ability to delay expensive work and start streaming a response NOW--simply by dropping awaitables and async generators into my htpy expressions--has been a huge boon for my app (which is specifically geared towards users on high latency/ packet-loss connections).

I like iter_chunks and aiter_chunks (I think I prefer iter_* over stream_* since it corresponds with Python's terminology), though I'd probably keep the dunder methods around (even just as aliases for (a)iter_chunks) for users who prefer the implicit/ 'duck-typing' style.

Happy to play around with this on my local branch and think about documentation (time permitting) over the next few days!

[pelme]

Improve these docs:

• Use the same example/output for for both sync and async examples
• Show the example code first in an inline code block instead of link to it. Show the video directly below.
• Remove/shorten the section on callables/lambda and generators.

[pelme]

@wbadart with #92 merged, I have rebased this and updated the naming to be .aiter_chunks(). The docs are still lacking and incomplete but it should work. Please give it a go and let me know if you have any feedback!

[Copilot]

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

[pelme]

See #160 for a updated PR.