PyGObject and asyncio

[lazka]

On the PyGObject IRC channel there is regular talk on how to best integrate with asyncio and if PyGObject should provide better integration out of the box.

What are your thoughts on integrating gblub into PyGObject?

(related bug report: https://bugzilla.gnome.org/show_bug.cgi?id=791591)

[gbtami]

It would be awesome! Now I have to use my "poor man integration": https://github.com/pychess/pychess/blob/master/pychess#L224 in PyChess because gbulb has unresolved issues.

If gbulb can be integrated into PyGObject I hope it will get more developers/contributors and may boost it's development.

[stuaxo]

As someone that has used both libraries: +1 one on this.

I'd imagine there may be potential to make the integration better / easier. There aren't a huge amount of developers on both projects, joining forces probably makes sense.

[nhoad]

I'm certainly interested in this! It'd be good to get some more people in the code base.

At the moment the code that's in master hasn't been released - it's essentially a reimplementation that gives a more full-fledged event loop that adds Windows support (instead of hacking support in on top of the Unix event loop), so I'm understandably paranoid about it. The end result is the same on Linux, however the biggest issue with it at the moment is that subprocesses don't work on Windows because non-blocking streams aren't supported by GObject's IOChannels (on Windows only, I think?). I don't know if this is something that better integration could help with, but more people working on the problem would help. I've contemplated just doing the release without subprocess support on Windows until I can figure out a workaround.

Also @gbtami, which unresolved issues are you referencing? The subprocess one?

Also also, if people want development to be a bit faster, they could help me out - I'm just one person working on this project for fun when I have some spare time, getting more people in who actually use it would be great.

[gbtami]

@nathan-hoad yes #24
Telling the truth my "poor man integration" which uses asyncio loop next to glib loop works OK. But it"s just a workaround of course.
Regarding contributing to gbulb you are absolutely right and I have to apologize. Last year I started to dig into windows subprocess issue and tried to figure out how quamash solved it, but I felt it needs more windows knowledge than I have. See https://github.com/harvimt/quamash/blob/master/quamash/_windows.py

[lazka]

That's great to hear!

I'll try to add an asyncio page to the PyGObject website which points to gbulb
and shows some examples. Maybe that will get more people interested.

One long term issue, if we ever try to move some of the code into PyGObject
itself is that it would have to be licensed under the LGPLv2.1+ (or something
compatible at least, like MIT) - Any thoughts on that?

[gzxu]

By the way, it will be great to add these two helper functions, though they are really simple. PyGObject lacks some sweet syntax sugars 😆

def connect_async(self, detailed_signal, handler_async, *args):
    def handler(self, *args):
        asyncio.ensure_future(handler_async(self, *args))
    self.connect(detailed_signal, handler, *args)


GObject.GObject.connect_async = connect_async


def wrap_asyncio(target, method, *, priority=False):
    async_begin = getattr(target, method + '_async')
    async_finish = getattr(target, method + '_finish')

    def wrapper(self, *args):
        def callback(self, result, future):
            future.set_result(async_finish(self, result))
        future = asyncio.get_event_loop().create_future()
        if priority:
            async_begin(self, *args, GLib.PRIORITY_DEFAULT, None, callback, future)
        else:
            async_begin(self, *args, None, callback, future)
        return future
    setattr(target, method + '_asyncio', wrapper)

An example usage will be

#!/usr/bin/env python3
import gi  # NOQA: E402
gi.require_versions({
    'Gtk': '3.0',
    'Soup': '2.4'
})  # NOQA: E402

import sys
import asyncio

import gbulb
from gi.repository import Gtk, Gio, Soup, GLib, GObject


def connect_async(self, detailed_signal, handler_async, *args):
    def handler(self, *args):
        asyncio.ensure_future(handler_async(self, *args))
    self.connect(detailed_signal, handler, *args)


GObject.GObject.connect_async = connect_async


def wrap_asyncio(target, method, *, priority=False):
    async_begin = getattr(target, method + '_async')
    async_finish = getattr(target, method + '_finish')

    def wrapper(self, *args):
        def callback(self, result, future):
            future.set_result(async_finish(self, result))
        future = asyncio.get_event_loop().create_future()
        if priority:
            async_begin(self, *args, GLib.PRIORITY_DEFAULT, None, callback, future)
        else:
            async_begin(self, *args, None, callback, future)
        return future
    setattr(target, method + '_asyncio', wrapper)


wrap_asyncio(Soup.Request, 'send')
wrap_asyncio(Gio.InputStream, 'read_bytes', priority=True)
wrap_asyncio(Gio.InputStream, 'close', priority=True)


class Window(Gtk.ApplicationWindow):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, title='Async Window', **kwargs)
        self.connect_async('realize', self.on_realize)

    async def on_realize(self, *args, **kwargs):
        button = Gtk.Button(label="Get")
        button.connect_async("clicked", self.on_button_clicked)

        entry = Gtk.Entry()
        entry.set_text('https://httpbin.org/get')

        text_view = Gtk.TextView()

        grid = Gtk.Grid()
        grid.attach(button, 1, 0, 1, 1)
        grid.attach(entry, 0, 0, 1, 1)
        grid.attach(text_view, 0, 1, 2, 1)
        self.add(grid)

        self.show_all()
        self.entry = entry
        self.text_view = text_view

    async def on_button_clicked(self, widget):
        session = Soup.Session()
        uri = Soup.URI.new(self.entry.get_text())
        request = session.request_http_uri('GET', uri)
        stream = await request.send_asyncio()
        data = await stream.read_bytes_asyncio(4096)
        self.text_view.get_buffer().set_text(data.get_data().decode())
        await stream.close_asyncio()


class Application(Gtk.Application):
    window = None

    def __init__(self, *args, **kwargs):
        super().__init__(*args, application_id=None, **kwargs)

    def do_activate(self):
        if not self.window:
            self.window = Window(application=self)
        self.window.present()


if __name__ == '__main__':
    gbulb.install(gtk=True)
    asyncio.get_event_loop().run_forever(Application(), sys.argv)

It is really fascinating to run every line of code inside just one thread, including networking and GUI 🎉 🎉

[nhoad]

Hello, I wanted to update everyone who's been involved in gbulb and I figured this was the best place to do it. Due to personal reasons and goings on in my life, I'm not able to give gbulb the attention it deserves. If people are interested in taking ownership and maintaining it, please discuss on here so we can come to an agreement, and I'll transfer ownership.

[gzxu]

👍 I think that gbulb is an excellent initiative, but IMHO overriding the undocumented SelectorEventLoop is not a good idea. I am submitting a patch wxWidgets/Phoenix#1103 to add asyncio support to WxPython, but currently I don't have time to write documentation and tests. BTW I am using my own implementation to use coroutines in my Python GTK applications.

[nhoad]

The SelectorEventLoop is by far the most common event loop. Nearly all of the documentation relates to it.

[gbtami]

@gzxu https://docs.python.org/3/library/asyncio-eventloop.html#event-loop-implementations

[gzxu]

Oh well, I meant that symbols starts with _ are extensively accessed here, which are private and may be subject to change. 🤷‍♂️

[benzea]

Soo, I randomly wondered about this again and played a bit with it over the weekend.

I think one of the major things we want here, is to turn allow GLib async functions to be run using asyncio using a nice syntax, my idea for that is the following:

• If we detect a callback that can be converted into a future (i.e. only one and has a closure and no destroy notify and we can guess the _finish function), then set the default value to gi.FutureCallback(finish_func) for it and gi.FutureCancellable() for the cancellable.
• Should the user pass additional user arguments, return them as part of the future result. My motivation for that is mostly to have a well defined behaviour
• HOWEVER: This would technically be an API break, as the defaults are usually None right now. In all cases where the return value is void it does not matter though, as the future is invisible and will just finish by itself. i.e. I think it is acceptable to do this.

Also, I guess we need to get GBulb into a shape where it is mergeable into pygobject.

For fun, I hacked up glib-asyncio to dispatch from the GLib mainloop. It is kind of neat as it is simple, but I suspect it is not portable (due to socket FDs not being pollable by GLib without wrapping them in a GIOChannelbasically). If someone is curious, it is here: jhenstridge/asyncio-glib#10

[benzea]

@lazka already asked earlier, a clarification of the gbulb license would be helpful. Without that one might need to start from scratch when trying to integrate it into pygobject.

EDIT: Ohh, looks like there is an Apache license file now. But it looks to me like that is not compatible with LGPL-2.1+.