Event loop load metrics

[Tinche]

Hello!

First, the motivation. We're running a fair number of asyncio services in production, and I'm trying to figure out how to expose some sort of load metrics. These metrics would be useful for tweaking and autoscaling said services. We're using UVLoop, of course (thank you very much!).

I think exposing a busyness factor from the event loop would be a very interesting notion. Something like the percentage of the time in the last n seconds or so that the loop was idle: no tasks to switch to, just waiting on external inputs.

I guess I was looking for something sort of similar to #267. I'm not sure it should be a debug build feature though, since these metrics are most important in a high load production environment, where you want your libraries operating at max efficiency.

I think over at that issue it was mentioned that UVLoop should have little to no non-asyncio functionality, so my first question is: should I take this discussion over to python-dev instead? My instinct was to first try proving the concept in a third party implementation, then see about making it part of the official API.

[1st1]

I think exposing a busyness factor from the event loop would be a very interesting notion. Something like the percentage of the time in the last n seconds or so that the loop was idle: no tasks to switch to, just waiting on external inputs.

Yes, it's a good metric to track. There are also others, off the top of my head:

• How long the loop spends in its internals (efficiently this is how you can see the overhead of uvloop/asyncio)

• How long the loop spends in various callbacks -- would be cool to track outliers

• How many open FDs / callbacks / transports the loop manages

etc.

There's also an experimental PR #171 by @pfreixes. I kind of dropped the ball on that one, hopefully Pau can excuse me for that and join this effort.

Ultimately we need to add monitoring capabilities to asyncio, that's for sure. I'd say we should try to discuss the API in https://discuss.python.org/c/async-sig/20. We can either target 3.9 asyncio, implement this as a third-party library, or in uvloop directly as experimental API. I'm potentially open to all three options, but would love to first discuss the API and the cons/pros.

[pfreixes]

@1st1 things take time :)

Happy to start again a discussion about how proper monitoring could be added into the Asyncio, Ive tried it once [1], iI can try to make a second attempt.

During the first attempt what I tried to do is create awareness of how a function could be implemented in the Python loop for inferring the load of the loop - similar to tracing the CPU- to be used later on for scaling your services or applying back-pressure.

After some feedback, I did another design, where the load function was decoupled from the loop implementation. This was done by adding some instrumentalization in the Python loop. More info here [2].

Later on, we started with @1st1 , with no success, the initiative for adding instrumentalization - monitor and tracing - for Uvloop, some of the work done resulted in this PR [3].

As @1st1 commented we could start by starting again a discussion in the SIG list [4], happy to contribute.

[1] https://mail.python.org/pipermail/async-sig/2017-August/000382.html
[2] https://github.com/pfreixes/asyncio_load_instrument
[3] #171
[4] https://discuss.python.org/c/async-sig/20