Microoptimize the walk() function #587
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I noticed that it takes pynvim about 4ms to attach to an nvim instance for me, and 3ms of that is due to the single line: metadata = walk(decode_if_bytes, metadata) This commit reduces the walk() time down to 1.5ms, which brings the total attach time down to 2.5ms. This is helpful for me because in my use case I end up connecting to all of the currently-running nvim processes and this starts to take a noticeable amount of time. Unfortunately parallelization does not help here due to the nature of the slowness. walk() is expensive because it does a very large amount of pure-python manipulation, so this commit is just some tweaks to reduce the overheads: - *args and **kw make the function call slow, and we can avoid needing them by pre-packing the args into fn via functools.partial - The comprehensions can be written to directly construct the objects rather than create a generator which is passed to a constructor - The typechecking is microoptimized by calling type() once and unrolling the `type_ in [list, tuple]` check I did notice that in my setup the metadata contains no byte objects, so the entire call is a noop. I'm not sure if that is something that could be relied on or detected, which could be an even bigger speedup.
|
I believe this is the same issue as was talked about in #250 |
|
Note that CI looks yucky right now, we're working on that :) |
|
Haha yeah at first I got worried about how I broke so much without realizing it Also wanted to say, thanks for the project! It's been very handy |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I noticed that it takes pynvim about 4ms to attach to an nvim instance for me, and 3ms of that is due to the single line:
metadata = walk(decode_if_bytes, metadata)
This commit reduces the walk() time down to 1.5ms, which brings the total attach time down to 2.5ms. This is helpful for me because in my use case I end up connecting to all of the currently-running nvim processes and this starts to take a noticeable amount of time. Unfortunately parallelization does not help here due to the nature of the slowness.
walk() is expensive because it does a very large amount of pure-python manipulation, so this commit is just some tweaks to reduce the overheads:
type_ in [list, tuple]checkI did notice that in my setup the metadata contains no byte objects, so the entire call is a noop. I'm not sure if that is something that could be relied on or detected, which could be an even bigger speedup.
fix #250