feat: optimize with iterators #3652
Open
+74
−73
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BobTheBuidler
commented
Mar 30, 2025
web3/_utils/abi.py
Outdated
|
|
||
| @curry | ||
| def abi_data_tree(types: Sequence[TypeStr], data: Sequence[Any]) -> List[Any]: | ||
| def abi_data_tree(types: Sequence[TypeStr], data: Sequence[Any]) -> "map[ABITypedData]": |
There was a problem hiding this comment.
idea for more optimization: rip out this func and replace it with a lambda which would have less overhead than using curry
BobTheBuidler
commented
Apr 5, 2025
web3/_utils/abi.py
Outdated
| [abi_data_tree(types)], | ||
| map(data_tree_map, normalizers), | ||
| [partial(recursive_map, strip_abi_type)], | ||
| return pipe( |
There was a problem hiding this comment.
this could be further optimized by caching the pipe per-method as a compose object
BobTheBuidler
force-pushed
the
patch-3
branch
7 times, most recently
from
ee5190b to
11a0672
Compare
This was originally part of my other PR but that is becoming quite cluttered and hard to review so I separated this one out. Previously, the code would create a new dict from `d` one time for each item in the iterable. Now, the code creates just one new dict from `d` at the beginning and uses that same dict to check membership for each item.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What was wrong?
The formatters init quite a few lists each run, and can be made much more efficient using iterators. This PR does so.
I figured this was more important than the logger stuff in my last PR so finished this one up first.
Related to Issue # N/A
Closes # N/A
How was it fixed?
creation of an IteratorProxy class that allows
mapobjects to be passed thru the formatters instead of fully materializing a list every step of the wayTodo:
Cute Animal Picture