Skip to content

Commit 79e0623

Browse files
committed
documented multithreaded, and make test runnable by pytest
1 parent b69c692 commit 79e0623

2 files changed

Lines changed: 28 additions & 12 deletions

File tree

README.md

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ A simple Python wrapper around [qoi](https://github.com/phoboslab/qoi), the "Qui
66

77
- Lossless with comparable compression to PNG, but fast! It encodes 10x faster and decodes around 5x faster than PNG in OpenCV or PIL.
88
- You can make it lossy with a simple trick, and then you can get similar compression to JPEG but a whole lot faster. (These number vary a lot depending on how "lossy" you make JPEG or QOI). That's cool.
9+
- Multi-threaded - no GIL hold-ups here.
910

1011
## Install
1112

@@ -39,7 +40,26 @@ from qoi.benchmark import benchmark
3940
benchmark() # Check out the arguments if you're interested
4041
```
4142

42-
## Benchmarks
43+
If you want to really max out your CPU:
44+
45+
```python
46+
from concurrent.futures import ThreadPoolExecutor, wait
47+
import numpy as np
48+
import qoi
49+
50+
RGB = np.random.randint(low=0, high=255, size=(224, 244, 3)).astype(np.uint8)
51+
52+
def worker():
53+
bites = bytearray(qoi.encode(RGB))
54+
img_decoded = qoi.decode(bites)
55+
56+
print("Go watch your CPU utilization ...")
57+
with ThreadPoolExecutor(8) as pool:
58+
futures = [pool.submit(worker) for _ in range(10000)]
59+
wait(futures)
60+
```
61+
62+
## (Single-threaded) Benchmarks
4363

4464
If we consider lossless, then we're generally comparing with PNG. Yup, there are others, but they're not as common. Benchmarks:
4565

@@ -113,7 +133,6 @@ When you're on `main` on your local, `git tag vX.X.X` then `git push origin vX.X
113133
- Benchmarks - add real images, and also compare performance with QOI to see overhead of python wrapper.
114134
- `setuptools_scm_git_archive`?
115135
- Code completion?
116-
- Investigate a simple 'lossy' compression with QOI - halve the image size and compress, and on decode, just upscale. It'll likely be very visually similar, but also much smaller, but should compare with JPEG.
117136

118137
## Discussion
119138

tests/test_multithread.py

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,21 @@
11
from concurrent.futures import ThreadPoolExecutor, wait
22

33
import numpy as np
4-
54
import qoi
65

76
RGB = np.random.randint(low=0, high=255, size=(224, 244, 3)).astype(np.uint8)
8-
RGBA = np.random.randint(low=0, high=255, size=(224, 244, 4)).astype(np.uint8)
97

108

119
def worker():
1210
bites = bytearray(qoi.encode(RGB))
13-
img_decoded = qoi.decode(bites)
14-
# print("done")
11+
decoded = qoi.decode(bites)
12+
assert np.array_equal(RGB, decoded)
1513

1614

17-
def main():
15+
def test_multithreaded():
16+
"""
17+
Note that this doesn't really test the performance, but it at least validates that it runs multithreaded.
18+
"""
1819
with ThreadPoolExecutor(8) as pool:
19-
futures = [pool.submit(worker) for _ in range(10000)]
20+
futures = [pool.submit(worker) for _ in range(100)]
2021
wait(futures)
21-
print("Did you see that in top? Congratulations you have bypass the gil")
22-
23-
if __name__ == "__main__":
24-
main()

0 commit comments

Comments
 (0)