You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+33-54Lines changed: 33 additions & 54 deletions
Original file line number
Diff line number
Diff line change
@@ -8,19 +8,16 @@
8
8
9
9
- 🔗 ***Fluent*** chainable lazy operations
10
10
- 🔀 ***Concurrent*** via *threads*/*processes*/`async`
11
-
- 🇹 ***Typed***, fully annotated, `Stream[T]` is both an `Iterable[T]` and an `AsyncIterable[T]`
12
-
- 🛡️ ***Tested*** extensively with **Python 3.7 to 3.14**
13
-
- 🪶 ***Light***, no dependencies
14
-
11
+
- 🇹 Fully ***Typed***, `Stream[T]` is both an `Iterable[T]` and an `AsyncIterable[T]`
12
+
- 🛡️ ***Battle-tested*** for prod, extensively tested with **Python 3.7 to 3.14**.
15
13
16
14
17
15
## 1. install
18
16
17
+
> no dependencies
19
18
```bash
20
19
pip install streamable
21
-
```
22
-
*or*
23
-
```bash
20
+
# or
24
21
conda install conda-forge::streamable
25
22
```
26
23
@@ -52,7 +49,7 @@ inverses: Stream[float] = (
52
49
53
50
## 5. iterate
54
51
55
-
Iterate over a `Stream[T]` just as you would over any other `Iterable[T]`/`AsyncIterable`, elements are processed *on-the-fly*:
52
+
Iterate over a `Stream[T]` just as you would over any other `Iterable[T]` (or `AsyncIterable`), elements are processed *on-the-fly*:
56
53
57
54
58
55
### as `Iterable[T]`
@@ -113,9 +110,7 @@ Iterate over a `Stream[T]` just as you would over any other `Iterable[T]`/`Async
113
110
114
111
# ↔ example: Extract-Transform-Load
115
112
116
-
Let's take an example showcasing most of the `Stream`'s operations:
117
-
118
-
**ETL scripts** can benefit from the expressiveness of this library. Below is a pipeline that extracts the 67 quadruped Pokémon from the first three generations using [PokéAPI](https://pokeapi.co/) and loads them into a CSV:
113
+
Let's take an example showcasing most of the `Stream`'s operations: Below is a pipeline that extracts the 67 quadruped Pokémon from the first three generations using [PokéAPI](https://pokeapi.co/) and loads them into a CSV:
119
114
120
115
```python
121
116
import csv
@@ -164,7 +159,7 @@ with open("./quadruped_pokemons.csv", mode="w") as file:
164
159
165
160
## or the `async` way
166
161
167
-
- use `.amap`: the `.map`'s `async` counterpart, see [`async` Operations](#-async-operations).
162
+
- use the `.amap` operation: the `.map`'s `async` counterpart, see [`async` Operations](#-async-operations).
168
163
-`await` the `Stream`: runs a full iteration over it as an `AsyncIterable[T]`.
169
164
170
165
```python
@@ -219,8 +214,6 @@ asyncio.run(main())
219
214
220
215
# 📒 ***Operations***
221
216
222
-
*A dozen expressive lazy operations and that’s it!*
> By default, all the concurrency modes presented below yield results in the upstream order (FIFO). Set the parameter `ordered=False` to yield results as they become available (***First Done, First Out***).
241
-
242
230
### thread-based concurrency
243
231
244
-
> Applies the transformation via `concurrency` threads:
232
+
> Applies the transformation via `concurrency` threads, yielding results in the upstream order (FIFO), set the parameter `ordered=False` to yield results as they become available (*First Done, First Out*).
245
233
246
234
<details ><summarystyle="text-indent: 40px;">👀 show snippet</summary></br>
> `concurrency` is also the size of the buffer containing not-yet-yielded results. **If the buffer is full, the iteration over the upstream is paused** until a result is yielded from the buffer.
264
-
265
-
> [!TIP]
266
-
> The performance of thread-based concurrency in a CPU-bound script can be drastically improved by using a [Python 3.13+ free-threading build](https://docs.python.org/3/using/configure.html#cmdoption-disable-gil).
251
+
> **Memory-efficient**: Only `concurrent` upstream elements are pulled for processing; the next upstream element is pulled only when a result is yielded downstream.
> Calls a function, passing the stream as first argument, followed by `*args/**kwargs` if any:
709
+
> Calls a function, passing the stream as first argument, followed by `*args/**kwargs` if any (inspired by the `.pipe` from [pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pipe.html) or [polars](https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.pipe.html)):
727
710
728
711
<details ><summarystyle="text-indent: 40px;">👀 show snippet</summary></br>
729
712
@@ -739,24 +722,20 @@ import pandas as pd
739
722
```
740
723
</details>
741
724
742
-
> Inspired by the `.pipe` from [pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pipe.html) or [polars](https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.pipe.html).
743
-
744
725
---
745
726
---
746
727
747
728
# 📒 ***`async` Operations***
748
729
749
-
Operations that accept a function as an argument have an `async` counterpart, which has the same signature but accepts `async` functions instead. These `async` operations are named the same as the original ones but with an `a` prefix.
730
+
The operations accepting a function as an argument have an `async` counterpart operation, which has the same signature but accepts `async` functions instead.
750
731
751
-
> [!TIP]
752
-
> One can mix regular and `async` operations on the same `Stream`, and then consume it as a regular `Iterable` or as an `AsyncIterable`.
732
+
**inter-operability**: Both regular and `async` operations can be mixed on the same `Stream`, and it can then be consumed as regular `Iterable` or as `AsyncIterable`.
753
733
754
-
## `.amap`
734
+
## `.amap`
755
735
756
736
> Applies an `async` transformation on elements:
757
737
758
-
759
-
> consumed as an `Iterable[T]`:
738
+
- consume as `Iterable[T]`:
760
739
761
740
<details ><summarystyle="text-indent: 40px;">👀 show snippet</summary></br>
0 commit comments