You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: examples/cache_hook/README.md
+14-5
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,9 @@
1
1
# Cache hook
2
-
This hook uses the [diskcache](https://grantjenks.com/docs/diskcache/tutorial.html) to cache node execution on disk. The cache key is a tuple of the function's `(source code, input a, ..., input n)`.
2
+
This hook uses the [diskcache](https://grantjenks.com/docs/diskcache/tutorial.html) to cache node execution on disk. The cache key is a tuple of the function's
3
+
`(source code, input a, ..., input n)`. This means, a function will only be executed once for a given set of inputs,
4
+
and source code hash. The cache is stored in a directory of your choice, and it can be shared across different runs of your
5
+
code. That way as you develop, if the inputs and the code haven't changed, the function will not be executed again and
6
+
instead the result will be retrieved from the cache.
3
7
4
8
> 💡 This can be a great tool for developing inside a Jupyter notebook or other interactive environments.
5
9
@@ -8,9 +12,13 @@ Disk cache has great features to:
8
12
- set automated eviction policy once maximum size is reached
9
13
- allow custom `Disk` implementations to change the serialization protocol (e.g., pickle, JSON)
10
14
11
-
> ⚠ The default `Disk` serializes objects using the `pickle` module. Changing Python or library versions could break your cache (both keys and values). Learn more about [caveats](https://grantjenks.com/docs/diskcache/tutorial.html#caveats).
15
+
> ⚠ The default `Disk` serializes objects using the `pickle` module. Changing Python or library versions could break your
16
+
> cache (both keys and values). Learn more about [caveats](https://grantjenks.com/docs/diskcache/tutorial.html#caveats).
12
17
13
-
> ❓ To store artifacts robustly, please use Hamilton materializers or the [CachingGraphAdapter](https://github.com/DAGWorks-Inc/hamilton/tree/main/examples/caching_nodes) instead. The `CachingGraphAdapter` stores tagged nodes directly on the file system using common formats (JSON, CSV, Parquet, etc.). However, it isn't aware of your function version and requires you to manually manage your disk space.
18
+
> ❓ To store artifacts robustly, please use Hamilton materializers or the
- DEBUG will return inputs for each node and specify if the value is `from cache` or `executed`
43
51
44
52
## Clear cache
45
-
The utility function `h_diskcache.evict_except_driver` allows you to clear cached values for all nodes except those in the passed driver. This is an efficient tool to clear old artifacts as your project evolves.
53
+
The utility function `h_diskcache.evict_all_except_driver` allows you to clear cached values for all nodes except those in the passed driver.
54
+
This is an efficient tool to clear old artifacts as your project evolves.
0 commit comments