Skip to content

Commit a376a56

Browse files
tjhunterTim Hunter
tjhunter
and
Tim Hunter
authored
[120] Documentation improvements (#124)
* doc revamp * cleanup * cleanup Co-authored-by: Tim Hunter <tjhunter@cs.stanford.edu>
1 parent 36f636a commit a376a56

33 files changed

+4033
-2749
lines changed

‎Makefile‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,4 +24,7 @@ release:
2424

2525

2626
doc:
27-
PYTHONPATH=$(PWD) JUPYTER_CONFIG_DIR=$(PWD)/doc_source/jupyter mkdocs build
27+
PYTHONPATH=$(PWD):$(PWD)/doc_source JUPYTER_CONFIG_DIR=$(PWD)/doc_source/jupyter mkdocs build
28+
29+
doc-serve:
30+
PYTHONPATH=$(PWD):$(PWD)/doc_source JUPYTER_CONFIG_DIR=$(PWD)/doc_source/jupyter mkdocs serve --no-livereload

‎dds/__init__.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,10 @@
1313
from ._version import version
1414
from .introspect import accept_module as _accept_module
1515
from .store import Store
16-
from .structures import DDSPath, ProcessingStage
16+
from .structures import DDSPath, ProcessingStage, DDSException
1717

1818
__all__ = [
19+
"DDSException",
1920
"DDSPath",
2021
"keep",
2122
"eval",

‎dds/fun_args.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -110,9 +110,9 @@ def dds_hash(x: Any) -> PyHash:
110110
# and its string
111111
return dds_hash(repr(x))
112112
msg = (
113-
f"The type {type(x)} is currently not supported. The only supported types are"
114-
f"'well-known' types that are part of the standard data structures in the python library."
115-
f"If you think your data type should be supported by DDS, please open a request ticket."
113+
f"The type {type(x)} is currently not supported. The only supported types are "
114+
f"'well-known' types that are part of the standard data structures in the python library. "
115+
f"If you think your data type should be supported by DDS, please open a request ticket. "
116116
f"General Python classes will not be supported since they can carry arbitrary state and "
117117
f"cannot be easily compared. Consider using a dataclass, a dictionary or a named tuple instead."
118118
)

‎doc_source/changelog.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,6 @@
11
# Changelog
22

3-
## v0.7.3
4-
5-
Fixes the usage of positional and keyworded arguments when used in conjunction
6-
with `dds.keep`.
7-
8-
## v0.8.0 (unreleased)
3+
## v0.8.0
94

105
A number of small improvements in ergonomics to this release:
116

@@ -16,6 +11,11 @@ dates (`datetime` objects), arbitrary named tuples and arbitrary data classes.
1611
should not take arguments (`dds.keep` should be used instead). Passing arguments
1712
now triggers an error.
1813

14+
## v0.7.3
15+
16+
Fixes the usage of positional and keyworded arguments when used in conjunction
17+
with `dds.keep`.
18+
1919
## v0.7.2
2020

2121
Small usability fixes in this release:
@@ -39,4 +39,4 @@ This is a major release that changes
3939
the algorithm of calculating the signatures.
4040
*Upgrading from a previous version will trigger the cache to be calculated again*.
4141

42-
This change is not expected to happen again except for localized bug fixes.
42+
This change is not expected to happen again except for localized bug fixes.

‎doc_source/tut_collab.ipynb‎

Lines changed: 41 additions & 114 deletions
Original file line numberDiff line numberDiff line change
@@ -13,16 +13,27 @@
1313
},
1414
{
1515
"cell_type": "code",
16-
"execution_count": 1,
17-
"metadata": {},
16+
"execution_count": null,
17+
"metadata": {
18+
"tags": [
19+
"hide_cell"
20+
]
21+
},
1822
"outputs": [],
1923
"source": [
2024
"! rm -rf /tmp/dds/tut_collab"
2125
]
2226
},
27+
{
28+
"cell_type": "markdown",
29+
"metadata": {},
30+
"source": [
31+
"We will mimic the switch between branches and computers with the two following functions. This assumes that at least the `internal_dir` of the stores are shared between all the collaborators. This is naturally the case when using a shared system such as Databricks' `DBFS` store or mounting a share drive such as NFS or even Microsoft SharePoint, Dropbox, .."
32+
]
33+
},
2334
{
2435
"cell_type": "code",
25-
"execution_count": 2,
36+
"execution_count": null,
2637
"metadata": {},
2738
"outputs": [],
2839
"source": [
@@ -37,29 +48,18 @@
3748
"store_main()"
3849
]
3950
},
51+
{
52+
"cell_type": "markdown",
53+
"metadata": {},
54+
"source": [
55+
"This is the code that we have in the main branch. Let's run it once to ensure that the content is in the store and is available to everyone."
56+
]
57+
},
4058
{
4159
"cell_type": "code",
42-
"execution_count": 3,
60+
"execution_count": null,
4361
"metadata": {},
44-
"outputs": [
45-
{
46-
"name": "stdout",
47-
"output_type": "stream",
48-
"text": [
49-
"calculating my_data\n"
50-
]
51-
},
52-
{
53-
"data": {
54-
"text/plain": [
55-
"'Alice'"
56-
]
57-
},
58-
"execution_count": 3,
59-
"metadata": {},
60-
"output_type": "execute_result"
61-
}
62-
],
62+
"outputs": [],
6363
"source": [
6464
"# main branch\n",
6565
"@dds.data_function(\"/my_data\")\n",
@@ -79,7 +79,7 @@
7979
},
8080
{
8181
"cell_type": "code",
82-
"execution_count": 4,
82+
"execution_count": null,
8383
"metadata": {},
8484
"outputs": [],
8585
"source": [
@@ -88,20 +88,9 @@
8888
},
8989
{
9090
"cell_type": "code",
91-
"execution_count": 5,
91+
"execution_count": null,
9292
"metadata": {},
93-
"outputs": [
94-
{
95-
"data": {
96-
"text/plain": [
97-
"'Alice'"
98-
]
99-
},
100-
"execution_count": 5,
101-
"metadata": {},
102-
"output_type": "execute_result"
103-
}
104-
],
93+
"outputs": [],
10594
"source": [
10695
"# fork branch\n",
10796
"\n",
@@ -117,27 +106,9 @@
117106
},
118107
{
119108
"cell_type": "code",
120-
"execution_count": 6,
109+
"execution_count": null,
121110
"metadata": {},
122-
"outputs": [
123-
{
124-
"name": "stdout",
125-
"output_type": "stream",
126-
"text": [
127-
"calculating my_data\n"
128-
]
129-
},
130-
{
131-
"data": {
132-
"text/plain": [
133-
"'Alice, Bob'"
134-
]
135-
},
136-
"execution_count": 6,
137-
"metadata": {},
138-
"output_type": "execute_result"
139-
}
140-
],
111+
"outputs": [],
141112
"source": [
142113
"# fork branch\n",
143114
"\n",
@@ -158,18 +129,9 @@
158129
},
159130
{
160131
"cell_type": "code",
161-
"execution_count": 7,
132+
"execution_count": null,
162133
"metadata": {},
163-
"outputs": [
164-
{
165-
"name": "stdout",
166-
"output_type": "stream",
167-
"text": [
168-
"728433a6899b293d14157e4a321431a59a66f85109086851f58bcd53ba984c8b\r\n",
169-
"ecf06a7b7943cda9620d652393d2067887ce53c816ea7f0a08de94bee2fa0290\r\n"
170-
]
171-
}
172-
],
134+
"outputs": [],
173135
"source": [
174136
"! ls /tmp/dds/tut_collab/internal/blobs | grep -v meta"
175137
]
@@ -183,17 +145,9 @@
183145
},
184146
{
185147
"cell_type": "code",
186-
"execution_count": 8,
148+
"execution_count": null,
187149
"metadata": {},
188-
"outputs": [
189-
{
190-
"name": "stdout",
191-
"output_type": "stream",
192-
"text": [
193-
"Alice"
194-
]
195-
}
196-
],
150+
"outputs": [],
197151
"source": [
198152
"! cat /tmp/dds/tut_collab/data_main/my_data"
199153
]
@@ -207,17 +161,9 @@
207161
},
208162
{
209163
"cell_type": "code",
210-
"execution_count": 9,
164+
"execution_count": null,
211165
"metadata": {},
212-
"outputs": [
213-
{
214-
"name": "stdout",
215-
"output_type": "stream",
216-
"text": [
217-
"Alice, Bob"
218-
]
219-
}
220-
],
166+
"outputs": [],
221167
"source": [
222168
"! cat /tmp/dds/tut_collab/data_fork/my_data"
223169
]
@@ -231,7 +177,7 @@
231177
},
232178
{
233179
"cell_type": "code",
234-
"execution_count": 10,
180+
"execution_count": null,
235181
"metadata": {},
236182
"outputs": [],
237183
"source": [
@@ -251,20 +197,9 @@
251197
},
252198
{
253199
"cell_type": "code",
254-
"execution_count": 11,
200+
"execution_count": null,
255201
"metadata": {},
256-
"outputs": [
257-
{
258-
"data": {
259-
"text/plain": [
260-
"'Alice, Bob'"
261-
]
262-
},
263-
"execution_count": 11,
264-
"metadata": {},
265-
"output_type": "execute_result"
266-
}
267-
],
202+
"outputs": [],
268203
"source": [
269204
"# main branch\n",
270205
"my_data()"
@@ -289,30 +224,22 @@
289224
"source": [
290225
"To conclude, `dds`'s philosophy of _data is code_ makes it easy to share and update data in a collaborative environment:\n",
291226
"* data is tracked in each branch\n",
292-
"* switching between code branches to retrieve code and views of the corresponding data\n",
227+
"* switching between code branches works just like normal code to retrieve views of the corresponding data\n",
293228
"* all the data can be pre-calculated before merging the code, making a code+data checkout always a fast operation for the target branch"
294229
]
295230
},
296231
{
297232
"cell_type": "code",
298-
"execution_count": 13,
233+
"execution_count": null,
299234
"metadata": {},
300-
"outputs": [
301-
{
302-
"name": "stdout",
303-
"output_type": "stream",
304-
"text": [
305-
"728433a6899b293d14157e4a321431a59a66f85109086851f58bcd53ba984c8b\r\n",
306-
"ecf06a7b7943cda9620d652393d2067887ce53c816ea7f0a08de94bee2fa0290\r\n"
307-
]
308-
}
309-
],
235+
"outputs": [],
310236
"source": [
311237
"! ls /tmp/dds/tut_collab/internal/blobs | grep -v meta"
312238
]
313239
}
314240
],
315241
"metadata": {
242+
"celltoolbar": "Tags",
316243
"kernelspec": {
317244
"display_name": "Python 3",
318245
"language": "python",

0 commit comments

Comments
 (0)