Skip to content

Commit 5a04c6b

Browse files
PeterKrausPeterKraus
authored
Stateless components (#175)
* Implement stateless components. * Ruff * More ruff. * Rework components, pipelines, drivers again. * Ruff * chore: ty * chore: more ty * chore: ty the third * docs: update part 1 * Modify workflow to pin ruff version. * fix failing tests * fix failing lint * Another change to ruff stuff and other workflow version stuff. * Fix out of order execution, again. * docs: rejig & update * debug prints in pytest
1 parent da4e6fe commit 5a04c6b

49 files changed

Lines changed: 1640 additions & 1605 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/push-main.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,16 +24,16 @@ jobs:
2424
needs: [call-pages]
2525
runs-on: ubuntu-latest
2626
steps:
27-
- uses: actions/checkout@v4
28-
- uses: actions/setup-python@v5
27+
- uses: actions/checkout@v7
28+
- uses: actions/setup-python@v7
2929
with:
3030
python-version: '3.10'
31-
- uses: actions/download-artifact@v4
31+
- uses: actions/download-artifact@v8
3232
with:
3333
name: public-ubuntu-latest-3.10
3434
path: public/main
3535
- uses: peaceiris/actions-gh-pages@v3
3636
with:
3737
github_token: ${{ secrets.GITHUB_TOKEN }}
3838
publish_dir: ./public
39-
keep_files: true
39+
keep_files: true

‎.github/workflows/workflow-build.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414
build:
1515
runs-on: ${{ inputs.os }}
1616
steps:
17-
- uses: actions/checkout@v4
17+
- uses: actions/checkout@v7
1818
with:
1919
fetch-depth: 0
20-
- uses: actions/setup-python@v5
20+
- uses: actions/setup-python@v7
2121
with:
2222
python-version: ${{ inputs.pyver }}
2323
- name: Install dependencies
@@ -29,7 +29,7 @@ jobs:
2929
shell: bash
3030
run: |
3131
python -m build
32-
- uses: actions/upload-artifact@v4
32+
- uses: actions/upload-artifact@v7
3333
with:
3434
name: dist-${{ inputs.os }}-${{ inputs.pyver }}
35-
path: dist
35+
path: dist

‎.github/workflows/workflow-lint.yml‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,10 @@ jobs:
77
lint:
88
runs-on: ubuntu-latest
99
steps:
10-
- uses: actions/checkout@v4
11-
- name: Run ruff
12-
uses: astral-sh/ruff-action@v3
10+
- uses: actions/checkout@v7
11+
- name: Run ruff check
12+
uses: astral-sh/ruff-action@v4.0.0
13+
with:
14+
version: "<0.16"
1315
- name: Run ruff format
14-
run: ruff format --check
16+
run: ruff format --check

‎.github/workflows/workflow-pages.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,11 @@ jobs:
1414
pages:
1515
runs-on: ${{ inputs.os }}
1616
steps:
17-
- uses: actions/checkout@v4
18-
- uses: actions/setup-python@v5
17+
- uses: actions/checkout@v7
18+
- uses: actions/setup-python@v7
1919
with:
2020
python-version: ${{ inputs.pyver }}
21-
- uses: actions/download-artifact@v4
21+
- uses: actions/download-artifact@v8
2222
with:
2323
name: dist-${{ inputs.os }}-${{ inputs.pyver }}
2424
- uses: jannekem/run-python-script-action@v1
@@ -33,7 +33,7 @@ jobs:
3333
run: |
3434
sphinx-apidoc -o docs/source/apidoc src/tomato -T -M -E -f --templatedir=docs/apidoc_t
3535
sphinx-build -W -b html docs/source public/main
36-
- uses: actions/upload-artifact@v4
36+
- uses: actions/upload-artifact@v7
3737
with:
3838
name: public-${{ inputs.os }}-${{ inputs.pyver }}
39-
path: public/main
39+
path: public/main

‎.github/workflows/workflow-test.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,11 @@ jobs:
1414
test:
1515
runs-on: ${{ inputs.os }}
1616
steps:
17-
- uses: actions/checkout@v4
18-
- uses: actions/setup-python@v5
17+
- uses: actions/checkout@v7
18+
- uses: actions/setup-python@v7
1919
with:
2020
python-version: ${{ inputs.pyver }}
21-
- uses: actions/download-artifact@v4
21+
- uses: actions/download-artifact@v8
2222
with:
2323
name: dist-${{ inputs.os }}-${{ inputs.pyver }}
2424
- uses: jannekem/run-python-script-action@v1
@@ -33,4 +33,4 @@ jobs:
3333
run: tomato --version
3434
- name: Run pytest
3535
shell: bash
36-
run: pytest -vvv --durations=0
36+
run: pytest -vvv --durations=0

‎docs/apidoc_t/package.rst_t‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,18 @@
1515

1616
{%- if is_namespace %}
1717
{{- [pkgname, "namespace"] | join(" ") | e | heading }}
18-
{% elif 'tomato.drivers.dummy' == pkgname %}
19-
{{- "**dummy**: A dummy driver module" | heading }}
20-
{% elif 'tomato.drivers.biologic' == pkgname %}
21-
{{- "**biologic**: Driver for BioLogic potentiostats" | heading }}
18+
{% elif 'tomato.daemon' == pkgname %}
19+
{{- "**tomato.daemon**: Functions and modules comprising the tomato daemon" | heading }}
20+
{% elif 'tomato.driverinterface_2_0' == pkgname %}
21+
{{- "**tomato.DriverInterface**: version 2.0" | heading }}
22+
{% elif 'tomato.driverinterface_2_1' == pkgname %}
23+
{{- "**tomato.DriverInterface**: version 2.1" | heading }}
24+
{% elif 'tomato.ketchup' == pkgname %}
25+
{{- "**tomato.ketchup**: CLI and API for the tomato job queue" | heading }}
26+
{% elif 'tomato.tomato' == pkgname %}
27+
{{- "**tomato.tomato**: CLI and API for the tomato daemon" | heading }}
28+
{% elif 'tomato.passata' == pkgname %}
29+
{{- "**tomato.passata**: CLI and API for tomato drivers and components" | heading }}
2230
{% else %}
2331
{{- [pkgname, "package"] | join(" ") | e | heading }}
2432
{% endif %}

‎docs/source/conf.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,3 +88,13 @@
8888
"dgbowl_schemas": ("https://dgbowl.github.io/dgbowl-schemas/main", None),
8989
"xarray": ("https://docs.xarray.dev/en/stable", None),
9090
}
91+
92+
rst_epilog = """
93+
.. |setfile| replace:: *settings file*
94+
95+
.. _setfile: quickstart.html#settings-file
96+
97+
.. |devfile| replace:: *devices file*
98+
99+
.. _devfile: quickstart.html#devices-file
100+
"""

‎docs/source/driver_develop.rst‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -88,8 +88,3 @@ DriverInterface ver. 2.0
8888
.. autoclass:: tomato.driverinterface_2_0.ModelDevice
8989
:no-index:
9090
:members:
91-
92-
93-
.. |setfile| replace:: *settings file*
94-
95-
.. _setfile: quickstart.html#settings-file

‎docs/source/quickstart.rst‎

Lines changed: 37 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -29,45 +29,63 @@ Where *appdir* is ``/home/kraus/.config/tomato/1.0a1/``. A custom *appdir* can b
2929

3030
**tomato** program flowchart
3131
````````````````````````````
32-
**tomato** currently contains two command line user-facing utilities/executables:
32+
**tomato** currently contains three command line user-facing utilities/executables:
3333

34-
- :mod:`~tomato.tomato`, responsible for management of the daemon process, and
35-
- :mod:`~tomato.ketchup`, responsible for job submission and management.
34+
- :mod:`~tomato.tomato`, responsible for management of the daemon process,
35+
- :mod:`~tomato.ketchup`, responsible for job submission and management, and
36+
- :mod:`~tomato.passata`, providing an interface to the hardware via drivers and components.
3637

37-
These two executables then internally spawn other processes, including the state daemon process ``tomato-daemon``, a ``tomato-job`` process for every running job, and a ``tomato-driver`` process for each driver type, managing all devices of a that type.
38+
These executables then internally spawn other processes, including the state daemon process ``tomato-daemon``, a ``tomato-job`` process for every running job, and a ``tomato-driver`` process for each driver type, managing all devices of a that type.
3839

3940
.. mermaid::
4041

4142
flowchart TB
4243
subgraph daemon[tomato-daemon]
43-
cmd{{port:cmd}} ==> st[(state)]
44-
st --> jm([job manager])
45-
st --> dm([driver manager])
46-
st <--> io[state io]
47-
end
44+
cmd{{port:cmd}} <==> cfg[(config)]
45+
cfg --> jm([job manager])
46+
cfg --> pm([pip manager])
47+
cfg --> dm([driver manager])
4848

49-
subgraph driver[tomato-driver]
50-
dcmd{{port:cmd}} ==> dst[(state)]
5149
end
5250

53-
dst o-.-o h1[hardware]
51+
jm -.-> job
5452

5553
subgraph job[tomato-job]
5654
mp([main loop]) --> tp>task process]
5755
end
5856

59-
cmd ==> db[(job db\nsqlite)]
60-
jm -.-> db
61-
mp -.-> db
57+
jm <-.-> jobdb[(job db\nsqlite)]
58+
mp -.-> jobdb
59+
60+
jm <-.-> pipdb[(pip db\nsqlite)]
61+
pm <-.-> pipdb
62+
mp -.-> pipdb
63+
64+
dm <-.-> drvdb[(drv db\nsqlite)]
65+
drvdb <-.-> dst
66+
67+
setf[settings file\ndevices file] ---> cfg
6268

6369
t>tomato] -.-> cmd
6470
k>ketchup] -.-> cmd
6571

66-
jm -.-> job
6772
dst -.-> driver
6873
dm -..-> driver
6974
tp -.-> dcmd
7075

76+
subgraph driver[tomato-driver]
77+
dcmd{{port:cmd}} ==> dst[(state)]
78+
end
79+
80+
dst o-.-o h1[hardware]
81+
82+
p>passata] -.-> cmd
83+
p -.-> dcmd
84+
85+
.. note::
86+
87+
Note that the :mod:`tomato.daemon` process is stateless, only responsible for reading of the configuration files and spawning the *job*, *driver* and *pipeline*--management threads. All configuration and state information that should be persistently stored (i.e. survive restart of the components) is stored in a :mod:`sqlite3` database.
88+
7189
.. _concepts:
7290

7391
**tomato** concepts flowchart
@@ -141,6 +159,7 @@ The following concepts are used in **tomato**:
141159

142160
pip3[pipeline 3] -.-> c3
143161

162+
.. _settings-file:
144163

145164
Settings file
146165
`````````````
@@ -212,6 +231,8 @@ Additional, *driver*-specific settings may be provided in this section. Each *dr
212231

213232
Further *driver*-specific settings, such as ``dllpath`` or ``calibration``, can be specified here. All of these *driver*-specific settings are passed to each *driver* when its process is launched and the :class:`DriverInterface` is initialised, and can therefore contain paths to various libraries or other files necessary for the *driver* to function.
214233

234+
.. _devices-file:
235+
215236
Devices file
216237
````````````
217238
This ``yaml``-formatted file contains information about each *device*, corresponding to an individual piece of hardware managed by **tomato**, as well as information about the organisation of the individually-addressable *components* of those *devices* into *pipelines*.
@@ -341,7 +362,3 @@ As of ``tomato-2.0``, the :obj:`task_params` specified in the *payload* are vali
341362

342363
.. autopydantic_model:: dgbowl_schemas.tomato.payload.Task
343364
:no-index:
344-
345-
.. |devfile| replace:: *devices file*
346-
347-
.. _devfile: quickstart.html#devices-file

‎docs/source/usage.rst‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -216,11 +216,3 @@ While the *job* is running, access to an up-to-date snapshot of the data is prov
216216
>>> ketchup snapshot <jobid>
217217
218218
This will create an up-to-date ``snapshot.<jobid>.nc`` file in the current working dir. The files are overwritten on subsequent invocations of ``ketchup snapshot``. An automated, periodic snapshotting stored in a custom location can be further configured within the *payload* of the *job*.
219-
220-
.. |setfile| replace:: *settings file*
221-
222-
.. _setfile: quickstart.html#settings-file
223-
224-
.. |devfile| replace:: *devices file*
225-
226-
.. _devfile: quickstart.html#devices-file

0 commit comments

Comments
 (0)