You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Replace per-handler legacy services and per-domain notification
controllers with a generic three-tier model: Analysis (use-case),
Workflow (step), AnalysisRun (execution attempt). Inspections become
first-class InspectionRecord entities. Result handlers move into
WorkflowResultHandlers and AnalysisResultHandlers folders.
The Argo trigger payload carries a stable core
(workflowId, inputBlobStorageLocations, outputBlobStorageLocation)
plus an extras object populated by ITriggerPayloadEnricher
implementations matched on WorkflowType (empty when no enricher
matches). This lets analyzer images extend their per-workflow
schema without coordinated changes to analytics-infrastructure
WorkflowTemplates and Sensors.
Workflow chains are supported via appsettings.json
(e.g. ["anonymizer", "thermal-reading"]); each step's output
becomes the next step's input by default. The anonymizer enricher
emits preProcessedBlobStorageLocation (TIFF on the anon storage
account), and AnonymizerResultHandler rewires the next workflow's
inputs to that TIFF when the next step is thermal-reading;
otherwise the chain is left untouched.
Add correlation IDs (workflow_id, analysis_run_id, analysis_id) to
the SaraVisualizationAvailable and SaraAnalysisResult MQTT messages.
Mirror the Pose/Position model with Flotilla.
Rewrite workflow-notifier as a generic CLI with started, result, and
exited subcommands; drop the per-domain commands. The exited
subcommand now takes error_message as a third positional so
{{workflow.failures}} can be passed verbatim. Drop the dead Error
value from WorkflowExitStatus on both Python and C# sides.
Add an integration test suite using Postgres Testcontainers with
per-test isolation. Add EF migration for the new schema.
Implements: #353
There will be several analysis available for SARA. These are triggered through an
32
-
Argo Workflow which can have a conditional flow based on the type of inspection.
33
-
34
-
```
35
-
anonymizer-->constant-level-oiler
36
-
\-->stid-uploader
3
+
SARA (Storage and Analysis of Robot Acquired plant data) is an ASP.NET Core
4
+
Web API that indexes inspection data published by ISAR and orchestrates
5
+
Argo-based analysis workflows on it, exposing the results to Flotilla. Each
6
+
incoming inspection becomes an `InspectionRecord`; one or more records are
7
+
grouped into an `Analysis` (the use-case), which executes as an `AnalysisRun`
8
+
made up of one or more sequential `Workflow` steps.
9
+
10
+
When running locally the endpoint is reachable at https://localhost:8100
11
+
(`/` redirects to Swagger).
12
+
13
+
## Architecture at a glance
14
+
15
+
-`InspectionRecord` -- one row per ISAR inspection result, persisted on
16
+
receipt of an `isar/+/inspection_result` MQTT message.
17
+
-`Analysis` -> `AnalysisRun` -> `Workflow` -- three-tier model where an
18
+
Analysis describes the use-case, an AnalysisRun is one execution attempt,
19
+
and each Workflow is a single Argo step.
20
+
-`AnalysisGroup` -- lets a single Analysis span multiple InspectionRecords.
21
+
The group is buffered until all expected records arrive or
22
+
`AnalysisGroupTimeoutMinutes` elapses.
23
+
- Workflow chains run sequentially. By default each step's output blob
24
+
becomes the next step's input; per-workflow rewiring lives in the matching
25
+
`IWorkflowResultHandler`.
26
+
-The Argo trigger payload has a stable core (`workflowId`,
27
+
`inputBlobStorageLocations`, `outputBlobStorageLocation`) plus an `extras`
28
+
object populated by `ITriggerPayloadEnricher` implementations matched on
29
+
workflow type.
30
+
- Result handling is split: `WorkflowResultHandlers/` runs per step,
31
+
`AnalysisResultHandlers/` runs once the whole Analysis is done.
32
+
33
+
## Run locally
34
+
35
+
```bash
36
+
make run# or: dotnet run --project api
37
37
```
38
38
39
-
## Analysis Mapping
40
39
41
-
Which analysis pipeline is run is chosen by the analysis mapping. A tag + an insepction descripts maps to an analysis type.
42
-
To add a new analysis type, add a value to the [AnalysisType](api/Database/Models/Analysis.cs) enum. Then add which tag + inspection description should map to the new AnalysisType. This can be done through the [AddOrCreateAnalysisMapping](api/Controllers/AnalysisMappingController.cs) endpoint. Then include code in the [MqttEventHandler](api/MQTT/MqttEventHandler.cs) to run the desiered pipeline for you AnalysisType. At the moment the supported analysis types are:
40
+
## Test & format
43
41
44
-
- Anonymizer
45
-
- ConstantLevelOiler
46
-
- ThermalReading
47
-
48
-
At the moment Anonymizer is configured to always run on IsarInspectionResultMessage
49
-
50
-
## Run
51
-
52
-
To build and run SARA, run the following command in the root folder:
53
-
54
-
```
55
-
dotnet run --project api
42
+
```bash
43
+
make test# xUnit; integration tests use Postgres Testcontainers (Docker required)
44
+
make format # CSharpier
56
45
```
57
46
58
-
## Running the argo workflow mock
59
47
60
-
`python mocks/argo_workflow_mock.py`
48
+
## Creating a new workflow
49
+
50
+
1. Register the workflow under `Analysis:Workflows` in `appsettings.json`
51
+
with its `TriggerUrl`, `OutputStorageAccount`, `OutputBlobContainer` and
52
+
(optionally) `OutputFileExtension`.
53
+
2. Reference it from one or more chains under `Analysis:Analyses`, e.g.
0 commit comments