|
1 | | -# Simple Forecaster |
| 1 | +# rawdataforecaster |
2 | 2 |
|
3 | | -Serves unmodified data from a forecast. The term 'simple' refers to the data not having been processed by forti. |
| 3 | +Serves raw (uncorrected) forecast data over gRPC from a blob store (Azure, S3, or local `file://`) in the [forti-internalformat](https://github.com/metno/forti-internalformat) format. |
4 | 4 |
|
5 | | -## Serving data |
| 5 | +## Loader strategies |
6 | 6 |
|
7 | | -```mermaid |
8 | | -graph LR; |
9 | | -server --> forecast; |
10 | | -forecast --> dataset; |
11 | | -dataset --> index; |
12 | | -dataset --> values; |
13 | | -``` |
| 7 | +The `loader.type` config key selects how forecast data is held in memory: |
14 | 8 |
|
15 | | -Four components are mainly involved in serving data: |
| 9 | +| Type | Behaviour | |
| 10 | +|---|---| |
| 11 | +| `memory` | Downloads the full grid blob at startup into CGo-allocated memory (bypassing GC pressure). Zero I/O at query time. Capped by `max_size_gib`. | |
| 12 | +| `blob` | Downloads nothing upfront; issues a byte-range read per query (2 s timeout). | |
16 | 13 |
|
17 | | -### server |
| 14 | +Use `memory` when latency matters and the dataset fits in RAM; use `blob` otherwise. |
18 | 15 |
|
19 | | -Handles incoming grpc requests, including protobuf serialization. |
| 16 | +## Native dependencies |
20 | 17 |
|
21 | | -### forecast |
22 | | - |
23 | | -Determines what is the correct group (and version) to serve data from. Forwards requests to the relevant `dataset` handler. |
24 | | - |
25 | | -### dataset |
26 | | - |
27 | | -Each object of type `dataset.Dataset` serves data for a single area/version. They maintain a list of grids for its area. A grid is a unique collection of latitude/longitude pairs within a single area. They exist because different parameters may have different grid resolutions. |
28 | | - |
29 | | -Handles requests for a given latitute/longitude pair. For each grid, lookup the correct index from `index`, and find relevant data from `values`. |
30 | | - |
31 | | -### index |
32 | | - |
33 | | -Handles lookup from latitude/longitude to a grid index. |
34 | | - |
35 | | -### values |
36 | | - |
37 | | -A collection of all data having the same area and grid id. |
38 | | - |
39 | | -Provides a `Reader` interface, for looking up data with a given index. The index is provided by the `geo` component. There are several implementations of this interface. |
40 | | - |
41 | | -## Loading data |
42 | | - |
43 | | -`forecast` component contains a function, `Forecast.update`, that is called periodically in a goroutine. It checks a blob store for updates, and loads data if needed, by calling `dataset.Download`. |
44 | | - |
45 | | -## Other modules |
46 | | - |
47 | | -### internal.health |
48 | | - |
49 | | -Provides grpc healthcheck, meant for kubernetes readiness probe. |
50 | | - |
51 | | -### pointdata |
52 | | - |
53 | | -Defines internal data format. Its placement reflects that several modules in various places in the hierarchy needs access to this. |
| 18 | +The geographic index uses [s2geometry](https://github.com/google/s2geometry) and [PROJ](https://proj.org/) via CGo. These are pre-installed in the devcontainer; building outside it requires both libraries. |
0 commit comments