Skip to content

Commit c7aabbd

Browse files
committed
simlpify and rewrite, to reflect current code
1 parent db0c9de commit c7aabbd

1 file changed

Lines changed: 11 additions & 46 deletions

File tree

rawdataforecaster/README.md

Lines changed: 11 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,18 @@
1-
# Simple Forecaster
1+
# rawdataforecaster
22

3-
Serves unmodified data from a forecast. The term 'simple' refers to the data not having been processed by forti.
3+
Serves raw (uncorrected) forecast data over gRPC from a blob store (Azure, S3, or local `file://`) in the [forti-internalformat](https://github.com/metno/forti-internalformat) format.
44

5-
## Serving data
5+
## Loader strategies
66

7-
```mermaid
8-
graph LR;
9-
server --> forecast;
10-
forecast --> dataset;
11-
dataset --> index;
12-
dataset --> values;
13-
```
7+
The `loader.type` config key selects how forecast data is held in memory:
148

15-
Four components are mainly involved in serving data:
9+
| Type | Behaviour |
10+
|---|---|
11+
| `memory` | Downloads the full grid blob at startup into CGo-allocated memory (bypassing GC pressure). Zero I/O at query time. Capped by `max_size_gib`. |
12+
| `blob` | Downloads nothing upfront; issues a byte-range read per query (2 s timeout). |
1613

17-
### server
14+
Use `memory` when latency matters and the dataset fits in RAM; use `blob` otherwise.
1815

19-
Handles incoming grpc requests, including protobuf serialization.
16+
## Native dependencies
2017

21-
### forecast
22-
23-
Determines what is the correct group (and version) to serve data from. Forwards requests to the relevant `dataset` handler.
24-
25-
### dataset
26-
27-
Each object of type `dataset.Dataset` serves data for a single area/version. They maintain a list of grids for its area. A grid is a unique collection of latitude/longitude pairs within a single area. They exist because different parameters may have different grid resolutions.
28-
29-
Handles requests for a given latitute/longitude pair. For each grid, lookup the correct index from `index`, and find relevant data from `values`.
30-
31-
### index
32-
33-
Handles lookup from latitude/longitude to a grid index.
34-
35-
### values
36-
37-
A collection of all data having the same area and grid id.
38-
39-
Provides a `Reader` interface, for looking up data with a given index. The index is provided by the `geo` component. There are several implementations of this interface.
40-
41-
## Loading data
42-
43-
`forecast` component contains a function, `Forecast.update`, that is called periodically in a goroutine. It checks a blob store for updates, and loads data if needed, by calling `dataset.Download`.
44-
45-
## Other modules
46-
47-
### internal.health
48-
49-
Provides grpc healthcheck, meant for kubernetes readiness probe.
50-
51-
### pointdata
52-
53-
Defines internal data format. Its placement reflects that several modules in various places in the hierarchy needs access to this.
18+
The geographic index uses [s2geometry](https://github.com/google/s2geometry) and [PROJ](https://proj.org/) via CGo. These are pre-installed in the devcontainer; building outside it requires both libraries.

0 commit comments

Comments
 (0)