Skip to content

Commit 8051504

Browse files
committed
docs(tsdb): write down what the esp_tsdb fork is and how to move it
The fork pin was documented only as a YAML comment on each board config, which said why the sidecar change exists but not what else the pin carries, how to rebase it onto a new upstream release, or which fork branches must not be deleted. That is the knowledge that decays first. docs/esp-tsdb-fork.md now records all three commits in the stack (sidecar header, tsdb_peek_span, esp32p4 manifest), each one's upstream status, the branch layout, and the rebase-not-merge refresh procedure. It also states plainly that no release blocks on an upstream merge — upstream last committed 2026-07-12 and merges PRs in bursts. Fixes a guide that had gone stale against the shipped configs: the tsdb-integration YAML example still pinned `zakery292/esp_tsdb^2.1.0` from the registry and sent P4 users to a `tigomonitor` fork branch, while every board config in the repo pins the fork SHA and the note 40 lines further down said so. The example now matches what we ship, and the branch it named is one of eight being pruned from the fork. Fork housekeeping done alongside this (no repo change — the pinned SHA is unmoved): RAR/esp_tsdb `main` was 23 commits behind upstream and 0 ahead, so anyone landing on it saw v2.0.2 and none of this work; it is now fast-forwarded to upstream 2.3.0. The pin's branch was `perf/sidecar-header-p4`, which no longer described its contents; the same commit is now the tip of `tigo/on-2.3.0`, a name that states its rebase base. Verified: `npx astro build` builds all 11 pages with the link validator clean. Screenshots not regenerated — no /api shape changed.
1 parent 982b735 commit 8051504

2 files changed

Lines changed: 99 additions & 5 deletions

File tree

docs/esp-tsdb-fork.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Why the board configs pin a fork of esp_tsdb, and how to move the pin
2+
3+
## One-line statement
4+
5+
The reference configs pin `RAR/esp_tsdb` at an immutable SHA that is upstream
6+
`main` plus exactly three commits. This is a deliberate, maintained state, not
7+
a stopgap waiting on a merge.
8+
9+
## What the fork actually contains
10+
11+
As of 2026-08-13 the pin is `ebfc360f00263ab90116ee3e556a9153ab4041a2`, which is
12+
`zakery292/esp_tsdb` 2.3.0 (`209bcca`) with three commits on top and **nothing
13+
behind** — it is a rebased topic stack, not a divergent history.
14+
15+
| Commit | What it does | Why we need it | Upstream |
16+
|---|---|---|---|
17+
| `dbf4ebf` | Writes the DB header to an alternating sidecar (`<db>.h0`/`.h1`) instead of in place at offset 0 | A snapshot took 21.4 s; 15.2 s of that was header rewrites. LittleFS overwrite cost is linear at ~20.4 ms/KB, so a byte-0 write rewrites the whole file. Now 633 ms median / 1,019 ms max over 132 commits on the rig, with no drift as the DBs fill. | [PR #6](https://github.com/zakery292/esp_tsdb/pull/6), open since 2026-08-09 |
18+
| `3fb785f` | Adds `tsdb_peek_span` — read a database's time span without opening it | The Diagnostics page needs each DB's span; opening every DB to get it is the expensive path we just removed. | **Never submitted.** No upstream path exists. |
19+
| `ebfc360` | Adds `esp32p4` to the manifest's `targets` list | Manifest-only. Without it the component manager refuses to install on a P4; the code itself is target-agnostic. | [PR #4](https://github.com/zakery292/esp_tsdb/pull/4), open since 2026-07-05 |
20+
21+
Upstream's last commit was 2026-07-12, and `development` is identical to `main`.
22+
PRs #1#3 were merged in a single batch on 2026-07-05, so the maintainer works in
23+
bursts. Treat a merge as welcome but unscheduled: **nothing in this project should
24+
block on one.** In particular, the v2.0.0 release does not.
25+
26+
## Branch layout on the fork
27+
28+
| Branch | Role |
29+
|---|---|
30+
| `main` | Mirror of upstream `main`. Never commit here — it exists so that anyone landing on the fork sees the real upstream code. |
31+
| `tigo/on-<upstream-version>` | The integration branch. Its name states the version it is rebased onto; its tip is the SHA the board configs pin. Currently `tigo/on-2.3.0`. |
32+
| `feat/manifest-esp32p4`, `upstream/sidecar-header` | PR heads for #4 and #6. Do not delete these — deleting a PR's head branch closes the PR. |
33+
34+
One branch per upstream base, rather than one long-lived branch that gets
35+
rebased in place, means a shipped pin's base is always readable from a branch
36+
name and old pins never dangle.
37+
38+
## Moving the pin to a new upstream release
39+
40+
The stack is three commits and must stay that way — **rebase, never merge**. A
41+
merge commit makes the stack unreadable and the next rebase painful.
42+
43+
```bash
44+
git clone git@github.com:RAR/esp_tsdb.git && cd esp_tsdb
45+
git remote add up https://github.com/zakery292/esp_tsdb.git && git fetch up
46+
47+
# 1. Mirror upstream, then branch the stack onto the new base.
48+
git checkout main && git merge --ff-only up/main && git push origin main
49+
git checkout -b tigo/on-<new-version> tigo/on-<old-version>
50+
git rebase --onto up/main <old-upstream-sha>
51+
52+
# 2. Sanity-check the shape before trusting it: expect "0<TAB>3".
53+
git rev-list --left-right --count up/main...HEAD
54+
55+
# 3. Verify on hardware, not by eye — see below.
56+
git push origin tigo/on-<new-version>
57+
```
58+
59+
Then update `ref:` in every board config that carries one (`esp32s3-atoms3r`,
60+
`esp32p4-evboard`, `test-p4-tigomonitor`, `test-p4-ble-tigomonitor`) **and** in
61+
the deployed rig config, which is a standalone file outside this repo.
62+
63+
**Verification is a rig run, not a compile.** A clean build proves the ref
64+
resolved, nothing more. The sidecar change is in the flash hot path, so confirm
65+
snapshot time is still sub-second in the History logs over several commits
66+
before calling a new pin good.
67+
68+
## If upstream merges the PRs
69+
70+
Retiring the fork means dropping to a registry version — but only once *all
71+
three* commits are upstream. Two of the three are in open PRs; `tsdb_peek_span`
72+
has never been submitted, so a full retirement needs that PR opened and merged
73+
first. Until then, a partial merge only shortens the stack; it does not remove it.
74+
75+
## Open item
76+
77+
`idf_component.yml` still declares `version: "2.3.0"`, identical to upstream
78+
despite the three extra commits, so a build's provenance is not self-describing.
79+
`2.3.0+tigo.1` is valid semver build metadata and would fix that. Not done yet —
80+
it touches the manifest that PR #4 also edits, so it is worth doing after that
81+
PR resolves rather than creating a conflict now.
82+
83+
## See also
84+
85+
- [Saving History to Flash](https://rar.github.io/esphome-tigomonitor/guides/tsdb-integration/) — the user-facing cost model and the reason the sidecar change exists
86+
- [`tsdb-flash-crash-issue.md`](tsdb-flash-crash-issue.md) — the separate flash-write crash investigation (cause was cabinet power, not the filesystem)

site/src/content/docs/guides/tsdb-integration.md

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -63,11 +63,12 @@ esp32:
6363
framework:
6464
type: esp-idf
6565
components:
66-
# Upstream 2.1.0+ has everything this project needs (handle-based
67-
# multi-instance API, wrapped-ring query fix). ESP32-P4 only: use the
68-
# RAR/esp_tsdb fork's `tigomonitor` branch instead (source/ref pin) —
69-
# 2.1.0's manifest doesn't list the esp32p4 target yet.
70-
- zakery292/esp_tsdb^2.1.0
66+
# A fork, pinned by commit SHA — see the note below for why. An
67+
# immutable SHA rather than a branch, because a branch can move under
68+
# a build and this is the config people copy.
69+
- name: zakery292/esp_tsdb
70+
source: https://github.com/RAR/esp_tsdb.git
71+
ref: ebfc360f00263ab90116ee3e556a9153ab4041a2
7172
- joltwallet/littlefs^1.16
7273
sdkconfig_options:
7374
CONFIG_PARTITION_TABLE_CUSTOM: "y"
@@ -111,6 +112,13 @@ The fork writes the header to an alternating sidecar file (`<db>.h0` / `<db>.h1`
111112
so the hot path appends instead. Measured on the reference rig over 132 commits:
112113
median 633 ms, maximum 1,019 ms, with no upward drift as the databases fill. The
113114
change is not upstream yet, so the board config pins the fork by commit SHA.
115+
116+
The pin is upstream 2.3.0 plus three commits and nothing behind it. On an
117+
ESP32-P4 it is required for a second reason: upstream's manifest doesn't list
118+
`esp32p4` as a target, so the component manager refuses to install there at all.
119+
[What the fork
120+
contains](https://github.com/RAR/esphome-tigomonitor/blob/main/docs/esp-tsdb-fork.md)
121+
has the details.
114122
:::
115123

116124
> **Board note:** the `board:` value above (`m5stack-atoms3`) is an example. The reference rig for this project is the **AtomS3R** — set `board:` to whatever board you actually run so you don't flash the wrong target.

0 commit comments

Comments
 (0)