Inconsistent status reporting

[tamlyn]

The current reported availability status in the table doesn't always seem to correspond with the historical availability charts. For example the table currently shows:

I take that to mean CX23 is not currently available in Helsinki, but was last available 8 hours ago. However the chart at the bottom of the page implies the reverse, that it is currently available and was last unavailable 8 hours ago.

Several other charts are similarly contradictory. Empirically I think the table is correct since none of those instance sizes are showing up for me in the cloud console.

Thanks for building this tracker - it's a great service!

[elsbrock]

Thanks for the detailed report — the screenshots made this straightforward to track down. Your instinct was right: the table is correct, the chart was wrong.

What was happening

The table and the chart come from two different data paths. The table reads a live snapshot from the worker, refreshed every 60s. The chart is reconstructed from Analytics Engine, which only ever receives transition events — "became available", "became unavailable" — never periodic samples. So the chart has to rebuild a step function from sparse edges: seed a state, then hold it until the next edge.

The history query aggregated those edges into hourly buckets with MAX(double1). That is fine for a metric and wrong for an edge stream. If a pair went available and then unavailable within the same hour, the 0 was erased and the whole hour reported a single "became available" event. The client took that at face value, and since nothing followed it, painted the pair green all the way to now.

That is exactly your CX23/Helsinki case. Stock appears for a few minutes, the worker catches it on one poll, it is gone by the next. Two events, one hour bucket, MAX = 1. Table: "not available, last seen 8h ago." Chart: "available since 8h ago." Same underlying event, opposite rendering — which is why it looked like a mirror image rather than noise.

It hit scarce, flappy types hardest, which is why several charts were contradictory at once and why the popular sizes were the worst offenders.

A second bug compounded it: the client inferred the state entering the window by inverting the first event in it. That only holds while edges strictly alternate — an invariant the bucketing had already broken — so the region before the blip rendered red as well.

Fix

Shipped in 4ad2595:

• The history query now returns raw transitions at full resolution. No bucketing, no MAX, no timestamps snapped to the top of the hour (which was also costing up to 60 minutes of accuracy per cell, independently of the collapse bug).
• The state entering the window is resolved by a separate argMax query over the preceding 30 days, instead of being guessed from the window's own first edge.
• The trailing cell of a live window is reconciled against the same snapshot the table renders — but only when it is saturated in the opposite direction, since that can only mean a transition never reached the dataset. A partly-available cell is consistent with either state and is left as measured.

The reconstruction logic is now a standalone module with tests covering the sub-hour blip case, so this specific regression can't come back quietly.

One caveat worth setting expectations on: this fixes how history is read, not what was already written. Hours that were recorded before the fix still only exist as their collapsed form in Analytics Engine, so charts will keep showing a bit of the old distortion until that data ages out of the 3-month retention window. Everything from here on is recorded and rendered at full resolution.

And thank you for the kind words — glad it's useful.

---

Generated by Claude Code

[elsbrock]

Correction to my last paragraph above: the caveat doesn't apply. Analytics Engine stores every transition as its own row — the collapse happened purely at query time, in the GROUP BY. The raw edges were never lost, so the new query reads existing history at full resolution too. Past charts should be correct as soon as the fix is deployed, not just data recorded from now on.

---

Generated by Claude Code