Skip to content

Populate ClickHouse timeseries schema from TOML definitions #5942

Open
@bnaecker

Description

@bnaecker

As part of #5941, we'll move timeseries schema from ClickHouse into CockroachDB as the primary source of truth. ClickHouse will still need these definitions, however, to support querying and as a self-contained record of how to understand all the data in the database. This issue tracks populating the schema in ClickHouse from the underlying TOML definitions, rather than deriving those from the samples oximeter receives.

I'm not entirely sure how this will work yet, but as a strawman:

  • Add an API to oximeter for setting the schema
  • oximeter will insert (or update) these in ClickHouse. It should be the case that it can simply insert or update them all, since the TOML definitions will have been checked both at compile time and by Nexus when inserting into CockroachDB. It's not clear how to express that in ClickHouse yet, given the rather confusing ways modifications can be made.
  • oximeter should probably drop its internal schema cache. This is an optimization, and has also led to some bugs. It should just read the schema from ClickHouse whenever it needs them.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions