Reconfigurator: Tracking issue for multi-node clickhouse deployment

[andrewjstone]

This is a tracking issue for all the work related to deploying multi-node clickhouse on a running system via reconfigurator. We expect that each system will run both single and multi-node for a while per RFD 468 and reconfigurator should allow us to transition between the two at runtime. We likely will need to change RSS to also deploy single and multi-node clickhouse to support this.

Reconfigurator:

• Update OmicronZoneConfig to support ClickhouseServer for replication #6298
• [Reconfigurator] Add policy for clickhouse replicas #6299
• Add CRDB blueprint tables related to clickhouse configuration
• Add planner code for deploying single node and multi-node clickhouse
  • Add logic and tests to generate a configuration - ClickhouseAllocator
  • Integrate ClickhouseAllocator into blueprint and db queries
• Add planner code for enabling multi-node clickhouse
• Add clickhouse configuration to RSS service plan (or punt to reconfigurator)
• Add executor code to deploy single and multi-node clickhouse
• Add executor code to enable multi-node clickhouse
• Add omdb support for reconfigurator deployment of clickhouse
• Add omdb support for inspection/debugging of clickhouse
  • keeper membership in inventory
  • blueprint support for cluster membership
  • General Debugging support

Sled-agent:

• Create new clickhouse_server zone and move all replicated clickhouse related code there [reconfigurator] clickhouse_server SMF service and oximeter replicated mode #6343
• Generate clickhouse XML configs with clickward instead of shell scripts [ClickHouse] Generate the configuration files instead of hardcoding them #3824
• Mechanism to deploy either replicated or single node clickhouse [reconfigurator] clickhouse_server SMF service and oximeter replicated mode #6343
• Deploy 3 server 5 keeper cluster instead of the current 2 server 3 keeper cluster
• Dropshot service to manage clickhouse server or keeper nodes [reconfigurator] clickhouse-admin SMF service with dropshot server #6304

Oximeter:

• Make it possible for oximeter to read/write to a clickhouse cluster [clickhouse] Make it possible for oximeter to read/write to a clickhouse cluster and/or single node #7338

[karencfv]

Related #6407