[DNM] Dashboard modification to detect fully Ceph-backed deployment

[H-M-Quang-Ngo]

Proposed changes to OpenStack Cloud Usage and OpenStack Compute Overview dashboards to address #154

This solution adds a template variable ceph_without_ephemeral for these dashboards to check these conditions:

• Ceph storage is detected
• All compute nodes report same local_gb storage (stddev < 1MB)
• That local_gb storage is very close to the total Ceph total storage (difference < 5GB)

If all checks are true, querying ceph_without_ephemeral will return true (1), which indicates Ceph is fully used as the storage backend on this deployment and there are likely no ephemeral/instance storage available. In that case, we need to use another expressions for disk calculation, otherwise (return 0) we fallback to the existing expressions.

Proposed panels to modify expression are: Free Disk by Aggregate (in OpenStack Cloud Usage), Hypervisor Disk Subscription and its Total (from OpenStack Compute Overview)

Some sample calculations

Case 1 : Current bug

Input:  20 nodes × 257TB each = 4.91 PiB (WRONG)
Detection: TRUE (Ceph exists + identical values + values match)
Output: 257TB (Correct)
Result: Correctly report storage

Case 2: Identical local storage

Input:  20 nodes × 1TB each = 20TB  
Detection: FALSE (no Ceph metrics)
Output: sum(20TB) (UNCHANGED)
Result: No false positive

Case 3: Mixed Ceph + Local

Input:  Ceph exists [1TB], nodes report [520GB, 500GB, 480GB]
Detection: FALSE (nodes' storage values not identical - stddev > 1MB)  
Output: sum(1500GB) (UNCHANGED)
Result: Correctly preserves mixed storage

Testing

Default configuration Environment (no Ceph)

• Jammy Yoga Openstack with local ephemeral storage, with 3 compute nodes from stsstack-bundles
• openstack-exporter-operator with grafana-agent subordinate.
• COS-lite on MicroK8s.
• 3 VM instances each with 20GiB disk created.

Brief check

$ openstack hypervisor list
+----+---------------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname       | Hypervisor Type | Host IP      | State |
+----+---------------------------+-----------------+--------------+-------+
|  1 | juju-1d5215-yoga-local-9  | QEMU            | 10.149.54.33 | up    |
|  2 | juju-1d5215-yoga-local-8  | QEMU            | 10.149.54.56 | up    |
|  3 | juju-1d5215-yoga-local-10 | QEMU            | 10.149.54.10 | up    |
+----+---------------------------+-----------------+--------------+-------+
openstack hypervisor show -c local_gb juju-1d5215-yoga-local-9
+----------+-------+
| Field    | Value |
+----------+-------+
| local_gb | 49    |
+----------+-------+
openstack hypervisor show -c local_gb juju-1d5215-yoga-local-8
+----------+-------+
| Field    | Value |
+----------+-------+
| local_gb | 49    |
+----------+-------+
openstack hypervisor show -c local_gb juju-1d5215-yoga-local-10
+----------+-------+
| Field    | Value |
+----------+-------+
| local_gb | 49    |
+----------+-------+

From Ceph storage panel (OpenStack Cloud Usage):

Result:

No Ceph shared storage back end is detected as:

OpenStack Cloud Usage

Unchanged:

OpenStack Compute Overview

Unchanged:

Ceph-shared storage Environment (the issue's case)

• Jammy Yoga Openstack with Ceph as the shared storage backend, with 3 compute nodes from stsstack-bundles

• openstack-exporter-operator with grafana-agent subordinate.

• COS-lite on MicroK8s.

• 3 VM instances each with 1GiB disk created.

• Add additional relations to scrape for Ceph metrics:
  - From OpenStack model:
  juju offer ceph-mon:metrics-endpoint

  From COS model:
  juju consume yoga-ceph.ceph-mon
juju relate ceph-mon:metrics-endpoint prometheus-k8s:metrics-endpoint

  (May need to workaround this bug)

Brief check

openstack volume service list -c Binary -c State -c Status -c Host  
+------------------+-----------------------------+---------+-------+
| Binary           | Host                        | Status  | State |
+------------------+-----------------------------+---------+-------+
| cinder-scheduler | juju-b7dbae-yoga-ceph-4     | enabled | down  |
| cinder-volume    | juju-b7dbae-yoga-ceph-4@LVM | enabled | down  |
| cinder-volume    | cinder@cinder-ceph          | enabled | up    |
| cinder-scheduler | cinder                      | enabled | up    |
+------------------+-----------------------------+---------+-------+
$ openstack hypervisor list
+----+--------------------------+-----------------+---------------+-------+
| ID | Hypervisor Hostname      | Hypervisor Type | Host IP       | State |
+----+--------------------------+-----------------+---------------+-------+
|  1 | juju-b7dbae-yoga-ceph-12 | QEMU            | 10.149.54.52  | up    |
|  2 | juju-b7dbae-yoga-ceph-21 | QEMU            | 10.149.54.17  | up    |
|  3 | juju-b7dbae-yoga-ceph-20 | QEMU            | 10.149.54.123 | up    |
+----+--------------------------+-----------------+---------------+-------+
$ juju run ceph-mon/0 pool-statistics
CLASS     SIZE    AVAIL    USED    RAW USED    %RAW USED
hdd       30 GiB  25 GiB   5.1 GiB  5.1 GiB    17.07%
TOTAL     30 GiB  25 GiB   5.1 GiB  5.1 GiB    17.07%
$ openstack hypervisor show -c local_gb juju-b7dbae-yoga-ceph-12 
+----------+-------+
| Field    | Value |
+----------+-------+
| local_gb | 29    |
+----------+-------+
$ openstack hypervisor show -c local_gb juju-b7dbae-yoga-ceph-21
+----------+-------+
| Field    | Value |
+----------+-------+
| local_gb | 29    |
+----------+-------+
$ openstack hypervisor show -c local_gb juju-b7dbae-yoga-ceph-20
+----------+-------+
| Field    | Value |
+----------+-------+
| local_gb | 29    |
+----------+-------+

From Ceph storage panel (OpenStack Cloud Usage):

Result:

Ceph shared storage back end is correctly detected for each dashboard, as:

OpenStack Cloud Usage

Pre:

Free disk usage is incorrectly reported: x3 (total 29 x 3 - 3 allocated = 84 GiB)

After:

Free Disk is correctly reported as 24.8 (29.9 - 5.10 used = 24.8 GiB)

OpenStack Compute Overview

Pre:

Total Disk is incorrectly reported: x3 from actual Ceph cluster storage (29x3 = 87 GiB)
Subscription percentage is incorrect accordingly: 3/87 = 3.45%

After:

Total is correct from actual Ceph cluster storage (29.9 GiB)
Subscription is 5.10/29.9 = 17% (same from the Ceph storage dashboard)

Note:

Actual changes should be made from sunbeam

[jneo8]

Hi @H-M-Quang-Ngo thanks for the PR.

It will be more nicer if you can:

• provide the screen shot of the dashboard
• (optional) provide some example calculation to the PR description to show this can work on both cases when bug exists/no-exists.

[jneo8]

Hi @H-M-Quang-Ngo,

I'm okay with the changes now—thanks for providing the calculation examples.

One thing I’d like to request is an update to the commit message. The current message makes it a bit difficult to trace the context of the change. Could you please include more details to make it clearer? Thanks!