Skip to content

openstack-cloud-controller charm crashes on transient k8s API unavailability #955

Description

@ktsakalozos-canonical

Summary

The openstack-cloud-controller charm's update-status hook crashes with an unhandled
httpx.ConnectError whenever the Kubernetes API is temporarily unreachable. The unit is then
permanently stuck in Juju error state and requires manual intervention (juju resolve) to recover.
This caused a CI pipeline failure in the canonicalk8s-openstack integration test run on 2026-06-26.

Affected component

Field | Value
Charm | openstack-cloud-controller
Channel | latest/stable
Revision | 48
Source file | src/charm.py
Method | _update_status → _check_node_provider_ids

Environment

Field | Value
k8s charm | 1.35/stable rev 1846 (k8s 1.35.0)
Cloud | OpenStack Caracal / Noble
Juju | 3.6.24
Base | ubuntu@24.04
Model | canonicalk8s-openstack on foundation-openstack

Observed behaviour

The update-status hook crashes with an uncaught exception:

unit-openstack-cloud-controller-0: 01:55:48 ERROR unit.openstack-cloud-controller/0.juju-log Uncaught exception while in charm code:
Traceback (most recent call last):

  File ".../src/charm.py", line 147, in _update_status
    nodes_without_provider_id = self._check_node_provider_ids()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../src/charm.py", line 126, in _check_node_provider_ids
    for node in client.list(Node):
  File ".../lightkube/core/generic_client.py", line 109, in __iter__
    for rv, chunk in self._inner_iter:
....
httpx.ConnectError: [Errno 113] No route to host

unit-openstack-cloud-controller-0: 01:55:48 ERROR juju.worker.uniter.operation hook "update-status" (via hook dispatching script: dispatch) failed: exit status 1

The unit is then permanently locked in Juju error state:

openstack-cloud-controller/0*  error  idle  hook failed: "update-status"

It does not self-recover. It must be manually resolved with juju resolve openstack-cloud-controller/0.

Expected behaviour

The _update_status / _check_node_provider_ids method should catch connection errors from the
Kubernetes API client (at minimum httpx.ConnectError, httpx.ConnectTimeout, and
httpcore.ConnectError) and set a transient waiting status instead of allowing the exception to
propagate and crash the hook. The k8s API being temporarily unreachable is a normal, expected
condition during cluster maintenance operations (leader replacement, snap upgrades, etc.).

Root cause

In src/charm.py, the _check_node_provider_ids method calls client.list(Node) via the
lightkube Kubernetes client with no error handling around network connectivity:

# charm.py ~line 126
def _check_node_provider_ids(self):
    for node in client.list(Node):   # <-- no try/except around this
        ...

When the k8s API endpoint is unreachable, lightkube → httpx → httpcore raises
httpx.ConnectError: [Errno 113] No route to host. This exception propagates all the way up to
ops.main() at charm.py:283, causing the hook to exit with status 1 and Juju to mark the unit
as errored.

Suggested fix

Wrap the Kubernetes API calls in _update_status (and any other hook handler that calls the k8s
API) to catch connectivity errors and set a waiting status:

from lightkube.core.exceptions import ApiError
import httpx

def _check_node_provider_ids(self):
    try:
        for node in client.list(Node):
            ...
    except (httpx.ConnectError, httpx.ConnectTimeout, httpx.TimeoutException) as e:
        raise ops.model.WaitingStatus(f"Kubernetes API temporarily unreachable: {e}")

Or, at the _update_status handler level:

def _update_status(self, event):
    try:
        nodes_without_provider_id = self._check_node_provider_ids()
        ...
    except (httpx.ConnectError, httpx.ConnectTimeout) as e:
        self.unit.status = ops.model.WaitingStatus(f"k8s API unreachable: {e}")
        return

Impact

Unit permanently stuck in error state after any k8s API disruption
CI pipeline fails: juju.errors.JujuUnitError: Unit in error: openstack-cloud-controller/0
Downstream tests (e.g. test_extra_sans) that call wait_for_idle(status="active") fail even when the test itself is otherwise correct
Manual juju resolve required to unblock the pipeline
Additional context: secondary test design gap
The test_k8s test suite (test_remove_leader_control_plane) performs a leader control-plane
replacement but only waits for k8s/* and k8s-worker/* units to become healthy. It does not
check openstack-cloud-controller or cinder-csi health after the operation. As a result:

The update-status hook failure occurs during test_remove_leader_control_plane (~01:47–01:53)
test_k8s reports integration-k8s: OK without detecting the broken cloud-controller
The errored unit is silently handed off to the test_openstack suite
test_extra_sans (the fourth openstack test) is the first test to call wait_for_idle(status="active") on the full model and fails

Recommendation: test_k8s (and any test that modifies k8s topology on a model with OpenStack
integrations) should include a post-operation assertion that all units — including subordinates —
return to a healthy state.

To Reproduce

Steps to reproduce

  • Deploy a canonicalk8s-openstack model with openstack-cloud-controller related to k8s
  • Trigger any operation that temporarily takes the Kubernetes API offline:
  • Remove the leader control-plane unit and add a replacement (juju remove-unit k8s/leader + juju add-unit k8s)
  • Apply a config change that triggers a snap reinstall (juju config k8s extra-sans=...)
  • Wait for Juju's periodic update-status hook to fire on openstack-cloud-controller/0 (every 5 minutes by default)

Environment

canonical k8s on charmed openstack caracal with noble

Relevant log output

Model                   Controller            Cloud/Region               Version  SLA          Timestamp
canonicalk8s-openstack  foundation-openstack  openstack_cloud/RegionOne  3.6.24   unsupported  01:54:38Z

App                         Version  Status  Scale  Charm                       Channel         Rev  Exposed  Message
cinder-csi                  v1.27.1  active      1  cinder-csi                  latest/stable    47  no       Versions: cinder-csi-driver=v1.27.1
k8s                         1.35.0   active      3  k8s                         1.35/stable    1846  yes      Ready
k8s-worker                  1.35.0   active      3  k8s-worker                  1.35/stable    1843  no       Ready
openstack-cloud-controller  v1.34.1  active      1  openstack-cloud-controller  latest/stable    48  no       Versions: openstack-cloud-controller-manager=v1.34.1
openstack-integrator        yoga     active      1  openstack-integrator        latest/stable   123  no       Ready

Unit                             Workload  Agent  Machine  Public address  Ports     Message
k8s-worker/1                     active    idle   1        10.243.37.107             Ready
k8s-worker/2*                    active    idle   2        10.243.37.111             Ready
k8s-worker/3                     active    idle   7        10.243.37.201             Ready
k8s/2*                           active    idle   5        10.243.37.191   6443/tcp  Ready
k8s/3                            active    idle   8        10.243.37.214   6443/tcp  Ready
k8s/4                            active    idle   9        10.243.37.227   6443/tcp  Ready
openstack-integrator/0*          active    idle   6        10.243.37.128             Ready
  cinder-csi/0*                  blocked   idle            10.243.37.128             Missing required certificates
  openstack-cloud-controller/0*  error     idle            10.243.37.128             hook failed: "update-status"

Machine  State    Address        Inst id                               Base          AZ    Message
1        started  10.243.37.107  f2c16312-2ceb-4555-a08c-2d7ee165214b  ubuntu@24.04  nova  ACTIVE
2        started  10.243.37.111  f84cb757-fd6a-4b6d-8056-d9e5083317fa  ubuntu@24.04  nova  ACTIVE
5        started  10.243.37.191  f8b70667-e942-46de-a1b3-682b8877581a  ubuntu@24.04  nova  ACTIVE
6        started  10.243.37.128  eedb69fc-aed5-4056-bce3-091c82fefb05  ubuntu@24.04  nova  ACTIVE
7        started  10.243.37.201  d2a70b3d-9808-4164-bbcc-bb86b4ff07dd  ubuntu@24.04  nova  ACTIVE
8        started  10.243.37.214  1790859c-331e-4cbb-941b-9d0643666ce0  ubuntu@24.04  nova  ACTIVE
9        started  10.243.37.227  84011a2d-4a18-415b-9a90-efeac2f1fd4c  ubuntu@24.04  nova  ACTIVE

Additional context

it fails to negotiate the version from keystone:

unit-openstack-integrator-0: 01:16:13 ERROR Failed to determine API version: unknown url type: ''
unit-openstack-integrator-0: 01:22:50 ERROR Failed to determine API version: unknown url type: ''
The charm attempted to negotiate the OpenStack API version from Keystone twice during deployment,
and failed both times with unknown url type: '' 

The Keystone auth URL was empty because the
relation data was not yet fully populated at those points. The charm then silently fell back to a
hardcoded default of "yoga" and transitioned to active: Ready without ever successfully
detecting the actual cloud version. No further version-detection attempts were made.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions