v2.4.0: Release 2.4.0

New Features

• Added udev rules for Pure Storage devices to optimize iSCSI LUN performance.
  The rules:
  - Set the I/O scheduler to none for improved throughput.
  - Reduce CPU usage by disabling entropy collection.
  - Balance CPU load by directing I/O completions to the originating CPU.
  - Increase the HBA timeout to 60 seconds for reliable I/O operations.

• Atmosphere previously deactivated the Keystone auth token cache due to bug
  https://tracker.ceph.com/issues/64094. This issue is now resolved upstream,
  making it safe to reactivate the cache in the new version of Ceph which
  includes the fix (18.2.7).

• Upgrade Percona XtraDB Cluster operator from 1.14.0 to 1.16.1 and Percona XtraDB Cluster from 8.0.36-28.1 to 8.0.41-32.1.
  This update includes performance improvements and bug fixes.

Upgrade Notes

• Bump Cert-Manager from v1.12.10 to v1.12.17 to address
  a breaking change in Cloudflare's API which impacted
  ACME DNS-01 challenges using Cloudflare.

• The max_allowed_packet setting increased from 4M (the default in
  MySQL 5.x) to 16M to support larger queries. Because MySQL 8.x uses
  a new default of 64M, the configuration no longer specifies this setting.

Bug Fixes

• The [cinder]/auth_type configuration value wasn't set resulting in
  the entire Cinder section not render in the configuration file, it
  is now set to password which will fully render the Cinder section
  for OpenStack Nova.

• Added a custom build of Cluster API driver for OpenStack which includes fixes unblocking upgrades of Magnum clusters created using a specific network or subnet configuration.

• Manila now uses Nova micro-version 2.60 by default. This change
  enables support for attaching multiple volumes to an instance.

• Manila now connects to the internal Nova and Glance endpoints
  instead of the public ones. This improves performance and reduces
  reliance on external network paths.

• Fixed the OAuth2 Proxy configuration to enable API access using valid JWT tokens
  without requiring interactive login. Previously, OAuth2 Proxy enforced login
  for all requests by default. This change lets the Alertmanager API and other
  services behind OAuth2 Proxy support programmatic access via JWT tokens.

• Increased the liveness probe timeouts for the Percona XtraDB Cluster.
  The configuration now sets timeoutSeconds to 60 and failureThreshold to 100.
  This change helps the cluster remain responsive and prevents unnecessary restarts during prolonged operations.

• Changed the liveness check from the MySQL exporter sidecar to a readiness check.
  The sidecar should wait indefinitely for the main containers and shouldn't terminate database pods.
  Especially during long SST operations. This change improves the cluster's stability during extended operations.

• Resolve the issue where the QEMU VNC and API TLS certificate fails to
  renew, preventing access to the virtual machine (VM) console via the
  dashboard and causing live migration failures.

Other Notes

• Add documentation about database backup and restore procedures.

v5.0.0-rc.3: Release 5.0.0-rc.3

Bug Fixes

• This change fixes a regression where Cinder volume creation fails with error
  FailedToDropPrivileges. Since update to Cinder 24.0.0, Cinder-Ceph
  container needs access to more capabilities for operations such as boot from
  volume or create a volume from an image.

• In an OVN deployment where external (baremetal) ports connect to VLAN
  networks, you need to bind the internal router port associated with
  the network to the same ha_chassis_group as the network. This setup
  mimics how the external port of the router functions in relation to
  the upstream gateway.

  In essence, the baremetal ports aren't able to communicate with their
  default gateway if either the internal router port is unbound or if
  the vrouter doesn't have an external gateway set, with the external
  router port bound to the same exact chassis and with the same exact
  priorities as the ha_chassis_group of the VLAN network.

v4.6.0: Release 4.6.0

New Features

• Add Neutron plugins for neutron-dynamic-routing and networking-generic-switch.
  These modules enable support for Neutron BGP agents and Ironic networking.

Bug Fixes

• The [cinder]/auth_type configuration value wasn't set resulting in
  the entire Cinder section not render in the configuration file, it
  is now set to password which will fully render the Cinder section
  for OpenStack Nova.

• This change fixes a regression where Cinder volume creation fails with error
  FailedToDropPrivileges. Since update to Cinder 24.0.0, Cinder-Ceph
  container needs access to more capabilities for operations such as boot from
  volume or create a volume from an image.

v3.4.1: Release 3.4.1

New Features

• Atmosphere previously deactivated the Keystone auth token cache due to bug
  https://tracker.ceph.com/issues/64094. This issue is now resolved upstream,
  making it safe to reactivate the cache in the new version of Ceph which
  includes the fix (18.2.7).

• Upgrade Percona XtraDB Cluster operator from 1.14.0 to 1.16.1 and Percona XtraDB Cluster from 8.0.36-28.1 to 8.0.41-32.1.
  This update includes performance improvements and bug fixes.

Upgrade Notes

• The max_allowed_packet setting increased from 4M (the default in
  MySQL 5.x) to 16M to support larger queries. Because MySQL 8.x uses
  a new default of 64M, the configuration no longer specifies this setting.

Bug Fixes

• The [cinder]/auth_type configuration value wasn't set resulting in
  the entire Cinder section not render in the configuration file, it
  is now set to password which will fully render the Cinder section
  for OpenStack Nova.

• Added a custom build of Cluster API driver for OpenStack which includes fixes unblocking upgrades of Magnum clusters created using a specific network or subnet configuration.

• Manila now uses Nova micro-version 2.60 by default. This change
  enables support for attaching multiple volumes to an instance.

• Manila now connects to the internal Nova and Glance endpoints
  instead of the public ones. This improves performance and reduces
  reliance on external network paths.

• Fixed the OAuth2 Proxy configuration to enable API access using valid JWT tokens
  without requiring interactive login. Previously, OAuth2 Proxy enforced login
  for all requests by default. This change lets the Alertmanager API and other
  services behind OAuth2 Proxy support programmatic access via JWT tokens.

• Increased the liveness probe timeouts for the Percona XtraDB Cluster.
  The configuration now sets timeoutSeconds to 60 and failureThreshold to 100.
  This change helps the cluster remain responsive and prevents unnecessary restarts during prolonged operations.

• Changed the liveness check from the MySQL exporter sidecar to a readiness check.
  The sidecar should wait indefinitely for the main containers and shouldn't terminate database pods.
  Especially during long SST operations. This change improves the cluster's stability during extended operations.

• Resolve the issue where the QEMU VNC and API TLS certificate fails to
  renew, preventing access to the virtual machine (VM) console via the
  dashboard and causing live migration failures.

Other Notes

• Add documentation about database backup and restore procedures.

v5.0.0-rc.2: Release 5.0.0-rc.2

New Features

• Add Neutron plugins for neutron-dynamic-routing and networking-generic-switch.
  These modules enable support for Neutron BGP agents and Ironic networking.

v5.0.0-rc.1: Release 5.0.0-rc.1

New Features

• Valkey service is now available on Atmosphere.
  This is required service for introduce Octavia Amphora V2 support.

• Add specific helm-toolkit patch on 0.2.78. This will allow DB drop and init job
  compatible with SQLAlchemy 2.0

• Octavia Amphere V2 is now supported and enable by default with Atmosphere.
  The Amphora V2 provider driver improves control plane resiliency.
  Should a control plane host go down during a load balancer provisioning
  operation, an alternate controller can resume the in-process provisioning
  and complete the request. This solves the issue with resources stuck in
  PENDING_* states by writing info about task states in persistent backend
  and monitoring job claims via jobboard.

• The Keystone role now supports additional parameters when creating the Keycloak realm to allow for the configuration of options such as password policy, brute force protection, and more.

• Added support for deploying the frr-k8s chart for BGP routing with
  OVN. Introduced the ovn_bgp_agent_enabled flag. When set to
  true, the frr-k8s chart will be automatically installed before
  OVN deployment.

• Add glance_image_tempfile_path variable to allow users for changing the temporary path for downloading images before uploading them to Glance.

• Keycloak is now configured to have the token-exchange and the admin-fine-grained-authz features enabled to allow for use of the OAuth Token Exchange protocol.

• The Keystone role now supports configuring multi-factor authentication for the users within the Atmosphere realm.

• Add support for Neutron policy check when perform port update with
  add address pairs. This will add a POST method /address-pair.
  It will check if both ports (to be paired) are created within same project.
  With this check, we can give non-admin user to operate address pair binding
  without risk on expose resource to other projects.

• The ovn-bgp-agent has been added to the chart. The ovn-bgp-agent
  is deployed as a DaemonSet within the OVN Helm chart.

• Add OVN BGP Agent image build.

• Introduced a new Rust-based binary ovsinit which focuses on handling the migration of IP addresses from a physical interface to an OVS bridge during the Neutron or OVN initialization process.

• It is now possible to configure DPDK interfaces using the interface names in addition to
  possibly being able to use the pci_id to ease deploying in heterogeneous environments.

• All roles that deploy Ingress resources as part of the deployment
  process now support the ability to specify the class name to use for the
  Ingress resource. This is done by setting the
  <role>_ingress_class_name variable to the desired class name.

• Introduced the ability to specify a prefix for image names. This allows for
  easier integration with image proxies and caching mechanisms, eliminating
  the need to maintain separate inventory overrides for each image.

• It's now possible to use the default TLS certificates configured within the
  ingress by using the ingress_use_default_tls_certificate variable which
  will omit the tls section from any Ingress resources managed by
  Atmosphere.

• Barbican now supports multiple keks in configuration. The configuration value
  .conf.simple_crypto_plugin_rewrap.old_kek now accepts comma-separated strings for
  kek lists, and multiple .conf.barbican.simple_crypto_plugin.kek values is now
  possible. The first key in the comma-separated .conf.simple_crypto_plugin_rewrap.old_kek
  string is for encrypting new data, while additional keys are for decrypting
  existing data. This behavior is consistent with .conf.barbican.simple_crypto_plugin.kek.

• The Barbican role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Storpool driver has been updated from the Bobcat release to the Caracal release.

• The Cinder role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Designate role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• Atmosphere previously deactivated the Keystone auth token cache due to bug
  https://tracker.ceph.com/issues/64094. This issue is now resolved upstream,
  making it safe to reactivate the cache in the new version of Ceph which
  includes the fix (18.2.7).

• Applied the same pod affinity rules used for OVN NB/SB sts's to northd deployment and
  changed the default pod affinity rules from preferred during scheduling to required
  during scheduling.

• The ovn-northd service did not have liveness probes enabled which can result in the pod failing readiness checks but not being automatically restarted. The liveness probe is now enabled by default which will restart any stuck ovn-northd processes.

• The Glance role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Heat role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Horizon role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Ironic role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Keystone role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The OpenStack database exporter has been updated and the collection of Octavia metrics happens through it only.

• Added alerting for amphoras to cover cases for when an Amphora becomes in ERROR state or not ready for an unexpected duration.

• The Magnum role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Manila role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Neutron role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Nova role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The Octavia role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• Neutron now supports using the built-in DHCP agent when using OVN (Open Virtual Network)
  for cases when DHCP relay is necessary.

• The Placement role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• The ovn-controller image is now being pre-pulled on the nodes prior to the Helm chart being deployed. This will help reduce the time it takes to switch over to the new version of the ovn-controller image.

• The Staffeln role now allows users to configure the priorityClassName and the runtimeClassName for all of the different components of the service.

• Update the frr-k8s webhook server runs on the control plane.

• Upgrade Percona XtraDB Cluster operator from 1.14.0 to 1.16.1 and Percona XtraDB Cluster from 8.0.36-28.1 to 8.0.41-32.1.
  This update includes performance improvements and bug fixes.

Known Issues

• The MTU for the metadata interfaces for OVN was not being set correctly, leading to a mismatch between the MTU of the metadata interface and the MTU of the network. This has been fixed with a Neutron change to ensure the neutron:mtu value in external_ids is set correctly.

Upgrade Notes

• Bump OVN from 24.03.1-44 to 24.03.2.34.

• The max_allowed_packet setting increased from 4M (the default in
  MySQL 5.x) to 16M to support larger queries. Because MySQL 8.x uses
  a new default of 64M, the configuration no longer specifies this setting.

• Upgrade Cluster API driver for Magnum to 0.26.0.

Security Issues

• The Horizon service now runs as the non-privileged user horizon in the container.

• The Horizon service ALLOWED_HOSTS setting is now configured to point to the configured endpoints for the service.

• The CORS headers are now configured to only allow requests from the configured endpoints for the service.

• Upgrade nginx ingress controller from 1.10.1 to 1.12.1 to fix CVE-2025-1097
  CVE-2025-1098, CVE-2025-1974, CVE-2025-24513, CVE-2025-24514.

Bug Fixes

• The [privsep_osbrick]/helper_command configuration value was not configured in both of the Cinder and Nova services, which lead to the inability to run certain CLI commands since it instead tried to do a plain sudo instead. This has been fixed by adding the missing helper command configuration to both services.

• The dmidecode package which is required by the os-brick library for certain operations was not installed on the images that needed it, which can cause NVMe-oF discovery issues. The package has been added to all images that require it.

• The [cinder]/auth_type configuration value was not set resulting in the entire Cinder section not being rendered in the configuration file, it is now set to password which will fully render the Cinder section for OpenStack Nova.

• The nova user within the nova-ssh image was missing the SHELL build argument which would c...

v4.5.1: Release 4.5.1

New Features

• Atmosphere previously deactivated the Keystone auth token cache due to bug
  https://tracker.ceph.com/issues/64094. This issue is now resolved upstream,
  making it safe to reactivate the cache in the new version of Ceph which
  includes the fix (18.2.7).

• Upgrade Percona XtraDB Cluster operator from 1.14.0 to 1.16.1 and Percona XtraDB Cluster from 8.0.36-28.1 to 8.0.41-32.1.
  This update includes performance improvements and bug fixes.

Upgrade Notes

• The max_allowed_packet setting increased from 4M (the default in
  MySQL 5.x) to 16M to support larger queries. Because MySQL 8.x uses
  a new default of 64M, the configuration no longer specifies this setting.

Bug Fixes

• Added a custom build of Cluster API driver for OpenStack which includes fixes unblocking upgrades of Magnum clusters created using a specific network or subnet configuration.

• Manila now uses Nova micro-version 2.60 by default. This change
  enables support for attaching multiple volumes to an instance.

• Manila now connects to the internal Nova and Glance endpoints
  instead of the public ones. This improves performance and reduces
  reliance on external network paths.

• Fixed the OAuth2 Proxy configuration to enable API access using valid JWT tokens
  without requiring interactive login. Previously, OAuth2 Proxy enforced login
  for all requests by default. This change lets the Alertmanager API and other
  services behind OAuth2 Proxy support programmatic access via JWT tokens.

• Increased the liveness probe timeouts for the Percona XtraDB Cluster.
  The configuration now sets timeoutSeconds to 60 and failureThreshold to 100.
  This change helps the cluster remain responsive and prevents unnecessary restarts during prolonged operations.

• Changed the liveness check from the MySQL exporter sidecar to a readiness check.
  The sidecar should wait indefinitely for the main containers and shouldn't terminate database pods.
  Especially during long SST operations. This change improves the cluster's stability during extended operations.

• Resolve the issue where the QEMU VNC and API TLS certificate fails to
  renew, preventing access to the virtual machine (VM) console via the
  dashboard and causing live migration failures.

• Checking DB transaction already starts in barbican kek rewrap.
  And use nested transaction if DB session already starts it's
  root transaction.

Other Notes

• Add documentation about database backup and restore procedures.

v4.5.0

New Features

• Valkey service is now available on Atmosphere.
  This is required service for introduce Octavia Amphora V2 support.

• Octavia Amphora V2 is now supported and enable by default with Atmosphere.
  The Amphora V2 provider driver improves control plane resiliency.
  Should a control plane host go down during a load balancer provisioning
  operation, an alternate controller can resume the in-process provisioning
  and complete the request. This solves the issue with resources stuck in
  PENDING_* states by writing info about task states in persistent data
  structure and monitoring job claims via Jobboard.

• The OpenStack database exporter has been updated and the collection of Octavia metrics happens through it only.

• Added alerting for amphoras to cover cases for when an Amphora becomes in ERROR state or not ready for an unexpected duration.

• Update the frr-k8s webhook server runs on the control plane.

Bug Fixes

• Backport fixes for Octavia Redis driver for support authentication
  and SSL for Redis Sentinel.

• Addressed an issue where instances not booted from volume would fail to resize. This issue was caused by a missing trailing newline in the SSH key, which led to misinterpretation of the key material during the resize operation. Adding proper handling of SSH keys ensures that the resize process works as intended for all instances.

v3.4.0

New Features

• Valkey service is now available on Atmosphere.
  This is required service for introduce Octavia Amphora V2 support.

• Octavia Amphora V2 is now supported and enable by default with Atmosphere.
  The Amphora V2 provider driver improves control plane resiliency.
  Should a control plane host go down during a load balancer provisioning
  operation, an alternate controller can resume the in-process provisioning
  and complete the request. This solves the issue with resources stuck in
  PENDING_* states by writing info about task states in persistent data
  structure and monitoring job claims via Jobboard.

• The OpenStack database exporter has been updated and the collection of Octavia metrics happens through it only.

• Added alerting for amphoras to cover cases for when an Amphora becomes in ERROR state or not ready for an unexpected duration.

Security Issues

• Upgrade nginx ingress controller from 1.10.1 to 1.12.1 to fix CVE-2025-1097
  CVE-2025-1098, CVE-2025-1974, CVE-2025-24513, CVE-2025-24514.

Bug Fixes

• Backport fixes for Octavia Redis driver for support authentication
  and SSL for Redis Sentinel and multiple Sentinel servers.

• The Cluster API driver for Magnum has been bumped to 0.28.0 to improve stability, fix bugs and add new features.

• Addressed an issue where instances not booted from volume would fail to resize. This issue was caused by a missing trailing newline in the SSH key, which led to misinterpretation of the key material during the resize operation. Adding proper handling of SSH keys ensures that the resize process works as intended for all instances.

• Improve alert generation for load balancers that have a non-ACTIVE provisioning state
  despite an ONLINE operational state. Previously, if a load balancer was in a
  transitional state such as PENDING_UPDATE (provisioning_state) while still marked
  as ONLINE (operational_state), the gauge metric
  openstack_loadbalancer_loadbalancer_status{provisioning_status!="ACTIVE"} did not
  trigger an alert. This update addresses the issue by ensuring that alerts are properly
  generated in these scenarios.

v2.3.0

New Features

• The OpenStack database exporter has been updated and the collection of Octavia metrics happens through it only.

• Added alerting for amphoras to cover cases for when an Amphora becomes in ERROR state or not ready for an unexpected duration.

Security Issues

• Upgrade nginx ingress controller from 1.10.1 to 1.12.1 to fix CVE-2025-1097
  CVE-2025-1098, CVE-2025-1974, CVE-2025-24513, CVE-2025-24514.

Bug Fixes

• The Cluster API driver for Magnum has been bumped to 0.28.0 to improve stability, fix bugs and add new features.

• Addressed an issue where instances not booted from volume would fail to resize. This issue was caused by a missing trailing newline in the SSH key, which led to misinterpretation of the key material during the resize operation. Adding proper handling of SSH keys ensures that the resize process works as intended for all instances.

• Improve alert generation for load balancers that have a non-ACTIVE provisioning state
  despite an ONLINE operational state. Previously, if a load balancer was in a
  transitional state such as PENDING_UPDATE (provisioning_state) while still marked
  as ONLINE (operational_state), the gauge metric
  openstack_loadbalancer_loadbalancer_status{provisioning_status!="ACTIVE"} did not
  trigger an alert. This update addresses the issue by ensuring that alerts are properly
  generated in these scenarios.