Feature Request: Persistent/Ephemeral Storage Selection for Stemcells

[kinjelom]

Feature Request: Persistent/Ephemeral Storage Selection for Stemcells

Detailed Description

Currently, virtual machine (VM) data and their disks are stored in datastores. In my setup, these datastores are replicated using storage array mechanisms. To optimize performance, only persistent data is replicated, while ephemeral data, including VM images, remains ephemeral.

In vSphere, VM files and stemcells are similar objects. However, in BOSH, these are treated as separate objects:

• Stemcells are considered persistent and easily restorable.
• VMs are ephemeral and should always be recreated using the stemcell.

To maintain this behavior after restoring an environment from persistent data, while restoring BOSH env (using BBL) we currently have to manually execute:

bosh upload-stemcell --fix

before deployments can proceed.

The requested feature would allow defining a persistent storage option for stemcells, ensuring that after an environment recovery, deployments can automatically use stemcells without requiring manual intervention.

Context:

• This feature is critical for setups where only persistent data is replicated, reducing unnecessary load and storage usage.
• Automating this process would eliminate the need for manual bosh upload-stemcell --fix, making BOSH more resilient to environment restoration scenarios.
• Other users managing large-scale vSphere environments with replication could benefit from this enhancement by ensuring that stemcells are always available without additional manual steps.

Possible Implementation

• Modify the BOSH vSphere CPI to allow configuration of persistent or ephemeral storage selection for stemcells.
• For exemple at this location in the code Cloud.rb#L219 allow the invocation of:

  target_datastore_pattern: StoragePicker.choose_global_stemcell_pattern(@datacenter),

• Implement StoragePicker.choose_global_stemcell_pattern(@datacenter) to determine whether a persistent or ephemeral storage pattern should be used based on the configuration.

If this feature is accepted, I am willing to implement it and submit a pull request. Let me know if this aligns with the project's vision.

Complexity

• Low - Simple enhancement or bug fix, no architectural changes or refactoring.
• Medium - Change requires some thought, but is relatively isolated.
• High - Significant architectural change or large refactor.

[jpalermo]

Hi @kinjelom

You're just suggesting different flags be used when the CPI creates the "VM" that's used as the stemcell?

I've certainly run into situations where I need to upload stemcell --fix before. But I'm not clear on the restoration workflow where this change would make it so you don't need to perform that.

Could you walk through exactly how the environment is being backed up and restored?

[kinjelom]

@jpalermo

You're just suggesting different flags be used when the CPI creates the "VM" that's used as the stemcell?

I'm not entirely sure, but the actual requirement is to store stemcells on persistent storage, while keeping the actual VM images on ephemeral storage.
In a BOSH-managed environment, VMs are always recreated using the stemcell, meaning the VM image itself is unnecessary and does not need to be backed up.

Below is the environment configuration for a Site 2 called recovery, which should be ready to restore the environment from a Site 1 called primary using backed-up persistent data.

Environment Setup

• S1 – Primary Data Center (DC1)
• S2 – Recovery Data Center (DC2)

In this configuration, we assume that only the necessary data for restoring services is backed up from S1 to S2:

• Entire jumpbox and BOSH Director VMs and stemcells, releases, and all deployment metadata required to recreate BOSH-managed deployments.
• Persistent data from BOSH-managed deployments.

The red connectors illustrate dependencies after disaster recovery restoration from S1 to S2.