Skip to content

Defend against bad volume sizes in marketplace AMIs #2422

Open
@gregharvey

Description

@gregharvey

If you try to build an ASG using an AMI that has a larger volume snapshot than the volume size in your Launch Configuration then it will perpetually fail to start instances with this message:

Launching a new EC2 instance. Status Reason: Volume of size 20GB is smaller than snapshot 'snap-0b38d043d1f7a0293', expect size >= 40GB.
Launching EC2 instance failed.

In this case 20GB is the LC volume size and 40GB is the AMI volume size, so the LC volume size needs to be at least 40GB to use this image. Effectively we must avoid trying to build ASGs using AMIs that have volumes that are larger than the volumes attached to the instances in the ASG.

We can defend against this. Already the ASG code makes sure, when in 'repack' mode, that the AMI volume size matches the ASG volume size. When using a marketplace AMI it's trickier though, we can't control things there. All we can do is create an Ansible assertion comparing the LC volume size - root_volume_size in the ASG vars - to the Block Devices string for the AMI's root device. The string looks like this for our usual Debian image:

/dev/xvda=snap-049a22b85789a8759:8:true:gp2

Which maps to:

name=snapshotId:volumeSize:encrypted:type

So we can see the volume size is 8GB. Therefore if our LC has a volume size of less than 8GB we should exit out of the process with a warning. We should be able to extract the block device string when we look up the AMI in the aws_ami role, regex over it to grab the volume size (it's always right after the snapshot ID) and compare it to our LC setting.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions