Security group not found in VPC

[hundredacres]

I'm receiving an error when trying declare a instance in a VPC subnet

[root@admin1001 ~]# puppet agent -t
Warning: Setting modulepath is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/rvm/gems/ruby-1.9.3-p551/gems/puppet-3.8.1/lib/puppet/settings.rb:1141:in `issue_deprecation_warning')
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for admin1001.usw2
Info: Applying configuration version '1435777315'
Info: Checking if VPC USW1 exists in us-west-1
Info: Checking if subnet Admin subnet exists in us-west-1
Notice: Using ec2 data and code
Notice: /Stage[main]/Main/Notify[Environment]/message: defined 'message' as 'Using ec2 data and code'
Info: Checking if subnet DB subnet exists in us-west-1
Info: Checking if security group mysql exists in region us-west-1
Info: Creating security group mysql in region us-west-1
Notice: /Stage[main]/Riskiq::Dr::Mysql/Ec2_securitygroup[mysql]/ensure: created
Info: Checking if instance db1001.usw2 is running in region us-west-1
Info: Checking if instance db1001.usw2 is stopped in region us-west-1
Info: Starting instance db1001.usw2 in region us-west-1
Error: Security groups 'mysql' not found in VPCs 'vpc-5c7f643e'
Error: /Stage[main]/Riskiq::Dr::Mysql/Ec2_instance[db1001.usw2]/ensure: change from absent to present failed: Security groups 'mysql' not found in VPCs 'vpc-5c7f643e'
Notice: Finished catalog run in 3.75 seconds

Below is part of the manifest that sets up the security group and the instance.

ec2_securitygroup { 'mysql':
    ensure      => $ensure,
    description => 'Security group for mysql instances',
    region      => $region,
    vpc         => $vpc,
    tags        => {
      'Name' => 'mysql'
    },
    ingress     => [{
      protocol => 'tcp',
      port     => 3306,
      cidr     => '0.0.0.0/0',
    }],
}
ec2_instance { 'db1001.usw2':
    ensure            => $ensure,
    region            => $region,
    availability_zone => $avail_zone,
    image_id          => 'ami-57cfc412',
    security_groups   => ['mysql'],
    instance_type     => 't2.medium',
    key_name          => 'riskiq-primary',
    subnet            => 'DB Subnet',
    tags              => {
      'Name' => 'db1001.usw2',
    },
}

When I query the ec2_securitygroup resource, I get back results that imply the security group is assigned to the VPN

[root@admin1001 ~]# puppet resource ec2_securitygroup mysql
Warning: Setting modulepath is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/rvm/gems/ruby-1.9.3-p551/gems/puppet-3.8.1/lib/puppet/settings.rb:1141:in `issue_deprecation_warning')
ec2_securitygroup { 'mysql':
  ensure      => 'present',
  description => 'Security group for mysql instances',
  ingress     => [{'cidr' => '0.0.0.0/0', 'from_port' => '3306', 'protocol' => 'tcp', 'to_port' => '3306'}],
  region      => 'us-west-1',
  vpc         => 'USW1',
}

I'm missing something, but not sure what.

[garethr]

Is the subnet "DB Subnet" in the same VPC?

You could try and check with puppet resource ec2_vpc_subnet "DB Subnet" I think?

Also, what is the value of $avail_zone?

It could be the error message here is not specific enough.

[hundredacres]

$avail_zone gets set by a params class

class riskiq::dr::params {
  $ensure = 'present'
  $region = 'us-west-1'
  $avail_zone = 'us-west-1b'
  $vpc = 'USW1'
}

VPC

# puppet resource ec2_vpc
Warning: Setting modulepath is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/rvm/gems/ruby-1.9.3-p551/gems/puppet-3.8.1/lib/puppet/settings.rb:1141:in `issue_deprecation_warning')
ec2_vpc { 'USW1':
  ensure           => 'present',
  cidr_block       => '10.9.0.0/16',
  instance_tenancy => 'default',
  region           => 'us-west-1',
}

Subnet

# puppet resource ec2_vpc_subnet "DB subnet"
Warning: Setting modulepath is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/rvm/gems/ruby-1.9.3-p551/gems/puppet-3.8.1/lib/puppet/settings.rb:1141:in `issue_deprecation_warning')
ec2_vpc_subnet { 'DB subnet':
  ensure                  => 'present',
  availability_zone       => 'us-west-1b',
  cidr_block              => '10.9.101.0/24',
  map_public_ip_on_launch => 'false',
  region                  => 'us-west-1',
  route_table             => 'DR Route Table',
  vpc                     => 'USW1',
}

[rayyagari]

I'm having the same issue while trying to launch an instance in a vpc using puppet open source, even though the security group is very much present.
Error: Security groups 'chef-launch' not found in VPCs 'vpc-XXXXXXXX'
Error: /Stage[main]/Main/Ec2_instance[test01.dev]/ensure: change from absent to present failed: Security groups 'chef-launch' not found in VPCs 'vpc-XXXXXXXX'

[evidex]

I have the same issue if I omit the 'subnet' attribute on my ec2_instance resources with Puppet OSE 4.2.1. If the 'subnet' attribute is set correctly, the instance is created without fault. However, this attribute is documented as being optional. I believe the issue is caused by the following line in lib/puppet/provider/ec2_instance/v2.rb, in determine_subnet

! s.tags.any? { |t| t.key == 'Name' }

If the ! is omitted, and subnet attribute on the ec2_instance is not set, the expected correct behavior is observed, of the first subnet being used, along with a warning about the ambiguous subnet choice.

[aequitas]

I have encountered this error message and found a solution to my problem. It is probably not related to the problem you are having or the solution for you. I also have not debugged it to catch the problem in source, so I don't have a patch for my particular problem.
But for future reference if anyone encounters the same error message with the same cause as me:

The problem I was having is that the subnet had an empty 'Name'. This resulted in the same error message as described above. I was able to deduct this from @evidex 's message above.
Simply adding a name to the subnet solved my problem.

[chamindg]

@garethr ,
I encountered the same problem (i.e. Error: Security groups ..... not found in VPCc). My manifest was very similar to what @hundredacres mentioned in the top.

Found out this happens when the particular subnet you try to use in the ec2_instance doesn't have a value set for its Name tag. It's hard to pinpoint the problem because the error doesn't mention about the subnet.

Thanks

[doyleyp]

I encountered this issue as well. The error message is misleading.
The issue I encountered was do to an invalid subnet ( I was using the ec2 id, and not the Tag Name ) not returning the id from determine_subnet()

subnet = determine_subnet(vpc_groups.keys)
if subnet.nil?
raise Puppet::Error,
"Security groups '#{groups.join(', ')}' not found in VPCs '#{vpc_groups.keys.join(', ')}'"
end

Suggest correcting error message.

[evidex]

Looks as though @DavidS has fixed this issue in 47091d4.

[DavidS]

It'd be great if someone could validate the fix in the 1.4.0 release.

[mvernimmen-CG]

I'm getting this problem with v1.4.0 :(
luckily I found the workaround here.

[DavidS]

@mvernimmen-CG which workaround did work for you?

[mvernimmen-CG]

Adding the subnet in the resource declaration. Then it works fine. Leaving it out means getting the "Error: Security groups 'xxxxx' not found in VPCs 'vpc-########'" message.

[daveside]

I just did a fresh new standalone install in order to try the puppetlabs-aws modules. I'm running a clean version of CentOs 7.2 with puppet 3.8.7 and all the updates installed. I followed the installation instructions and installed aws-sdk-core, retries and the puppetlabs-aws module.

So when I tried a simple ec2 instance creation with my AWS credentials export, I got the following error message:

Notice: Compiled catalog for ip-172-31-49-17.ec2.internal in environment production in 0.03 seconds
WARNING: jmespath gem requires json gem >= 1.8.1; json 1.7.7 already loaded
Error: Security groups 'default' not found in VPCs 'vpc-0e038a6b'
Error: /Stage[main]/Main/Ec2_instance[my-instance-name]/ensure: change from absent to present failed: Security groups 'default' not found in VPCs 'vpc-0e038a6b'
Notice: Finished catalog run in 1.02 seconds

The content of the manifest file is:

ec2_instance { 'my-instance-name':
  ensure        => present,
  region        => 'us-east-1',
  image_id      => 'ami-d97897b4',
  instance_type => 't2.micro',
} 

When I try the stack test example, I'm able to create all the security groups but not instances. Here is the output error:

Notice: Compiled catalog for ip-172-31-49-17.ec2.internal in environment production in 0.08 seconds
WARNING: jmespath gem requires json gem >= 1.8.1; json 1.7.7 already loaded
Notice: /Stage[main]/Main/Ec2_securitygroup[lb-sg]/ensure: created
Notice: /Stage[main]/Main/Ec2_securitygroup[web-sg]/ensure: created
Error: Security groups 'web-sg' not found in VPCs 'vpc-0e038a6b'
Error: /Stage[main]/Main/Ec2_instance[web-2]/ensure: change from absent to present failed: Security groups 'web-sg' not found in VPCs 'vpc-0e038a6b'
Error: Security groups 'web-sg' not found in VPCs 'vpc-0e038a6b'
Error: /Stage[main]/Main/Ec2_instance[web-1]/ensure: change from absent to present failed: Security groups 'web-sg' not found in VPCs 'vpc-0e038a6b'
Notice: /Stage[main]/Main/Elb_loadbalancer[lb-1]: Dependency Ec2_instance[web-1] has failures: true
Notice: /Stage[main]/Main/Elb_loadbalancer[lb-1]: Dependency Ec2_instance[web-2] has failures: true
Warning: /Stage[main]/Main/Elb_loadbalancer[lb-1]: Skipping because of failed dependencies
Notice: /Stage[main]/Main/Ec2_securitygroup[db-sg]/ensure: created
Error: Security groups 'db-sg' not found in VPCs 'vpc-0e038a6b'
Error: /Stage[main]/Main/Ec2_instance[db-1]/ensure: change from absent to present failed: Security groups 'db-sg' not found in VPCs 'vpc-0e038a6b'
Notice: Finished catalog run in 5.60 seconds

I tried the subnet workaround without any success. Any help would be appreciate