Whats the right way to error templates across regions

[kddejong]

cfn-lint version: (cfn-lint --version)
0.12.1

Description of issue.
When Linting a template against multiple regions its possible to miss errors that could be of value.

Take this template.

Resources:
  Cluster:
    Type: "AWS::EKS::Cluster"
    Properties:
      Name: String
      ResourcesVpcConfig: EKS Cluster ResourcesVpcConfig
      RoleArn: String
      Version: String

when linting it seems weird to have different errors depending on the region.

➜  cfn-lint test.yaml --region us-east-1
E3002 Expecting an object at Resources/Cluster/Properties/ResourcesVpcConfig
test.yaml:6:7

➜  cfn-lint test.yaml --region us-west-1 us-east-1
E3001 Invalid or unsupported Type AWS::EKS::Cluster for resource Cluster in us-west-1
test.yaml:3:5

I'm wondering if it should be.

➜  cfn-lint test.yaml --region us-west-1 us-east-1
E3001 Invalid or unsupported Type AWS::EKS::Cluster for resource Cluster in us-west-1
test.yaml:3:5

E3002 Expecting an object at Resources/Cluster/Properties/ResourcesVpcConfig
test.yaml:6:7

[fatbasstard]

Definitely agree on the suggested output.

The region specific processing is tweaked a bit in #586. Currently the actual region (or first in a multiregion) approach is used.

Downside is: we don't want to run ALL rules again, that's a waiste of time. So we need to look for an approach to make "some" region dependent rules run twice. 🤔

[PatMyron]

Another example of a missed error when linting across multiple regions:

$ cfn-lint --version
cfn-lint 0.23.5

$ cfn-lint test.yaml -r sa-east-1
E3030 You must specify a valid value for InstanceType (r5.large).
Valid values are ['c1.medium', 'c1.xlarge', 'c3.2xlarge', 'c3.4xlarge', 'c3.8xlarge', 'c3.large', 'c3.xlarge', 'c4.2xlarge', 'c4.4xlarge', 'c4.8xlarge', 'c4.large', 'c4.xlarge', 'c5.18xlarge', 'c5.2xlarge', 'c5.4xlarge', 'c5.9xlarge', 'c5.large', 'c5.xlarge', 'd2.2xlarge', 'd2.4xlarge', 'd2.8xlarge', 'd2.xlarge', 'g2.2xlarge', 'g2.8xlarge', 'i2.2xlarge', 'i2.4xlarge', 'i2.8xlarge', 'i2.xlarge', 'i3.16xlarge', 'i3.2xlarge', 'i3.4xlarge', 'i3.8xlarge', 'i3.large', 'i3.xlarge', 'm1.large', 'm1.medium', 'm1.small', 'm1.xlarge', 'm2.2xlarge', 'm2.4xlarge', 'm2.xlarge', 'm3.2xlarge', 'm3.large', 'm3.medium', 'm3.xlarge', 'm4.10xlarge', 'm4.16xlarge', 'm4.2xlarge', 'm4.4xlarge', 'm4.large', 'm4.xlarge', 'm5.12xlarge', 'm5.16xlarge', 'm5.24xlarge', 'm5.2xlarge', 'm5.4xlarge', 'm5.8xlarge', 'm5.large', 'm5.xlarge', 'r3.2xlarge', 'r3.4xlarge', 'r3.8xlarge', 'r3.large', 'r3.xlarge', 'r4.16xlarge', 'r4.2xlarge', 'r4.4xlarge', 'r4.8xlarge', 'r4.large', 'r4.xlarge', 't1.micro', 't2.2xlarge', 't2.large', 't2.medium', 't2.micro', 't2.nano', 't2.small', 't2.xlarge', 't3.2xlarge', 't3.large', 't3.medium', 't3.micro', 't3.nano', 't3.small', 't3.xlarge', 'x1.16xlarge', 'x1.32xlarge']
test.yaml:6:7

$ cfn-lint test.yaml -r ALL_REGIONS

$ cfn-lint test.yaml -r us-east-1 sa-east-1

Resources:
  MyEC2Instance: 
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: 'r5.large'
      ImageId: 'ami-a'

[PatMyron]

Where do we configure which rules run in multiple regions?

Time doesn't seem to be a concern yet:

$  time cfn-lint template
real	0m0.939s
$  time cfn-lint template -r ALL_REGIONS
real	0m0.979s