Lambda as ALB target in Cloudformation

[ssoulier]

Hi,

Could you please add the cloudformation and SAM the ability to specify an events for lambda which is an Application Load Balancer ?

Thanks.

[rsram312]

Went through the documentations. Not sure if it is updated either. Can you please add the SAM template ability to specify ALB feature for Lambda

[lorddelicious]

Yes please SAM ALB is needed. Discontinuing use of API Gateway due to high cost, transitioning to ALB.

[ssoulier]

Any update on this feature request ?

[kauphylover]

Another request to the AWS team to prioritize this!

[dbettin]

Another request. Thanks!

[brettstack]

Absolutely. At a minimum we could add a new Event for ALB which create the permission for you and you could specify your existing ALB. However, it's probably more likely that you want SAM to create the ALB target group for you, or maybe even the entire ALB (similar to how we create the API).

Would love to get some more feedback from everyone on this, including some SAM syntax.

[rsram312]

@brettstack Ideally, what I was hoping is if SAM can create the entire ALB and register Lambda as target. Probably following might be capabilities required:

1. Creating an ALB
2. Creating a target group with target type as Lambda with options to enable health check
3. Adding listeners to the ALB
4. Adding listener rules to the ALB
5. Adding permissions to the Lambda to access the ALB
6. Register the targets with the target group

I am not exactly sure which of the above features are already available in SAM. So, please correct me if anything above is redundant.

Currently, I was using a workaround by creating a Lambda function which does all these operations and was invoking that lambda wherever required. However, the feature being available in SAM would be really handy in directly leveraging it within the SAM template.

[brettstack]

@rsram312 that's very useful. Do you have an existing minimal template that includes all of those required resources?

[rsram312]

@brettstack The sample template I was using as workaround leverages boto3 to create those resources. If it might be of any use, I could probably send it over.

[deleugpn]

For me the preference is in specifying an existing ALB and getting the target group and permissions created by SAM.

[kauphylover]

I like what @deleugpn suggests - seems like the most natural way to go about it.

[brettstack]

We will try to accommodate all scenarios.

1. Create everything for you (ALB, Group, Permissions)
2. Create just Group and Permission
3. Create just Permission

[deleugpn]

I'm not sure if it would be possible, but I would guess that the following would be quite an amazing syntax:

      Events: 
        MyEventName:
          Type: ALB
          Properties:
            LoadBalancerArn: !ImportValue LoadBalancer
            ListenerArn: !ImportValue Listener
            CertificateArn: !ImportValue Certificate
            Condition: [...]

I don't know if I would need to specify anything else other than this.

The expected result would be something similar to this:

  HttpsListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
      - Type: forward
        TargetGroupArn: !Ref TargetGroup
      Conditions: [Available on the Event Property]
      ListenerArn: [Available on the Event Property]
      Priority: [This is a tricky one. I think we have no other option than to delegate this to the user to define, unfortunately]

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      TargetType: lambda

[luketn]

  Priority: [This is a tricky one. I think we have no other option than to delegate this to the user to define, unfortunately]

The valid range of priorities is from 1 - 50,000.

One approach we could take to this is to default the value to a random number 40,000 to 50,000 catch the exception of an already used priority and choose again.

As long as the path / domain is unique, the priority should not matter.

[deleugpn]

SAM would be limited to deploy up to 10,000 lambdas on a single Listener, I guess that's fine.

[mneil]

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS. The examples in this rails app do not show how to use SourceAccount or SourceArn to restrict access. I have been unable to launch a lambda using either of those restrictions behind an ALB successfully. Could someone update the example, or provide an example, of how to use lambda behind an ALB that does not allow access to the world the entire account?

Edited: The lambda is access to the entire account, not the world

[raaone7]

Hey Michael, you can use lambda behind lambda, using few methods - alb subnets in private subnet - restrict alb with WAF + ipwhitelist to your VPC and NAT cidr - restrict alb with WAF + api gateway header check rule - restrict the alb sg to your vpc cidr only hope above helps. regards, ram

…

On Sat, Aug 10, 2019 at 8:24 AM Michael Neil ***@***.***> wrote: I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from *any* loadbalancer on AWS. The examples in this rails app do not show how to use SourceAccount or SourceArn to restrict access. I have been unable to launch a lambda using either of those restrictions behind an ALB successfully. Could someone update the example, or provide an example, of how to use lambda behind an ALB that does not allow access to the world? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#721>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACRZW6ZKNC574PM3GV7JFUDQDXVDPANCNFSM4GKMLMNQ> .

[praneetap]

Thanks for the feedback @mneil! Could you send us some links to the examples you are referring to?

[luketn]

BTW due to the long timeframe on this I’m using Serverless for ALB/lambda projects:
https://serverless.com/framework/docs/providers/aws/events/alb/

I’d suggest taking inspiration from their syntax - it’s pretty nice!

[deleugpn]

I also stopped recommending SAM as the go-to tool for serverless deployment. AWS is developing and marketing serverless A LOT but lagging behind on CloudFormation / SAM A LOT as well. 3rd party tools are doing a much better job at supporting AWS-provided features than AWS themselves.

[brysontyrrell]

Plus one from me on this.

I'm breaking up a stack currently and taking a set of my endpoints off an API Gateway and moving them behind an ALB. Based on another stack where I did an ALB->Lambda stack, I need to build the following resources into my template to make this work:

AWS::EC2::VPC (and all the stuff that goes with that like subnets, IGW, EIP, etc.)
AWS::EC2::SecurityGroup
AWS::Lambda::Permission
AWS::ElasticLoadBalancingV2::LoadBalancer
AWS::ElasticLoadBalancingV2::TargetGroup (per Lambda)
AWS::ElasticLoadBalancingV2::ListenerRule (per Lambda)

That's... a lot of stuff. Really, AWS needs to make ALBs a true serverless service and remove the requirements for an underlying VPC. This all becomes much simpler then. Until such time, SAM has an opportunity to abstract this boilerplate madness away in a simplified syntax.

[andrewdmay]

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS.

@mneil The only way I've found to restrict access to the Lambda to a specific target group is to name the target group and use a wildcard in the SourceArn on the Permission:

SourceArn: !Sub arn:aws:elasticloadbalancing:${AWS::Region}:${AWS::AccountId}:targetgroup/MyTargetGroupName/*

This allows the permission to be created before the Target Group (which otherwise fails because of the missing permission) and prevents a circular reference. Unfortunately it does mean naming the target group, which isn't idea, and you can't match on the full ARN.

You can modify the permission to add the SourceARN after the fact (and that could be done within a custom resource).

I took a look at what the Serverless Framework was doing for this because at first it appeared they had somehow worked around this problem - but actually what they are doing is first creating a permission without a SourceARN that the Target Group depends upon, and then they add a second permission that uses the SourceARN after the Target Group has been created. This doesn't increase security, but does make the ALB show up as the event source for the Lambda in the Console.

(which if you use the wildcard SourceARN, you get an error about it not matching a Target group, but it does appear to work, and I assume the restriction works, but I haven't verified that).

[andrewdmay]

For supporting ALBs as a source in SAM, I've been experimenting with a CloudFormation macro that accepts events like this:

      Events:
        AlbApi:
          Type: ALB
          Properties:
            Host: abc.def.com
            ListenerArn: !Ref ListenerArn
            Priority: 123

I'm following the basic pattern of how the different events work, and then using the Serverless Framework as inspiration for the property names. I'm assuming that the Listener already exists and is shared for multiple lambdas, so I'm just generating the permission, target group and listener rule.

I think automatically generating the ALB and listeners with an appropriate security group etc. just from the events would be difficult.

[aidansteele]

@andrewdmay I did a similar thing (and so far have made the same assumption re: pre-existing listener) and it’s been working very well for us.

https://github.com/glassechidna/sam-alb

[samirhsnv]

Any updates here? We need this feature because API Gateway has a hard timeout limit of 30 seconds

[deleugpn]

@samirhsnv I believe this ticket is about making it easier, however it's already possible to deploy your Lambda behind ALB using CFN. You can see an example here: https://blog.deleu.dev/running-lambda-behind-application-load-balancer/

[arleyyap]

Any updates here?

[samlamgithub]

Hi, is there any progress on this? Would to very useful to have this support, thanks

[SimonCMoore]

Adding ALB as an Eventsource to Lambda seems consistent.
Please +1 parent discussion to show your interest and drive our prioritization process.

[oharaandrew314]

I have no idea why this wasn't added to SAM as soon as the feature was released. If the feature isn't in SAM, then it may as well not exist for me. This is why it took my 5 years to discover the capability. While API Gateways are a cheap option to get started, there are some situations where I prefer an ALB:

1. I already have an ALB and want to use hostname routing across all services, for consistency
2. I want access to be whitelisted by IP. Private APIGWs are only accessible within the VPC, and local port forwarding doesn't work, because the TLS handshake fails

In the meantime, it's possible to use plain cloudformation to add the trigger to your function. Add and adapt this to your existing template. Enjoy.

 TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: <tg-name>
      HealthCheckEnabled: false # May want to enable after you add a health check to your lambda
      TargetType: lambda
      Targets:
        - Id: !GetAtt MyApiFunction.Arn

  ListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
        - Type: forward
          TargetGroupArn: !Ref TargetGroup
      Conditions:  # Replace with the condition of your choice
        - Field: host-header
          HostHeaderConfig:
            Values: <hostname>
      ListenerArn: <listener-arn>
      Priority: <routing-order>

  LoadBalancerPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref MyApiFunction
      Principal: elasticloadbalancing.amazonaws.com

[dzmitry-kankalovich]

I've got to be honest, these kind of problems made me eventually give up on CloudFormation and SAM, and use exclusively Terraform.

I wish AWS would address them timely, but this one was open since like 2018...