Underlying Lambda OS changing May 21 2019

[gene1wood]

https://aws.amazon.com/blogs/compute/upcoming-updates-to-the-aws-lambda-execution-environment/

I expect all packages in this repo will need to be rebuild on the new Amazon Linux OS and possibly create multiple areas for packages that work with the old OS and those that work with the new OS

[celeduc]

I'm very worried, given the crickets echoing in this repo...

[naamancampbell]

Now receiving a bcrypt - missing libc.musl library error.

Workaround (until July 22): Manually add old Lambda layer (as per AWS article):
arn:aws:lambda:::awslayer:AmazonLinux1703

[amorphic]

FYI for anyone that gets really stuck, we've had good results using serverless with the serverless-python-requirements plugin. We've only used it for a fairly simple application so far though.

[youcandanch]

A lot of the folks actively working on Zappa aren't as active in this repo, to my knowledge. Let me try to give a bit of background based off what we've been talking about in the Slack org, and I'll ask @jneves / @mcrowson / @scoates / @mathom etc. to correct me where I'm wrong.

There's been a lot of discussion around divorcing the deployment aspects of Zappa from the handler aspects, albeit with a decent amount of talk and not too much in the way of PRs. (Not by anyone's ill intent or active orphaning; everyone who could be working on this actively has a slew of other things they're working on as well, most of which are, you know, jobs.) One of the things that would definitely be involved in that separation is a revisiting of the concept of lambda-packages.

One of the other things involved here is the introduction of layers, which could in theory obviate a lot of what's being accomplished here. I'd posit that we could avoid the heavy refactoring and architectural separation of deployer/handler and accommodate for the new execution environment changes given the following:

a) Add native layer support into Zappa. We should likely be doing this anyway.
b) Introduce a Zappa-specific layer that aims to be a replacement for this repository, as it's effectively accomplishing the same goals.
c) Release a new major version of Zappa that removes the dependency on lambda-packages.

This is work definitely worth doing, and given that - even now that the industry has caught up with things like SAM, Apex, etc. - I don't want Zappa to stall out, I'd love to take this on and try to get it done. I'd want a bit of validation from folks who've been actively maintaining before moving on, though. I'll dump a link to this comment in the Slack org to start some discussion and given some buy-in, I'll make time to work on it over the next week or so.

Edit: Turns out layers aren't an option due to size requirements, which is super annoying. Other options are presenting themselves, though, so I'll keep chewing on it.

[wagmiwiz]

FYI, and I know this isn't a solution, but if you can run on Python 3.7, then there are no lambda-packages for it, which means that you effectively get your option c) :D

[naamancampbell]

Now receiving a bcrypt - missing libc.musl library error.

Workaround (until July 22): Manually add old Lambda layer (as per AWS article):
arn:aws:lambda:::awslayer:AmazonLinux1703

The Lambda layer workaround to delay the update has stopped working today.

As a selling point of Zappa ("precompiled C-extensions"), lambda-packages isn't getting the attention required to avoid breaking Zappa projects.

[naamancampbell]

Upgrading to Python 3.7 did not fix the lambda-packages issue.
I've documented how I've replaced lambda-packages functionality in my project:
https://engineering.strobla.com/posts/replacing-lambda-packages/