Document install pinned python versions in container for terraform-aws-modules/lambda/aws module

[nitrocode]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

• I'd be willing to implement this feature (contributing guide)

Describe the user story

The terraform-aws-modules/lambda/aws module requires a python interpretter that matches the runtime so python3.8 interpretter for runtime python3.8, etc.

Describe the solution you'd like

Document installation of pyenv or similar to pin down runtime versions for python lambdas to use this common upstream module

Describe the drawbacks of your solution

N/A

Describe alternatives you've considered

N/A

---

I believe this should work. Some of the commands can be removed since the sh -l or bash -l is not supported by gosu and so it will not run the .profile file. Thus, we need to softlink into /usr/local/bin for the binaries to be available to all users (root and atlantis).

RUN apk add --no-cache \
        build-base \
        libffi-dev \
        openssl-dev \
        bzip2-dev \
        zlib-dev \
        readline-dev \
        sqlite-dev

RUN cd $HOME \
    && git clone https://github.com/pyenv/pyenv.git $HOME/.pyenv \
    && cd $HOME/.pyenv \
    && git branch pyenv-2.3.27 v2.3.27 \
    && git checkout pyenv-2.3.27 \
    && cd $HOME \
    && git clone https://github.com/pyenv/pyenv-virtualenv.git $HOME/.pyenv/plugins/pyenv-virtualenv \
    && cd $HOME/.pyenv/plugins/pyenv-virtualenv \
    && git branch virtualenv-1.2.1 v1.2.1 \
    && git checkout virtualenv-1.2.1 \
    && cd $HOME \
    && echo 'export PYENV_ROOT="$HOME/.pyenv"' >> $HOME/.profile \
    && echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> $HOME/.profile \
    && echo 'eval "$(pyenv init -)"' >> $HOME/.profile \
    && echo 'eval "$(pyenv virtualenv-init -)"' >> $HOME/.profile \
    && source $HOME/.profile \
    && pyenv install \
        3.8.18 \
        3.9.18 \
        3.10.13 \
        3.11.5 \
    && ln -sf $HOME/.pyenv/versions/3.8.18/bin/python3.8 /usr/local/bin/ \
    && ln -sf $HOME/.pyenv/versions/3.9.18/bin/python3.9 /usr/local/bin/ \
    && ln -sf $HOME/.pyenv/versions/3.10.13/bin/python3.10 /usr/local/bin/ \
    && ln -sf $HOME/.pyenv/versions/3.11.5/bin/python3.11 /usr/local/bin/

[jarviliam]

Ran into this problem over here 🙏 .
Just wanted to add that it would be convenient if atlantis published a docker image with python bundled alongside the base docker image. That way users that don't need the python binary can continue using the base image while others that need python can use the above (atlantis-python).

[jamengual]

you can now install whatever you want using the entrypoint.d dir: https://github.com/runatlantis/atlantis/blob/main/docker-entrypoint.sh#L35

…

On Tue, Mar 5, 2024 at 2:12 AM Liam Jarvis ***@***.***> wrote: Ran into this problem over here 🙏 . Just wanted to add that it would be convenient if atlantis published a docker image with python bundled alongside the base docker image. That way users that don't need the python binary can continue using the base image while others that need python can use the above (atlantis-python). — Reply to this email directly, view it on GitHub <#3778 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ3ERAYYDB5V6SHS62F5FDYWWK77AVCNFSM6AAAAAA45NIFCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGQYDKMZRGU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[RulerOf]

@jamengual I've been exploring that option, but it requires putting a file into the container to make that work. I use an ephemeral container configured through environment variables, and don't have any means at my disposal to place files directly into my fargate containers, especially not when using the terraform-aws-atlantis module.

I could rewrite the module to facilitate... something... but it still looks like the only good option here is to maintain my own downstream image, which I'd rather not have to do.

Am I missing something?

[jamengual]

Well, the file needs to be somewhere. That does not stop you from putting that into an EBS or EFS volume to mount on ECS.
I think that is far more simple than running ECR and a pipeline etc.

[RulerOf]

Edit: I misread the error I received and thought the EFS mount wouldn't work if the path didn't exist.

The EFS solution works, but I am either going to have to fork the module or complete a painful upgrade to v4—there's not enough customization in the v3 module to make this work as-is.

[RulerOf]

@jamengual Okay... so now I've got EFS mounted in the container's /docker-entrypoint.d folder. I created a script to install python.

The container runs the entrypoint script as the atlantis user, and the python installation fails:

  | 2024-05-10T21:52:28.870Z | /docker-entrypoint.d/ is not empty, will attempt to perform script execition
  | 2024-05-10T21:52:28.870Z | Looking for shell scripts in /docker-entrypoint.d/
  | 2024-05-10T21:52:28.872Z | Launching /docker-entrypoint.d/install-python.sh
  | 2024-05-10T21:52:28.988Z | ERROR: Unable to lock database: Permission denied
  | 2024-05-10T21:52:28.989Z | ERROR: Failed to open apk database: Permission denied

It does not appear to be possible to become root to complete the install:

╰─❯ docker run -it --rm ghcr.io/runatlantis/atlantis:latest bash
No files found in /docker-entrypoint.d/, skipping
4f33a6f957e4:/$ sudo su
bash: sudo: command not found
4f33a6f957e4:/$ su
su: must be suid to work properly
4f33a6f957e4:/$

Should I open a separate issue?

[jamengual]

we do not have sudo in the atlantis image.
You basically need to install it using the atlantis user

[RulerOf]

Uh... am I misunderstanding the point of the entrypoint scripts feature? Is it intentional that users of this image should never be able to perform actions as root without rebuilding it?

If I'm doing this wrong, could you point me in the direction of what the right method would be? I'm fairly sure that even if I wanted to compile Python in the home folder, I'd still have to install libraries and such that'd require root to install anyway. And if I have to run a docker build instead, that really defeats the point of this implementation because I'm back to basically forking this thing.

[nitrocode]

Hmm it seems that the docker entry script update using the directory method was probably setup prior to removing default root user from the image.

Perhaps the directory file run needs to put into a separate script that is run before switching the user to atlantis so it can manage installations as root

[RulerOf]

The user never switches. The dockerfile sets the user to atlantis so pid 1 starts that way:

atlantis/Dockerfile

Line 170 in 11afe2b

USER atlantis

I'm hesitant to work on a PR to fix this because overhauling the container's init and build processes would be a very significant change, and if this was done for security reasons, I don't have a gauge on how receptive the maintainers would be to that.

[RulerOf]

@nitrocode I took the time, would appreciate any thoughts you have on this: main...RulerOf:atlantis:main

I'll file a separate issue and submit a PR after lunch.

Edit: Nevermind. They explicitly removed this exact behavior due to perceived vulnerabilities in gosu operation. 🤦