Enable packaging of curl|bash and other wild stuff. by jordansissel · Pull Request #1957 · jordansissel/fpm

jordansissel · 2022-11-07T01:42:54Z

For #1853

References: https://gist.github.com/jordansissel/0e0fc22344a17eadd1a3b9a1bbb7ed60

Attempts that worked out well enough:

✅ rustup
✅ sdkman
✅ sdkman's sdk install java

jordansissel · 2022-11-07T01:44:50Z

I picked Podman to start with because that's what I have right now.

Here's converting rustup's curl|bash to an rpm.

The rustup docs say the following:

Run the following in your terminal, then follow the onscreen instructions.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

I'll run it with |sh -s -- -y to run the installer non-interactively (accepting "yes" to all questions)
To make it faster/smaller, we'll also use rustup-init's flag, --profile=minimal
Add --no-modify-path so the installer doesn't mess with any .bashrc/.profile
Set a custom install path RUSTUP_HOME=/usr/local/rustup

% fpm --verbose -f -s curlbash -t rpm  -n rustup \
  --depends gcc \
  --curlbash-setup "apt-get update && apt-get install -y curl" \
  "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --profile=minimal --no-modify-path -y"
... [ some time passes ] ...
Created package {:path=>"rustup-1.0-1.x86_64.rpm"}

% rpm -qlp rustup-1.0-1.x86_64.rpm| less
...
/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo
/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rust-gdb
/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rust-gdbgui
/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rust-lldb
/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc
...

🫡 We've got an rpm. Let's try it.

# Start with a fresh fedora 36 container
% podman run -v "$PWD/rustup-1.0-1.x86_64.rpm:/rustup-1.0-1.x86_64.rpm" -it fedora:36

# Install the rpm.
[root@7bcb72eab5fa /]# dnf install rustup-1.0-1.x86_64.rpm 
...

Dependencies resolved.
==========================================================================================================
 Package                             Architecture    Version                  Repository             Size
==========================================================================================================
Installing:
 rustup                              x86_64          1.0-1                    @commandline          146 M
Installing dependencies:
...
 gcc                                 x86_64          12.2.1-2.fc36            updates                33 M
 glibc-devel                         x86_64          2.35-20.fc36             updates                70 k
 glibc-headers-x86                   noarch          2.35-20.fc36             updates               475 k
...

Transaction Summary
==========================================================================================================
Install  19 Packages

Total size: 206 M
Total download size: 59 M
Installed size: 695 M
Is this ok [y/N]: 
...
Installed:
  binutils-2.37-36.fc36.x86_64                 binutils-gold-2.37-36.fc36.x86_64                         
  cpp-12.2.1-2.fc36.x86_64                     elfutils-debuginfod-client-0.187-4.fc36.x86_64            
  gc-8.0.6-3.fc36.x86_64                       gcc-12.2.1-2.fc36.x86_64                                  
  glibc-devel-2.35-20.fc36.x86_64              glibc-headers-x86-2.35-20.fc36.noarch                     
  guile22-2.2.7-5.fc36.x86_64                  kernel-headers-6.0.5-200.fc36.x86_64                      
  libmpc-1.2.1-4.fc36.x86_64                   libpkgconf-1.8.0-2.fc36.x86_64                            
  libtool-ltdl-2.4.7-1.fc36.x86_64             libxcrypt-devel-4.4.28-1.fc36.x86_64                      
  make-1:4.3-7.fc36.x86_64                     pkgconf-1.8.0-2.fc36.x86_64                               
  pkgconf-m4-1.8.0-2.fc36.noarch               pkgconf-pkg-config-1.8.0-2.fc36.x86_64                    
  rustup-1.0-1.x86_64

Cool, eh?

Let's try running rustc?

[root@9b80162d68a3 /]# PATH=/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin:"$PATH"

[root@9b80162d68a3 /]# echo 'fn main() { println!("Hello world!"); }' > hello.rs
[root@9b80162d68a3 /]# rustc hello.rs
[root@9b80162d68a3 /]# ./hello 
Hello world!

jordansissel · 2022-11-07T05:50:31Z

sdkman --

% bundle exec bin/fpm --verbose -f -s curlbash -t rpm -n sdkman \
  --depends zip --depends unzip \
  --curlbash-setup "apt-get update" \
  --curlbash-setup "apt-get install -y curl unzip zip"  'curl -s "https://get.sdkman.io?rcupdate=false" |  bash'

And installing it...

[root@693dc2883f2c /]# dnf install sdkman-1.0-1.x86_64.rpm 
...
Installed:
  sdkman-1.0-1.x86_64             unzip-6.0-57.fc36.x86_64             zip-3.0-32.fc36.x86_64            

Complete!

[root@693dc2883f2c /]# source "$HOME/.sdkman/bin/sdkman-init.sh"
[root@693dc2883f2c /]# sdk install java
Downloading: java 17.0.5-tem

In progress...

#############################                                                                        29.3%
Repackaging Java 17.0.5-tem...

Done repackaging...

Installing: java 17.0.5-tem
Done installing!


Setting java 17.0.5-tem as default.

👍

jordansissel · 2022-11-07T05:57:37Z

Heh, what if we package what sdkman does instead?

% bundle exec bin/fpm --verbose -f -s curlbash -t rpm -n sdkman-java \
  --curlbash-setup "apt-get update" \
  --curlbash-setup "apt-get install -y curl unzip zip" \
  --curlbash-setup 'curl -s "https://get.sdkman.io?rcupdate=false" |  bash' \
  'bash -c ". $HOME/.sdkman/bin/sdkman-init.sh; sdk install java"'

[root@77d5815e9be6 /]# rpm -Uvh sdkman-java-1.0-1.x86_64.rpm 
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:sdkman-java-1.0-1                ################################# [100%]

[root@77d5815e9be6 /]# /root/.sdkman/candidates/java/17.0.5-tem/bin/java --version
openjdk 17.0.5 2022-10-18
OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)
OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (build 17.0.5+8, mixed mode, sharing)

jordansissel · 2022-11-07T06:18:09Z

Chef's installer

% bundle exec bin/fpm --verbose -f -s curlbash -t rpm -n chef \
  --curlbash-container-image fedora:36 \
  'curl -L https://omnitruck.chef.io/install.sh | bash'

% podman run -v "$PWD/chef-1.0-1.x86_64.rpm:/chef-1.0-1.x86_64.rpm" -it fedora:36
[root@3a51dd0abbfc /]# rpm -i chef-1.0-1.x86_64.rpm 
[root@3a51dd0abbfc /]# ohai platform
[root@3a51dd0abbfc /]# ohai platform
[
  "fedora"
]

jordansissel · 2022-11-07T06:33:06Z

Trying ohmyzsh:

% bundle exec bin/fpm --verbose -f -s curlbash -t rpm -n ohmyzsh \
  --depends zsh --depends git \
  --curlbash-container-image fedora:36 \
  --curlbash-setup "dnf install -y zsh git" \
  'curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh | sh'

Created package {:path=>"ohmyzsh-1.0-1.x86_64.rpm"}

[root@4a8f07e439b5 /]# dnf install ohmyzsh-1.0-1.x86_64.rpm 
Dependencies resolved.
==========================================================================================================
 Package                        Architecture   Version                         Repository            Size
==========================================================================================================
Installing:
 ohmyzsh                        x86_64         1.0-1                           @commandline         2.1 M
Installing dependencies:
...
Transaction Summary
==========================================================================================================
Install  79 Packages

Total size: 24 M
Total download size: 22 M
Installed size: 94 M
Is this ok [y/N]: y


[root@4a8f07e439b5 /]# zsh -ls
➜  / echo $ZSH_THEME
robbyrussell

jordansissel · 2022-11-07T06:39:05Z

Trying nodesource. This script seems to add a apt repository on Ubuntu then offers apt-get install nodejs. Let's capture the result of installing nodejs instead.

% bundle exec bin/fpm --verbose -f -s curlbash -t deb -n nodesource -v 19 \
  --curlbash-setup 'apt-get update' \
  --curlbash-setup 'apt-get install -y curl' \
  --curlbash-setup 'curl -fsSL https://deb.nodesource.com/setup_19.x | bash -' \
  'apt-get install -y nodejs'
Created package {:path=>"nodesource_19_amd64.deb"}

This is fun because it actually captures the nodejs install instead of just updating your apt repos.

% podman run -v "$PWD/nodesource_19_amd64.deb:/nodesource_19_amd64.deb" -it ubuntu:latest 
root@28e3fc58ef68:/# dpkg -i nodesource_19_amd64.deb 
...
root@28e3fc58ef68:/# node 
Welcome to Node.js v19.0.1.
Type ".help" for more information.
>

jordansissel · 2022-11-07T06:46:19Z

It's not perfect, but it is funny :)

jordansissel · 2022-11-13T07:01:10Z

Added Dockerfile/Containerfile support

For #1853

I was originally using tar's --exclude (didn't get committed), but '--exclude tmp' will exclude any path with "tmp" in the name when I really want to exclude just the toplevel directory "/tmp". Then I rememered fpm has its own --exclude which is based on Ruby's File::fnmatch, and that works differently than tar's --exclude and also does what I want. This bug was found by trying to package sdkman: ``` % bundle exec bin/fpm --verbose -f -s curlbash -t rpm -n sdkman \ --depends zip --depends unzip \ --curlbash-setup "apt-get update" \ --curlbash-setup "apt-get install -y curl unzip zip" 'curl -s "https://get.sdkman.io?rcupdate=false" | bash' ```

I found that sometimes fpm would crash during the cleanup step because some directories had their write permission removed. I'm rather surprised I don't remember seeing this problem in the past, though it's possible my memory is bad and I just forget having experienced this. The solution I came up with is to ensure all directories have both execute and write permissions before attempting `Fileutils.rm_r` As a cheat, I reuse a FileUtils internal class (`FileUtils::Entry_`) and its `preorder_traversal` methods to walk all directories, ensuring that top-level and parent directories are modified first. I know it's taboo to use internal/non-public code. It works on Ruby 2.7, and if we need to change implementations to make things work better on other rubies, then we'll do that. It works.

Example: ``` FROM fedora:36 RUN ls RUN touch /usr/bin/hello-world ``` ``` % bundle exec bin/fpm -f -s curlbash -t rpm -n example /tmp/Dockerfile Created package {:path=>"example-1.0-1.x86_64.rpm"} % rpm -qlvp example-1.0-1.x86_64.rpm -rw-r--r-- 1 root root 0 Nov 12 21:43 /usr/bin/hello-world ```

…s added

Also added some scaffholding for docker support whenever that gets added to fpm.

and also fixed bugs found by the tests. :)

Also ensure we clean up the temporary container image we created

I feel like this better suits the purpose, as we could curlbash things but this feature can also just do regular Dockerfile/Containerfile things.

jordansissel marked this pull request as ready for review November 14, 2022 01:45

jordansissel mentioned this pull request Oct 21, 2024

Request: Docker image package type #2079

Open

jordansissel added 13 commits October 10, 2025 23:30

Enable packaging of curl|bash and other wild stuff.

7a8b934

For #1853

Skip /var/tmp also

d42a2ad

Clarify that execute perms are also needed.

c13a296

Remove unused code leftover from an earlier version

e5328b0

Remove debug print

87e376c

The 'command' variable was renamed 'entry' when Dockerfile support wa…

111e8c8

…s added

Add checks for podman.

664b122

Also added some scaffholding for docker support whenever that gets added to fpm.

Wrote some tests...

6c3af96

and also fixed bugs found by the tests. :)

Move the safety checks to the top.

4993a86

Also ensure we clean up the temporary container image we created

Rename curlbash to container.

b969afa

I feel like this better suits the purpose, as we could curlbash things but this feature can also just do regular Dockerfile/Containerfile things.

jordansissel force-pushed the issue/1853 branch from 2d4bf9b to b969afa Compare October 11, 2025 06:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable packaging of curl|bash and other wild stuff.#1957

Enable packaging of curl|bash and other wild stuff.#1957
jordansissel wants to merge 13 commits intomainfrom
issue/1853

jordansissel commented Nov 7, 2022 •

edited

Loading

Uh oh!

jordansissel commented Nov 7, 2022 •

edited

Loading

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jordansissel commented Nov 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jordansissel commented Nov 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 7, 2022

Uh oh!

jordansissel commented Nov 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jordansissel commented Nov 7, 2022 •

edited

Loading

jordansissel commented Nov 7, 2022 •

edited

Loading