Skip to content

Conversation

@juliankuners
Copy link
Contributor

@juliankuners juliankuners commented Apr 24, 2025

Following up on #4096, evm-semantics#2739, evm-semantics#2745, and kontrol#1013, I measured the install time of kontrol with kup on a fresh Ubuntu virtual machine. Compared to a previous measurement, the install time has been reduced to 10 minutes from previously 20 minutes.

During the latest measurement I noticed that Haskell dependencies are still being downloaded, even though the import-from-derivation anti-pattern has already been removed in #4096. I investigated and noticed that the Haskell dependencies are also considered runtime dependencies. This is caused by the withZ3 wrapper that links to binaries in Haskell derivations. These Haskell derivations also contain references to Haskell dependencies, namely kore and hs-backend-booster.

This pull request changes the withZ3 wrapper to copy the respective binary prior to wrapping.

E.g., for kore-rpc-booster, this reduces the total size to download by more than 5GB, when installing from nix cache. In addition, these downloads were spread among many smaller derivations to download.

Output of `nix-tree` before the change:

┌─────────────────────────────────────────────────────┬────────────────────────────────────────────────────┬────────────────────────────────────────────────────┐
│kore-rpc-booster                  5.15 GiB (5.15 GiB)│hs-backend-booster-0.1.0         5.12 GiB (5.08 GiB)│kore-0.1.0                       4.89 GiB (1.31 GiB)│
│                                                     │z3-4.13.4                      72.03 MiB (32.24 MiB)│kore-rpc-types-0.1.0            3.34 GiB (80.65 MiB)│
│                                                     │bash-5.2p26                     32.51 MiB (1.54 MiB)│json-rpc-1.0.4                  3.21 GiB (15.82 MiB)│
│                                                     │                                                    │aeson-pretty-0.8.10              3.04 GiB (21.2 MiB)│
│                                                     │                                                    │deriving-aeson-0.2.9           3.02 GiB (605.28 KiB)│
│                                                     │                                                    │aeson-2.1.2.1                   3.02 GiB (52.64 MiB)│
│                                                     │                                                    │monad-logger-0.3.40               2.98 GiB (4.6 MiB)│
│                                                     │                                                    │stm-conduit-4.0.1                2.97 GiB (2.49 MiB)│
│                                                     │                                                    │conduit-extra-1.3.6               2.95 GiB (5.0 MiB)│
│                                                     │                                                    │cereal-conduit-0.8.0            2.88 GiB (441.2 KiB)│
│                                                     │                                                    │conduit-1.3.5                    2.88 GiB (13.4 MiB)│
│                                                     │                                                    │semialign-1.3                    2.86 GiB (3.73 MiB)│
│                                                     │                                                    │mono-traversable-1.0.17.0       2.85 GiB (22.06 MiB)│
│                                                     │                                                    │recursion-schemes-5.2.2.5        2.82 GiB (6.79 MiB)│
│                                                     │                                                    │cryptonite-0.30                 2.81 GiB (54.26 MiB)│
│                                                     │                                                    │free-5.2                        2.81 GiB (22.79 MiB)│
│                                                     │                                                    │vector-algorithms-0.9.0.1       2.81 GiB (25.24 MiB)│
│                                                     │                                                    │witherable-0.4.2                   2.8 GiB (4.6 MiB)│
│                                                     │                                                    │indexed-traversable-instances- 2.79 GiB (358.65 KiB)│
│                                                     │                                                    │bitvec-1.1.5.0                  2.78 GiB (10.49 MiB)│
│                                                     │                                                    │vector-0.13.1.0                 2.77 GiB (49.75 MiB)│
│                                                     │                                                    │semigroupoids-6.0.1             2.77 GiB (12.16 MiB)│
│                                                     │                                                    │language-c-0.9.3                2.76 GiB (63.64 MiB)│
│                                                     │                                                    │profunctors-5.6.2               2.75 GiB (10.68 MiB)│
│                                                     │                                                    │memory-0.18.0                    2.74 GiB (5.69 MiB)│
│                                                     │                                                    │basement-0.0.16                 2.74 GiB (28.09 MiB)│
│                                                     │                                                    │bifunctors-5.6.2                2.73 GiB (16.04 MiB)│
│                                                     │                                                    │optparse-applicative-0.18.1.0    2.72 GiB (8.72 MiB)│
│                                                     │                                                    │streaming-commons-0.2.2.6        2.72 GiB (4.91 MiB)│
│                                                     │                                                    │attoparsec-0.14.4               2.72 GiB (17.16 MiB)│
│                                                     │                                                    │QuickCheck-2.14.3               2.72 GiB (19.19 MiB)│
│                                                     │                                                    │network-run-0.2.8                2.72 GiB (1.29 MiB)│
│                                                     │                                                    │hpp-0.6.5                       2.71 GiB (13.35 MiB)│
└─────────────────────────────────────────────────────┴────────────────────────────────────────────────────┴────────────────────────────────────────────────────┘
/nix/store/y51pfqn8fcmnimhkswfc1fda2pan3kmj-hs-backend-booster-0.1.0                                                                                             
NAR Size: 108.81 MiB | Closure Size: 5.12 GiB | Added Size: 5.08 GiB                                                                                             
Immediate Parents (1): kore-rpc-booster                                                                                                                          

Output of `nix-tree` after the change:

┌─────────────────────────────────────────────────────┬────────────────────────────────────────────────────┬────────────────────────────────────────────────────┐
│kore-rpc-booster              113.87 MiB (113.87 MiB)│z3-4.13.4                      72.03 MiB (32.24 MiB)│gcc-13.2.0-lib                  39.78 MiB (8.81 MiB)│
│                                                     │elfutils-0.191                 58.27 MiB (16.82 MiB)│glibc-2.39-52                  30.96 MiB (28.88 MiB)│
│                                                     │gmp-with-cxx-6.3.0            40.51 MiB (741.04 KiB)│                                                    │
│                                                     │ncurses-6.4                     34.51 MiB (3.54 MiB)│                                                    │
│                                                     │bash-5.2p26                     32.51 MiB (1.54 MiB)│                                                    │
│                                                     │secp256k1-0.4.1                 32.22 MiB (1.25 MiB)│                                                    │
│                                                     │zlib-1.3.1                    31.09 MiB (125.16 KiB)│                                                    │
│                                                     │libffi-3.4.6                   31.03 MiB (71.87 KiB)│                                                    │
│                                                     │glibc-2.39-52                  30.96 MiB (28.88 MiB)│                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
│                                                     │                                                    │                                                    │
└─────────────────────────────────────────────────────┴────────────────────────────────────────────────────┴────────────────────────────────────────────────────┘
/nix/store/ih51sgk8g57fnkbd5r82ddi8k5vln8cl-z3-4.13.4                                                                                                            
NAR Size: 32.24 MiB | Closure Size: 72.03 MiB | Added Size: 32.24 MiB                                                                                            
Immediate Parents (1): kore-rpc-booster                                                                                                                          

@juliankuners juliankuners requested a review from jberthold April 24, 2025 12:14
Copy link
Member

@jberthold jberthold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great discovery!

@automergerpr-permission-manager automergerpr-permission-manager bot merged commit 362dab3 into master Apr 24, 2025
6 checks passed
@automergerpr-permission-manager automergerpr-permission-manager bot deleted the nix-runtime-dependencies branch April 24, 2025 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants