perf(dracut-functions): optimize _inst_rule_programs()#2352
Open
aafeijoo-suse wants to merge 1 commit into
Open
perf(dracut-functions): optimize _inst_rule_programs()#2352aafeijoo-suse wants to merge 1 commit into
aafeijoo-suse wants to merge 1 commit into
Conversation
bdrung
reviewed
Apr 9, 2026
bdrung
reviewed
Apr 9, 2026
|
|
||
| # cache installed rule programs | ||
| if ! declare -p _rule_programs_cache 2> /dev/null | grep -q "declare -A"; then | ||
| declare -gxA _rule_programs_cache=() |
Member
There was a problem hiding this comment.
I am +1 for calling inst_multiple once, I am not sure about the added cache.
How much difference does this cache make? Could we do the caching in inst_multiple instead? I would go one step further for #1419: Add staging functions similar to the inst_* functions. These functions would just collect what should be added. Then call an dracut-intsall equivalent to only write a manifest for 3cpio.
Contributor
Author
There was a problem hiding this comment.
I am +1 for calling
inst_multipleonce, I am not sure about the added cache.How much difference does this cache make?
I don't think hyperfine can show a real improvement with these kinds of minor performance patches. That depends on multiple factors. But IMO it seems like a logical and small code improvement. I remember when I implemented 80f2caf, I didn't see anything important with hyperfine, but that turned out to be critical in some systems (https://bugzilla.redhat.com/show_bug.cgi?id=2278534#c8).
Could we do the caching in
inst_multipleinstead? I would go one step further for #1419: Add staging functions similar to theinst_*functions. These functions would just collect what should be added. Then call an dracut-intsall equivalent to only write a manifest for 3cpio.
This is a simple, localized performance improvement. I know you would like to improve how the inst* functions work, but that would be another story. You would need to track the name of the module and all the specific options passed to dracut-install for each call, to be able to say "this module cannot be installed because...".
Member
There was a problem hiding this comment.
We had very slow dracut-install executions on some ARM systems. So reducing the amount of calls we do to dracut-install helped there. That difference could be measured by hyperfine back then. So maybe I should do my benchmarks on a Raspberry Pi instead. I did the hyperfine benchmark to see if we should include this patch in the upcoming Ubuntu 26.04.
Member
|
I did a benchmark with my |
Let's start analyzing the output of a non-hostonly build, counting how many
times `_inst_rule_programs()` calls an `inst*()` function and with what
arguments:
```
$ dracut -f -N --debug testn.img 2>&1 &> testn.txt
$ grep -r -c "(_inst_rule_programs): inst_" testn.txt
59
$ grep -r "(_inst_rule_programs): inst_" testn.txt
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/systemctl
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/udevadm
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/dmsetup
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/dmsetup
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/dmsetup
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/sbin/dmsetup
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/sbin/kpartx
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /bin/sh
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /bin/sh
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /bin/sh
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/dmsetup
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/bin/readlink
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/bin/basename
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/bin/basename
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/bin/basename
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /bin/sh
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /sbin/partx
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/rm
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/systemd-run
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/systemctl
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/sbin/multipath
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/sbin/kpartx
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/bin/logger
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/sbin/multipath
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/bin/logger
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/kpartx_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/kpartx_id
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /bin/sh
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /bin/sh
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /bin/logger
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/lib/udev/cdrom_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/cdrom_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/fido_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/scsi_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/scsi_id
/usr/lib/dracut/dracut-functions.sh@1244(_inst_rule_programs): inst_binary /usr/lib/udev/scsi_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/ata_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/ata_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/ata_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/scsi_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/scsi_id
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/v4l_id
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/udevadm
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/dmi_memory_id
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/udevadm
/usr/lib/dracut/dracut-functions.sh@1259(_inst_rule_programs): inst_binary /usr/bin/loginctl
/usr/lib/dracut/dracut-functions.sh@1274(_inst_rule_programs): inst_multiple /usr/lib/udev/mtd_probe
```
This means that `_inst_rule_programs()` spawns 59 dracut-install processes,
without grouping the binaries of the same rule, and sometimes trying to install
the same binary. Therefore, this can be optimized:
- Instead of calling `inst_binary()` and `inst_multiple()` within the 3 loops
for each PROGRAM/RUN/IMPORT element, collect the different binaries to be
installed and call `inst_multiple()` once at the end. Note that `inst_binary()`
only differs from `inst_multiple()` because it only expects 1 argument.
- Cache the binaries already installed from previous rules, to avoid unnecessary
calls.
After this simple patch, the number of spawned dracut-install processes drops
from 59 to 18, i.e., 41 less.
```
$ grep -r -c "(_inst_rule_programs): inst_" testn2.txt
18
$ grep -r "(_inst_rule_programs): inst_" testn2.txt
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/bin/systemctl
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/bin/udevadm
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/sbin/dmsetup
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/sbin/kpartx
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /bin/sh
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/bin/readlink /usr/bin/basename /usr/sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /sbin/partx /sbin/mdadm
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/sbin/multipath /usr/bin/systemd-run /usr/bin/rm
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/kpartx_id /usr/bin/logger
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /bin/logger /usr/bin/sg_inq
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/cdrom_id
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/fido_id
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/scsi_id
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/ata_id
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/v4l_id
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/dmi_memory_id
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/bin/loginctl
/usr/lib/dracut/dracut-functions.sh@1291(_inst_rule_programs): inst_multiple /usr/lib/udev/mtd_probe
```
Double-check that we are installing the same binaries:
```
$ grep -r "(_inst_rule_programs): inst_" testn.txt | cut -d ' ' -f 3 | sort -u | wc -l
25
$ grep -r "(_inst_rule_programs): inst_" testn2.txt | awk -F"inst_multiple " '{ print $2 }' | tr ' ' '\n' | sort | wc -l
25
$ diff \
> <(grep -r "(_inst_rule_programs): inst_" testn.txt | cut -d ' ' -f 3 | sort -u) \
> <(grep -r "(_inst_rule_programs): inst_" testn2.txt | awk -F"inst_multiple " '{ print $2 }' | tr ' ' '\n' | sort)
$ echo $?
0
```
aafeijoo-suse
force-pushed
the
inst-perf
branch
from
1e5ecfa to
de20685
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Let's start analyzing the output of a non-hostonly build, counting how many times
_inst_rule_programs()calls aninst*()function and with what arguments:This means that
_inst_rule_programs()spawns 59 dracut-install processes, without grouping the binaries of the same rule, and sometimes trying to install the same binary. Therefore, this can be optimized:inst_binary()andinst_multiple()within the 3 loops for each PROGRAM/RUN/IMPORT element, collect the different binaries to be installed and callinst_multiple()once at the end. Note thatinst_binary()only differs frominst_multiple()because it only expects 1 argument.After this simple patch, the number of spawned dracut-install processes drops from 59 to 18, i.e., 41 less.
Double-check that we are installing the same binaries:
Checklist