Error while running on x86_64 machine

[i-am-yuvi]

I am running:

• ubuntu 24.04

yuvic@yuvic-ulinux:~/fuzzamoto$ docker build -t fuzzamoto .
[+] Building 1718.6s (48/48) FINISHED                                                                                                                                          docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                     0.0s
 => => transferring dockerfile: 4.95kB                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/debian:bookworm                                                                                                                       9.1s
 => [internal] load .dockerignore                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                          0.0s
 => [internal] load build context                                                                                                                                                        0.0s
 => => transferring context: 126.52kB                                                                                                                                                    0.0s
 => [ 1/44] FROM docker.io/library/debian:bookworm@sha256:264982ff4d18000fa74540837e2c43ca5137a53a83f8f62c7b3803c0f0bdcd56                                                               9.9s
 => => resolve docker.io/library/debian:bookworm@sha256:264982ff4d18000fa74540837e2c43ca5137a53a83f8f62c7b3803c0f0bdcd56                                                                 0.0s
 => => sha256:264982ff4d18000fa74540837e2c43ca5137a53a83f8f62c7b3803c0f0bdcd56 8.52kB / 8.52kB                                                                                           0.0s
 => => sha256:6bc30d909583f38600edd6609e29eb3fb284ab8affce8d0389f332fc91c2dd91 1.02kB / 1.02kB                                                                                           0.0s
 => => sha256:b2ab84c007feae81d95c5350d44ad7a54ea4693a79cb40fb05bd3fe00cbd4d26 453B / 453B                                                                                               0.0s
 => => sha256:cf05a52c02353f0b2b6f9be0549ac916c3fb1dc8d4bacd405eac7f28562ec9f2 48.49MB / 48.49MB                                                                                         8.3s
 => => extracting sha256:cf05a52c02353f0b2b6f9be0549ac916c3fb1dc8d4bacd405eac7f28562ec9f2                                                                                                1.4s
 => [ 2/44] RUN apt-get update                                                                                                                                                           4.2s
 => [ 3/44] RUN apt-get install -y software-properties-common                                                                                                                           26.0s 
 => [ 4/44] RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 15CF4D18AF4F7421                                                                                                3.4s 
 => [ 5/44] RUN apt-add-repository "deb http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-19 main"                                                                                   6.0s 
 => [ 6/44] RUN apt-add-repository "deb-src http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-19 main"                                                                               2.8s 
 => [ 7/44] RUN apt-get update && apt-get install -y   ninja-build   libgtk-3-dev   pax-utils   python3-msgpack   python3-jinja2   curl   lld-19   llvm-19   llvm-19-dev   clang-19    114.6s 
 => [ 8/44] RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y                                                                                                 41.2s 
 => [ 9/44] RUN rustup install nightly && rustup default nightly                                                                                                                        35.9s 
 => [10/44] RUN git clone --depth 1 --branch "v0.6.0" https://github.com/0xricksanchez/AFL_Runner.git                                                                                    2.9s 
 => [11/44] RUN cd AFL_Runner && cargo install --path .                                                                                                                                 47.4s 
 => [12/44] RUN mkdir -p /root/.config/tmux/ &&   echo "set -g prefix C-y" > /root/.config/tmux/tmux.conf                                                                                0.1s 
 => [13/44] RUN git clone https://github.com/AFLplusplus/AFLplusplus                                                                                                                    18.8s 
 => [14/44] RUN cd AFLplusplus && make PERFORMANCE=1 install -j$(nproc --ignore 1)                                                                                                      86.6s 
 => [15/44] RUN cd AFLplusplus/nyx_mode/ && ./build_nyx_support.sh                                                                                                                     182.7s 
 => [16/44] RUN cd AFLplusplus/nyx_mode/packer/packer/linux_x86_64-userspace &&   ./compile_64.sh                                                                                        0.7s 
 => [17/44] RUN git clone --depth 1 --branch master https://github.com/bitcoin/bitcoin.git                                                                                               7.3s 
 => [18/44] RUN make -C bitcoin/depends NO_QT=1 NO_BDB=1 NO_ZMQ=1 NO_UPNP=1 NO_NATPMP=1 NO_USDT=1 download-linux SOURCES_PATH=/tmp/bitcoin-depends                                     351.3s 
 => [19/44] RUN sed -i --regexp-extended '/.*rm -rf .*extract_dir.*/d' ./bitcoin/depends/funcs.mk &&     make -C ./bitcoin/depends DEBUG=1 NO_QT=1 NO_BDB=1 NO_ZMQ=1 NO_USDT=1       S  76.9s 
 => [20/44] COPY ./target-patches/bitcoin-core-rng.patch bitcoin/                                                                                                                        0.0s 
 => [21/44] RUN cd bitcoin/ &&       git apply bitcoin-core-rng.patch                                                                                                                    0.2s 
 => [22/44] RUN cd bitcoin/ && cmake -B build_fuzz       --toolchain ./depends/$(./depends/config.guess)/toolchain.cmake       -DSANITIZERS="address"       -DAPPEND_CPPFLAGS="-DFUZZI  14.3s 
 => [23/44] RUN cmake --build bitcoin/build_fuzz -j$(nproc) --target bitcoind                                                                                                          543.1s 
 => [24/44] WORKDIR /fuzzamoto/fuzzamoto-nyx-sys                                                                                                                                         0.1s 
 => [25/44] COPY ./fuzzamoto-nyx-sys/Cargo.toml .                                                                                                                                        0.2s 
 => [26/44] COPY ./fuzzamoto-nyx-sys/src/ src/                                                                                                                                           0.0s 
 => [27/44] COPY ./fuzzamoto-nyx-sys/build.rs .                                                                                                                                          0.0s 
 => [28/44] WORKDIR /fuzzamoto/fuzzamoto                                                                                                                                                 0.0s 
 => [29/44] COPY ./fuzzamoto/Cargo.toml .                                                                                                                                                0.0s 
 => [30/44] COPY ./fuzzamoto/src/ src/                                                                                                                                                   0.0s
 => [31/44] WORKDIR /fuzzamoto/fuzzamoto-cli                                                                                                                                             0.0s
 => [32/44] COPY ./fuzzamoto-cli/Cargo.toml .                                                                                                                                            0.0s
 => [33/44] COPY ./fuzzamoto-cli/src/ src/                                                                                                                                               0.0s
 => [34/44] WORKDIR /fuzzamoto/fuzzamoto-scenarios                                                                                                                                       0.0s
 => [35/44] COPY ./fuzzamoto-scenarios/Cargo.toml .                                                                                                                                      0.0s
 => [36/44] COPY ./fuzzamoto-scenarios/bin/ bin/                                                                                                                                         0.0s
 => [37/44] COPY ./fuzzamoto-scenarios/grammars/ grammars/                                                                                                                               0.0s
 => [38/44] WORKDIR /fuzzamoto                                                                                                                                                           0.0s
 => [39/44] COPY ./Cargo.toml .                                                                                                                                                          0.0s
 => [40/44] RUN mkdir .cargo && cargo vendor > .cargo/config                                                                                                                            10.3s
 => [41/44] RUN cargo build --workspace --verbose --features nyx,reduced_pow --release                                                                                                 101.9s
 => [42/44] RUN clang-19 -fPIC -DENABLE_NYX -D_GNU_SOURCE -DNO_PT_NYX     ./fuzzamoto-nyx-sys/src/nyx-crash-handler.c -ldl -I. -shared -o libnyx_crash_handler.so                        0.3s
 => [43/44] RUN for scenario in /fuzzamoto/target/release/scenario-*; do       if [ -f "$scenario" ] && [ -x "$scenario" ]; then       scenario_name=$(basename $scenario);       expor  2.7s
 => exporting to image                                                                                                                                                                  17.3s
 => => exporting layers                                                                                                                                                                 17.3s
 => => writing image sha256:c133d900bd065081e939f0737c6670d7808e2af7fe82db17c5637fb5d00cba5f                           0.0s
 => => naming to docker.io/library/fuzzamoto                                                                                                                                             0.0s

yuvic@yuvic-ulinux:~/fuzzamoto$ docker run --privileged -it fuzzamoto bash
root@38030cbbe1f5:/# mkdir /tmp/in && echo "AAA" > /tmp/in/A
root@38030cbbe1f5:/# ls
AFL_Runner  AFLplusplus  bin  bitcoin  boot  dev  etc  fuzzamoto  home	lib  lib32  lib64  media  mnt  opt  proc  root	run  sbin  srv	sys  tmp  usr  var
root@38030cbbe1f5:/# ls -a
.  ..  .dockerenv  AFL_Runner  AFLplusplus  bin  bitcoin  boot	dev  etc  fuzzamoto  home  lib	lib32  lib64  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
root@38030cbbe1f5:/# afl-fuzz -X -i /tmp/in -o /tmp/out -- /tmp/fuzzamoto_scenario-http-server
afl-fuzz++4.33a based on afl by Michal Zalewski and a large online community
[+] AFL++ is maintained by Marc "van Hauser" Heuse, Dominik Maier, Andrea Fioraldi and Heiko "hexcoder" Eißfeldt
[+] AFL++ is open source, get it at https://github.com/AFLplusplus/AFLplusplus
[+] NOTE: AFL++ >= v3 has changed defaults and behaviours - see README.md
[+] AFL++ Nyx mode is enabled (developed and maintained by Sergej Schumilo)
[+] Nyx is open source, get it at https://github.com/Nyx-Fuzz
[+] No -M/-S set, autoconfiguring for "-S default"
[*] Getting to work...
[+] Using exploration-based constant power schedule (EXPLORE)
[+] Enabled testcache with 50 MB
[+] Generating fuzz data with a length of min=1 max=1048576

[-] PROGRAM ABORT : Library 'libnyx.so' not found
         Location : find_afl_binary(), src/afl-common.c:565

[i-am-yuvi]

@dergoegge

[Crypt-iQ]

Can be fixed by using LD_PRELOAD and pointing to libnyx.so @i-am-yuvi

[i-am-yuvi]

Hey @Crypt-iQ thanks for this. I was able to run it!

[dergoegge]

I've also seen this before, I think I've worked around this by specifying the full path to afl-fuzz, so /AFLplusplus/afl-fuzz -X -i /tmp/in -o /tmp/out -- /tmp/fuzzamoto_scenario-http-server.

Thanks for giving it a try!

[i-am-yuvi]

Another issue persists, tried everything but couldn't find anything:

[+] Found a free CPU core, try binding to #0.
[*] Validating target binary...
[*] Scanning '/tmp/in'...
[*] Creating hard links for all input files...
[+] Loaded a total of 1 seeds.
[*] Spinning up the NYX backend...
[!] libnyx: spawning qemu with:
 /AFLplusplus/nyx_mode/QEMU-Nyx/x86_64-softmmu/qemu-system-x86_64 -kernel /AFLplusplus/nyx_mode/packer/linux_initramfs/bzImage-linux-4.15-rc7 -initrd /AFLplusplus/nyx_mode/packer/linux_initramfs/init.cpio.gz -append nokaslr oops=panic nopti ignore_rlimit_data -display none -serial none -enable-kvm -net none -k de -m 4096 -chardev socket,server,path=/tmp/out/workdir/interface_0,id=nyx_interface -device nyx,chardev=nyx_interface,bitmap_size=65536,input_buffer_size=1048576,worker_id=0,workdir=/tmp/out/workdir,sharedir=/tmp/fuzzamoto_scenario-http-server,aux_buffer_size=4096 -machine kAFL64-v1 -cpu kAFL64-Hypervisor-v1 -fast_vm_reload path=/tmp/out/workdir/snapshot/,load=off,skip_serialization=on
[QEMU-Nyx] Could not access KVM-PT kernel module!
[QEMU-Nyx] Trying vanilla KVM...

[QEMU-Nyx] ERROR: vmware backdoor is not enabled...

	Run the following commands to fix the issue:
	-----------------------------------------
	sudo modprobe -r kvm-intel
	sudo modprobe -r kvm
	sudo modprobe  kvm enable_vmware_backdoor=y
	sudo modprobe  kvm-intel
	cat /sys/module/kvm/parameters/enable_vmware_backdoor
	-----------------------------------------

qemu-system-x86_64: /AFLplusplus/nyx_mode/QEMU-Nyx/accel/kvm/kvm-all.c:2238: kvm_init: Assertion `ret < 0' failed.
[!] libnyx failed to initialize QEMU-Nyx: cannot launch QEMU-Nyx...

[-] PROGRAM ABORT : Something went wrong ...
         Location : afl_fsrv_start(), src/afl-forkserver.c:732

[Crypt-iQ]

@i-am-yuvi what OS are you on?

[i-am-yuvi]

@i-am-yuvi what OS are you on?

Ubuntu 24.04

[dergoegge]

Have you tried running the commands that are suggested?

	sudo modprobe -r kvm-intel
	sudo modprobe -r kvm
	sudo modprobe  kvm enable_vmware_backdoor=y
	sudo modprobe  kvm-intel
	cat /sys/module/kvm/parameters/enable_vmware_backdoor

If your CPU is an amd cpu then it'd be kvm-amd not kvm-intel.

[i-am-yuvi]

With lots of trail and effort, it worked, my machine's CPU is AMD and the issue persists for Qemu-Nyx (see here open PR). I have shared some insights for someone having issue with AMD type cpu:

• First check if the host system has loaded kvm modules with

lsmod | grep kvm

if not then, load it

sudo modprobe kvm
sudo modprobe kvm-intel  # or kvm-amd for AMD CPUs

• Run docker container by explicitly mounting the kvm module volume, as running modprobe kvm will output kvm not found

docker run -it --privileged -v /dev:/dev -v /lib/modules/$(uname -r):/lib/modules/$(uname -r) -v /sys:/sys fuzzamoto bash

• Since modprobe is not installed inside container, install it first

apt install kmod 

• Now load kvm inside the container

modprobe -r kvm-intel
modprobe -r kvm
modprobe  kvm enable_vmware_backdoor=y
modprobe  kvm-intel
cat /sys/module/kvm/parameters/enable_vmware_backdoor

should print Y, if it does then you are good to go!

• Now run it

AFLplusplus/afl-fuzz -X -i /tmp/in -o /tmp/out -- /tmp/fuzzamoto_scenario-http-server

Thanks @dergoegge @Crypt-iQ !!

[dergoegge]

Glad you could figure this out!

I normally run all the modprobe stuff outside of the container (I'm also mostly using amd cpus) and that usually works.