|
| 1 | +# Deterministic Build Guide |
| 2 | + |
| 3 | +The following is a step-by-step guide to build your own deterministic Ordisrespector Bitcoin Core. |
| 4 | +If you complete these steps you should get exactly the same binaries (byte-per-byte) as distributed in this repository. |
| 5 | + |
| 6 | +## Requirements |
| 7 | + |
| 8 | +* Ubuntu 22.04 (not a hard requirement, but to be able to follow the guide verbatim) |
| 9 | +* An Apple Developer account (KYCed, but optional. Only required for MacOS binaries) |
| 10 | +* A powerful CPU |
| 11 | +* Lots of RAM |
| 12 | +* About 100GB of free space for intermediary build artifacts |
| 13 | +* A bit of proficiency in Linux |
| 14 | +* Patience |
| 15 | + |
| 16 | +For reference, my shiny new [Ryzen 7 7700X] workstation with 32GB of DDR5 RAM and an NVMe SSD takes about 2 and a half hours to complete a full build of the 9 different binaries of Bitcoin Core 24.0.1. |
| 17 | + |
| 18 | +## Install Guix |
| 19 | + |
| 20 | +Start by installing the `guix` package provided by Ubuntu, then update it to the latest stable version with its own update command. |
| 21 | + |
| 22 | +```sh |
| 23 | +$ sudo apt install guix |
| 24 | +$ guix pull |
| 25 | +``` |
| 26 | + |
| 27 | +After the second command ends, it will suggest to add the following to lines to your `$HOME/.profile` file, do it: |
| 28 | + |
| 29 | +```sh |
| 30 | +# append the following to $HOME/.profile: |
| 31 | + |
| 32 | +export GUIX_PROFILE="/home/marcel/.config/guix/current" |
| 33 | +. "$GUIX_PROFILE/etc/profile" |
| 34 | +``` |
| 35 | + |
| 36 | +Apply the changes by running `source $HOME/.profile`, then run `guix --version`. It will suggest to install a Guix package and add one more line to your `.profile` file. |
| 37 | + |
| 38 | +```sh |
| 39 | +$ guix install glibc-locales |
| 40 | +``` |
| 41 | +```sh |
| 42 | +# append the following to $HOME/.profile: |
| 43 | + |
| 44 | +export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale" |
| 45 | +``` |
| 46 | + |
| 47 | +Reload your environment again with `source $HOME/.profile`, now `guix --version` should show no warnings. |
| 48 | +At this point you can log out and log in again to your user session so that the `.profile` changes are automatically applied to all your shell sessions instead of just the current one. |
| 49 | + |
| 50 | +## Installing the MacOS SDK (optional) |
| 51 | + |
| 52 | +You can fully skip this section if you don't intend to build MacOS binaries. |
| 53 | + |
| 54 | +Warning: only Bitcoin Core v24, v23 and v22 can do MacOS Guix builds. |
| 55 | +Download the appropriate Xcode.xip file for the version of Bitcoin Core you want to build. |
| 56 | +To be able to download these files you'll need to use a browser that is logged in into the [Apple developer portal]. |
| 57 | + |
| 58 | +| Bitcoin Core Version | XCode | |
| 59 | +|-------------------------|--------------------------------------------------------------------------------------------------| |
| 60 | +| Bitcoin Core v24 or v23 | [Xcode_12.2.xip](https://download.developer.apple.com/Developer_Tools/Xcode_12.2/Xcode_12.2.xip) | |
| 61 | +| Bitcoin Core v22 | [Xcode_12.1.xip](https://download.developer.apple.com/Developer_Tools/Xcode_12.1/Xcode_12.1.xip) | |
| 62 | + |
| 63 | +Assuming you downloaded `Xcode_12.2.xip` into your `$HOME/Downloads` folder, now install the `git` and `cpio` packages, clone Bitcoin Core's [`apple-sdk-tools`] repository and use its `extract_xcode.py` script to extract the archive. |
| 64 | + |
| 65 | +```sh |
| 66 | +$ cd $HOME |
| 67 | +$ sudo apt install cpio git |
| 68 | +$ git clone https://github.com/bitcoin-core/apple-sdk-tools |
| 69 | +$ python3 apple-sdk-tools/extract_xcode.py -f Downloads/Xcode_12.2.xip | cpio -d -i |
| 70 | +``` |
| 71 | + |
| 72 | +This will create a huge `Xcode.app` folder in your current directory. |
| 73 | +Now clone the Bitcoin repository to run another helper script that will take this directory and prepare an "Xcode SDK" from it. |
| 74 | +Since we are cloning the bitcoin repo take the opportunity to check out the specific version you want to build. |
| 75 | +In this guide we are building Bitcoin Core 24.0.1 so we'll check out the `v24.0.1` tag. |
| 76 | + |
| 77 | +```sh |
| 78 | +$ cd $HOME |
| 79 | +$ git clone https://github.com/bitcoin/bitcoin |
| 80 | +$ cd bitcoin |
| 81 | +$ git checkout v24.0.1 |
| 82 | +$ cd .. |
| 83 | +$ ./bitcoin/contrib/macdeploy/gen-sdk $HOME/Xcode.app |
| 84 | +``` |
| 85 | + |
| 86 | +The last command will generate an `Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz` in your home directory. |
| 87 | +It contains the Apple bits that Bitcoin Core v24 and v23 needs to perform a deterministic MacOS build. |
| 88 | +Now create dedicated folder in your home directory for MacOS SDKs and extract this .tar.gz in it: |
| 89 | + |
| 90 | +```sh |
| 91 | +$ cd $HOME |
| 92 | +$ mkdir MacOS-SDKs |
| 93 | +$ mv Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz MacOS-SDKs |
| 94 | +$ cd MacOS-SDKs |
| 95 | +$ tar zxf Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz |
| 96 | +``` |
| 97 | + |
| 98 | +With this folder structure in place we're ready to do Guix MacOS builds. |
| 99 | +If you want you can now delete the `XCode.app` directory and the .xip download to reclaim some 70GB of space in your hard drive. |
| 100 | +But save the .tar.gz archive (only 70MB) and keep it close to your heart. |
| 101 | + |
| 102 | +## Building Bitcoin Core |
| 103 | + |
| 104 | +Start by creating a couple of temporary folders in your home directory that the build process will use. |
| 105 | + |
| 106 | +```sh |
| 107 | +$ cd $HOME |
| 108 | +$ mkdir depends-SOURCES_PATH depends-BASE_PATH |
| 109 | +``` |
| 110 | + |
| 111 | +If you skipped the MacOS section now clone the Bitcoin Core repository and check out the version you want to build. |
| 112 | +As stated elsewhere in the walkthrough we're building Bitcoin Core 24.0.1. |
| 113 | + |
| 114 | +If you already cloned the bitcoin repository and checked out the version you want earlier, simply enter the `bitcoin` directory. |
| 115 | + |
| 116 | +```sh |
| 117 | +$ cd $HOME |
| 118 | +$ sudo apt install git |
| 119 | +$ git clone https://github.com/bitcoin/bitcoin |
| 120 | +$ cd bitcoin |
| 121 | +$ git checkout v24.0.1 |
| 122 | +``` |
| 123 | + |
| 124 | +Now for the most disrespectful part of the guide, while still inside the bitcoin repository, download the `ordinals-filter.patch` file and apply it to the source tree! |
| 125 | + |
| 126 | +```sh |
| 127 | +$ wget https://gist.githubusercontent.com/luke-jr/4c022839584020444915c84bdd825831/raw/555c8a1e1e0143571ad4ff394221573ee37d9a56/filter-ordinals.patch |
| 128 | +$ git apply filter-ordinals.patch |
| 129 | +``` |
| 130 | + |
| 131 | +You can use `git diff` to check the changes you've made to the v24.0.1 codebase. They must look like this: |
| 132 | + |
| 133 | +```sh |
| 134 | +$ git diff |
| 135 | +diff --git a/src/script/interpreter.cpp b/src/script/interpreter.cpp |
| 136 | +index 5f4a1aceb..3eec7373f 100644 |
| 137 | +--- a/src/script/interpreter.cpp |
| 138 | ++++ b/src/script/interpreter.cpp |
| 139 | +@@ -479,6 +479,14 @@ bool EvalScript(std::vector<std::vector<unsigned char> >& stack, const CScript& |
| 140 | + return set_error(serror, SCRIPT_ERR_MINIMALDATA); |
| 141 | + } |
| 142 | + stack.push_back(vchPushValue); |
| 143 | ++ if ((flags & SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS) && opcode == OP_FALSE) { |
| 144 | ++ auto pc_tmp = pc; |
| 145 | ++ opcodetype next_opcode; |
| 146 | ++ valtype dummy_data; |
| 147 | ++ if (script.GetOp(pc_tmp, next_opcode, dummy_data) && next_opcode == OP_IF) { |
| 148 | ++ return set_error(serror, SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS); |
| 149 | ++ } |
| 150 | ++ } |
| 151 | + } else if (fExec || (OP_IF <= opcode && opcode <= OP_ENDIF)) |
| 152 | + switch (opcode) |
| 153 | + { |
| 154 | +``` |
| 155 | +
|
| 156 | +We're ready to start the build proper. |
| 157 | +The Guix build is governed with a series of environment variables that we'll now define. |
| 158 | +We start with the mandatory ones. |
| 159 | +
|
| 160 | +```sh |
| 161 | +$ export SOURCES_PATH="$HOME/depends-SOURCES_PATH" |
| 162 | +$ export BASE_CACHE="$HOME/depends-BASE_CACHE" |
| 163 | +$ export FORCE_DIRTY_WORKTREE=1 |
| 164 | +``` |
| 165 | +
|
| 166 | +This will instruct Guix to use the temporary build directories we created earlier, and to not be scared to build Bitcoin core with "uncommitted changes" (the patch we just applied). |
| 167 | +
|
| 168 | +Now, if you did the MacOS section also define this env var so that Guix can find the MacOS SDK: |
| 169 | +
|
| 170 | +```sh |
| 171 | +$ export SDK_PATH="$HOME/MacOS-SDKs" |
| 172 | +``` |
| 173 | +
|
| 174 | +Lastly, we choose which binaries we want to build. |
| 175 | +This is controlled with the `HOSTS` environment variable. |
| 176 | +If you don't set it, it will build Bitcoin Core for all supported targets. |
| 177 | +The exact targets supported vary for each release, you can find the specific list for 24.0.1 [in this link]. |
| 178 | +
|
| 179 | +If you skipped the MacOS section it is important to define the HOSTS variable without any Apple targets, otherwise Guix won't start. |
| 180 | +In this example we'll tell Guix to only build binaries for Linux x86_64, Linux ARM64 (e.g. for a Raspberry Pi 4) and Apple Silicon (since we went through the pain of preparing the Xcode SDK). |
| 181 | +This is also make the build run faster than if it builds all 9 targets. |
| 182 | +
|
| 183 | +```sh |
| 184 | +$ export HOSTS="x86_64-linux-gnu aarch64-linux-gnu arm64-apple-darwin" |
| 185 | +``` |
| 186 | +
|
| 187 | +...and we're ready to buidl. |
| 188 | +
|
| 189 | +```sh |
| 190 | +$ ./contrib/guix/guix-build |
| 191 | +``` |
| 192 | +
|
| 193 | +A very long and scary build log will start running through your screen. |
| 194 | +
|
| 195 | +Now it'd be a good time to let your machine do the deed and take a break. |
| 196 | +
|
| 197 | +## Verifying the result |
| 198 | +
|
| 199 | +After the build completes it will have created a `guix-build-24.0.1` directory inside the `bitcoin` project. |
| 200 | +In turn, this directory will have a subdirectory for each target you built (in our case `x86_64-linux-gnu`, `aarch64-linux-gnu` and `arm64-apple-darwin`). |
| 201 | +We are interested in the contents of `ìnstalled/bin` of each of these subdirectories. |
| 202 | +You should find our `bitcoind` binary there along with some other ones such as `bitcoin-cli`. |
| 203 | +
|
| 204 | +Use `sha256sum` to calculate the `bitcoind` sha256 hashes and compare them to the ones in the corresponding `SHASUMS256` file. |
| 205 | +In this example we should compare our hashes against these lines from the `SHASUMS256-24.0.1.txt` file: |
| 206 | +
|
| 207 | +```sh |
| 208 | +5a63f7cfa64e7d21005b4fb3956a9192442a4565bd7c03824f5559b9b8499552 bitcoind-24.0.1-aarch64-linux-gnu |
| 209 | +f70da220ebda9ba57dea5c531e1fa04122bb1fb3b11e9bf336b8996a448bf0b1 bitcoind-24.0.1-arm64-apple-darwin |
| 210 | +d31f48b0a06ff754bdc691b88a091bfac88af24a2d75ebf65383ab52b49155ba bitcoind-24.0.1-x86_64-linux-gnu |
| 211 | +``` |
| 212 | +
|
| 213 | +## Submit your signature |
| 214 | +
|
| 215 | +If you go through all this trouble and get exactly the same bitcoind binaries that I'm distributing I encourage you to GPG sign the corresponding SHASUMS256 file like I did and submit it in an issue. |
| 216 | +I'll add your name in the main README.md under the [Build Signers] section and include your signature file in the releases page. |
| 217 | +
|
| 218 | +### Other Resources |
| 219 | +
|
| 220 | +* Guix binary installation document: [https://guix.gnu.org/manual/en/html_node/Binary-Installation.html](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html) |
| 221 | +* Guix README, Bitcoin Core: [https://github.com/bitcoin/bitcoin/blob/master/contrib/guix](https://github.com/bitcoin/bitcoin/blob/master/contrib/guix) |
| 222 | +* Guix installation docs, Bitcoin Core: [https://github.com/bitcoin/bitcoin/blob/master/contrib/guix/INSTALL.md](https://github.com/bitcoin/bitcoin/blob/master/contrib/guix/INSTALL.md) |
| 223 | +* MacOS SDK extraction guide: [https://github.com/bitcoin/bitcoin/tree/master/contrib/macdeploy](https://github.com/bitcoin/bitcoin/tree/master/contrib/macdeploy) |
| 224 | +
|
| 225 | +
|
| 226 | +[Ryzen 7 7700X]: https://www.amd.com/en/products/cpu/amd-ryzen-7-7700 |
| 227 | +[Apple developer portal]: https://developer.apple.com/ |
| 228 | +[`apple-sdk-tools`]: https://github.com/bitcoin-core/apple-sdk-tools |
| 229 | +[in this link]: https://github.com/bitcoin/bitcoin/tree/v24.0.1/contrib/guix#recognized-environment-variables |
| 230 | +[Build Signers]: https://github.com/BcnBitcoinOnly/ordisrespector-binaries#build-signers |
0 commit comments