Update 'How to backport Rust', describe orig-vendor tarball

blkerby · blkerby · commit 8fa1a5c6e250 · 2026-02-12T17:22:29.000-07:00
diff --git a/docs/maintainers/niche-package-maintenance/rustc/backport-rust.md b/docs/maintainers/niche-package-maintenance/rustc/backport-rust.md
@@ -87,7 +87,7 @@ For example, if you were backporting `rustc-1.82` to Jammy...
 
 `<N>` is the suffix for `~bpo` in the {ref}`changelog version number <rust-version-strings>` and signals which crucial Rust dependencies (if any) were re-included in the source tarball.
 
-`<lp_bug_number>` refers to the bug number on Launchpad.
+`<lp_bug_number>` refers to the bug number on Launchpad requesting this backport (if applicable).
 
 ```{include} common/local-repo-setup.md
 
@@ -96,22 +96,13 @@ For example, if you were backporting `rustc-1.82` to Jammy...
 
 ## Backport process
 
-The baseline backport process is essentially trivial on its own and has few distinguishing features from a regular Rust toolchain update. The majority of these docs is taken up by the {ref}`rust-common-backporting-changes` section, which details things you'll often have to do in order to get the backport to build properly.
+The baseline backport process is simple, but there are many different things that can cause a backport to fail to build. The majority of these docs is taken up by the {ref}`rust-common-backporting-changes` section, which details things you'll often have to do in order to get the backport to build properly.
 
+### Launchpad bug report
 
-### Bug report
-
-To keep track of backport progress and status, a Launchpad bug report is absolutely necessary.
-
-It's quite likely that there's a specific _reason_ why the backport was needed (e.g., a Rust-based application in an old Ubuntu release has an SRU that needs a newer toolchain to build). In this case, simply reference that bug report throughout the process, assigning the bug to yourself.
-
-If no bug exists, you'll need to create your own. You can find a {lpbug}`good example here <2100492>`. If you need to go back multiple Ubuntu releases, target the bug to _all_ series along the way as well, so each of the intermediate backports can be monitored. Additionally, if you need to go back multiple Rust versions, a separate bug report must be filed for each Rust version.
-
-- Going back to our {ref}`Jammy 1.86 example <rust-example-backport>`, we'd have to create three bug reports:
-  1. `rustc-1.84` bug targeting Noble and Jammy
-  2. `rustc-1.85` bug targeting Noble and Jammy
-  3. `rustc-1.86` bug targeting Plucky, Noble, and Jammy
+In some cases a backport is requested for a specific reason, e.g., a Rust-based application in an old Ubuntu release has an SRU that needs a newer toolchain to build. In this case a Launchpad bug should be created, if one does not already exist. You can find a {lpbug}`good example here <2100492>`. The Launchpad bug helps to keep track of backport progress and status, which is essential if it is planned for the backport to be uploaded to the Ubuntu Archive. If you need to go back multiple Ubuntu releases, target the bug to _all_ series along the way as well, so each of the intermediate backports can be monitored.
 
+In other cases, backports are prepared proactively in case they may be needed in the future. In this case, a Launchpad bug does not need to be created for the backport. Even if a given backport is not needed in the Ubuntu Archive, it will likely still be needed to bootstrap later Rust versions. We upload all backports to the ["Rust Toolchain" Staging PPA](https://launchpad.net/~rust-toolchain/+archive/ubuntu/staging/). A backport which successfully builds in this PPA and passes its {term}`autopkgtest` suite may later be copied into the Ubuntu Archive as needed.
   
 ### Setup
 
@@ -137,44 +128,51 @@ $ git checkout -b jammy-1.85
 
 ### Changelog version
 
-The first thing we should do on our new branch is create a new changelog entry right off the bat. Before we change anything, however, it's important to understand the meaning of every component of the version number. Ensure you read and understand the {ref}`rust-version-strings` article before proceeding.
+The first thing we should do on our new branch is create a new changelog entry. Before we change anything, however, it's important to understand the meaning of every component of the version number. Ensure you read and understand the {ref}`rust-version-strings` article before proceeding.
 
 
 #### Creating the new changelog entry
 
-To begin, you only have to add/change `<release_number>` in the changelog version number. Don't forget to decrement it! You can leave any `~bpo<N>`s (or lack thereof) as-is for now, as you haven't made any changes to which dependencies have been {term}`vendored <vendored dependency>` yet.
+To begin, run the command `dch`:
 
-`<existing_version_number>` is the full version number of the latest changelog entry.
-
-```none
-$ dch -bv \
-    <existing_version_number>.<decremented_release_number> \
-    --distribution "<release>"
+```bash
+$ dch
 ```
 
+ This adds a new entry to the changelog and opens an editor allowing you to modify it:
+
+- Change "UNRELEASED" to the Ubuntu series that you are backporting to, using its short name, e.g. `jammy`. 
+- Update the version string to reference the series you are backporting to, using its numeric value, e.g. `22.04`, and reset any revision number at the end of the version string.
+
+You can leave the part of the version string before the first hyphen unchanged for now, as this refers to the version of the orig tarballs, which you haven't yet changed.
+
 **Examples:**
 
 | Existing release | Backport | `<existing_version_number>` | New version number |
 | --- | --- | --- | --- |
-| 1.82 Devel | 1.82 Oracular | `1.82.0+dfsg0ubuntu1-0ubuntu2` | `1.82.0+dfsg0ubuntu1-0ubuntu0.24.09` |
-| 1.81 Jammy | 1.81 Focal | `1.81.0+dfsg0ubuntu1~bpo0-0ubuntu0.22.03` | `1.81.0+dfsg0ubuntu1~bpo0-0ubuntu0.20.03` |
+| 1.93 Devel | 1.93 Noble | `1.93.0+dfsg-0ubuntu1` | `1.93.0+dfsg-0ubuntu1.24.04` |
+| 1.89 Noble | 1.89 Jammy | `1.89.0+dfsg~24.04.1-0ubuntu0.24.04.2 ` | `1.89.0+dfsg~24.04.1-0ubuntu0.22.04` |
 
-As you can see, we leave everything untouched except for the addition of the decremented release number at the very end.
-
-Make the changelog entry description something like this:
+Make the initial changelog entry description something like this:
 
 ```none
   * Backport to <release> (LP: <lp_bug_number>)
 ```
 
+The part with the Launchpad bug number is optional but should be included if an applicable bug exists.
+
 (rust-generating-the-orig-tarball)=
 ### Generating the orig tarball
 
 ```{include} common/uscan.md
 
 ```
 
-If you've had to vendor LLVM or `libgit2`, add the {ref}`relevant ~bpo <rust-version-strings>` to the end of the orig tarball's version number too.
+### Generating the orig-vendor tarball
+
+```{include} common/orig-vendor.md
+
+```
 
 ```{include} common/local-build.md
 
diff --git a/docs/maintainers/niche-package-maintenance/rustc/common/local-repo-setup.md b/docs/maintainers/niche-package-maintenance/rustc/common/local-repo-setup.md
@@ -16,10 +16,11 @@ rustc
 │   ├── Cargo.toml
 │   ├── debian
 │   └── [...]
-└── rustc-<...>.orig.tar.xz
+├── rustc-<...>.orig.tar.xz
+└── rustc-<...>.orig-vendor.tar.xz
 ```
 
-Naturally, your higher-level `rustc` directory won't have any {term}`.orig.tar.xz <orig tarball>` files yet, but they will be stored there once you start working on the package.
+Naturally, your higher-level `rustc` directory won't have the {term}`.orig.tar.xz <orig tarball>` files yet, but they will be stored there once you start working on the package.
 
 ### Cloning the Git repository
 
diff --git a/docs/maintainers/niche-package-maintenance/rustc/common/orig-vendor.md b/docs/maintainers/niche-package-maintenance/rustc/common/orig-vendor.md
@@ -0,0 +1,40 @@
+For Rust versions 1.89 and later, we use a separate "vendor tarball" for the filtered contents of the upstream `vendor` directory. The `vendor` directory contains the source code for all external crates that the Rust toolchain depends on. This filename for this separate tarball ends in `.orig-vendor.tar.xz`. 
+
+The vendor tarball needs to be rebuilt any time that we make a change that affects which crate versions are needed, or when working on a new upstream Rust version. Similar to the main "orig tarball", the vendor tarball is saved as a package artifact on the Ubuntu Archive or PPA, and it can be downloaded from Launchpad, if no such changes are needed.
+
+To rebuild the vendor tarball, first replace the `vendor` directory with the unfiltered upstream version. If `uscan` has been run previously for this Rust version, the parent directory should contain a file `rustc-<X.Y.Z>-src.tar.xz`. Extract this into a temporary location, then copy over the `vendor` directory:
+
+```bash
+~/rustc/rustc$ cd ..
+~/rustc$ tar xf rustc-<X.Y.Z>-src.tar.xz
+~/rustc$ rm -rf rustc/vendor/
+~/rustc$ cp -ra rustc-<X.Y.Z>-src/vendor/ rustc/
+```
+
+It is expected that you have a copy of the toolchain installed locally (e.g. using `rustup`), with toolchain version matching the version of the toolchain you are packaging. When invoking commands using `debian/rules`, it will expect the environment variable `RUST_BOOTSTRAP_DIR` to point to the sysroot of the local toolchain, which you can locate using `rustup`. For example, if packaging version 1.92.0 of Rust, you can run
+
+```bash
+~/rustc/rustc$ rustup install 1.92.0
+~/rustc/rustc$ rustup +1.92.0 which rustc
+/home/<user>/.rustup/toolchains/1.92.0-x86_64-unknown-linux-gnu/bin/rustc
+~/rustc/rustc$ export RUST_BOOTSTRAP_DIR=/home/<user>/.rustup/toolchains/1.92.0-x86_64-unknown-linux-gnu
+```
+
+The process of filtering the vendored crates relies on a custom cargo subcommand [`cargo-vendor-filterer`](https://github.com/coreos/cargo-vendor-filterer), which we will run indirectly via the `vendor-tarball` target of `debian/rules`. You will first need to install the `cargo-vendor-filterer` binary:
+
+```bash
+cargo install cargo-vendor-filterer
+```
+
+Then run the `debian/rules` script using the `vendor-tarball` target:
+
+```bash
+~/rustc/rustc$ debian/rules vendor-tarball
+```
+
+When it finishes, there should be a file in the parent directory named `rustc-<X.Y>_<version>.orig-vendor.tar.xz`, which is the vendor tarball. Extract it to replace the unfiltered `vendor` directory with the filtered version:
+
+```bash
+~/rustc/rustc$ rm -rf vendor
+~/rustc/rustc$ tar xf ../rustc-<X.Y>_<version>.orig-vendor.tar.xz
+```
diff --git a/docs/maintainers/niche-package-maintenance/rustc/common/uscan.md b/docs/maintainers/niche-package-maintenance/rustc/common/uscan.md
@@ -1,13 +1,17 @@
-We can use {manpage}`uscan(1)` (from the {lpsrc}`devscripts` package) to get the new source code and generate the new {term}`orig tarball`.
+We can use {manpage}`uscan(1)` (from the {lpsrc}`devscripts` package) to generate the new {term}`orig tarball`. This is the upstream source code for the Rust toolchain, filtered to exclude any files listed in `Files-Excluded` of the `debian/copyright` file. Files can be excluded for various reasons:
 
-The log of this should be saved somewhere because `uscan` will warn you if any files you've excluded from `debian/copyright` aren't actually in the original source. You must consult the upstream Rust changes and see what happened to that file, updating `debian/copyright` accordingly depending on if it was removed, renamed, or refactored.
+- They are vendored dependencies (e.g. `src/gcc`) where we prefer to instead use a system version.
+- They are only relevant for executing on platforms unsupported by Ubuntu (e.g. Windows).
+- They apply to unstable Rust features which we do not yet support, e.g. `src/tools/enzyme`.
 
-Download the orig tarball from the upstream Rust source, yanking out all excluded files:
+The orig tarball needs to be rebuilt anytime we make a change to the `Files-Excluded` or when working on a new upstream Rust version. If no such changes are made, the orig tarball can instead be downloaded from Launchpad; it will be listed as a file ending in `.orig.tar.xz` under the "Package files" for the package, either on the Ubuntu Archive or on the PPA to which it was uploaded (e.g. the ["Rust Toolchain" Staging PPA](https://launchpad.net/~rust-toolchain/+archive/ubuntu/staging/+packages)).
+
+To rebuild the orig tarball, run `uscan` while saving its log somewhere:
 
 ```none
 $ uscan --download-version <X.Y.Z> -v 2>&1 | tee <path_to_log_output>
 ```
 
-This process can take a while. Once it is complete, you will find a file with an `.orig.tar.xz` suffix in your parent `rustc` directory. That is your orig tarball. It contains the new upstream source code for the new Rust version.
+The output will will warn you if any files you've excluded from `debian/copyright` aren't actually in the original source. You must consult the upstream Rust changes and see what happened to that file, updating `debian/copyright` accordingly depending on if it was removed, renamed, or refactored.
 
-You must then rename the orig tarball to match the first part of your package version number, i.e., `rustc-<X.Y>_<X.Y.Z>+dfsg0ubuntu0`.
+This process can take a while. Once it is complete, you will find a file with an `.orig.tar.xz` suffix in your parent `rustc` directory. You must then rename it to match the part of your package version number before the first hyphen, e.g., if your package name is `rustc-1.89` and package version is `1.89.0+dfsg-0ubuntu1`, you would rename the orig tarball to `rustc-1.89_1.89.0+dfsg.orig.tar.xz`.
