|
| 1 | ++++ |
| 2 | +title = "Fixing our own problems in the Rust compiler" |
| 3 | +slug = "fixing-our-own-problems-in-the-rust-compiler" |
| 4 | +authors = ["Folkert de Vries"] |
| 5 | +date = "2026-03-30" |
| 6 | + |
| 7 | +[taxonomies] |
| 8 | +tags = ["bzip2-rs", "zlib-rs", "zstd-rs", "data compression"] |
| 9 | + |
| 10 | ++++ |
| 11 | + |
| 12 | +In our data compression projects, we use Rust where C is traditionally used. During the work, we've hit limitations in Rust itself and in the surrounding tooling. Over the years, we've become increasingly comfortable with fixing these issues ourselves. |
| 13 | + |
| 14 | +<!-- more --> |
| 15 | + |
| 16 | +Previously, we felt stuck at times by missing functionality in stable Rust, without a clear path forward except to wait. In practice, waiting has not been a fruitful strategy: the features we need are niche and rarely make it to the top of Rust maintainers' to-do lists. |
| 17 | + |
| 18 | +In this post, I'll share some of the steps we took to get unstuck. It goes over some of the fixes and improvements that we've made as a part of Trifecta Tech's [Data compression initiative](/initiatives/data-compression) (`zlib-rs`, `libbzip2-rs` and `libzstd-rs-sys`) over the past year. |
| 19 | + |
| 20 | +In the same spirit, I'll give a talk about **["Stabilizing decade-old features"](https://2026.rustweek.org/talks/folkert/)** at RustWeek, with advice on how to start making these kinds of contributions yourself. |
| 21 | + |
| 22 | +### Contributions to `clippy` |
| 23 | + |
| 24 | +We often use [c2rust](https://github.com/immunant/c2rust), an automatic translation tool that converts C to Rust. The Rust it produces is behaviorally equivalent to the C code, but far from idiomatic. |
| 25 | + |
| 26 | +We get a lot of value out of clippy not only flagging issues, but increasingly being able to fix them. We often run commands like the one below to only fix one or a couple of lints at a time, and bundle those in a commit that is easier to review than just fixing all lints at once. |
| 27 | + |
| 28 | +``` |
| 29 | + cargo clippy --fix -- --allow clippy::all --warn clippy::manual_div_ceil |
| 30 | +``` |
| 31 | + |
| 32 | +Sometimes the code that `c2rust` produces is so cursed that applying the `clippy --fix` produces invalid code. In [https://github.com/rust-lang/rust-clippy/pull/15304](https://github.com/rust-lang/rust-clippy/pull/15304) we fixed a bug where parentheses would cause `clippy::collapsible_if` to apply an incorrect suggestion: |
| 33 | + |
| 34 | +``` |
| 35 | +warning: this `if` statement can be collapsed |
| 36 | + --> src/main.rs:6:5 |
| 37 | + | |
| 38 | + 6 | / if true { |
| 39 | + 7 | | (if true { |
| 40 | + 8 | | () |
| 41 | + 9 | | }) |
| 42 | +10 | | } |
| 43 | + | |_____^ |
| 44 | + | |
| 45 | + = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_if |
| 46 | + = note: `#[warn(clippy::collapsible_if)]` on by default |
| 47 | +help: collapse nested if block |
| 48 | + | |
| 49 | + 6 ~ if true |
| 50 | + 7 ~ &&f true { |
| 51 | + 8 | () |
| 52 | + 9 ~ }) |
| 53 | + | |
| 54 | +``` |
| 55 | + |
| 56 | +In other cases we spot a rewrite that isn't covered by an existing lint, but that does seem generally useful. For instance, the `c2rust` output contains lots of pointer arithmetic, like this: |
| 57 | + |
| 58 | +```rust |
| 59 | +foo.offset(4) |
| 60 | +``` |
| 61 | + |
| 62 | +The `ptr_offset_by_literal` lint, added in [https://github.com/rust-lang/rust-clippy/pull/15606](https://github.com/rust-lang/rust-clippy/pull/15606), rewrites this to: |
| 63 | + |
| 64 | +```rust |
| 65 | +foo.add(4) |
| 66 | +``` |
| 67 | + |
| 68 | +Code using only `ptr::add` and not `ptr::sub` is much easier to convert to using slices. |
| 69 | + |
| 70 | +Along the way it turned out that the implementation of the related `ptr_offset_with_cast` lint, on which I heavily based my initial implementation, had accumulated some technical debt. My changes fell right in the middle of the [clippy feature freeze](https://blog.rust-lang.org/inside-rust/2025/06/21/announcing-the-clippy-feature-freeze/), so it was a perfect moment to fix that up too in [https://github.com/rust-lang/rust-clippy/pull/15613](https://github.com/rust-lang/rust-clippy/pull/15613). |
| 71 | + |
| 72 | +## Contributions to Miri |
| 73 | + |
| 74 | +### Improved intrinsic support |
| 75 | + |
| 76 | +We use Miri to test the unsafe code that we still have. I recently wrote about our work on [emulating avx-512 intrinsics in Miri](https://trifectatech.org/blog/emulating-avx-512-intrinsics-in-miri/). Since then we've added support for a couple additional instructions for our `avx512vnni` implementation of adler32. |
| 77 | + |
| 78 | +My next goal is to also be able to run the zlib-rs AArch64 SIMD tests with Miri. For now that'll need some additional support in Miri itself, but with LLVM 23 we'll be able to only use portable intrinsics and the custom Miri support should no longer be needed. |
| 79 | + |
| 80 | +I want to stress that the Miri implementation is really only half the work. The other half is testing the implementation (sometimes giving rise to improving tests in e.g. `rust-lang/stdarch` as well), to make sure that the behavior is correct and that support never regresses. |
| 81 | + |
| 82 | +### ICE when reading from a static array of function pointers |
| 83 | + |
| 84 | +In the first week of working on [libzstd-rs-sys](https://github.com/trifectatechfoundation/libzstd-rs-sys) we ran into `c2rust` producing some Rust code that Miri was unable to handle. |
| 85 | + |
| 86 | +[https://github.com/rust-lang/miri/issues/4501](https://github.com/rust-lang/miri/issues/4501) |
| 87 | + |
| 88 | +This (cursed) static initialization code threw an internal compiler error in Miri. This was fixed by Ralf Jung. |
| 89 | + |
| 90 | +```rust |
| 91 | +unsafe extern "C" fn run_static_initializers() {} |
| 92 | + |
| 93 | +#[used] |
| 94 | +#[cfg_attr(target_os = "linux", unsafe(link_section = ".init_array"))] |
| 95 | +#[cfg_attr(target_os = "windows", unsafe(link_section = ".CRT$XIB"))] |
| 96 | +#[cfg_attr(target_os = "macos", unsafe(link_section = "__DATA,__mod_init_func"))] |
| 97 | +static INIT_ARRAY: [unsafe extern "C" fn(); 1] = [run_static_initializers]; |
| 98 | + |
| 99 | +fn main() {} |
| 100 | +``` |
| 101 | + |
| 102 | +### Missing support for `libc::memset` |
| 103 | + |
| 104 | +[https://github.com/rust-lang/miri/issues/4503](https://github.com/rust-lang/miri/issues/4503) |
| 105 | + |
| 106 | +Miri supports some `libc` functions, like `memcpy`, but `memset` was missing. Newly `c2rust`-translated code would often hit a call to `memset` and fail early without running (and potentially finding errors in) the majority of the program. Support for `memset` was added by Vishruth Thimmaiah. |
| 107 | + |
| 108 | +## Larger Features |
| 109 | + |
| 110 | +We've also contributed substantially to some larger features that we believe Rust should have to be an effective systems programming language. Note that these bigger features are never solo projects, they contain the (indirect) work of many contributors. |
| 111 | + |
| 112 | +### `#![feature(cfg_select)]` |
| 113 | + |
| 114 | +[https://github.com/rust-lang/rust/issues/152944](https://github.com/rust-lang/rust/issues/152944) |
| 115 | + |
| 116 | +A much nicer way to write configuration predicates: |
| 117 | + |
| 118 | +```rust |
| 119 | +cfg_select! { |
| 120 | + unix => { |
| 121 | + fn foo() { /* unix specific functionality */ } |
| 122 | + } |
| 123 | + target_pointer_width = "32" => { |
| 124 | + fn foo() { /* non-unix, 32-bit functionality */ } |
| 125 | + } |
| 126 | + _ => { |
| 127 | + fn foo() { /* fallback implementation */ } |
| 128 | + } |
| 129 | +} |
| 130 | + |
| 131 | +let is_unix_str = cfg_select! { |
| 132 | + unix => "unix", |
| 133 | + _ => "not unix", |
| 134 | +}; |
| 135 | +``` |
| 136 | + |
| 137 | +We already use a custom version of this macro in some of our crates, but in rust 1.95 it will finally be available from std on stable Rust. I worked on implementing this macro as a built-in macro in the Rust compiler, and getting it through the stabilization process. |
| 138 | + |
| 139 | +### `#![feature(c_variadic)]` |
| 140 | + |
| 141 | +[https://github.com/rust-lang/rust/issues/44930](https://github.com/rust-lang/rust/issues/44930) |
| 142 | + |
| 143 | +Rust can call c-variadic functions (like `libc::printf`), but defining them is unstable. This feature is required for completing the zlib-rs C api where we have these functions: |
| 144 | + |
| 145 | +```rust |
| 146 | +pub unsafe extern "C" fn gzprintf( |
| 147 | + file: gzFile, |
| 148 | + format: *const c_char, |
| 149 | + va: ...) -> c_int { |
| 150 | + unsafe { gzvprintf(file, format, va) } |
| 151 | +} |
| 152 | + |
| 153 | +unsafe extern "C" fn gzvprintf( |
| 154 | + file: gzFile, |
| 155 | + format: *const c_char, |
| 156 | + va: core::ffi::VaList, |
| 157 | +) -> c_int { |
| 158 | + /* ... */ |
| 159 | +} |
| 160 | +``` |
| 161 | + |
| 162 | +My work on this feature started after the 2025 all-hands. Like any large feature this is a team effort, where over this past year we've [figured out ABI-compatibility](https://github.com/rust-lang/rust/pull/144529) and most recently [added const evaluation support](https://github.com/rust-lang/rust/pull/150601). I'm currently working on updating the [reference](https://github.com/rust-lang/reference/pull/2177), and plan to open a stabilization PR soon. |
| 163 | + |
| 164 | +## Conclusion |
| 165 | + |
| 166 | +When you do cursed things, problems find you. |
| 167 | + |
| 168 | +The interplay between low-level systems work and compiler development has been very fruitful for us. It has been enormously gratifying to be able to tackle some of these problems ourselves, and we have lots of exciting things lined up for 2026. |
| 169 | + |
| 170 | +*These contributions to the compiler resulted from Trifecta Tech Foundation's [Data compression initiative](/initiatives/data-compression). Please [contact us](/support) if you are interested in financially supporting this work.* |
0 commit comments