Evaluate whether we can make miri a rustc backend

[oli-obk]

Right now miri is a rustc driver, but we may be able to make miri a codegen backend like llvm and cranelift. Not sure what benefits this brings, but it seems to me like miri behaves more like a backend than anything else. The miri binary would then become just a thin wrapper invoking rustc with a bunch of flags.

[bjorn3]

Disadvantages

• You have to provide your own metadata loader for rlibs and dylibs (CodegenBackend::metadata_loader). Feel free to copy https://github.com/bjorn3/rustc_codegen_cranelift/blob/169140506366d5aa96967a71b6af98563a9c2988/src/metadata.rs#L1-L59 though.
• You have to provide the target feature whitelist yourself by providing the target_features_whitelist for LOCAL_CRATE. (The default list should probably be moved out of the codegen backend though)

Advantages

• You can decide which target features are presented to the rust code using #[cfg(target_feature = "...")] by overriding the target_features method of CodegenBackend, or using the default of none. This means you have to worry less about supporting all SIMD intrinsics.

[RalfJung]

Could we still have all Miri-specific code in this separate repo? I think that helps with making Miri easier to work on.

You can decide which target features are presented to the rust code using #[cfg(target_feature = "...")] by overriding the target_features method of CodegenBackend, or using the default of none. This means you have to worry less about supporting all SIMD intrinsics.

That would be nice indeed! Is there no way to get that effect with a driver? (Or maybe we could have a minimal backend on the rustc side that does just that, but still otherwise keep e.g. all the shims and Stacked Borrows separately in this repo.)

[bjorn3]

Could we still have all Miri-specific code in this separate repo? I think that helps with making Miri easier to work on.

Yes, all you need to do is compile miri as a dylib exporting the function

/// This is the entrypoint for a hot plugged rustc_codegen_cranelift
#[no_mangle]
pub fn __rustc_codegen_backend() -> Box<dyn CodegenBackend> {
    Box::new(MiriCodegenBackend)
}

then implement rustc_codegen_ssa::traits::CodegenBackend for the unit struct MiriCodegenBackend. Finally to use it you pass -Zcodegen-backend=/path/to/miri.so to rustc.

Is there no way to get that effect with a driver?

Not that I know of. create_session already adds the target features:

https://github.com/rust-lang/rust/blob/40008dcb494a00571123b7d59115068a200ebe34/src/librustc_interface/util.rs#L77

Once that function returns, Session can no longer by modified. (except using unsafe of course, but that would be a stacked borrow violation)

---

I just thought about another disadvantage that would also be nice to solve for cg_clif: There is no way to accept options that are not already recognized by rustc. In cg_clif, I use env vars as workaround, but it would be nice if it could be real options behind for example a new -X flag.

[BatmanAoD]

Would this make it easier to use MIRI as a general-purpose Rust interpreter? I'm wondering if that might be a path toward providing a faster debug/test cycle than Cranelift.

[oli-obk]

I don't think this has any effect on the usability of miri, all we do is shuffle around a few things in the parts of miri that the user doesn't see.

[RalfJung]

I'm wondering if that might be a path toward providing a faster debug/test cycle than Cranelift.

Given how slow Miri is, I don't think Miri can fill that niche, even for those libraries/applications where Miri supports all the APIs they use.

[bjorn3]

@BatmanAoD I have got a WIP branch for cg_clif that lazily compiles functions as necessary in JIT mode: https://github.com/bjorn3/rustc_codegen_cranelift/tree/wip_lazy_jit.

[BatmanAoD]

Thanks all! I understand. @bjorn3 that sounds really promising!