Macros testing

[pellico]

I have the following top level question:
Is it possible to validate and verify a macro and then use the macro in my code without any worries ?

Procedural Macros
I don't think it is possible to prove that any procedural macro is correctly generate code for any input.
A procedural macro by manipulating the AST is almost as powerful as a compiler.
Usually compiler safety manual requires to test the generated code because it is not possible to prove the correctness of generated code for any input program. Therefore I see the following top level direction:

1. Require that all code generated by the macro call is tested following the same rules (depends on safety standard) as regular Rust code. (I read somewhere that the code generated by cargo expand of procedural macro can be compiled as regular code)
2. Restrict the usage of procedural macro until we are quite sure that the macro is correct and there is no need to test generated code.

Declarative Macros
I have the feeling that it should be possible in principle to validate and verify because they are not very powerful and they more a template.

[cpetig]

If I recall correctly procedural macros are compiled to shared libraries (proc-macro crate type) which are loaded into rustc, so using code coverage tools to gauge test coverage on proc macros could be possible. This came up as an idea with macros compiled to wasm, but it should work in principle with the current implementation as well.

We probably need a compiler flag to compile macros with -C instrument-coverage, then compile a test crate exercising the macro and dumping coverage files. Ideally if the coverage in the safety crates is a subset of the test crate coverage, your testing should cover the needed functionality.

[felix91gr]

In broad strokes, I think widely used Procedural Macros should perhaps be audited, vetted to be correct, and then pinned to a version that we know is OK for Safety Critical contexts.

Maybe some kind of Formal Verification tool could help streamline this, so that proofs of correctness can be automatically checked and then updated over time.

But yeah, I don't see any other way to vet the correctness of a Proc Macro other than these, because as you said PMs are almost as powerful as the compiler.

[felix91gr]

Regarding Declarative Macros, I think we share the same feeling.

I also think that making a tool that at least detected some dangerous uses of declarative macros should in principle be possible. Clippy has one lint that I know of that does this: macro_metavars_in_unsafe.

I still need a lot more experience with declarative macros to tell whether or not this is in general, true, though.