Usage question: a small test with Botan results in a lot of warnings

[lieser]

In https://github.com/lieser/botan/tree/pl/test-microwalk I'm currently trying to setup a small test Microwalk CI for Botan.

I however get a lot of warnings (https://github.com/lieser/botan/actions/runs/4701855851/jobs/8338408892).

A few trace messages similar to:

[dbug] [trace:pin] Pin tool log: Error: Cannot resolve image of basic block 7fa6a0631220 (instrumenting as 'interesting')

A lot of warnings similar to:

[warn] [preprocess:prefix] Free of address 00005629e46ad420 does not correspond to any heap allocation, skipping

A massive amount of message similar to

[warn] [preprocess:0] Could not resolve target of memory access 00007fa6a100af38 -> [00005629e46a4a88] (read), skipping
[warn] [preprocess:0] Could not resolve target of memory access 00007fa6a0f0070e -> [00005629e4697170] (write), skipping

For a pretty small test the complete log is over 1.000.000 lines long, which does not seem normal to me.

Would be nice if you can give me a hint on what is going wrong. Let me know if I should open an issue in the main repository for this usage question.

(Note: this was split from microwalk-project/microwalk-pin-action#1)

[JanWichelmann]

I apologize for the delay in answering - I wanted to get to the bottom of the issue first. I've now pushed a number of commits that solve some of the problems causing the aforementioned warnings, and generally improve compatibility to C++ programs.

Pin tool log: Error ...

That message is a bit misleading, and it can be safely ignored. This is caused by Intel Pin executing portions of libc before it tells Microwalk that the library is loaded, and the message is mostly a sanity check. I've changed the wording appropriately.

Free of address ...

This was caused due to a tail call in calloc, which the compiler converted into a jmp and thus led Microwalk to miss the allocation. I've made some updates to the Pin tool to catch such cases, so the warning should no longer appear (well, there may still be a handful instances, but you can safely ignore those).

Could not resolve target ...

Many of those were caused by the missing heap allocations above, but not all. I've done some more testing with Botan, and it seems that the C++ runtime uses a memory region that is neither part of the heap nor the stack. I'm still investigating this.

---

Apart from the concrete issues, I noticed that you are using Microwalk for analyzing Botan's blinded EC point multiplication. Blinded implementations use randomness, which is not supported by Microwalk at the moment (i.e., it only yields reliable results for deterministic code).

A workaround is forcing use of processor-based randomness and then setting the return value of rdrand to a constant through the corresponding configuration option, but this may yield lots of false positives by exposing leakage which is normally hidden by blinding.

Instead, you may want to wait until I have added an analysis module that supports randomness, which is something I definitely plan to do in the near future.

[Zhang-Yang-Sustech]

Hello author! I'm a graduate student developer, and I'm also hoping to use this tool to analyze some cryptography libraries primarily written in C/C++. When I analyzed libraries like OpenSSL and libgcrypt, I encountered similar warnings. I'd like to know whether these warnings might affect the correctness of the analysis, as this determines whether my current research results are trustworthy. If you have time, I'd really appreciate your reply to my question. Thank you!

[JanWichelmann]

Hi @Zhang-Yang-Sustech! Can you attach a sample repository and/or the logs you see? This would allow me to investigate what is causing these warnings and whether you can safely ignore them.

[Zhang-Yang-Sustech]

Thank you for your reply! Sorry for my presumptuous. I will attach some pieces.

At first, when I saw the warnings, I thought they could be safely ignored. But after reading your previous comment, I now feel that they might actually affect the results, so I wanted to ask you about it.

[JanWichelmann]

Oh ok, you can ignore those. As the message states, some parts of libc may be instrumented before Pin detects the library load, which leads to some BBs having no parent image. We decided to not silently discard this error, but it is very very unlikely that these parts contain relevant code, as they are usually restricted to dynamic linking and library loading.

[Zhang-Yang-Sustech]

Thanks! If I meet other problem, I will contact you.