ISO 26262 and Rust Gap Analysis for the front-page to evaluate Rust safety-critical readiness

[fried-gluttony]

This pull request should update the main website after clicking the button "find out more".

I imagined a typical Functional Safety Manager that has to decide whether it's viable to use Rust for an project and what he has to know to make that decision.

My suggestion is to give on the first page a small overview, to get everyone on the same track on what safety-critical is and how to judge the readiness of Rust.
Next I propose to give an overview using a simple traffic-light score in terms of how much additional effort to expect when using Rust for safety critical - depending on different standards - purely from a standards-conformity perspective. That means it doesn't account for all the savings due to the awesome language. I guess that still needs to be stated for fair comparison.

It is not yet ripe, I just wanted to outline the methodology and give some examples. Would love any feedback - is it well structured and understandable?

[netlify]

✅ Deploy Preview for safety-critical-rust-consortium ready!

Name	Link
🔨 Latest commit	6538224
🔍 Latest deploy log	https://app.netlify.com/projects/safety-critical-rust-consortium/deploys/69b87174ab024500088aabfb
😎 Deploy Preview	https://deploy-preview-562--safety-critical-rust-consortium.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[DaleEMartin]

There are some misspellings in the new text, it would be good to run it through a spell-checker. For this kind of document I don't know if there is a github workflow for auto spelling and grammar checks?

[fried-gluttony]

@DaleEMartin Thanks for checking it out!
You are right - the text is not yet spellchecked or in best quality, I should've checked that before.

Could you comment a bit on the content:

• was it clear to understand?
• is the methodology good to give an overview?
• anything else that comes up?

I wanted to gather first some comments through this PR on the overall structure, before investing more time to perfect it.

[PLeVasseur]

Thanks for putting this up, @fried-gluttony!

The target audience (Functional Safety Manager deciding if Rust is viable) and the traffic-light framing are both good fits. Main blocking issue is a docs build failure in iso26262.md line 29 (Option'<T'> parsed as invalid JSX in MDX).

Beyond that, several ratings appear overly optimistic or technically inverted (notably concurrency guarantees, "disallow safe rust", process methods marked n.A., and requirements-based testing claims).

[PLeVasseur]

There are some misspellings in the new text, it would be good to run it through a spell-checker. For this kind of document I don't know if there is a github workflow for auto spelling and grammar checks?

This should help: #566

It's the same typo checker we're using on the coding guidelines repo.

[felix91gr]

I see some items where you've pointed out there are things you don't know yet. I'm going to go through this to help with what I can. Please gimme some time, I might be able to do this today. If you don't hear from me before Sunday, feel free to ping me @fried-gluttony.

BTW I love what I'm seeing so far :)

[felix91gr]

Table 6, point 1a One entry and one exit point in subprograms and functions, I have some questions.

Do we know what the reasoning is behind this rule? I have the feeling that this might be very C-specific, in the sense that it's trying to address a very important shortcoming of the C language. If that's the case, then we could perhaps make an argument for this not being necessary for Safety Critical Rust in practice.

[felix91gr]

Table 6 again 1b No dynamic objects or variables, or else online test during their creation

I believe this should be enforceable at compile-time by configuring the project to not have any form of allocation. Lemme get back to you on that one, but if memory serves, then this should have been already solved through compile-time enforced configuration

[felix91gr]

Quick note on Table 6 1j No recursions: we have a guideline in place for that https://coding-guidelines.arewesafetycriticalyet.org/coding-guidelines/associated-items/gui_ot2Zt3dd6of1.html

There is however, not a clippy lint for that. But there's hope that for this year we will have one that's good enough for most cases!

[felix91gr]

This review has a bit of everything. I hope it helps :)

[fried-gluttony]

Table 6, point 1a One entry and one exit point in subprograms and functions, I have some questions.

Do we know what the reasoning is behind this rule? I have the feeling that this might be very C-specific, in the sense that it's trying to address a very important shortcoming of the C language. If that's the case, then we could perhaps make an argument for this not being necessary for Safety Critical Rust in practice.

Yes I agree. Even for C++ I already heard that some disprove exception handling due to this rule. This is a typical case where I suggest to tailor it, as it does more harm than good.
I edited it as such

[vjonaswolf]

I got some comments:

• Figure 2: Take care with copyrights. Even extracts are not allowed without written consent.
• Table 1: 1d: Defensive programming is to my understanding unrelated to Option and Result. Non-Null helps, but to me the most important thing with Rust that supports defensive programming are the range checks when accessings slices and vec's. And the overflow-checks (even they are typically only enabled in debug builds).
• 6.5 General Considerations: I would be careful with mentioning specific commercial tools.
• Table 3: 1f: This typically has nothing to do wih available runtime environments like Embassy, RTIC or an OS. The more important part to me here is: In Rust it is very clear when things allocate. This is something that is partially very hard to control in C++, but is essential if you want to give worst-case guarantees.
• Table 6: The giant difference of the ? operator to C++ is exceptions that ? is explicit, while keeping resource cleanup from C++. However, I would still discourage large Rust functions with early returns.
• 6.8 General Conisderations: I would not re-list availability here, but rather reference the Rust tier support.
• Table 9: There is no branch coverage in llvm for Rust. Doesn't work ;)
• Table 10: 1c Fault-injection tests: you can manually do that. I would be more optimistic here.
• Table 10: 1e I don't see why that requires a qualified model generator.
• Table 14: 1b see comment to Table 10 1c

[fried-gluttony]

Awesome, thanks a lot for the review @vjonaswolf . I just edited your remarks and added my opinion below.

I got some comments:

• Figure 2: Take care with copyrights. Even extracts are not allowed without written consent.

Right, I'll just go ahead and remove it for now, I think it would be nice as reference but is not critical. I just also requested the usage of the figure from ISO for the website - let's see what they reply.

• Table 1: 1d: Defensive programming is to my understanding unrelated to Option and Result. Non-Null helps, but to me the most important thing with Rust that supports defensive programming are the range checks when accessings slices and vec's. And the overflow-checks (even they are typically only enabled in debug builds).

Well, it depends I would say on the usage. At the core, defensive programming is a style to cover and check for conditions, that shouldn't happen during intended use and still having defined behavior.
I would say that Option/Result Types aid in that, together with assert!() guards. I agree though that the things you mentioned also aid (arguably more) in that and are even inherent in the language. I additionally mentioned them as:

• 6.5 General Considerations: I would be careful with mentioning specific commercial tools.

Why? I think it assists the safety development and for some points there are no open solutions yet.
A full list of supportive tooling (incl. many commercial tools) is maintained by the subcommittee and I referenced it in the beginning.

• Table 3: 1f: This typically has nothing to do wih available runtime environments like Embassy, RTIC or an OS. The more important part to me here is: In Rust it is very clear when things allocate. This is something that is partially very hard to control in C++, but is essential if you want to give worst-case guarantees.

As for memory this is covered already by Table 6 - 1b. By using no_std you don't have that issue of unpredictable dynamic allocation and it's more explicit - I just added that.
From my developers lens I think the frameworks are also very critical, because they help immensely to not having to write a scheduler etc. from scratch? And RTIC would aid a lot to give these guarantees, looking at more complexe software with more tasks.

• Table 6: The giant difference of the ? operator to C++ is exceptions that ? is explicit, while keeping resource cleanup from C++. However, I would still discourage large Rust functions with early returns.

Changed that

• 6.8 General Conisderations: I would not re-list availability here, but rather reference the Rust tier support.

Which availability do you mean here? Rust tier support?

• Table 9: There is no branch coverage in llvm for Rust. Doesn't work ;)

Oh, I wasn't aware of that. I changed it for now - does maybe Ada have a solution for that?

• Table 10: 1c Fault-injection tests: you can manually do that. I would be more optimistic here.

agreed and edited.

• Table 10: 1e I don't see why that requires a qualified model generator.

You are right, actually a strategy could be to use back-to-back tests to actually prove the equivalence between model and code.

• Table 14: 1b see comment to Table 10 1c

edited.

[felix91gr]

Well, it depends I would say on the usage. At the core, defensive programming is a style to cover and check for conditions, that shouldn't happen during intended use and still having defined behavior.

That is my understanding of the term as well. And to the following bit,

I would say that Option/Result Types aid in that

I would add that:

In general, Rust's type system reduces the need to add defensive programming on top of a project, because programs that type check are, in a sense, "defensive" by construction.

Enums and lifetimes especially make it possible to express correctness conditions in the types of the variables.

[PLeVasseur]

I'm going to focus a bit more on what I think should happen to make the "status" more visible.

I think we should have our landing page working a bit more for us in helping us show: "Are We Safety Critical Yet?"

To that end, I think it makes sense to move all the contents of this page:
https://deploy-preview-562--safety-critical-rust-consortium.netlify.app/docs/intro

To actually be the contents of the front page:
https://deploy-preview-562--safety-critical-rust-consortium.netlify.app/

Now that we have a more rigorous methodology and a first set of work on ISO 26262, I think the more hand-wavey nature of this can be somewhat cofusing towards "where are we?":

I think this is good, and we should keep it:

but, let's move it down further on the page.

@fried-gluttony -- if you're okay with giving this a try I think it'd be good to do in this PR. If not, I wouldn't block your PR on it. I'd figure out how to do it afterwards and then do it myself.

[PLeVasseur]

Thanks so much @fried-gluttony for taking the time and care to support ISO 26262 fleshing out! The front page is also looking rather snazzy too :)