Preemptive and/or automated handling of CVEs

[ben-polinsky]

As a library with a large number of dependencies, iTwin.js has frequent PR + releases blocked/delayed by CVEs. Not only can this disrupt normal development and our release cadence, but it takes significant effort from our team to mitigate week after week. (We also suspect the number of CVEs will start to increase as both AI development and AI bug reports become more frequent h/t @tcobbs-bentley.)

An attempted first line of defense has been Dependabot. This alerts us to some vulnerabilities and attempts to resolve. It fails almost always during backports due to failed cherry-picks. Other times, it isn't on top of CVEs picked up by Mend.

We've discussed running rush update --full more often which updates all dependencies to the latest version compatible with semver.

Open to ideas and suggestions on how to relieve some of the burden on our team.

List of ideas:

• run rush update --full via automation (commit back to repo) or require as part of PR workflow.
• Look into Github Copilot "repository tasks" https://github.blog/ai-and-ml/automate-repository-tasks-with-github-agentic-workflows/

[tcobbs-bentley]

I think that rush update --full hasn't been run in a long time, so perhaps step one is to do that in a PR of its own.

[ben-polinsky]

Months ago, I attempted to engage with AppSec about some of Mend's offerings (specifically remediate and removal) which solve this issue (some of which seems available to us at our current subscription level). They didn't seem too interested, but I'll follow up. Mend advertises solutions for our exact case: https://www.mend.io/code-scanning/

It would be good if we could quantify the number of person-hours this takes us a week, on average.

[anmolshres98]

maybe rush update --full can be run with the nightly builds?

[tcobbs-bentley]

maybe rush update --full can be run with the nightly builds?

I think that's too likely to cause nightly builds to fail (for example due to lint rules failing after an update). I could be wrong, though.

[hl662]

I think that rush update --full hasn't been run in a long time, so perhaps step one is to do that in a PR of its own.

This is our first line of defense when we get cve alerts in core. For a long time, we were unable to do so, but I've fixed that issue manually back in December: #8861

Since then, we've been able and I believe we are using rush update --full whenever we can.

maybe rush update --full can be run with the nightly builds?

Having an automated audit check like this is good, but in my experience:

it depends... some dependencies don't really adhere to semver, like eslint rules (that's why I had to make a PR to manually fix eslint rules). If eslint is patched and a new rule comes in that would cause an error, then our automated pipelines would fail and no nightly release. Thats what caused our rush update --full to collapse last year, requiring manual intervention.

So, if we want to introduce some form of automation, we'll need to also setup the automation so it can alert us maintainers when it can't resolve things by itself.

[ben-polinsky]

we are using rush update --full whenever we can.

What does this mean in practice?

[hl662]

What does this mean in practice?

In context of cves, we run rush update --full when the cve is found in a transient dependency.

• For example: This audit agent I made is specific to itwinjs core and I gave it instructions on how to resolve different dependencies, including transient deps. And if it can't resolve on its own, an explicit instruction was given to notify the agent invoker what the issue is.

An attempted first line of defense has been Dependabot. This alerts us to some vulnerabilities and attempts to resolve. It fails almost always during backports due to failed cherry-picks.

I previously wrote a skill.md that I believe would help, I haven't circled back to keep working on this yet. As you mentioned, failed cherry-picks are an issue with both dependabot and mergifyio bots - so I gave these skills instructions on how to resolve cherry picks we commonly face in our repo. The idea is this skill can be used to invoke an agent that can fix cherry pick conflicts from mergifyio backports, or we can forego mergifyio altogether and invoke a custom agent that can backport a PR and resolve conflicts on its own. (Our own mergify bot). We could use natural language instructions to tell the agent to backport a PR across many different branches, rather than having to always follow a rigid structure Mergifyio backport <branch> and make the comment multiple times

[ben-polinsky]

In context of cves, we run rush update --full when the cve is found in a transient dependency.

This is reactive, though. From my understanding, CVEs are discovered and not publicly disclosed. They give the library maintainer time to add a fix and often wait until the end of a specified waiting period (30; 45 days). So the mitigations are often available before the CVE announcement.