Add a way to cancel queued perf run

[klensy]

Hi.

There should be the way to cancel queued perf run - for example, if we accidentally queued one task few times, or if build fails.

For example: rust-lang/rust#83207 here task was queued, but build failed. Build was fixed, and perf queued again, so it was executed 2 times.