[GreenDragon] Duplicate and templatize clang-stage2-Rthinlto for bisection

[jroelofs]

The new approach modularizes and templatizes the job so that we can automatically bisect build failures down to the commit responsible for them. The new jobs written in this style use the name-inspecting symlink trick that clang itself uses to determine driver mode, for organizational simplicity and deduplication of configuration. For now, I am duplicating the job wholesale, until we're comfortable with replacing all of the existing clang-* jobs with this style.

As part of this, we're adding a templatized bisection job and corresponding runner. Currently, these cooperate to bisect the clang-stage2-Rthinlto-v2 job's failures, but going forward we should be set up to re-use that for any other job's bisection needs. We will also be well set up to add a test bisection runner, which could leverage the artifacts saved to S3 as a caching mechanism to speed up the search.

[tomershafir]

Initial thoughts:

• The pipeline declarations in pipeline() and createTemplatedPipeline() are very similar. Given that we want to enable composability, consistency and reusability, maybe we should parametrize both into a single pipeline.
• Similarly, I think we should align the design of bisection to the one of Clang jobs. Extract a parametrized BisectBuilder that is called post a Clang job. The builder would create the trigger pipeline (build-bisect), embed the utils, and possibly call a parametrized ClangBuilder (instead build-bisect-run). Currently, the call graph is: Clang job -> build-bisect -> bisect utils -> build-bisect-run which can be simplified.

[jroelofs]

Initial thoughts:

• The pipeline declarations in pipeline() and createTemplatedPipeline() are very similar. Given that we want to enable composability, consistency and reusability, maybe we should parametrize both into a single pipeline.

Agreed. There is quite a bit of duplication along that axis. I'll clean that up.

• Similarly, I think we should align the design of bisection to the one of Clang jobs. Extract a parametrized BisectBuilder that is called post a Clang job. The builder would create the trigger pipeline (build-bisect), embed the utils, and possibly call a parametrized ClangBuilder (instead build-bisect-run). Currently, the call graph is: Clang job -> build-bisect -> bisect utils -> build-bisect-run which can be simplified.

I deliberately chose to arrange the jobs that way so that:

• job instances from bisection don't contaminate the timeline for the main job, and
• bisection as a high-level task has its own run in its own job.

... and later on, we can add a parallel test-bisect / test-bisect-run for bisection of test failures, as a distinct task from bisection of build failures.

[tomershafir]

I think we can still extract a BisectBuilder and have a thin build-bisect job that consumes it, which is self contained and not contaminating. Then later we can add a hook such that a test-bisect job would reuse this builder as a thin consumer. Similarly to the Clang infrastructure.