Auto Filling Infrastructure

[Lxiamail]

Work items:

• Detection of regressions in app sizes across all measured platforms (in addition to the existing throughput regressions)
• Issues auto-filed into the fork repository for triage during the weekly triage meeting
• That weekly triage meeting being able to incorporate app size regression issue triage into the existing workflow
• Legitimate app size regression issues being set up for easy transfer into the runtime repo with actionable details in place
• Add test project (preview2)
• Add way to add test data from file instead of table
• Add end to end test
• Runtime integration (preview3)
• Add bot account to runtime repo
• Add tests to ensure max number of filed issues

[jeffhandley]

@DrewScoggins Is it correct that the first item ("Testing framework initial bring up") would unblock the ability for us to contribute the "contract tests" that we had talked about @adamsitnik working on?

[DrewScoggins]

Yes, that is entirely correct.

[jeffhandley]

@Lxiamail I wanted to capture a couple additional notes on this one based on our conversation last week.

Turn on auto-filing for all scenario tests (preview3)

From the Libraries' team perspective, the definition of done for this would include:

• Detection of regressions in app sizes across all measured platforms (in addition to the existing throughput regressions)
• Issues auto-filed into the fork repository for triage during the weekly triage meeting
• That weekly triage meeting being able to incorporate app size regression issue triage into the existing workflow
• Legitimate app size regression issues being set up for easy transfer into the runtime repo with actionable details in place

We are already a couple months into our trimming work and we have some notable features delivering in Preview 3+ where potential for app size regression is high and we'll need to be able to detect that through the auto-filing, so having that in place for Preview 3 is important (and more important than the testing framework item at this point).

Thanks!

[Lxiamail]

@jeffhandley Thanks Jeff! We will take it as high priority in preview3