-
Notifications
You must be signed in to change notification settings - Fork 76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add flat flow test functionality to the ReproTool #4559
Open
dkurepa
wants to merge
11
commits into
dotnet:main
Choose a base branch
from
dkurepa:dkurepa/FlatFlowTest
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+647
−390
Open
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
816c23a
Hmm
dkurepa b4e962e
Add new files..
dkurepa eb4d650
changes
dkurepa d886ae0
hmm
dkurepa cecf943
Merge remote-tracking branch 'source/main' into dkurepa/FlatFlowTest
dkurepa 28eb736
Add Tools.Common lib, add flat flow test functionality to ReproTool
dkurepa 24cd5b5
Update tools/ProductConstructionService.ReproTool/Options/FullBackflo…
dkurepa 5b1ae2a
Update tools/ProductConstructionService.ReproTool/Options/ReproOption…
dkurepa 881e852
Address PR feedback
dkurepa 3029637
Merge remote-tracking branch 'origin/dkurepa/FlatFlowTest' into dkure…
dkurepa 485c1b0
Get repo name from DefaultRemote
dkurepa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
68 changes: 68 additions & 0 deletions
68
tools/ProductConstructionService.ReproTool/Operations/FlatFlowTestOperation.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using Maestro.Data; | ||
| using Microsoft.DotNet.DarcLib; | ||
| using Microsoft.DotNet.ProductConstructionService.Client; | ||
| using Microsoft.Extensions.DependencyInjection; | ||
| using Microsoft.Extensions.Logging; | ||
| using Tools.Common; | ||
| using GitHubClient = Octokit.GitHubClient; | ||
|
|
||
| namespace ProductConstructionService.ReproTool.Operations; | ||
|
|
||
| internal class FlatFlowTestOperation( | ||
| VmrDependencyResolver vmrDependencyResolver, | ||
| ILogger<FlatFlowTestOperation> logger, | ||
| GitHubClient ghClient, | ||
| DarcProcessManager darcProcessManager, | ||
| IBarApiClient prodBarClient, | ||
| [FromKeyedServices("local")] IProductConstructionServiceApi localPcsApi) : Operation(logger, ghClient, localPcsApi) | ||
| { | ||
| internal override async Task RunAsync() | ||
| { | ||
| await darcProcessManager.InitializeAsync(); | ||
|
|
||
| var vmrRepos = await vmrDependencyResolver.GetVmrRepositoriesAsync( | ||
| "https://github.com/dotnet/dotnet", | ||
| "https://github.com/dotnet/sdk", | ||
| "main"); | ||
|
|
||
| var vmrTestBranch = await PrepareVmrForkAsync("main", skipCleanup: true); | ||
|
|
||
| var channelName = $"repro-{Guid.NewGuid()}"; | ||
| await using var channel = await darcProcessManager.CreateTestChannelAsync(channelName, true); | ||
|
|
||
| foreach (var vmrRepo in vmrRepos) | ||
| { | ||
| var productRepoForkUri = $"{ProductRepoFormat}{vmrRepo.Mapping.DefaultRemote.Split('/', StringSplitOptions.RemoveEmptyEntries).Last()}"; | ||
| var latestBuild = await prodBarClient.GetLatestBuildAsync(vmrRepo.Mapping.DefaultRemote, vmrRepo.Channel.Channel.Id); | ||
|
|
||
| var productRepoTmpBranch = await PrepareProductRepoForkAsync(vmrRepo.Mapping.DefaultRemote, productRepoForkUri, latestBuild.GetBranch(), false); | ||
|
|
||
| var testBuild = await CreateBuildAsync( | ||
| productRepoForkUri, | ||
| productRepoTmpBranch.Value, | ||
| latestBuild.Commit, | ||
| []); | ||
|
|
||
| await UpdateVmrSourceFiles( | ||
| vmrTestBranch.Value, | ||
| vmrRepo.Mapping.DefaultRemote, | ||
| productRepoForkUri); | ||
|
|
||
| await using var testSubscription = await darcProcessManager.CreateSubscriptionAsync( | ||
| channel: channelName, | ||
| sourceRepo: productRepoForkUri, | ||
| targetRepo: VmrForkUri, | ||
| targetBranch: vmrTestBranch.Value, | ||
| sourceDirectory: null, | ||
| targetDirectory: vmrRepo.Mapping.Name, | ||
| skipCleanup: true); | ||
|
|
||
| await darcProcessManager.AddBuildToChannelAsync(testBuild.Id, channelName, skipCleanup: true); | ||
|
|
||
| await TriggerSubscriptionAsync(testSubscription.Value); | ||
| } | ||
| } | ||
| } | ||
74 changes: 74 additions & 0 deletions
74
tools/ProductConstructionService.ReproTool/Operations/FullBackflowTestOperation.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using Maestro.Data; | ||
| using Microsoft.DotNet.DarcLib; | ||
| using Microsoft.DotNet.ProductConstructionService.Client; | ||
| using Microsoft.DotNet.ProductConstructionService.Client.Models; | ||
| using Microsoft.Extensions.DependencyInjection; | ||
| using Microsoft.Extensions.Logging; | ||
| using ProductConstructionService.ReproTool.Options; | ||
| using Tools.Common; | ||
| using GitHubClient = Octokit.GitHubClient; | ||
|
|
||
| namespace ProductConstructionService.ReproTool.Operations; | ||
| internal class FullBackflowTestOperation : Operation | ||
| { | ||
| private readonly IBarApiClient _prodBarClient; | ||
| private readonly FullBackflowTestOptions _options; | ||
| private readonly DarcProcessManager _darcProcessManager; | ||
| private readonly VmrDependencyResolver _vmrDependencyResolver; | ||
|
|
||
| public FullBackflowTestOperation( | ||
| ILogger<Operation> logger, | ||
| GitHubClient ghClient, | ||
| [FromKeyedServices("local")] IProductConstructionServiceApi localPcsApi, | ||
| IBarApiClient prodBarClient, | ||
| FullBackflowTestOptions options, | ||
| DarcProcessManager darcProcessManager, | ||
| VmrDependencyResolver vmrDependencyResolver) | ||
| : base(logger, ghClient, localPcsApi) | ||
| { | ||
| _prodBarClient = prodBarClient; | ||
| _options = options; | ||
| _darcProcessManager = darcProcessManager; | ||
| _vmrDependencyResolver = vmrDependencyResolver; | ||
| } | ||
|
|
||
| internal override async Task RunAsync() | ||
| { | ||
| await _darcProcessManager.InitializeAsync(); | ||
| Build vmrBuild = await _prodBarClient.GetBuildAsync(_options.BuildId); | ||
|
|
||
| Build testBuild = await CreateBuildAsync( | ||
| VmrForkUri, | ||
| _options.VmrBranch, | ||
| _options.Commit, | ||
| [ ..CreateAssetDataFromBuild(vmrBuild).Take(1000)]); | ||
|
|
||
| var channelName = $"repro-{Guid.NewGuid()}"; | ||
| await using var channel = await _darcProcessManager.CreateTestChannelAsync(channelName, skipCleanup: true); | ||
| await _darcProcessManager.AddBuildToChannelAsync(testBuild.Id, channelName, skipCleanup: true); | ||
|
|
||
| var vmrRepos = (await _vmrDependencyResolver.GetVmrRepositoriesAsync( | ||
| "https://github.com/dotnet/dotnet", | ||
| "https://github.com/dotnet/sdk", | ||
| "main")); | ||
|
|
||
| foreach (var vmrRepo in vmrRepos) | ||
| { | ||
| var productRepoForkUri = $"{ProductRepoFormat}{vmrRepo.Mapping.DefaultRemote.Split('/', StringSplitOptions.RemoveEmptyEntries).Last()}"; | ||
|
|
||
| var subscription = await _darcProcessManager.CreateSubscriptionAsync( | ||
| channel: channelName, | ||
| sourceRepo: VmrForkUri, | ||
| targetRepo: productRepoForkUri, | ||
| targetBranch: _options.TargetBranch, | ||
| sourceDirectory: vmrRepo.Mapping.Name, | ||
| targetDirectory: null, | ||
| skipCleanup: true); | ||
|
|
||
| await _darcProcessManager.TriggerSubscriptionAsync(subscription.Value); | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably come from the
Options.VmrBranch?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm if we go with that, then the source mappings should already have the maestro-auth-test fork URI, you're right I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually this will only work in the backflow