This document describes the concepts of channels, branches and subscriptions in the .NET Core 3.0 (ProdCon v2) world as well as workflow and management.
For .NET Core 3, the engineering system must be able to support a number of core construction scenarios, including:
- Builds involving custom branches for various parts of the stack (e.g. long lived feature work).
- Builds for a new release.
- Builds involving stabilized assets (that are only published after validation).
- Internal only builds.
- Non-validated builds (only requirement for producing a new product is that "it builds").
- Speculative builds (see Maestro.md).
- Upstack builds on PRs.
In Prodcon v2, there are a number of central principals that shape how the product is constructed:
-
The product is spread among many repositories, with inter-repo dependencies expressed as pointers to build assets. The pointers normally take the form of version number and name of asset.
-
All product changes must be commits. These commits alter Branches.
In prodcon v1, inter-repo dependency updates were forced via a dependency info file that was passed into the build.
-
If the inter-repo dependency graph has a maximal depth N, then N commits are required for a complete alteration of the product output.
-
Branches do not generally define the intended use of the code in the branch. This is defined by the Channels (one or more) a build of that branch is assigned.
-
Alterations in product composition (new repo dependencies) are done via a pull model, pulling new assets based on the intended purpose of those assets. These pulls are automated using Subscriptions
A Git branch is a moving pointer to the head of a tree of commits that define potential product content. Branches are created and destroyed based on the individual needs of a repository and thus are not consistent across the product stack. It is also important to recognize that branches are primarily a way to improve developer productivity and understanding of a repository. The git repo is really just a set of commits, and these commits are what defines potential product content, not branches.
A "tag" that defines the purpose of a set of build assets. Whereas commits define potential content, a build creates content based on a commit, and channels define what the created content is intended for.
Channels have the following characteristics:
- Should be consistent across the product stack, though they may not necessarily apply to all repos. For example, all repos may produce bits for a ".NET Core 3.0.0" channel, but only some repos (e.g. core-sdk) might produce output for a ".NET SDK 3.0.1xx" channel
- Build assets are applied to a set channels after the build is done. This may be manual or automatic.
- A build may be applied to multiple channels.
- Channels may be public or internal. This differentiates what may be done with outputs from a build. The following rules must be followed:
- Builds of internal, non public branches may not be assigned to public channels.
- Build assets that are assign to an internal channel may not publish/display outputs in a public location (e.g. nuget.org, dev.azure.com/dnceng/public)
Defines the process of product transformation (creation of new commits). Inter-repo dependencies are expressed as dependencies on created content (e.g. core-sdk commit abcd depends on core-setup build asset Microsoft.NetCore.App.123456). Subscriptions define a set of rules under which those inter-repo dependencies should be altered in new commits to a target branch.
Subscriptions have the following characteristics:
-
They define a mapping between assets produced by a build, and new commits to branches in another repo.
source repo + channel + assets (optional) -> target branch + target repo -
They have a desired quality metric for when the mapping should be applied (e.g. should tests have passed?)
-
They have a trigger for when the mapping should be applied.
-
They may optionally include a list of GitHub logins (users who must be a public member of the Microsoft organization) or team aliases (for notification to work, this team must exist in the target repository's organization). When included in a non-batched subscription, failed policies for the pull requests produced will tag these users. As such, these users should be teams or users who represent subject-matter experts for the source repository of the subscription.
A subscription can be visualized with the following pseudocode
function runSubscription(newBuild, subscription) {
// Determine whether the build applies to the subscription
if (subscription.sourceRepo != newBuild.sourceRepo ||
subscription.sourceChannel != newBuild.channel) {
return
}
// Determine whether trigger should be run
// For example, this might return false if we've already run once today and the subscription only runs once a day.
if (!subscription.isTriggered(newBuild)) {
return
}
// Check out the target repo and branch.
// git clone targetRepo; git checkout targetBranch
repo = checkOutSources(targetRepo, targetBranch)
// Check out a new branch in which to make a commit
// git checkout -b update-dependencies
repo.checkOutBranchForChanges()
// Map assets existing in target's
foreach (dependency in repo.Dependencies) {
if (newBuild.assets.contains(dependency)) {
if (subscription.assets.count == 0 || subscription.assets.contains(dependency)) {
repo.updateAsset(newBuild.assets)
}
}
}
// Check quality of new repo content (do a build, etc.)
if (subscription.isDesiredQuality(repo)) {
mergeChanges()
}
else if (subscription.hasFailureNotificationTags and subscription.isNotBatched) {
tagTheseUsersOnDependencyFlowPullRequest()
}
}
Those who have worked in a monolithic repo world in the past can have trouble understanding the issues that a distributed repository world introduces (though it has other significant benefits). To better understand these issues, let's look at branches, channels and subscriptions in the monolithic world. A mono repo world is one where the vast majority of changes do not cross repository lines. Even in monolithic repositories, there are still some distributed qualities. For example, an update of a checked in toolset effectively is a pull of an external content channel (like a build of a compiler). However, these types of changes tend to be fewer, and thus the product repository can generally be viewed as self-contained.
When a product repository can be viewed as self-contained, the branches, channels and subscriptions concepts tend to blend together.
- A single commit (and moving branch head by extension) covers the entire product stack. All components are versioned in a single atomic fashion. This makes it possible to imply a channel based on a branch:
- "Main produces vNext"
- "Dev15.6 produces Dev15.6"
- Because a single commit describes all components at a single specific point in time, no subscription is needed to update dependencies between components
- There are perhaps a few remaining 'subscriptions' for the remaining distributed components, but there are few enough that they tend to be manually dealt with and have little overhead.
-
External team checks in lots of new commits into branch X (potential content)
-
External team produces new toolset from branch X (created content)
-
External team declares new toolset as for a specific channel (e.g. C++ Compiler 16.x LKG). Much of the time this is done through human communication channels (e.g. email).
-
Product team infers quality level of new external toolset based on the channel (LKG)
-
Product maps new toolset into their repository
Starting from: source repo (optional) + channel + asset (optional) -> target branch + target repo source repo = toolset team's repo channel = C++ Compiler 16.x LKG asset = new compiler exe target branch = what branches needs a 16.x LKG compiler? E.g. Main because we need stable bits target repo = product repo Gives: toolset team's repo + C++ Compiler 16.x LKG + new compiler exe -> Main + product repo -
Product team decides whether to commit new toolset based on quality results on their repo.
-
After combining the product construction principals with the .NET Core 3 scenarios, it's clear that:
- There will be many branches in each repo, some long lived, some short lived.
- There will be many commits required for product alteration.
- There will be many builds for various purposes.
- Managing product alteration cannot be manual.
This increased complexity needs to be tooled where possible.
There are a variety of scenarios that require changes in channels, branches and subscriptions. This section walks through these core scenarios, explains the manual steps that would normally be involved, and the expected behavior of tooling (if any). These scenarios are explained in terms of specific .NET Core repositories.
Onboarding of new repositories adds new nodes to the product dependency graph. Initially the number of nodes in the dependency graph is 0. There are no branches of any repos producing assets for any channel, and thus there are no subscriptions consuming those assets (because there would be no place to move them). Onboarding the initial repository involves the following:
-
Define potential content - The dotnet/runtime repo's main branch should be onboarded.
-
Onboard dotnet/runtime to Arcade style publishing. Arcade publishing will push build assets to shared storage locations and notify the Build Asset Registry (BAR) of new builds.
-
Create content - Build dotnet/runtime's main branch (let's say its ID is 'dotnet/runtime#1') Through arcade publishing a new entry for this build will be created in the BAR. 'dotnet/runtime#1' will not initially have a channel..
-
Create a new channel for content - Create '.NET Core 3.0.0' channel in the BAR
darc add-channel --name '.NET Core 3.0.0' -
Assign existing dotnet/runtime builds to the channel if desired - dotnet/runtime's main branch creates content for '.NET Core 3.0.0' channel, so use the BAR to assign 'dotnet/runtime#1' to '.NET Core 3.0.0'
darc add-build-to-channel --id '<barId>' --channel '.NET Core 3.0.0' -
Assign future builds of a specific branch a default channel (if desired) - Use the BAR to map dotnet/runtime main onto '.NET Core 3.0.0'.
darc add-default-channel --repo https://github.com/dotnet/runtime --branch main --channel '.NET Core 3.0.0'
Once the repository graph has more than one node (or if there is a circular dependency, like in the case of dotnet/arcade), it becomes possible to create subscriptions. Onboarding new repositories after the initial node involves the steps above with the following alterations/additions:
- No new channel is necessary if repository/branch being onboarded is producing assets for an existing channel
- Repository should be onboarded onto the formal dependency description format, enabling automated update of dependencies via Darc.
Let's say dotnet/corefx's main branch is the second branch to be onboarded. dotnet/corefx has a dependency on dotnet/runtime's output assets.
- Define potential content - The dotnet/corefx repo's main branch should be onboarded.
- Onboard dotnet/corefx to Arcade style publishing. Arcade publishing will push build assets to shared storage locations and notify the BAR of new builds.
- Onboard dotnet/corefx to Arcade style dependency management - Create a Dependency.Versions.xml and associated dependency props files in the main branch. Use Darc to add dependencies for specific dotnet/runtime dependencies (e.g. Microsoft.NETCore.Runtime.Coreclr). These dependencies were produced by dotnet/runtime builds and assigned to the '.NET Core 3.0.0' channel.
- Create content - Build dotnet/corefx's main branch (let's say its ID is 'dotnet/corefx#3') Through arcade publishing a new entry for this build will be created in the BAR. 'dotnet/corefx#3' will not initially have a channel..
- Assign new dotnet/corefx build to the channel - dotnet/corefx's main branch creates content for '.NET Core 3.0.0' channel, so use the BAR to assign 'dotnet/corefx#3' to '.NET Core 3.0.0'
- Assign future builds of a specific branch a default channel (if desired) - Use the BAR to map dotnet/corefx main onto '.NET Core 3.0.0'.
- Add a subscription to automatically pull new dotnet/runtime dependencies into dotnet/corefx's main branch. Subscription info
- Mapping - Any dotnet/runtime .NET Core 3.0.0 asset -> dotnet/corefx's main branch
- Quality metric - No quality metric (implies build passes)
- Trigger - Nightly at 2am
Using this workflow, developers will slowly build up the product dependency graph of repos, branches, etc. This could be done manually with other branches, creating new channels, subscriptions as needed. However, these operations become tedious and error prone as the graph becomes larger. In a lot of cases, we can infer what branches channels and subscriptions are needed based on the current set and some outside input. Operations to create these can then be performed based on the desired outcome.
In this scenario, there is an existing repository dependency graph, a set of subscriptions, and a set of output channels. There will usually only be one output channel, but this is not necessarily true. In branching for a release, or creating a new dev build involving some of the repositories involved, the goal is to produce a new graph that describes new input branches and flows their outputs in a similar manner to the original graph. This involves:
- Create new channels for outputs
- Create new subscriptions to flow those outputs
- Creating or identifying new branches
- Because there is most likely a relationship between original and new branches, it may be desirable to set up automatic merges between branches.
Let's look at a simple example of branching for stabilization and release. This is a subset of a sample repository graph for .NET Core v3.0
dotnet/core-sdk
- branch: main
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- output channels: .NET Core 3.0 Dev, .NET Core SDK 3.0.1xx Dev
- subscriptions
- dotnet/core-setup .NET Core 3.0 Dev
- aspnet/universe .NET Core 3.0 Dev
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: main
- depends on
- none
- output channel: .NET Core 3.0 Dev
- subscriptions
- none
aspnet/universe
- branch: main
- depends on
- dotnet/core-setup
- output channel: .NET Core 3.0 Dev
- subscriptions
- dotnet/core-setup .NET Core 3.0 Dev
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
In branching for release, we want to isolate the input code from risky changes, allow mainline development to continue, etc. This typically involves creating a release branch for each repository, like "release/3.0". The outputs for .NET Core 3.0 switch to coming from this set of branches, and the original set of branches is now producing .NET Core 3.1 development bits. Looking at this action from the perspective of channels and subscriptions:
- The .NET Core 3.0 Dev channel is now defunct (we are in release shutdown)
- A .NET Core 3.0 release channel should produced from a set of yet-created branches. These branches would be based off the branches producing .NET Core 3.0 Dev bits.
- Subscriptions intaking .NET Core 3.0 assets should map onto new branches and flow in the same manner as the .NET Core 3.0 Dev subscriptions
- The .Net Core 3.1 Dev channel should be created and assigned as the default output channel of the branches that previously defaulted to the .NET Core 3.0 Dev channel.
- Because fixes in the release branches should flow back into the mainline branches, there should be automatic merge PRs opened.
The new .NET Core 3.0 graph should then look like:
dotnet/core-sdk
- branch: release/3.0.1xx
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- automerge
- release/3.0.1xx -> main
- output channels: .NET Core 3.0, .NET Core SDK 3.0.1xx
- subscriptions
- dotnet/core-setup .NET Core 3.0
- aspnet/universe .NET Core 3.0
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: release/3.0
- depends on
- none
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0
- subscriptions
- none
aspnet/universe
- branch: release/3.0
- depends on
- dotnet/core-setup
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0
- subscriptions
- dotnet/core-setup .NET Core 3.0
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
And the newly created .NET Core 3.1.0 Dev graph looks like:
dotnet/core-sdk
- branch: main
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- output channels: .NET Core 3.1 Dev, .NET Core SDK 3.1.1xx Dev
- subscriptions
- dotnet/core-setup .NET Core 3.1 Dev
- aspnet/universe .NET Core 3.1 Dev
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: main
- depends on
- none
- output channel: .NET Core 3.1 Dev
- subscriptions
- none
aspnet/universe
- branch: main
- depends on
- dotnet/core-setup
- output channel: .NET Core 3.1 Dev
- subscriptions
- dotnet/core-setup .NET Core 3.1 Dev
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
To perform this operation, we want to do two operations: channel branch, then channel rename. We branch the .NET Core 3.0 Dev channel, to create .NET Core 3.0 Release, then rename .NET Core 3.0 Dev to .NET Core 3.1 Dev.
darc channel branch '.NET Core 3.0 Dev' '.NET Core 3.0'
darc channel rename '.NET Core 3.0 Dev' '.NET Core 3.1 Dev'
Dev builds are very similar to a new release, except for the following:
- There is a subset of input repos where active work will happen
- The original channel will not be renamed.
Let's start with the same input graph as before:
dotnet/core-sdk
- branch: main
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- output channels: .NET Core 3.0 Dev, .NET Core SDK 3.0.1xx Dev
- subscriptions
- dotnet/core-setup .NET Core 3.0 Dev
- aspnet/universe .NET Core 3.0 Dev
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: main
- depends on
- none
- output channel: .NET Core 3.0 Dev
- subscriptions
- none
aspnet/universe
- branch: main
- depends on
- dotnet/core-setup
- output channel: .NET Core 3.0 Dev
- subscriptions
- dotnet/core-setup .NET Core 3.0 Dev
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
Let's say a user wants to do work in core-setup and aspnet/universe on previously created branches for feature/foo. Basically that means that they probably want to have the output of those branches appear in the SDK. Remember the Product Construction v2 rules:
- Since all changes involve commits, that means that there must be a new branch of core-sdk that can take the output aspnet/universe and core-setup branches.
- No more than one channel per input repo may flow input a repo+branch combo (e.g. cannot have dotnet/core-setup's .NET Core 3.0 and dotnet/core-setup's .NET Core 3.1 assets flow to dotnet/core-sdk main as there will be collisions)
- A single channel may not receive input from two different branches in the same repo.
This means that we need:
- A new Dev channel that builds from the new branches are assigned to
- New branches for any repos downstream of the input repositories.
- Duplicated subscriptions targeting the new branches, replacing the old channel with the new dev channel where it appears. Other subscriptions unrelated to the dev channel should remain the same (e.g. a new dotnet/roslyn for Dev 16.0 should be mapped into the new core-sdk branch)
If we want to create a channel called ".NET Core 3.0 Feature/Foo"
dotnet/core-sdk
- branch: feature/foo (forked from main)
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- automerge
- main -> feature/foo
- output channels: .NET Core 3.0 Feature/Foo (.NET Core 3.0.1xx eliminated because main already produces this)
- subscriptions
- dotnet/core-setup .NET Core 3.0 Feature/Foo
- aspnet/universe .NET Core 3.0 Feature/Foo
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: feature/foo
- depends on
- none
- automerge
- main -> feature/foo
- output channel: .NET Core 3.0 Feature/Foo
- subscriptions
- none
aspnet/universe
- branch: feature/foo
- depends on
- dotnet/core-setup
- automerge
- main -> feature/foo
- output channel: .NET Core 3.0 Feature/Foo
- subscriptions
- dotnet/core-setup .NET Core 3.0 Feature/Foo
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
To perform this operation, we want to do a restricted channel branch operation. We branch the .NET Core 3.0 Dev channel, to create .NET Core 3.0 Feature/Foo for only aspnet/universe, dotnet/core-setup and downstream repos.
darc channel branch '.NET Core 3.0 Dev' '.NET Core 3.0 Feature/Foo' dotnet/core-setup aspnet/universe
Non-validated builds are useful for getting an early read on quality or ensuring that the path between repos deep in the stack flow bits as often as possible. In this mode, there is no quality gate between the repos aside from whether the repo builds or not. This effectively looks like a channel branch into a new channel with:
- Subscriptions that do not gate on quality to move bits
- Auto-merges between upstream branches and the new channel branches.
Starting from the original graph again:
dotnet/core-sdk
- branch: main
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- output channels: .NET Core 3.0 Dev, .NET Core SDK 3.0.1xx Dev
- subscriptions
- dotnet/core-setup .NET Core 3.0 Dev
- aspnet/universe .NET Core 3.0 Dev
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: main
- depends on
- none
- output channel: .NET Core 3.0 Dev
- subscriptions
- none
aspnet/universe
- branch: main
- depends on
- dotnet/core-setup
- output channel: .NET Core 3.0 Dev
- subscriptions
- dotnet/core-setup .NET Core 3.0 Dev
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
We want to branch into '.NET Core 3.0 Unstable', producing a graph that looks the same, yet with additional auto-merges and subscription options. Like with dev builds, this means that we need homes for all those commits. The primary difference here from a dev build is that we aren't intending to do work in these branches. So any leaf repositories, those without inter-repo dependencies (or at least non-backedges), need not be branched. The original branch simply outputs to a new channel. In the example, core-setup and roslyn are both leaves, and only core-setup produces for .NET Core 3.0 Dev
dotnet/core-sdk
- branch: main-unstable
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- output channels: .NET Core 3.0 Unstable
- automerge
- main -> main-unstable
- subscriptions
- dotnet/core-setup .NET Core 3.0 Unstable, no validation
- aspnet/universe .NET Core 3.0 Unstable, no validation
- dotnet/roslyn Dev16.0, no validation
dotnet/core-setup
- branch: main
- depends on
- none
- output channel: .NET Core 3.0 Dev, .NET Core 3.0 Unstable
- subscriptions
- none
aspnet/universe
- branch: main
- depends on
- dotnet/core-setup
- automerge
- main -> main-unstable
- output channel: .NET Core 3.0 Unstable
- subscriptions
- dotnet/core-setup .NET Core 3.0 Unstable
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
To perform this operation, we want to do a channel branch operation. We branch the .NET Core 3.0 Dev channel, to create .NET Core 3.0 Unstable. We then edit the configuration so that newly created subscriptions do not gate on test results.
darc channel branch '.NET Core 3.0 Dev' '.NET Core 3.0 Unstable'
Stabilization for release can be done one of two ways:
- Gradual stabilization - Having the main channel produce and flow stabilized assets. This means each individual build would switch to a stable mode, run the build, and normal flow would occur. As each repo flowed assets to other repos, we'd change subscriptions to manual flow to avoid overwriting those dependencies. The downside to this is that you cannot get non-stabilized builds from the same set of branches in parallel (because the checked in inputs are stabilized), so during the window where stabilized assets are being produced, the builds cannot easily be shared externally.
- Parallel stabilization - Having a parallel set of branches to the release branch which produce and consume stabilized assets. These builds would not be published externally. The downside is increased complexity and resource usage.
Both approaches are valid and could be used in various stages of the product. Let's look at the parallel set of branches approach, starting with the release graph for .NET Core 3.0
dotnet/core-sdk
- branch: release/3.0.1xx
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- automerge
- release/3.0.1xx -> main
- output channels: .NET Core 3.0, .NET Core SDK 3.0.1xx
- subscriptions
- dotnet/core-setup .NET Core 3.0
- aspnet/universe .NET Core 3.0
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: release/3.0
- depends on
- none
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0
- subscriptions
- none
aspnet/universe
- branch: release/3.0
- depends on
- dotnet/core-setup
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0
- subscriptions
- dotnet/core-setup .NET Core 3.0
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
Like with an unstable build, each node with dependencies needs a new branch to pull in those stabilized dependencies. Lead nodes can go one of two ways. They can either set up a build (with different queueing param) that produces the stabilized assets, and then only those builds would get assigned to the stabilized channel, or they could make code alterations in a new branch that causes those assets to be stabilized by default. The latter is probably simpler.
Like an unstable build, we need merges between the main branches and the new branch
dotnet/core-sdk
- branch: release/3.0.1xx-stabilized
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- automerge
- release/3.0.1xx -> release/3.0.1xx-stabilized
- output channels: .NET Core 3.0 Stabilized, .NET Core SDK 3.0.1xx Stabilized
- subscriptions
- dotnet/core-setup .NET Core 3.0 Stabilized
- aspnet/universe .NET Core 3.0 Stabilized
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: release/3.0-stabilized
- depends on
- none
- automerge
- release/3.0 -> release/3.0-stabilized
- output channel: .NET Core 3.0 Stabilized
- subscriptions
- none
aspnet/universe
- branch: release/3.0-stabilized
- depends on
- dotnet/core-setup
- automerge
- release/3.0 -> release/3.0-stabilized
- output channel: .NET Core 3.0 Stabilized
- subscriptions
- dotnet/core-setup .NET Core 3.0 Stabilized
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
To perform this operation, we want to do a channel branch operation. We branch the .NET Core 3.0 channel, to create .NET Core 3.0 Stabilized.
darc channel branch '.NET Core 3.0 Dev' '.NET Core 3.0 Stabilized'
In gradual stabilization, we gradually edit the configuration for a channel or channels to remove subscriptions as components shut down and flow stable dependencies. This is done through a series of channel edit operations.
darc channel edit '.NET Core 3.0'
Internal only builds are much like regular builds, just that the channel they output to is an internal-only channel. This has implications for where outputs from that channel can be published (e.g. an internal channel). Other than that, a simple channel branch operation should suffice. Starting with the .NET Core 3.0 graph:
dotnet/core-sdk
- branch: release/3.0.1xx
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- automerge
- release/3.0.1xx -> main
- output channels: .NET Core 3.0, .NET Core SDK 3.0.1xx
- subscriptions
- dotnet/core-setup .NET Core 3.0
- aspnet/universe .NET Core 3.0
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: release/3.0
- depends on
- none
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0
- subscriptions
- none
aspnet/universe
- branch: release/3.0
- depends on
- dotnet/core-setup
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0
- subscriptions
- dotnet/core-setup .NET Core 3.0
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
We want:
- A new .NET Core 3.0 Internal channel, set to private
- A set of branches based off the .NET Core 3.0 branches
- Automatic merges between .NET Core 3.0 branches and the internal only branches
An output graph might look like:
dotnet/core-sdk
- branch: release/3.0.1xx-internal
- depends on
- dotnet/core-setup
- aspnet/universe
- dotnet/roslyn
- automerge
- release/3.0.1xx -> main
- output channels: .NET Core 3.0 Internal
- subscriptions
- dotnet/core-setup .NET Core 3.0 Internal
- aspnet/universe .NET Core 3.0 Internal
- dotnet/roslyn Dev16.0
dotnet/core-setup
- branch: release/3.0-internal
- depends on
- none
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0 Internal
- subscriptions
- none
aspnet/universe
- branch: release/3.0-internal
- depends on
- dotnet/core-setup
- automerge
- release/3.0 -> main
- output channel: .NET Core 3.0 Internal
- subscriptions
- dotnet/core-setup .NET Core 3.0 Internal
dotnet/roslyn
- branch: main
- depends on
- none
- output channel: Dev16.0
- subscriptions
-none
This can be represented by the following channel branch operation
darc channel branch '.NET Core 3.0' '.NET Core 3.0 Internal' --internal
These two scenarios are very much like the previous scenarios (simple channel branch operations), though with different subscriptions. The primary difference is that the creation and deletion of these graphs is fully automated (no user interaction). One way this might be done is to have the automated system call the darc tool, but have it output the configuration to a json file. The bot would then alter the configuration as needed and submit.
A speculative build is a typical channel branch with the following modifications:
- Restricted channel branch (single repo)
- No inter-branch merges
- Subscriptions flow outputs on build completion, open PRs and merge PRs upon quality gate passes.
An upstack build on a PR is a typical channel branch with the following modifications
- Restricted channel branch (single repo)
- No inter-branch merges
- Builds of upstack components are not signed
- Subscriptions flow outputs on build completion, not gated by quality gates, though quality is reported.
Unified Build is a new way of constructing .NET used for .NET 10 onwards.
Documentation on the tooling used to create and manipulate channels and subscriptions can be found at Darc.md