Skip to content

Latest commit

 

History

History
162 lines (128 loc) · 6.75 KB

PROCESSES.md

File metadata and controls

162 lines (128 loc) · 6.75 KB

Processes, release procedures etc

This repository includes multiple projects at multiple versions, some of which depend on each other. That makes things tricky, and leads to rather more complicated procedures than a simple "single package" repository. Please read this guide carefully to understand how releases happen, and why they happen that way.

The API catalog

Only the source within the apis directory is published, although not every project within that directory is published. (There are test projects and tools alongside the production code.) Nothing in the tools directory is published as a package.

Most project files under apis are at least partially generated. Metadata used for generation is in apis.json - the API catalog file. There's an entry for each API, containing:

  • The kind of API (grpc, rest, other)
  • The version number
  • A product title and home page for documentation
  • Dependencies
  • Additional test project dependencies
  • Target framework versions
  • Package description for NuGet
  • Tags for NuGet (in addition to default ones)
  • Proto path within the googleapis repository
  • Service YAML file

The catalog is used to generate project files and also during the release process described below. Running the project generator is very simple: from the root directory, in a bash shell, run

./generateprojects.sh

The CI systems run this before building, to ensure that the project files are in a stable state.

Generating the project files allows for broad changes (such as adding Source Link support) to be made very simply, just by changing the generator. Modifying every project file by hand simply doesn't scale.

However, sometimes manual editing of project files is required. The project generator supports this by assuming it "owns":

  • The first PropertyGroup element (for general properties)
  • The first ItemGroup element (for dependencies)
  • The ItemGroup element for Source Link (with a label of "dotnet pack instructions")
  • The common import to only attempt to build desktop assemblies on Windows

Any other elements are left as they are - so if you wish to add an ItemGroup such as for file grouping, just add it anywhere after the generated one, and it should do the right thing.

Additionally, for each API, the project generator adds all projects under the API directory (and project references) to the solution for that API. It will create project files from scratch as well - so when adding a new package from autogenerated API sources, the simplest approach is to copy the source files, modify the API catalog, and run the project generator. The project files and solution file will be generated and should immediately be usable.

Versioning

All releasable packages follow Semantic Versioning. Non-releasable code (tests, tools, analyzers) are not versioned. The precise meaning of a breaking change (dictating version number increments) is out of the scope of this document. (Jon is writing a blog post about this topic and will link to it when it is published.)

The version number in the API catalog (and therefore in project files) is one of:

  • The mostly recently released package version
  • The version that's about to be released (see below)
  • A prerelease with a suffix of 00 to indicate that the next release should be the first prerelease for that major/minor version.

The last of these cases can happen if either an existing version has gone to GA and new features have been merged since (to ensure that we don't add features within a minor GA version) or for brand-new APIs that haven't been released at all. The tooling for tagging is aware of this convention, and doesn't create tags for alpha00 or beta00 suffixes.

Typically version numbers should be changed in a commit which does nothing else, for clarity. Include both the apis.json change and the project file changes that occur when the project generator has been run, in the same commit.

Dependencies

Dependencies in the API catalog are specified as properties where the property name is the package name and the value is the version number.

If the version number is set to "project", then a project reference is used. If project X has a production dependency on project Y, then both X and Y must be released together, to avoid misaligned versions. The tagging tool enforces this.

If the version number is set to "default", then the version number is determined by the project generator, to keep these dependencies in sync for all appropriate packages.

Two project types gain dependencies automatically if they're not specified:

  • "grpc" projects always have dependencies on Google.Api.Gax.Grpc and Grpc.Core
  • "rest" projects always have a dependency on Google.Api.Gax.Rest

Note that as of 2024-03-18, even GA versions can use default dependencies. This means that when creating a patch release (which is relatively rare) the previous version's dependencies should be checked in NuGet and the same versions (or patch-level only changes) should be explicitly listed. This is checked within the project generator.

Releasing

Releasing is performed automatically via internal processes.

Turning down a library

When an API has been deprecated, our process for turning down the corresponding library is:

  • Release a new version with a modified docs/index.md file to indicate the deprecation. This should also be indicated in the release notes. See this PR as an example.
  • Mark all published versions as deprecated and delisted in NuGet. The deprecation step can easily be done manually as the NuGet web site allows marking all versions as deprecated in one go. For delisting, either delist each version manually, or use the release manager delist-package command to delist all versions.
  • Create a PR to remove the API from the API catalog, delete the source code, and regenerate projects (which will update Renovate and the README). See this PR as an example.

Updating the .NET SDK

Periodically, we need to update the version of the .NET SDK we use for building. This might be to gain access to new C# features, or due to security patches being available. This must be done carefully in order to avoid other processes breaking. Only start this process when you are confident that all the steps below can be completed in a reasonable timeframe.

  1. Update internal CI systems to include the new SDK.
  2. Create a PR with the following changes:
    1. Update global.json to specify the .NET release version required.
    2. Update Dockerfile.generator to specify the .NET release version required.
    3. For new major SDK versions, update GitHub actions
  3. Merge the above PR.
  4. Ensure the new Docker image is used.