improve experimental configurability of gapic generator

[shollyman]

The entrypoint we use for configuring the gapic generator outputs is mediated for the most part via our bazel build definition for go_gapic_library, defined here:

gapic-generator-go/rules_go_gapic/go_gapic.bzl

Lines 91 to 105 in d0bf8c3

	def go_gapic_library(
	name,
	srcs,
	importpath,
	deps,
	release_level = "",
	grpc_service_config = None,
	service_yaml = None,
	gapic_yaml = None,
	metadata = False,
	transport = "grpc",
	diregapic = False,
	rest_numeric_enums = False,
	omit_snippets = False,
	**kwargs):

The arguments to that rule are currently:

def go_gapic_library(
  name,
  srcs,
  importpath,
  deps,
  release_level = "",
  grpc_service_config = None,
  service_yaml = None,
  gapic_yaml = None,
  metadata = False,
  transport = "grpc",
  diregapic = False,
  rest_numeric_enums = False,
  omit_snippets = False,
  **kwargs):

To enable easier configuration of uncommon/experimental options, I'm proposing the following:

• accept a list of plugin args directly as part of the bazel rule definition (amended via the other existing options)
• amend existing BUILD.bazel rules for affected definitions to use new plugin arguments.
• switch the various hardcoded configs in the generator to use plugin arguments instead
  • ex:

    gapic-generator-go/internal/gengapic/generator.go

    Lines 39 to 55 in d0bf8c3

    	// keyed by proto package name, e.g. "google.cloud.foo.v1".
    	var enableWrapperTypesForPageSize = map[string]bool{
    	"google.cloud.bigquery.v2": true,
    	}
    	// keyed by the API + service name joined by "/", e.g. "storage.googleapis.com/StorageControl".
    	var enableDirectPath = map[string]bool{
    	"storage.googleapis.com/StorageControl": true,
    	// for test purposes.
    	"showcase.googleapis.com/Foo": true,
    	}
    	var enableMtlsHardBoundTokens = map[string]bool{
    	"cloudkms.googleapis.com": true,
    	"secretmanager.googleapis.com": true,
    	"showcase.googleapis.com": true,
    	}

  • ex:

    gapic-generator-go/internal/gengapic/paging.go

    Lines 154 to 166 in d0bf8c3

    	var paginationOverrides = []struct {
    	pkgName string
    	disallowedMethods []string // methods explicitly denied from pagination
    	}{
    	{
    	pkgName: "google.cloud.talent.v4beta1",
    	disallowedMethods: []string{"SearchProfiles", "SearchJobs"},
    	},
    	{
    	pkgName: "google.cloud.bigquery.v2",
    	disallowedMethods: []string{"GetQueryResults"},
    	},
    	}

If we do this in the correct order, we should be able to refactor with a minimal amount of churn.

As options/configs become more standard, we can also hoist them out of the plugin_args and make them first class parts of the bazel definition, or move them into the service config if they're so general that they should be configured through that mechanism.

[noahdietz]

move them into the service config if they're so general that they should be configured through that mechanism.

In general, I think the language-specific sections of the service config publishing aspect are a great place for language-specific or experimental features. GoSettings are described here.

Python even has an experimental features subfield for especially unique things.

I can't speak to each example here, but I think it would be reasonable to move these things into GoSettings even just temporarily to consolidate, then deprecate and move to a higher-level property if other languages need the same thing.

[noahdietz]

Disabling pagination will definitely be up-scoped at some point as there is already a lot of talk re:configifying AIP-158 heuristics/duck typing.

DP and MTLS are probably good candidates for up scoping as well, but I think there is uneven support for DP in each language so a lang-specific config would probably last a while.

[shollyman]

Based on internal discussions, I'm going to write this up in more detail in a doc so we can get better consensus.