Preventing Import of Dependencies in Go Generate

[shiron-dev]

I currently push a module containing only proto files (buf.build/shiron-dev/protoc-gen-connect-map) to the BSR.

When I include this module in the deps section of my buf.yaml file and import a proto file from that module into a proto file in my current project, the generated Go code (pb.go) produced by buf generate unintentionally includes an import statement for buf.build/shiron-dev/protoc-gen-connect-map.

However, buf.build/shiron-dev/protoc-gen-connect-map is not a Go package, so this import statement is unnecessary.

Is there a way to prevent buf generate from generating import statements for modules included as dependencies? I'm particularly interested in controlling import statement generation through settings in buf.gen.yaml.

Specifically, I'd appreciate advice on the following:

• How to configure buf.yaml to prevent a specific module from influencing go generate.
• Configuration options in buf.gen.yaml to control the generation of import statements.

I will paste the contents of my buf.yaml and buf.gen.yaml files below this post. Your review of these configurations would be greatly appreciated.

Thank you for your assistance.

# For details on buf.yaml configuration, visit https://buf.build/docs/configuration/v2/buf-yaml
version: v2
lint:
  use:
    - STANDARD
breaking:
  use:
    - FILE

deps:
  - buf.build/shiron-dev/protoc-gen-connect-map

# buf.gen.yaml
version: v2
managed:
  enabled: true
  disable:
    - module: buf.build/shiron-dev/protoc-gen-connect-map

plugins:
  - remote: buf.build/protocolbuffers/go
    out: .

[doriable]

So this is a little bit difficult to explain without seeing a representation of your proto sources, but hopefully the following explanation is helpful.

Basically this import statement is generated because your proto sources import proto definitions from your dependency, and it needs to reference Go code for this dependency. And it is expecting that you are handling the required Go code through some kind of import.

A common example from our docs is the use of googleapis as a dependency -- we expect users to import the appropriate googleapis package for their language (e.g. genproto for Go), so we advise users to disable the go_package_prefix in managed mode for the googleapis module.

In your case, you can disable the setting of the appropriate file option in your external module (assuming that it has the appropriate go_package file option set on its proto sources) or provide a value that would map to the Go import statement you intend to use.

[shiron-dev]

Thank you for the detailed code example and your thorough response.

However, when I ran the code you provided, I encountered the following error:

gen/example.com/dkeung/override/example.pb.go:162:2: undefined: file_connect_map_proto_init

It appears that this error occurs because example.proto imports connect_map.proto, and consequently, the generated example.pb.go expects connect_map.pb.go to exist. In other words, it seems that the error arises because gen/example.com/dkeung/override/connect_map.pb.go, which is the generated output of connect_map.proto found in buf.build/dkeung/connect-map-example, is missing.

Therefore, I would greatly appreciate it if you could advise me on either of the following approaches:

1. A method to prevent example.pb.go from expecting the generation of connect_map.pb.go in the first place.
2. A method to generate connect_map.pb.go in the same path as example.pb.go (i.e., gen/example.com/dkeung/override/).

Thank you for your assistance.

[doriable]

1. A method to prevent example.pb.go from expecting the generation of connect_map.pb.go in the first place.

As noted above, the generated code will always expect that you have access to all the definitions you import, so this is basically not possible, so it's just a matter of resolving the second option.

2. A method to generate connect_map.pb.go in the same path as example.pb.go (i.e., gen/example.com/dkeung/override/).

There are a couple of ways this can be addressed, and it really depends on your setup:

Option 1: Import the generated code through Go

If protoc-gen-connect-map dependency provides some Go package that you want to import separately, you would need to adjust the managed mode options to basically generate that import statement for you and you would import it through Go as you would any other Go package.

Option 2: Generate your imports also

So buf generate is designed to support the above use-case, where by default, it only generates your definitions, and allowing you to handle the imports somehow (e.g. the googleapis example I linked above, where commonly, a user would import the genproto package). However, you can choose to generate the imports in this case using buf generate --include-imports.

Using the example setup I have above, since both connect_map.proto and example.proto are both at the root of their respective packages, running buf generate --include-imports in project-a gives me the following:

project-a @ $ tree .
.
├── buf.gen.yaml
├── buf.lock
├── buf.yaml
├── gen
│   └── example.com
│       └── dkeung
│           └── override
│               ├── connect_map.pb.go
│               └── example.pb.go
└── proto
    └── example.proto

6 directories, 6 files

[shiron-dev]

Thanks for your guidance.
I discovered that using buf generate --include-imports solved the problem.
I really appreciate your help throughout this process.

[shiron-dev]

I apologize for posting this after the issue has been marked as resolved. However, I've observed some unexpected behavior and would appreciate your clarification.

Using the example from https://github.com/shiron-dev/buf-sample, I've found that when paths=source_relative and include_imports: true are specified together, the generated code for imported proto files appears to ignore the go_package option. Instead, the code is generated directly into the output directory (in the gen directory).

However, when I remove the paths=source_relative option and only specify include_imports: true, the code is generated correctly according to the go_package setting (in the gen2 directory).

Is this the intended behavior? Could this be a bug in the buf CLI, or is there a issue with my configuration?

[doriable]

This is intended behaviour, from protoc-gen-go docs: https://protobuf.dev/reference/go/go-generated/

If the paths=source_relative flag is specified, the output file is placed in the same relative directory as the input file. For example, an input file protos/buzz.proto results in an output file at protos/buzz.pb.go.

When the flag is specified, it uses the same relative directory structure as your proto source files, buf generate is just respecting the options in this case.