Enhance Support for Multi-file Protobuf Imports to Handle Large .proto Files More Gracefully

[panbadi]

When working with large-scale microservices using go-zero, we often need to split large .proto files into multiple sub-files (by business module, common types, etc.) and import them via Protobuf's native import syntax. This is a standard best practice in Protocol Buffers to improve maintainability, reusability, and reduce file bloat.
However, go-zero's goctl tool has strict and restrictive constraints on multi-file Protobuf imports. Unlike native Protobuf or other gRPC frameworks (e.g., gRPC-Go), go-zero requires imported .proto files to be placed in specific directories (e.g., a flat base directory under the proto root), enforces package names to match directory names, and fails to properly resolve imports with nested directories or non-flat structures. These limitations make it difficult to manage complex protobuf definitions in large projects and go against Protobuf's design philosophy of modularity.

Current Pain Points

1. Strict Directory Constraints: Imported proto files cannot be organized in nested directories (e.g., proto/common/enums.proto, proto/user/model.proto), forcing a flat structure that becomes unmanageable as the number of proto files grows.

2. Package Name Rigidity: The package name of the imported proto file must exactly match its parent directory name, which limits flexibility and may conflict with business naming conventions.

3. Poor Compatibility with Standard Protobuf Workflows: Developers familiar with standard Protobuf multi-file patterns need to adjust their workflows significantly to fit go-zero's constraints, increasing learning costs and potential errors.

4. Difficulty in Reusing Common Protos Across Services: Sharing common proto definitions (e.g., cross-service enums, base message types) across multiple RPC/API services requires cumbersome directory adjustments, rather than simple relative imports.

Expected Improvement

We hope go-zero can enhance its support for multi-file Protobuf imports to align with native Protobuf behavior, including:

1. Support nested directory structures for imported proto files (e.g., import "common/enums.proto" where common is a subdirectory under the proto root).

2. Relax the package name constraint, allowing package names to be independent of directory names (as long as they are unique and properly referenced).

3. Ensure goctl correctly resolves relative imports based on the proto root directory (specified via -I/--proto_path), consistent with native protoc behavior.

4. Support importing proto files from external dependencies (e.g., via Go modules or local paths outside the project's proto root) for better reusability.

Use Case Example

A typical modular proto structure we want to use (but currently cannot with go-zero without workarounds):

proto/
├── common/ # Nested common directory
│ ├── enums.proto # Common enums (package: common)
│ └── base.proto # Base messages (package: common)
├── user/ # User module
│ ├── model.proto # User models (package: user)
│ └── service.proto# User RPC service (imports "common/base.proto")
└── order/ # Order module
├── model.proto # Order models (imports "common/enums.proto")
└── service.proto# Order RPC service (imports "common/base.proto" and "user/model.proto")

With improved import support, goctl should generate code correctly for this structure when the proto root is set to proto/ (via -I=./proto), without requiring flat directories or package-name-directory-name matching.

Additional Context

This improvement would greatly benefit large-scale go-zero projects by enabling better protobuf organization, reducing technical debt, and aligning go-zero with industry-standard Protobuf practices. It would also lower the barrier for developers migrating from other gRPC frameworks to go-zero.

Thank you for considering this enhancement!