Skip to content

Commit 515b70a

Browse files
spencercjhclaude
andauthored
feat: add gRPC-protoc framework support (#14)
* feat(grpc): create grpcprotoc package structure Add package constants and Info type for gRPC-protoc extractor. Signed-off-by: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpc): implement grpcprotoc detector Add detector that: - Rejects buf-managed projects with clear error - Finds all .proto files - Detects import paths - Identifies google.api.http annotations Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpc): implement grpcprotoc patcher Add patcher that verifies: - protoc is installed - protoc-gen-connect-openapi is installed - Returns clear installation hints if missing Signed-off-by: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpc): implement grpcprotoc generator Add generator that: - Builds protoc command with import paths - Executes protoc-gen-connect-openapi - Returns path to generated OpenAPI file Signed-off-by: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpc): register grpcprotoc extractor in builtin registry Add grpcprotoc to builtin extractor registry. Implements full Extractor interface. Signed-off-by: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpc): add --proto-import-path CLI flag Add flag for specifying additional protoc import paths. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * docs: add gRPC-protoc support to README Document gRPC-protoc extractor usage, requirements, and CLI flags. Signed-off-by: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpcprotoc): improve proto file handling and add demo project - Add ServiceProtoFiles field to Info struct to identify main entry points - Update Detector to find proto files with service definitions - Update Generator to: - Only compile service proto files (avoids duplicate definition errors) - Handle relative output directories correctly by converting to absolute paths - Search recursively for generated OpenAPI files - Add integration-tests/grpc-protoc-demo with sample UserService proto - Include generated OpenAPI spec for demo project Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(grpcprotoc): add HTTP annotations support for REST endpoints - Add google/api/annotations.proto and http.proto to demo project - Update user.proto with google.api.http annotations for all RPC methods - Generator now conditionally adds --connect-openapi_opt=features=google.api.http when HasGoogleAPI is detected, generating REST endpoints alongside gRPC The generated OpenAPI spec now includes both gRPC-style paths and REST endpoints: - GET /v1/users - ListUsers - POST /v1/users - CreateUser - GET /v1/users/{id} - GetUser - PUT /v1/users/{id} - UpdateProfile - POST /v1/users/{user_id}/files - UploadFile Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * test(e2e): add gRPC-protoc integration test - Add TestE2E_GrpcProtoc_Generate to verify gRPC-protoc detection and generation - Verify HTTP annotations detection (HasGoogleAPI) - Verify generated spec contains REST paths from google.api.http annotations - Fix existing tests to properly extract FrameworkData for spring.Info Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * docs: update grpc-protoc documentation and tests - Add grpc_protoc_test.go with dedicated e2e test for gRPC-protoc - Update integration-tests/README.md with grpc-protoc-demo project - Update grpc-protoc-demo/README.md with: - third_party directory structure - HTTP annotations usage - spec-forge integration notes - Update CLAUDE.md with: - grpcprotoc package in architecture - publisher package - Remove duplicate tests from spring_maven_test.go (now in separate files) Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * docs(grpc): update design and implementation plan to reflect actual implementation - Add ServiceProtoFiles field to Info struct documentation - Document conditional HTTP annotations support (only when HasGoogleAPI is true) - Update demo project structure to include third_party/google/api - Mark both documents as completed - Add implementation summary with key design decisions Signed-off-by: spencercjh <spencercjh@gmail.com> * ci: add Protoc installation step in CI configuration Signed-off-by: spencercjh <spencercjh@gmail.com> * ci: add setup step for protoc-gen-connect-openapi Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(grpcprotoc): remove duplicate google/api/annotations.proto check The hasGoogleAPIImport method had two identical conditions checking for 'google/api/annotations.proto'. Since protobuf only uses double quotes for imports, the single-quote check is unnecessary and has been removed. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * refactor(grpcprotoc): simplify findOutputFile logic Remove redundant fallback check that could match files with unexpected format. Now the method only looks for files with the expected extension based on the requested format. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * refactor(grpcprotoc): remove duplicate FrameworkName constant Keep only ExtractorName to extractor.go to follow the existing convention (spring, gozero). Remove FrameworkName from grpcprotoc.go and update all references. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * refactor(grpcprotoc): use strings.Contains instead of custom helper Replace custom contains function with standard library strings.Contains for consistency and clarity. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(grpcprotoc): add service definition requirement in detector The detector now rejects projects that have .proto files but no service definitions. This prevents the grpc-protoc extractor from being selected for non-gRPC projects that leading to confusing errors during generation. - Update detector to return ErrNotProtocProject when no service definitions found - Update test cases to include service definitions in proto files - Use proper proto file formatting with newlines for service definitions Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(grpcprotoc): pass OutputFile to protoc plugin and simplify findOutputFile - Add output_name option to protoc-gen-connect-openapi when OutputFile is specified - Remove recursive search fallback in findOutputFile to avoid finding unrelated files - Only search output directory and service proto file directories Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(e2e): update grpc_protoc_test to use ExtractorName Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(grpcprotoc): handle comments in hasServiceDefinition and avoid redundant search in findOutputFile - Add proper comment handling (// and /* */) in hasServiceDefinition - Use set to skip already-searched directories in findOutputFile - Address code review feedback from PR #14 Signed-off-by: spencercjh <spencercjh@gmail.com> --------- Signed-off-by: Claude <claude@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com> Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <claude@anthropic.com>
1 parent 8554511 commit 515b70a

28 files changed

Lines changed: 5519 additions & 26 deletions

.github/workflows/ci.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,5 +99,11 @@ jobs:
9999
java-version: "25"
100100
distribution: "temurin"
101101

102+
- name: Set up Protoc
103+
uses: arduino/setup-protoc@v3
104+
105+
- name: Set up protoc-gen-connect-openapi
106+
run: go install github.com/sudorandom/protoc-gen-connect-openapi@latest
107+
102108
- name: Run E2E tests
103109
run: make test-e2e

CLAUDE.md

Lines changed: 30 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,11 @@ make verify
2929

3030
## Architecture Overview
3131

32-
Spec Forge is a CLI tool that generates enriched OpenAPI specifications from Spring Boot projects.
32+
Spec Forge is a CLI tool that generates enriched OpenAPI specifications from various frameworks (Spring Boot, go-zero, gRPC-protoc).
3333

3434
**Core workflow:**
3535
```
36-
Source Code → Detect → Patch → Generate → Validate → Enrich → Restore
36+
Source Code → Detect → Patch → Generate → Validate → Enrich → Publish
3737
```
3838

3939
### Package Structure
@@ -43,27 +43,41 @@ cmd/ # Cobra CLI commands
4343
├── root.go # Entry point, config initialization
4444
├── generate.go # `spec-forge generate` - full pipeline
4545
├── enrich.go # `spec-forge enrich` - standalone enrichment
46-
├── spring.go # `spec-forge spring` - patch/detect subcommands
46+
├── publish.go # `spec-forge publish` - publish to platforms
4747
4848
internal/
4949
├── config/ # Viper configuration loading
5050
├── executor/ # Shell command execution with timeout
5151
├── extractor/ # OpenAPI spec extraction
5252
│ ├── types.go # GenerateOptions, GenerateResult, etc.
53-
│ └── spring/ # Spring Boot specific implementation
54-
│ ├── detector.go # Project type detection (Maven/Gradle)
55-
│ ├── patcher.go # springdoc dependency injection
56-
│ ├── generator.go # Maven/Gradle command execution
57-
│ ├── maven.go # POM parsing, spring-boot plugin config
58-
│ └── gradle.go # build.gradle parsing
53+
│ ├── builtin/ # Built-in extractor registry
54+
│ ├── spring/ # Spring Boot implementation
55+
│ │ ├── detector.go # Project type detection (Maven/Gradle)
56+
│ │ ├── patcher.go # springdoc dependency injection
57+
│ │ ├── generator.go # Maven/Gradle command execution
58+
│ │ ├── maven.go # POM parsing, spring-boot plugin config
59+
│ │ └── gradle.go # build.gradle parsing
60+
│ ├── gozero/ # go-zero implementation
61+
│ │ ├── detector.go # go.mod parsing, dependency detection
62+
│ │ ├── patcher.go # go-swagger installation check
63+
│ │ └── generator.go # goctl command execution
64+
│ └── grpcprotoc/ # gRPC-protoc implementation
65+
│ ├── detector.go # .proto file detection, buf.yaml rejection
66+
│ ├── patcher.go # protoc tools check
67+
│ ├── generator.go # protoc command execution
68+
│ └── grpcprotoc.go # Info struct with ProtoFiles, ServiceProtoFiles
5969
├── validator/ # kin-openapi validation
60-
└── enricher/ # LLM-based description enrichment
61-
├── enricher.go # Main enricher interface
62-
├── config.go # Enricher configuration
63-
├── prompt/ # Prompt templates
64-
├── processor/ # Batching and concurrent processing
65-
└── provider/ # LLM providers (factory pattern)
66-
└── factory.go # Use NewProvider(cfg Config) to create providers
70+
├── enricher/ # LLM-based description enrichment
71+
│ ├── enricher.go # Main enricher interface
72+
│ ├── config.go # Enricher configuration
73+
│ ├── prompt/ # Prompt templates
74+
│ ├── processor/ # Batching and concurrent processing
75+
│ ├── specctx/ # Spec context extraction (reserved for future)
76+
│ └── provider/ # LLM providers (factory pattern)
77+
│ └── factory.go # Use NewProvider(cfg Config) to create providers
78+
└── publisher/ # OpenAPI spec publishing
79+
├── local.go # Local file publishing
80+
└── readme.go # ReadMe.com publishing via rdme CLI
6781
```
6882

6983
### Data Flow

README.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,9 +99,33 @@ LLM_API_KEY="sk-xxx" spec-forge enrich ./openapi.json \
9999
|----------------------------------------------------------------------------------------------------------------------------------------------|----------------|----------------|
100100
| [Spring Boot](https://springdoc.org/#plugins) | Java | ✅ Supported |
101101
| [go-zero](https://go-zero.dev/reference/cli-guide/swagger/) | Go | ✅ Supported |
102+
| [gRPC (protoc)](https://github.com/sudorandom/protoc-gen-connect-openapi) | Multi-language | ✅ Supported |
102103
| [Hertz](https://github.com/hertz-contrib/swagger-generate/tree/main/protoc-gen-http-swagger) | Go | 🚧 Coming soon |
103104
| [Kitex](https://github.com/hertz-contrib/swagger-generate/tree/main/protoc-gen-rpc-swagger) | Go | 🚧 Coming soon |
104-
| [gRPC](https://github.com/grpc-ecosystem/grpc-gateway?tab=readme-ov-file#6-optional-generate-openapi-definitions-using-protoc-gen-openapiv2) | Multi-language | 🚧 Coming soon |
105+
106+
### gRPC Projects (Native protoc)
107+
108+
For gRPC projects using native protoc (not buf-managed):
109+
110+
```bash
111+
# Generate OpenAPI spec from proto files
112+
spec-forge generate ./my-grpc-project
113+
114+
# With additional import paths
115+
spec-forge generate ./my-grpc-project --proto-import-path ./third_party --proto-import-path ./vendor
116+
117+
# Generate with AI enrichment
118+
LLM_API_KEY="your-key" spec-forge generate ./my-grpc-project --enrich --language zh
119+
```
120+
121+
**Requirements:**
122+
- `protoc` installed ([install guide](https://github.com/protocolbuffers/protobuf/releases))
123+
- `protoc-gen-connect-openapi` installed:
124+
```bash
125+
go install github.com/sudorandom/protoc-gen-connect-openapi@latest
126+
```
127+
128+
**Note:** buf-managed projects are not supported in this mode. Use `buf generate` with the plugin, then use `spec-forge enrich` on the generated OpenAPI spec.
105129

106130
## Supported Publishers
107131

cmd/generate.go

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ var (
4444
generatePublishTarget string
4545
// generatePublishOverwrite controls whether to overwrite existing remote spec
4646
generatePublishOverwrite bool
47+
// generateProtoImportPaths are additional import paths for protoc (-I flags)
48+
generateProtoImportPaths []string
4749
)
4850

4951
// generateCmd represents the generate command
@@ -123,10 +125,11 @@ func runGenerate(cmd *cobra.Command, args []string) error { //nolint:gocyclo //
123125
}
124126

125127
genOpts := &extractor.GenerateOptions{
126-
OutputDir: outputDir,
127-
Format: config.Get().Output.Format,
128-
Timeout: generateTimeout,
129-
SkipTests: true,
128+
OutputDir: outputDir,
129+
Format: config.Get().Output.Format,
130+
Timeout: generateTimeout,
131+
SkipTests: true,
132+
ProtoImportPaths: generateProtoImportPaths,
130133
}
131134

132135
genResult, err := extractorImpl.Generate(ctx, path, info, genOpts)
@@ -262,6 +265,8 @@ func init() {
262265
"publish target (local, readme)")
263266
generateCmd.Flags().BoolVar(&generatePublishOverwrite, "publish-overwrite", false,
264267
"overwrite existing spec (applies to both local and remote publishers)")
268+
generateCmd.Flags().StringSliceVar(&generateProtoImportPaths, "proto-import-path", nil,
269+
"additional import paths for protoc (-I flags), can be specified multiple times")
265270
}
266271

267272
// enrichGeneratedSpec enriches the generated spec with AI-generated descriptions

0 commit comments

Comments
 (0)