Skip to content

Commit d304a4f

Browse files
committed
feat(grpc): detect Go grpc-go client call sites as client-role routes
1 parent 8b4b142 commit d304a4f

14 files changed

Lines changed: 606 additions & 7 deletions

File tree

ARCHITECTURE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -554,7 +554,7 @@ Each extractor is detected by characteristic project files and then parses what
554554

555555
**OpenAPI** scans for spec files independently of the main walker (so it finds them even when `*.yaml`/`*.json` are globally ignored), confirming candidates by an `openapi:`/`swagger:` key. It emits one `route` per operation enriched with method, `operationId`, summary, tags, and a spec back-reference; specs under an `openapi/client/` directory are marked `role:"client"`. Gateway extensions (`x-gateway-config`, `x-gateway-capabilities`) are parsed into props.
556556

557-
**gRPC** models Protocol Buffers services the same way HTTP endpoints are modeled, so a gRPC surface answers the same cross-repo and unused-endpoint questions as a REST one. A small dependency-free proto3 scanner (comment-stripped, brace-depth aware — the same class of parser as OpenAPI's) reads each `.proto` and emits, for every `rpc`, a **server-role `route`** whose `Name` is the gRPC wire path `/pkg.Service/Method` (e.g. `/users.v1.UserService/GetUser`) with `method:"POST"`, `framework:"grpc"`, `source:"grpc-proto"`, `type:"grpc"`, and `rpc_service`/`rpc_method`/`streaming` props — the exact path+method a gRPC-web client hits over HTTP, so these flow through the cross-repo linker's normalized path+method matching and the `unused-routes` explainer with no linker special-casing. Each service also emits an `interface` symbol, each RPC a `method` symbol (`has_method`-linked to its service), and each message a `struct`/`enum` symbol, and proto `import`s become `dependency` facts — so proto participates in `traverse`, `find_path`, and `impact_analysis`. **Client-side detection lives in the TypeScript extractor**: a repo-wide pre-pass resolves generated gRPC-web stubs (`@protobuf-ts`, and any stub carrying the `@generated from protobuf service`/`new ServiceType(...)`/connect-es `typeName` markers) into a service→method map, then per-file it binds `new XxxServiceClient(...)` variables (including typed constructor-injected fields) and emits a **client-role `route`** (`source:"ts-grpc-client"`) for each `client.method(...)` **call site** — only for methods actually called, so an RPC the frontend never invokes correctly surfaces as unmatched by clients. Cross-repo gRPC edges are tagged `via:"grpc"`. **Go handler binding:** a post-extraction pass connects each gRPC server route to the Go method that serves it via a `handled_by` edge (route → `pkg.Type.Method`) and a `handler` prop, so `impact_analysis`/`find_path` traverse from the RPC to its implementation (and, through the cross-repo edges, on to its clients). The bridge is the `protoc-gen-go-grpc` forward-compatibility convention — a server impl embeds `Unimplemented<Service>Server`, which the Go extractor already records as an `implements` edge, so the service short name matches the route's `rpc_service` with no new Go parsing; ambiguous or non-embedding impls are left unbound. *Scope:* Go gRPC *clients* are not yet detected (the client side is TypeScript gRPC-web only for now).
557+
**gRPC** models Protocol Buffers services the same way HTTP endpoints are modeled, so a gRPC surface answers the same cross-repo and unused-endpoint questions as a REST one. A small dependency-free proto3 scanner (comment-stripped, brace-depth aware — the same class of parser as OpenAPI's) reads each `.proto` and emits, for every `rpc`, a **server-role `route`** whose `Name` is the gRPC wire path `/pkg.Service/Method` (e.g. `/users.v1.UserService/GetUser`) with `method:"POST"`, `framework:"grpc"`, `source:"grpc-proto"`, `type:"grpc"`, and `rpc_service`/`rpc_method`/`streaming` props — the exact path+method a gRPC-web client hits over HTTP, so these flow through the cross-repo linker's normalized path+method matching and the `unused-routes` explainer with no linker special-casing. Each service also emits an `interface` symbol, each RPC a `method` symbol (`has_method`-linked to its service), and each message a `struct`/`enum` symbol, and proto `import`s become `dependency` facts — so proto participates in `traverse`, `find_path`, and `impact_analysis`. **Client-side detection** covers both TypeScript and Go. In the **TypeScript** extractor a repo-wide pre-pass resolves generated gRPC-web stubs (`@protobuf-ts`, and any stub carrying the `@generated from protobuf service`/`new ServiceType(...)`/connect-es `typeName` markers) into a service→method map, then per-file it binds `new XxxServiceClient(...)` variables (including typed constructor-injected fields) and emits a **client-role `route`** (`source:"ts-grpc-client"`) for each `client.method(...)` **call site** — only for methods actually called, so an RPC the frontend never invokes correctly surfaces as unmatched by clients. The **Go** extractor does the same for grpc-go consumers: because a Go call site (`client.GetUser(ctx, req)`) carries no wire path, a repo-wide pre-pass reads the authoritative `/pkg.Service/Method` from the *generated* concrete client's `Invoke`/`NewStream` string literal (unary + streaming), builds a client-interface→method→path index, then binds `NewXxxClient(...)` variables per file and emits a **client-role `route`** (`source:"go-grpc-client"`) per call site. Cross-repo gRPC edges are tagged `via:"grpc"`. **Go handler binding:** a post-extraction pass connects each gRPC server route to the Go method that serves it via a `handled_by` edge (route → `pkg.Type.Method`) and a `handler` prop, so `impact_analysis`/`find_path` traverse from the RPC to its implementation (and, through the cross-repo edges, on to its clients). The bridge is the `protoc-gen-go-grpc` forward-compatibility convention — a server impl embeds `Unimplemented<Service>Server`, which the Go extractor already records as an `implements` edge, so the service short name matches the route's `rpc_service` with no new Go parsing; ambiguous or non-embedding impls are left unbound. *Scope:* client detection targets protoc-gen-go-grpc / `@protobuf-ts`-style generated stubs; other client libraries (e.g. connect-go/connect-es) are not yet recognized.
558558

559559
**Ruby** is parsed with tree-sitter, replacing the former line-based regex scanner — the grammar handles heredocs, endless methods (`def x = expr`), multi-line expressions, and the nested scopes that tripped up the line scanner. It is Rails-aware: ActiveRecord models (`has_many`/`has_one`/`belongs_to`/`has_and_belongs_to_many`, scopes, table inference, explicit `self.table_name`) emit `storage` facts; the route DSL in `config/routes.rb` (plus `config/routes/*.rb` and packwerk `draw`) is walked from the real block structure, so nested `namespace`/`scope`/`resources`/`member`/`collection` blocks produce one `route` per RESTful action (honoring `only:`/`except:`); and Packwerk package boundaries (`package.yml` dependency enforcement, `app/public/` privacy) are parsed. It tracks modules, classes, methods with `public`/`private`/`protected` visibility, `class << self` eigenclass and `module_function` methods — now correctly typed as class methods rather than instance methods — mixins (`include`/`extend`/`prepend` → `implements` edges), `ActiveSupport::Concern` (flagged `concern: true`), constants, and `attr_*` accessors. Like the other AST extractors, it walks method bodies for call sites, emitting `calls` edges (qualified `Const.method`/`Ns::Class.method` and receiver `var.method`, deduplicated) and `implements` edges for superclasses — so Ruby participates in `traverse`, `find_path`, and `impact_analysis`.
560560

internal/cachecov/coverage_test.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ var versionCoverage = map[int][]string{
105105
71: {"TestExtractStoredMethodEndpointFacts"}, // Swift stored-method endpoints
106106
72: {"TestWrapperEndpoint_PathAndVerbFromCallSite", "TestRoutes_NestedSingularResource"},// Swift request-wrapper + Ruby nested resources
107107
73: {"TestExtract_ServerRoutesPerRPC", "TestGRPCClient_OnlyCalledMethodsEmitted"}, // gRPC proto server routes + TS gRPC-web client routes
108+
74: {"TestGoGRPCClient_EmitsClientRoutes"}, // Go gRPC client call-site routes
108109
}
109110

110111
func TestCacheVersionCoverage(t *testing.T) {

internal/engine/cache.go

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,12 @@ import (
147147
// extractor detects gRPC-web client call sites as client-role routes
148148
// (source "ts-grpc-client"), so gRPC flows through the cross-repo linker and
149149
// unused-routes like HTTP.
150-
const cacheVersion = "v73"
150+
// v74: Go extractor detects gRPC client call sites (NewXxxClient(...) +
151+
// client.Method(...)) and emits client-role routes (source "go-grpc-client"),
152+
// resolving the wire path from the generated concrete client's Invoke/NewStream
153+
// literal. Documentation-only bump — goextractor is not a FileOwner, so its
154+
// facts are never cached; recorded for changelog continuity.
155+
const cacheVersion = "v74"
151156

152157
// extractorCache holds per-extractor facts keyed by a content hash of the files
153158
// the extractor depends on. It is loaded from disk at the start of a snapshot and

internal/engine/golden_test.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ var fixtures = []fixture{
5757
{name: "openapi_sample", subRepos: []string{"."}},
5858
{name: "multirepo", subRepos: []string{"repoA", "repoB"}},
5959
{name: "php_multirepo", subRepos: []string{"provider", "consumer"}},
60+
{name: "go_grpc_multirepo", subRepos: []string{"server", "client"}},
6061
}
6162

6263
func TestGolden(t *testing.T) {
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{"kind":"dependency","name":". -\u003e context","file":"client/main.go","line":4,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"context"}]}
2+
{"kind":"dependency","name":". -\u003e google.golang.org/grpc","file":"client/main.go","line":8,"repo":"client","props":{"language":"go","source":"external"},"relations":[{"kind":"imports","target":"google.golang.org/grpc"}]}
3+
{"kind":"dependency","name":". -\u003e grpcclient/gen/users/v1","file":"client/main.go","line":6,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"gen/users/v1"}]}
4+
{"kind":"dependency","name":"client -\u003e server","repo":"client","props":{"confidence":"verified","endpoint_count":1,"endpoints":["POST /users.v1.UserService/CreateUser"],"synthetic":"crossrepo","type":"cross_repo","via":["grpc"]}}
5+
{"kind":"dependency","name":"gen/users/v1 -\u003e context","file":"client/gen/users/v1/users_grpc.pb.go","line":6,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"context"}]}
6+
{"kind":"dependency","name":"gen/users/v1 -\u003e google.golang.org/grpc","file":"client/gen/users/v1/users_grpc.pb.go","line":7,"repo":"client","props":{"language":"go","source":"external"},"relations":[{"kind":"imports","target":"google.golang.org/grpc"}]}
7+
{"kind":"module","name":".","file":"client/.","repo":"client","props":{"language":"go","modulePath":"grpcclient","package":"main"}}
8+
{"kind":"module","name":"gen/users/v1","file":"client/gen/users/v1","repo":"client","props":{"language":"go","package":"usersv1"}}
9+
{"kind":"module","name":"proto/users/v1","file":"server/proto/users/v1","repo":"server","props":{"language":"grpc","package":"users.v1"}}
10+
{"kind":"route","name":"/users.v1.UserService/CreateUser","file":"client/main.go","line":21,"repo":"client","props":{"framework":"grpc","language":"go","method":"POST","role":"client","rpc_method":"CreateUser","rpc_service":"users.v1.UserService","source":"go-grpc-client","type":"grpc"},"relations":[{"kind":"declares","target":"."}]}
11+
{"kind":"route","name":"/users.v1.UserService/CreateUser","file":"server/proto/users/v1/users.proto","line":12,"repo":"server","props":{"framework":"grpc","language":"grpc","method":"POST","role":"server","rpc_method":"CreateUser","rpc_service":"users.v1.UserService","source":"grpc-proto","streaming":"none","type":"grpc"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
12+
{"kind":"route","name":"/users.v1.UserService/GetUser","file":"server/proto/users/v1/users.proto","line":11,"repo":"server","props":{"framework":"grpc","language":"grpc","method":"POST","role":"server","rpc_method":"GetUser","rpc_service":"users.v1.UserService","source":"grpc-proto","streaming":"none","type":"grpc","unmatched_by_clients":true},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
13+
{"kind":"service","name":"client","repo":"client","props":{"edge_coverage":[{"detected":1,"edge_type":"http_client","resolved":1,"unresolved":0}],"synthetic":"crossrepo"},"relations":[{"kind":"depends_on","target":"server"}]}
14+
{"kind":"service","name":"server","repo":"server","props":{"synthetic":"crossrepo"}}
15+
{"kind":"symbol","name":"..main","file":"client/main.go","line":13,"repo":"client","props":{"cyclomatic":2,"exported":false,"language":"go","symbol_kind":"function"},"relations":[{"kind":"calls","target":"conn.Close"},{"kind":"calls","target":"context.Background"},{"kind":"calls","target":"gen/users/v1.NewUserServiceClient"},{"kind":"calls","target":"gen/users/v1.UserServiceClient.CreateUser"},{"kind":"calls","target":"google.golang.org/grpc.Dial"},{"kind":"calls","target":"google.golang.org/grpc.WithInsecure"},{"kind":"declares","target":"."}]}
16+
{"kind":"symbol","name":"gen/users/v1.CreateUserRequest","file":"client/gen/users/v1/users_grpc.pb.go","line":12,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
17+
{"kind":"symbol","name":"gen/users/v1.CreateUserResponse","file":"client/gen/users/v1/users_grpc.pb.go","line":13,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
18+
{"kind":"symbol","name":"gen/users/v1.GetUserRequest","file":"client/gen/users/v1/users_grpc.pb.go","line":10,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
19+
{"kind":"symbol","name":"gen/users/v1.GetUserResponse","file":"client/gen/users/v1/users_grpc.pb.go","line":11,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
20+
{"kind":"symbol","name":"gen/users/v1.NewUserServiceClient","file":"client/gen/users/v1/users_grpc.pb.go","line":28,"repo":"client","props":{"cyclomatic":1,"exported":true,"language":"go","symbol_kind":"function"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
21+
{"kind":"symbol","name":"gen/users/v1.User","file":"client/gen/users/v1/users_grpc.pb.go","line":14,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
22+
{"kind":"symbol","name":"gen/users/v1.UserServiceClient","file":"client/gen/users/v1/users_grpc.pb.go","line":19,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"interface"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
23+
{"kind":"symbol","name":"gen/users/v1.userServiceClient","file":"client/gen/users/v1/users_grpc.pb.go","line":24,"repo":"client","props":{"exported":false,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
24+
{"kind":"symbol","name":"gen/users/v1.userServiceClient.CreateUser","file":"client/gen/users/v1/users_grpc.pb.go","line":41,"repo":"client","props":{"cyclomatic":2,"exported":true,"language":"go","receiver":"userServiceClient","symbol_kind":"method"},"relations":[{"kind":"calls","target":"google.golang.org/grpc.ClientConnInterface.Invoke"},{"kind":"declares","target":"gen/users/v1"}]}
25+
{"kind":"symbol","name":"gen/users/v1.userServiceClient.GetUser","file":"client/gen/users/v1/users_grpc.pb.go","line":32,"repo":"client","props":{"cyclomatic":2,"exported":true,"language":"go","receiver":"userServiceClient","symbol_kind":"method"},"relations":[{"kind":"calls","target":"google.golang.org/grpc.ClientConnInterface.Invoke"},{"kind":"declares","target":"gen/users/v1"}]}
26+
{"kind":"symbol","name":"proto/users/v1.CreateUserRequest","file":"server/proto/users/v1/users.proto","line":17,"repo":"server","props":{"exported":true,"language":"grpc","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
27+
{"kind":"symbol","name":"proto/users/v1.CreateUserResponse","file":"server/proto/users/v1/users.proto","line":18,"repo":"server","props":{"exported":true,"language":"grpc","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
28+
{"kind":"symbol","name":"proto/users/v1.GetUserRequest","file":"server/proto/users/v1/users.proto","line":15,"repo":"server","props":{"exported":true,"language":"grpc","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
29+
{"kind":"symbol","name":"proto/users/v1.GetUserResponse","file":"server/proto/users/v1/users.proto","line":16,"repo":"server","props":{"exported":true,"language":"grpc","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
30+
{"kind":"symbol","name":"proto/users/v1.User","file":"server/proto/users/v1/users.proto","line":5,"repo":"server","props":{"exported":true,"language":"grpc","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
31+
{"kind":"symbol","name":"proto/users/v1.UserService","file":"server/proto/users/v1/users.proto","line":10,"repo":"server","props":{"exported":true,"framework":"grpc","language":"grpc","rpc_service":"users.v1.UserService","symbol_kind":"interface"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
32+
{"kind":"symbol","name":"proto/users/v1.UserService.CreateUser","file":"server/proto/users/v1/users.proto","line":12,"repo":"server","props":{"exported":true,"framework":"grpc","language":"grpc","streaming":"none","symbol_kind":"method"},"relations":[{"kind":"declares","target":"proto/users/v1"},{"kind":"has_method","target":"proto/users/v1.UserService"}]}
33+
{"kind":"symbol","name":"proto/users/v1.UserService.GetUser","file":"server/proto/users/v1/users.proto","line":11,"repo":"server","props":{"exported":true,"framework":"grpc","language":"grpc","streaming":"none","symbol_kind":"method"},"relations":[{"kind":"declares","target":"proto/users/v1"},{"kind":"has_method","target":"proto/users/v1.UserService"}]}

internal/engine/testdata/repos/go_grpc_multirepo/client/gen/users/v1/users_grpc.pb.go

Lines changed: 48 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module grpcclient
2+
3+
go 1.21
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
package main
2+
3+
import (
4+
"context"
5+
6+
usersv1 "grpcclient/gen/users/v1"
7+
8+
"google.golang.org/grpc"
9+
)
10+
11+
// main dials the user service and creates a user. It calls only CreateUser —
12+
// GetUser is never invoked, so the server's GetUser RPC stays unmatched.
13+
func main() {
14+
conn, err := grpc.Dial("localhost:8080", grpc.WithInsecure())
15+
if err != nil {
16+
panic(err)
17+
}
18+
defer conn.Close()
19+
20+
client := usersv1.NewUserServiceClient(conn)
21+
client.CreateUser(context.Background(), &usersv1.CreateUserRequest{Name: "ada"})
22+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module grpcserver
2+
3+
go 1.21

0 commit comments

Comments
 (0)