Skip to content

Commit a82cf2e

Browse files
committed
feat(grpc): detect struct-field-injected, connect-go, and connect-es clients
1 parent d304a4f commit a82cf2e

10 files changed

Lines changed: 370 additions & 132 deletions

File tree

ARCHITECTURE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -554,7 +554,7 @@ Each extractor is detected by characteristic project files and then parses what
554554

555555
**OpenAPI** scans for spec files independently of the main walker (so it finds them even when `*.yaml`/`*.json` are globally ignored), confirming candidates by an `openapi:`/`swagger:` key. It emits one `route` per operation enriched with method, `operationId`, summary, tags, and a spec back-reference; specs under an `openapi/client/` directory are marked `role:"client"`. Gateway extensions (`x-gateway-config`, `x-gateway-capabilities`) are parsed into props.
556556

557-
**gRPC** models Protocol Buffers services the same way HTTP endpoints are modeled, so a gRPC surface answers the same cross-repo and unused-endpoint questions as a REST one. A small dependency-free proto3 scanner (comment-stripped, brace-depth aware — the same class of parser as OpenAPI's) reads each `.proto` and emits, for every `rpc`, a **server-role `route`** whose `Name` is the gRPC wire path `/pkg.Service/Method` (e.g. `/users.v1.UserService/GetUser`) with `method:"POST"`, `framework:"grpc"`, `source:"grpc-proto"`, `type:"grpc"`, and `rpc_service`/`rpc_method`/`streaming` props — the exact path+method a gRPC-web client hits over HTTP, so these flow through the cross-repo linker's normalized path+method matching and the `unused-routes` explainer with no linker special-casing. Each service also emits an `interface` symbol, each RPC a `method` symbol (`has_method`-linked to its service), and each message a `struct`/`enum` symbol, and proto `import`s become `dependency` facts — so proto participates in `traverse`, `find_path`, and `impact_analysis`. **Client-side detection** covers both TypeScript and Go. In the **TypeScript** extractor a repo-wide pre-pass resolves generated gRPC-web stubs (`@protobuf-ts`, and any stub carrying the `@generated from protobuf service`/`new ServiceType(...)`/connect-es `typeName` markers) into a service→method map, then per-file it binds `new XxxServiceClient(...)` variables (including typed constructor-injected fields) and emits a **client-role `route`** (`source:"ts-grpc-client"`) for each `client.method(...)` **call site** — only for methods actually called, so an RPC the frontend never invokes correctly surfaces as unmatched by clients. The **Go** extractor does the same for grpc-go consumers: because a Go call site (`client.GetUser(ctx, req)`) carries no wire path, a repo-wide pre-pass reads the authoritative `/pkg.Service/Method` from the *generated* concrete client's `Invoke`/`NewStream` string literal (unary + streaming), builds a client-interface→method→path index, then binds `NewXxxClient(...)` variables per file and emits a **client-role `route`** (`source:"go-grpc-client"`) per call site. Cross-repo gRPC edges are tagged `via:"grpc"`. **Go handler binding:** a post-extraction pass connects each gRPC server route to the Go method that serves it via a `handled_by` edge (route → `pkg.Type.Method`) and a `handler` prop, so `impact_analysis`/`find_path` traverse from the RPC to its implementation (and, through the cross-repo edges, on to its clients). The bridge is the `protoc-gen-go-grpc` forward-compatibility convention — a server impl embeds `Unimplemented<Service>Server`, which the Go extractor already records as an `implements` edge, so the service short name matches the route's `rpc_service` with no new Go parsing; ambiguous or non-embedding impls are left unbound. *Scope:* client detection targets protoc-gen-go-grpc / `@protobuf-ts`-style generated stubs; other client libraries (e.g. connect-go/connect-es) are not yet recognized.
557+
**gRPC** models Protocol Buffers services the same way HTTP endpoints are modeled, so a gRPC surface answers the same cross-repo and unused-endpoint questions as a REST one. A small dependency-free proto3 scanner (comment-stripped, brace-depth aware — the same class of parser as OpenAPI's) reads each `.proto` and emits, for every `rpc`, a **server-role `route`** whose `Name` is the gRPC wire path `/pkg.Service/Method` (e.g. `/users.v1.UserService/GetUser`) with `method:"POST"`, `framework:"grpc"`, `source:"grpc-proto"`, `type:"grpc"`, and `rpc_service`/`rpc_method`/`streaming` props — the exact path+method a gRPC-web client hits over HTTP, so these flow through the cross-repo linker's normalized path+method matching and the `unused-routes` explainer with no linker special-casing. Each service also emits an `interface` symbol, each RPC a `method` symbol (`has_method`-linked to its service), and each message a `struct`/`enum` symbol, and proto `import`s become `dependency` facts — so proto participates in `traverse`, `find_path`, and `impact_analysis`. **Client-side detection** covers both TypeScript and Go. In the **TypeScript** extractor a repo-wide pre-pass resolves generated gRPC-web stubs (`@protobuf-ts`, and any stub carrying the `@generated from protobuf service`/`new ServiceType(...)`/connect-es `typeName` markers) into a service→method map, then per-file it binds `new XxxServiceClient(...)` variables (including typed constructor-injected fields) and emits a **client-role `route`** (`source:"ts-grpc-client"`) for each `client.method(...)` **call site** — only for methods actually called, so an RPC the frontend never invokes correctly surfaces as unmatched by clients. The **Go** extractor does the same for grpc-go consumers: because a Go call site (`client.GetUser(ctx, req)`) carries no wire path, a repo-wide pre-pass reads the authoritative `/pkg.Service/Method` from the *generated* code — the concrete client's `Invoke`/`NewStream` string literal (grpc-go, unary + streaming) or the `…Procedure` const (connect-go) — and builds a client-interface→method→path index. Per-file, it reuses the Go extractor's own receiver/field/local-variable type resolution (`resolveChain`) so a client is recognized whether it's a **local variable**, an **inline construction**, or a **struct field** (`s.users.GetUser(...)` — dependency injection), emitting a **client-role `route`** (`source:"go-grpc-client"`) per call site. Both **grpc-go** and **connect-go** consumers are covered (a client held only in a package-level var is the one unhandled case). On the TypeScript side, **connect-es** consumers using `createClient(Service, transport)` / `createPromiseClient(...)` are detected alongside the `new XxxClient(...)` form. Cross-repo gRPC edges are tagged `via:"grpc"`. **Go handler binding:** a post-extraction pass connects each gRPC server route to the Go method that serves it via a `handled_by` edge (route → `pkg.Type.Method`) and a `handler` prop, so `impact_analysis`/`find_path` traverse from the RPC to its implementation (and, through the cross-repo edges, on to its clients). The bridge is the `protoc-gen-go-grpc` forward-compatibility convention — a server impl embeds `Unimplemented<Service>Server`, which the Go extractor already records as an `implements` edge, so the service short name matches the route's `rpc_service` with no new Go parsing; ambiguous or non-embedding impls are left unbound. *Scope:* client detection targets protoc-gen-go-grpc, connect-go, `@protobuf-ts`, and connect-es generated stubs; grpc-web JS and other bespoke client libraries are not yet recognized.
558558

559559
**Ruby** is parsed with tree-sitter, replacing the former line-based regex scanner — the grammar handles heredocs, endless methods (`def x = expr`), multi-line expressions, and the nested scopes that tripped up the line scanner. It is Rails-aware: ActiveRecord models (`has_many`/`has_one`/`belongs_to`/`has_and_belongs_to_many`, scopes, table inference, explicit `self.table_name`) emit `storage` facts; the route DSL in `config/routes.rb` (plus `config/routes/*.rb` and packwerk `draw`) is walked from the real block structure, so nested `namespace`/`scope`/`resources`/`member`/`collection` blocks produce one `route` per RESTful action (honoring `only:`/`except:`); and Packwerk package boundaries (`package.yml` dependency enforcement, `app/public/` privacy) are parsed. It tracks modules, classes, methods with `public`/`private`/`protected` visibility, `class << self` eigenclass and `module_function` methods — now correctly typed as class methods rather than instance methods — mixins (`include`/`extend`/`prepend` → `implements` edges), `ActiveSupport::Concern` (flagged `concern: true`), constants, and `attr_*` accessors. Like the other AST extractors, it walks method bodies for call sites, emitting `calls` edges (qualified `Const.method`/`Ns::Class.method` and receiver `var.method`, deduplicated) and `implements` edges for superclasses — so Ruby participates in `traverse`, `find_path`, and `impact_analysis`.
560560

internal/cachecov/coverage_test.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@ var versionCoverage = map[int][]string{
106106
72: {"TestWrapperEndpoint_PathAndVerbFromCallSite", "TestRoutes_NestedSingularResource"},// Swift request-wrapper + Ruby nested resources
107107
73: {"TestExtract_ServerRoutesPerRPC", "TestGRPCClient_OnlyCalledMethodsEmitted"}, // gRPC proto server routes + TS gRPC-web client routes
108108
74: {"TestGoGRPCClient_EmitsClientRoutes"}, // Go gRPC client call-site routes
109+
75: {"TestConnectES_CreateClient", "TestGoGRPCClient_ConnectGo"}, // connect-es + connect-go + struct-field-injected clients
109110
}
110111

111112
func TestCacheVersionCoverage(t *testing.T) {

internal/engine/cache.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,11 @@ import (
152152
// resolving the wire path from the generated concrete client's Invoke/NewStream
153153
// literal. Documentation-only bump — goextractor is not a FileOwner, so its
154154
// facts are never cached; recorded for changelog continuity.
155-
const cacheVersion = "v74"
155+
// v75: broadened gRPC client detection — Go now resolves struct-field-injected
156+
// clients (via the field-type map) and connect-go (procedure-const paths); the
157+
// TypeScript extractor detects connect-es createClient/createPromiseClient(...)
158+
// call sites. Bump required because the TS extractor is a FileOwner (cached).
159+
const cacheVersion = "v75"
156160

157161
// extractorCache holds per-extractor facts keyed by a content hash of the files
158162
// the extractor depends on. It is loaded from disk at the start of a snapshot and

internal/engine/testdata/golden/go_grpc_multirepo.facts.jsonl

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,22 @@
11
{"kind":"dependency","name":". -\u003e context","file":"client/main.go","line":4,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"context"}]}
2+
{"kind":"dependency","name":". -\u003e context","file":"client/repo.go","line":4,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"context"}]}
23
{"kind":"dependency","name":". -\u003e google.golang.org/grpc","file":"client/main.go","line":8,"repo":"client","props":{"language":"go","source":"external"},"relations":[{"kind":"imports","target":"google.golang.org/grpc"}]}
34
{"kind":"dependency","name":". -\u003e grpcclient/gen/users/v1","file":"client/main.go","line":6,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"gen/users/v1"}]}
4-
{"kind":"dependency","name":"client -\u003e server","repo":"client","props":{"confidence":"verified","endpoint_count":1,"endpoints":["POST /users.v1.UserService/CreateUser"],"synthetic":"crossrepo","type":"cross_repo","via":["grpc"]}}
5+
{"kind":"dependency","name":". -\u003e grpcclient/gen/users/v1","file":"client/repo.go","line":6,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"gen/users/v1"}]}
6+
{"kind":"dependency","name":"client -\u003e server","repo":"client","props":{"confidence":"verified","endpoint_count":2,"endpoints":["POST /users.v1.UserService/CreateUser","POST /users.v1.UserService/GetUser"],"synthetic":"crossrepo","type":"cross_repo","via":["grpc"]}}
57
{"kind":"dependency","name":"gen/users/v1 -\u003e context","file":"client/gen/users/v1/users_grpc.pb.go","line":6,"repo":"client","props":{"language":"go","source":"stdlib"},"relations":[{"kind":"imports","target":"context"}]}
68
{"kind":"dependency","name":"gen/users/v1 -\u003e google.golang.org/grpc","file":"client/gen/users/v1/users_grpc.pb.go","line":7,"repo":"client","props":{"language":"go","source":"external"},"relations":[{"kind":"imports","target":"google.golang.org/grpc"}]}
79
{"kind":"module","name":".","file":"client/.","repo":"client","props":{"language":"go","modulePath":"grpcclient","package":"main"}}
810
{"kind":"module","name":"gen/users/v1","file":"client/gen/users/v1","repo":"client","props":{"language":"go","package":"usersv1"}}
911
{"kind":"module","name":"proto/users/v1","file":"server/proto/users/v1","repo":"server","props":{"language":"grpc","package":"users.v1"}}
1012
{"kind":"route","name":"/users.v1.UserService/CreateUser","file":"client/main.go","line":21,"repo":"client","props":{"framework":"grpc","language":"go","method":"POST","role":"client","rpc_method":"CreateUser","rpc_service":"users.v1.UserService","source":"go-grpc-client","type":"grpc"},"relations":[{"kind":"declares","target":"."}]}
1113
{"kind":"route","name":"/users.v1.UserService/CreateUser","file":"server/proto/users/v1/users.proto","line":12,"repo":"server","props":{"framework":"grpc","language":"grpc","method":"POST","role":"server","rpc_method":"CreateUser","rpc_service":"users.v1.UserService","source":"grpc-proto","streaming":"none","type":"grpc"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
12-
{"kind":"route","name":"/users.v1.UserService/GetUser","file":"server/proto/users/v1/users.proto","line":11,"repo":"server","props":{"framework":"grpc","language":"grpc","method":"POST","role":"server","rpc_method":"GetUser","rpc_service":"users.v1.UserService","source":"grpc-proto","streaming":"none","type":"grpc","unmatched_by_clients":true},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
13-
{"kind":"service","name":"client","repo":"client","props":{"edge_coverage":[{"detected":1,"edge_type":"http_client","resolved":1,"unresolved":0}],"synthetic":"crossrepo"},"relations":[{"kind":"depends_on","target":"server"}]}
14+
{"kind":"route","name":"/users.v1.UserService/GetUser","file":"client/repo.go","line":16,"repo":"client","props":{"framework":"grpc","language":"go","method":"POST","role":"client","rpc_method":"GetUser","rpc_service":"users.v1.UserService","source":"go-grpc-client","type":"grpc"},"relations":[{"kind":"declares","target":"."}]}
15+
{"kind":"route","name":"/users.v1.UserService/GetUser","file":"server/proto/users/v1/users.proto","line":11,"repo":"server","props":{"framework":"grpc","language":"grpc","method":"POST","role":"server","rpc_method":"GetUser","rpc_service":"users.v1.UserService","source":"grpc-proto","streaming":"none","type":"grpc"},"relations":[{"kind":"declares","target":"proto/users/v1"}]}
16+
{"kind":"service","name":"client","repo":"client","props":{"edge_coverage":[{"detected":2,"edge_type":"http_client","resolved":2,"unresolved":0}],"synthetic":"crossrepo"},"relations":[{"kind":"depends_on","target":"server"}]}
1417
{"kind":"service","name":"server","repo":"server","props":{"synthetic":"crossrepo"}}
18+
{"kind":"symbol","name":"..UserRepo","file":"client/repo.go","line":11,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"."}]}
19+
{"kind":"symbol","name":"..UserRepo.Fetch","file":"client/repo.go","line":15,"repo":"client","props":{"cyclomatic":1,"exported":true,"language":"go","receiver":"UserRepo","symbol_kind":"method"},"relations":[{"kind":"calls","target":"gen/users/v1.UserServiceClient.GetUser"},{"kind":"declares","target":"."}]}
1520
{"kind":"symbol","name":"..main","file":"client/main.go","line":13,"repo":"client","props":{"cyclomatic":2,"exported":false,"language":"go","symbol_kind":"function"},"relations":[{"kind":"calls","target":"conn.Close"},{"kind":"calls","target":"context.Background"},{"kind":"calls","target":"gen/users/v1.NewUserServiceClient"},{"kind":"calls","target":"gen/users/v1.UserServiceClient.CreateUser"},{"kind":"calls","target":"google.golang.org/grpc.Dial"},{"kind":"calls","target":"google.golang.org/grpc.WithInsecure"},{"kind":"declares","target":"."}]}
1621
{"kind":"symbol","name":"gen/users/v1.CreateUserRequest","file":"client/gen/users/v1/users_grpc.pb.go","line":12,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
1722
{"kind":"symbol","name":"gen/users/v1.CreateUserResponse","file":"client/gen/users/v1/users_grpc.pb.go","line":13,"repo":"client","props":{"exported":true,"language":"go","symbol_kind":"struct"},"relations":[{"kind":"declares","target":"gen/users/v1"}]}
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
package main
2+
3+
import (
4+
"context"
5+
6+
usersv1 "grpcclient/gen/users/v1"
7+
)
8+
9+
// UserRepo holds the gRPC client as a struct field (dependency injection). The
10+
// call goes through s.users, exercising field-type resolution.
11+
type UserRepo struct {
12+
users usersv1.UserServiceClient
13+
}
14+
15+
func (r *UserRepo) Fetch(ctx context.Context, id string) (*usersv1.GetUserResponse, error) {
16+
return r.users.GetUser(ctx, &usersv1.GetUserRequest{UserId: id})
17+
}

internal/extractors/goextractor/go.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -238,7 +238,7 @@ func (e *GoExtractor) extractFile(fset *token.FileSet, f *ast.File, relFile, pkg
238238
result = append(result, extractHTTPClientFacts(fset, f, relFile, pkgDir)...)
239239

240240
// Extract outbound gRPC-client calls
241-
result = append(result, extractGRPCClientFacts(fset, f, relFile, pkgDir, grpcStubs)...)
241+
result = append(result, extractGRPCClientFacts(fset, f, relFile, pkgDir, modulePath, fileImports, fieldTypes, grpcStubs)...)
242242

243243
// Extract storage patterns
244244
result = append(result, extractStorage(fset, f, relFile, pkgDir)...)

0 commit comments

Comments
 (0)