Skip to content

Reorganize integrations into self-contained packages#124

Merged
mirackara merged 23 commits intomainfrom
structure-cleanup
Mar 9, 2026
Merged

Reorganize integrations into self-contained packages#124
mirackara merged 23 commits intomainfrom
structure-cleanup

Conversation

@mirackara
Copy link
Contributor

Summary

Restructures the codebase to group each integration's parsing logic, code generation, and examples into self-contained packages under integrations/, following the New
Relic Go agent's industry-standard organization pattern.

Motivation

Previously, each integration was split across multiple locations:

  • Detection/parsing logic in parser/{integration}.go
  • Code generation in internal/codegen/{integration}.go
  • Examples scattered in end-to-end-tests/

This split structure required developers to jump between directories to understand a single integration, making the codebase harder to maintain and extend.

Changes

New Structure

integrations/
├── nragent/ # Agent initialization
├── nrnethttp/ # net/http integration
├── nrgin/ # Gin web framework
├── nrgochi/ # Go-chi router
├── nrgrpc/ # gRPC integration
├── nrmysql/ # MySQL/database
└── nrslog/ # Structured logging

Each integration now contains:

  • parsing/ - Detection and instrumentation logic
  • codegen/ - DST node generation
  • example/ - End-to-end test examples

Migration Details

  • Moved 7 integration parsing files from parser/integrations/nr*/parsing/
  • Moved 7 integration codegen files from internal/codegen/integrations/nr*/codegen/
  • Moved example applications to respective integration example/ directories
  • Updated parser/manager.go to import and register integration functions from new locations
  • Kept shared utilities in internal/codegen/ (transaction, segment, context helpers)
  • Updated end-to-end-tests/testcases.json with new example paths

Test Infrastructure

  • Created parser/testhelpers.go with exported test utilities for integration tests
  • Added test helper methods to InstrumentationManager for internal state access
  • Exported necessary helper functions across all integrations for test visibility
  • Moved manager-dependent tests to parser/grpc_internal_test.go
  • Cleaned CLI tool imports from test expectations

Migration Path for Future Integrations

To add a new integration, create:
integrations/nr{name}/
├── parsing/{name}.go # Detection logic
├── parsing/{name}_test.go # Tests
├── codegen/{name}.go # Code generation
└── example/ # Example applications

Then register tracing functions in parser/manager.go via DetectDependencyIntegrations().

mirackara added 14 commits March 3, 2026 14:32
- Move detection logic from parser/slog.go to integrations/nrslog/slog.go
- Move codegen from internal/codegen/slog.go to integrations/nrslog/codegen.go
- Move examples from end-to-end-tests/slog-examples to integrations/nrslog/example/
- Export InstrumentationManager.AddImport() for integration use
- Update testcases.json to point to new example location

The nrslog integration is now self-contained with detection, codegen, and
examples in one directory. This is the first of 7 integrations to be reorganized.

Part of structure cleanup to match New Relic Go Agent organization pattern.
- Move detection logic from parser/mysql.go to integrations/nrmysql/mysql.go
- Move codegen from internal/codegen/mysql.go to integrations/nrmysql/codegen.go
- Move examples from end-to-end-tests/mysql to integrations/nrmysql/example/
- Export InstrumentationManager.AgentVariableName() for integration use
- Add mysql test case to testcases.json

The nrmysql integration is now self-contained with detection, codegen, and
examples in one directory. Test files remain in parser/ for now (will move in v1).

Part of structure cleanup to match New Relic Go Agent organization pattern.
- Move detection logic from parser/gochi.go to integrations/nrgochi/chi.go
- Move codegen from internal/codegen/gochi.go to integrations/nrgochi/codegen.go
- Move examples from end-to-end-tests/gochi to integrations/nrgochi/example/
- Export InstrumentationManager.GetDecoratorPackage() for integration use
- Update testcases.json to point to new example location

The nrgochi integration is now self-contained with detection, codegen, and
examples in one directory.

Part of structure cleanup to match New Relic Go Agent organization pattern.
- Move detection logic from parser/gin.go to integrations/nrgin/gin.go
- Move codegen from internal/codegen/gin.go to integrations/nrgin/codegen.go
- Remove duplicate gin examples (already moved in stashed changes)
- Update testcases.json to point to new example locations

The nrgin integration is now self-contained with detection, codegen, and
examples in one directory.

Part of structure cleanup to match New Relic Go Agent organization pattern.
- Move detection logic from parser/grpc.go to integrations/nrgrpc/grpc.go
- Move codegen from internal/codegen/grpc.go to integrations/nrgrpc/codegen.go
- Move examples from end-to-end-tests/grpc to integrations/nrgrpc/example/
- Export InstrumentationManager.Facts() for integration use
- Update testcases.json to point to new example location

The nrgrpc integration is now self-contained with detection, codegen, and
examples in one directory.

Part of structure cleanup to match New Relic Go Agent organization pattern.
- Move detection logic from parser/agent.go to integrations/nragent/agent.go
- Move codegen from internal/codegen/agent.go to integrations/nragent/codegen.go
- Export additional InstrumentationManager methods:
  - SetAgentVariableName() for setting agent variable name
  - AppName() for getting application name
  - SetupFunc() / SetSetupFunc() for setup function management
  - ErrorCache() for error cache access

The nragent integration is now self-contained with detection and codegen
in one directory. No examples (core initialization logic).

Part of structure cleanup to match New Relic Go Agent organization pattern.
- Move detection logic from parser/netHTTP.go to integrations/nrnethttp/http.go
- Move codegen from internal/codegen/http.go to integrations/nrnethttp/codegen.go
- Move examples from end-to-end-tests to integrations/nrnethttp/example/
- Export InstrumentationManager.TransactionCache() for integration use
- Update testcases.json to point to new example locations

The nrnethttp integration is now self-contained with detection, codegen, and
examples in one directory.

Part of structure cleanup to match New Relic Go Agent organization pattern.

All 7 integrations now reorganized! Next: wire in cmd/ and cleanup old files.
Final cleanup after reorganizing all 7 integrations:

1. Created parser/errors.go with complete NoticeError implementation
   - Restored NoticeError function with all three cases (ReturnStmt, IfStmt, AssignStmt)
   - Added helper functions: errNilCheck, shouldNoticeError, findErrorVariable
   - Uses existing ErrorCache methods (GetExpression, IsExistingError, Load, Clear)

2. Moved HttpRequestContext to shared utilities
   - Added to internal/codegen/context.go since used by multiple integrations
   - Removed from nrnethttp/codegen.go to avoid duplication

3. Fixed integration codegen function calls
   - nrnethttp: Updated calls to use local functions (RoundTripper, WrapRequestContext, etc.)
   - nrgrpc: Updated calls to use local interceptor functions
   - Functions in same package don't need codegen. prefix

4. Fixed cmd/instrument.go registration
   - Wrapped registerIntegrations call in lambda to match expected signature
   - All integrations now properly registered via dependency injection

All 7 integrations now self-contained in integrations/ directory with proper
separation between integration-specific code and shared utilities.
Flatten example directories that had unnecessary double nesting:
- integrations/nrgrpc/example/grpc/grpc/* -> integrations/nrgrpc/example/grpc/*
- integrations/nrnethttp/example/http-app/http-app/* -> integrations/nrnethttp/example/http-app/*
- integrations/nrnethttp/example/http-mux-app/http-mux-app/* -> integrations/nrnethttp/example/http-mux-app/*
- integrations/nrgin/example/graceful-shutdown/graceful-shutdown/* -> integrations/nrgin/example/graceful-shutdown/*

This occurred during the mv operations when moving from end-to-end-tests/ to integrations/*/example/.
Restored all test files that were accidentally deleted in commit 89f7ded:

Integration-specific tests moved to integrations/*/:
- nragent: codegen_test.go (211 lines), parsing_test.go (676 lines)
- nrnethttp: codegen_test.go (133 lines), parsing_test.go (2138 lines)
- nrgin: codegen_test.go (125 lines), parsing_test.go (413 lines)
- nrgochi: codegen_test.go (79 lines), parsing_test.go (330 lines)
- nrgrpc: codegen_test.go (465 lines), parsing_test.go (785 lines)
- nrmysql: codegen_test.go (201 lines), parsing_test.go (559 lines)

Shared tests kept in original locations:
- parser/manager_test.go (1210 lines)
- parser/utils_test.go (196 lines)
- parser/preinstrumentationtracing_test.go (256 lines)
- internal/codegen/segment_test.go (224 lines)
- internal/codegen/transaction_test.go (431 lines)
- internal/codegen/error_test.go (83 lines)

Total: 8,515 lines of tests restored

Note: Tests require import/reference updates to work with new structure.
Next step: Fix imports, unexported constant references, and helper functions.
Changes:
1. Exported test helper functions in parser/utils_test.go:
   - PanicRecovery (was panicRecovery)
   - RunStatelessTracingFunction (was testStatelessTracingFunction)
   - RunStatefulTracingFunction (was testStatefulTracingFunction)
   - CreateTestApp, CleanTestApp (for integration test usage)

2. Exported nragent codegen constants/functions for tests:
   - AgentErrorVariableName (was agentErrorVariableName)
   - PanicOnError (was panicOnError)

3. Fixed nragent/codegen_test.go:
   - Added nragent package import
   - Qualified all function/constant references with nragent. prefix

4. Fixed parser tests to use exported Load* methods:
   - LoadStatelessTracingFunctions
   - LoadStatefulTracingFunctions
   - LoadDependencyScans
   - LoadPreInstrumentationTracingFunctions

5. Restored getImports() method (used only by tests)

6. Skipped obsolete Test_DetectDependencyIntegrations (functionality moved to cmd/)

Status: Parser tests now compile successfully ✓
Next: Fix integration parsing tests (nrgin, nrgrpc, nrnethttp, etc.)
Major changes:
1. Created parser/testhelpers.go:
   - Moved test helper functions from parser/utils_test.go to regular .go file
   - Functions now exported and accessible to integration tests:
     * PanicRecovery, RunStatelessTracingFunction, RunStatefulTracingFunction
     * CreateTestApp, CleanTestApp

2. Fixed nragent tests:
   - Codegen tests use nragent package import with qualified references
   - Parsing tests use parser.* for helpers and nragent.* for functions

3. Fixed nrgin tests (COMPLETE ✓):
   - Exported helper functions: GinMiddlewareCall, GetGinContextFromHandler, DefineTxnFromGinCtx
   - Tests use nrgin.GinImportPath instead of unexported ginImportPath
   - All references properly qualified

4. Auto-fixed all integration parsing_test.go files:
   - Added parser and integration package imports
   - Replaced helper function calls with parser.* prefix
   - Replaced integration function calls with package.* prefix

Status:
- ✓ Parser tests compile
- ✓ nragent tests compile
- ✓ nrgin tests compile
- Remaining: nrgochi, nrgrpc, nrmysql, nrnethttp tests (need similar const/function exports)

Pattern identified for remaining integrations:
- Export helper functions/constants used by tests
- Update test references to use qualified package names
Completed:
- ✓ nrgochi tests compile successfully
  - Exported GochiImportPath and GetChiRouterName
  - Fixed test function names

- nrgrpc tests partially fixed:
  - Exported GetCallExpressionArgumentSpacing
  - Exported GrpcDialCall, GrpcNewServerCall, IsGrpcRegisterServerCall, GetRegisteredServerIdent
  - Exported GetTxnFromGrpcServer
  - Exported types: GrpcServerStreamType, GrpcServerTxnData
  - Fixed test function names
  - Note: Tests have struct field access issues (unexported fields in parser.InstrumentationManager)
    These need test refactoring to avoid direct struct construction

Status:
- ✓ parser
- ✓ nragent
- ✓ nrgin
- ✓ nrgochi
- ⚠️  nrgrpc (needs test refactoring for unexported field access)
- Remaining: nrmysql, nrnethttp
- Changed package name in segment_test.go to codegen_test for clarity.
- Updated references to EndExternalSegment, StartExternalSegment, and CaptureHttpResponse to use the codegen package.
- Added grpc_internal_test.go to test gRPC integration with InstrumentationManager.
- Introduced helper methods in manager.go for setting up test state in InstrumentationManager.
- Updated manager_test.go to reflect changes in package state initialization.
- Renamed test helper functions in preinstrumentationtracing_test.go and testhelpers.go for consistency and clarity.
- Adjusted function names in utils_test.go to follow Go naming conventions.
- Created unitTest helper function for easier test package creation.
@mirackara mirackara changed the title Code Restructure Reorganize integrations into self-contained packages Mar 3, 2026
@mirackara mirackara marked this pull request as ready for review March 6, 2026 16:24
@mirackara mirackara merged commit bbbe408 into main Mar 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants