test: split e2e tests and add go-zero integration tests#11
Conversation
- spring_maven_test.go: Maven Spring Boot tests - spring_gradle_test.go: Gradle Spring Boot tests - gozero_test.go: Go-zero e2e tests - mock_provider_test.go: Shared mock provider - error_test.go: Error handling tests Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com>
Review Summary by QodoSplit e2e tests into framework-specific files and add go-zero tests
WalkthroughsDescription• Split monolithic e2e test file into framework-specific test files • Added go-zero framework integration tests with detection and generation • Extracted error handling and mock provider into separate test files • Improved test organization and maintainability Diagramflowchart LR
A["e2e_test.go<br/>monolithic"] -->|split| B["spring_maven_test.go"]
A -->|split| C["spring_gradle_test.go"]
A -->|split| D["gozero_test.go"]
A -->|split| E["error_test.go"]
A -->|split| F["mock_provider_test.go"]
B -->|Maven tests| G["Framework-specific<br/>test suite"]
C -->|Gradle tests| G
D -->|Go-zero tests| G
E -->|Error handling| G
F -->|Shared utilities| G
File Changes1. integration-tests/spring_maven_test.go
|
Code Review by Qodo
1.
|
Signed-off-by: spencercjh <spencercjh@gmail.com>
75873b6 to
9842941
Compare
There was a problem hiding this comment.
Pull request overview
This PR reorganizes the integration-tests e2e suite by splitting Spring tests into separate Maven/Gradle files and introducing go-zero integration coverage.
Changes:
- Split Spring Boot e2e tests into Maven- and Gradle-specific test files.
- Add go-zero e2e tests covering detection, generation, and “no goctl” behavior.
- Extract the mock LLM provider into a shared test helper file.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| integration-tests/spring_maven_test.go | Removes Gradle + error-handling + mock provider code from Maven Spring test file after split. |
| integration-tests/spring_gradle_test.go | New Gradle-specific Spring Boot generation/validation e2e test. |
| integration-tests/mock_provider_test.go | New shared mock provider used by enrich flow tests. |
| integration-tests/gozero_test.go | New go-zero e2e tests (detect/generate/no-goctl). |
| integration-tests/error_test.go | New dedicated error-handling e2e test for missing command execution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: spencercjh <spencercjh@gmail.com>
Signed-off-by: spencercjh <spencercjh@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: spencercjh <spencercjh@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 11 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- error_test.go: fail test on wrong error type instead of just logging - error_test.go: remove unused targetDir variable - README.md: add prerequisites for Java 25 and goctl installation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: spencercjh <spencercjh@gmail.com>
7115958 to
193ec31
Compare
Signed-off-by: spencercjh <spencercjh@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 11 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| result, err := gen.Generate(ctx, projectPath, info, &extractor.GenerateOptions{ | ||
| Format: "yaml", |
There was a problem hiding this comment.
This e2e test writes openapi.yaml into the checked-in demo project directory (hence the added .gitignore entries). To avoid polluting the working tree and to make runs more hermetic, set GenerateOptions.OutputDir (and optionally OutputFile) to a t.TempDir() location.
| result, err := gen.Generate(ctx, projectPath, info, &extractor.GenerateOptions{ | |
| Format: "yaml", | |
| outputDir := t.TempDir() | |
| result, err := gen.Generate(ctx, projectPath, info, &extractor.GenerateOptions{ | |
| Format: "yaml", | |
| OutputDir: outputDir, | |
| OutputFile: "openapi.yaml", |
| - name: Set up goctl | ||
| run: go install github.com/zeromicro/go-zero/tools/goctl@latest | ||
|
|
There was a problem hiding this comment.
Installing goctl@latest makes CI non-reproducible (a new goctl release can break the pipeline without code changes). Consider pinning to a specific version (ideally matching the go-zero version used in integration-tests/gozero-demo/go.mod) or centralizing the version in a workflow/env var.
| - **goctl** - Required for go-zero projects. Install with: | ||
| ```bash | ||
| go install github.com/zeromicro/go-zero/tools/goctl@latest | ||
| ``` | ||
|
|
There was a problem hiding this comment.
The docs recommend go install .../goctl@latest, which is convenient but can lead to non-reproducible e2e runs when goctl behavior changes. Consider documenting a pinned goctl version (matching the demo project's go-zero dependency) or at least noting that @latest may cause test drift over time.
| - **goctl** - Required for go-zero projects. Install with: | |
| ```bash | |
| go install github.com/zeromicro/go-zero/tools/goctl@latest | |
| ``` | |
| - **goctl** - Required for go-zero projects. Install with a version pinned to the go-zero dependency used in the `gozero-demo` project, for example: | |
| ```bash | |
| go install github.com/zeromicro/go-zero/tools/goctl@vX.Y.Z |
Note: Using @latest can change goctl behavior over time and may cause end-to-end tests to drift.
| go.opentelemetry.io/otel/trace v1.24.0 // indirect | ||
| go.opentelemetry.io/proto/otlp v1.3.1 // indirect | ||
| go.uber.org/automaxprocs v1.6.0 // indirect | ||
| go.yaml.in/yaml/v2 v2.4.2 // indirect |
There was a problem hiding this comment.
The require entry on go.yaml.in/yaml/v2 introduces a likely typosquatted and potentially malicious replacement for the legitimate gopkg.in/yaml.v2 module, creating a supply-chain risk. Including this module in go.mod allows it to be fetched and built in developer or CI environments, where it could run arbitrary code with access to secrets. Remove this requirement and, if YAML parsing is needed, depend on the official gopkg.in/yaml.v2 module instead.
Review feedback addressed: - CustomProvider: use p.name instead of hardcoded CustomProviderName in error messages - StreamWriter: optimize bytes.Contains to bytes.IndexByte for hot path - E2E tests: handle enrich.enabled=false config, pass --custom-api-key-env for custom provider - Design doc: fix StreamWriter file path, update Stream to *bool type, correct prefix casing Fixes review comments #11, #12, #13, #16, #17 on PR #57 Signed-off-by: caijiahao <caijh@inesa.com> Signed-off-by: spencercjh <spencercjh@gmail.com>
* docs: add Phase 4.1 Streaming design document Design P4.1: Real-time streaming support for LLM enrichment Key features: - Option pattern for Provider interface (WithStreamingFunc) - StreamWriter for thread-safe output with batch prefix - Concurrent processing with mutex-protected output - CLI flag --no-stream to disable streaming (enabled by default) Signed-off-by: spencercjh <spencercjh@gmail.com> * docs: add P4.1 Streaming implementation plan 8 tasks with TDD approach: 1. Provider interface refactoring (Option pattern) 2. StreamWriter implementation 3. Provider streaming implementation 4. BatchProcessor streaming integration 5. Enricher streaming support 6. CLI integration (--no-stream flag) 7. Integration testing 8. Final verification Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(provider): add Option pattern for streaming support - Add GenerateOptions, Option, WithStreamingFunc, applyOptions - Update Provider interface to accept ...Option - Update all provider implementations with new signature (streaming logic in next commit) Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(processor): add StreamWriter for thread-safe streaming output - StreamWriter with mutex-protected writes - WriteWithPrefix adds batch type prefix - Tested for concurrent access safety Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(provider): add streaming support to all providers - Replace GenerateFromSinglePrompt with GenerateContent - Pass StreamingFunc to langchaingo via llms.WithStreamingFunc - Return content from response.Choices for non-streaming callers Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(processor): integrate streaming into BatchProcessor - Add StreamWriter field and WithStreamWriter option - Pass streaming callback to provider with batch type prefix - Update NewBatchProcessor to accept functional options Task 4 of P4.1 streaming implementation Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(enricher): add Stream option to EnrichOptions - Add Stream (bool) and Writer (io.Writer) to EnrichOptions - Create StreamWriter when streaming is enabled - Default: streaming enabled to os.Stdout Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(cli): add --no-stream flag to enrich command - Streaming enabled by default - Use --no-stream to disable and wait for complete response Signed-off-by: spencercjh <spencercjh@gmail.com> * test(enricher): add streaming integration test - Test that streaming callback is invoked - Test that StreamWriter output contains batch prefix [api] Signed-off-by: spencercjh <spencercjh@gmail.com> * style(enricher): fix whitespace alignment Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(batch): update streaming function to ignore context parameter Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(processor): add nil check to NewStreamWriter Prevent runtime panic by validating writer parameter is not nil. Add test to verify panic behavior with nil writer. Signed-off-by: spencercjh <spencercjh@gmail.com> * test(enricher): add test for Stream: false path Add TestEnricher_WithStreamingDisabled to verify that no streaming callback is passed to the provider when Stream option is disabled. Signed-off-by: spencercjh <spencercjh@gmail.com> * test(processor): add streaming callback verification in batch tests Add TestBatchProcessor_ProcessBatch_WithStreaming to verify that streaming callback is properly invoked when StreamWriter is configured. Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(processor): add chunk buffering to StreamWriter - Add buffer accumulation for streaming chunks - Auto-flush on newline, buffer threshold, or prefix change - Add WithFlushThreshold option for configurable buffering - Add Flush method for explicit buffer clearing - Update enricher to flush StreamWriter after processing - Add tests for buffering behavior Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(processor): add streaming debug logging and metrics - Add StreamWriterMetrics struct to track streaming statistics - Add GetMetrics() method for retrieving metrics - Add WithDebug option for enabling debug logging - Add WithFlushThreshold option for configurable buffering - Track total chunks, bytes, flushes, and unique prefixes - Debug logging shows chunk details and flush events Signed-off-by: spencercjh <spencercjh@gmail.com> * style: fix formatting issues from golangci-lint Signed-off-by: spencercjh <spencercjh@gmail.com> * docs: clarify make verify usage in CLAUDE.md Explain that make verify checks for uncommitted changes and should only be used after committing. Recommend individual commands (make fmt, lint, test) before committing to avoid false failures from git diff check. Signed-off-by: spencercjh <spencercjh@gmail.com> * test(e2e): add conditional enrich E2E tests with streaming verification - Add skipIfNoConfig helper to skip tests without valid E2E config - Check for .spec-forge.e2e.local.yaml or .spec-forge.local.yaml - Verify API key environment variable is set before running - Test streaming output with prefix verification ([api], [schema], [param]) - Test --no-stream flag disables streaming prefixes - Test local config file loading mechanism - Add .spec-forge.e2e.example.yaml as configuration template - Update .gitignore to exclude E2E local configs Signed-off-by: spencercjh <spencercjh@gmail.com> * test(e2e): add conditional enrich E2E tests with streaming verification - Add loadE2EConfig helper to load config from integration-tests/.spec-forge.e2e.local.yaml - Skip tests gracefully if no valid config found (file missing or API key env not set) - Add TestE2E_Enrich_NoStreamFlag to verify --no-stream disables streaming prefixes - Add TestE2E_Enrich_WithStreaming to test real LLM enrichment with streaming output - Add TestE2E_Enrich_WithLocalConfig to test local config file loading - Add .spec-forge.e2e.example.yaml as configuration template - Update .gitignore to exclude integration-tests/.spec-forge.e2e.local.yaml - Document E2E enrich test setup in integration-tests/README.md Signed-off-by: spencercjh <spencercjh@gmail.com> * test(e2e): fix enrich E2E tests for streaming verification - Remove streaming prefix check in stdout (StreamWriter writes to os.Stdout directly, not Cobra's buffer) - Remove TestE2E_Enrich_WithLocalConfig (tests Viper config loading, not enrich logic) - Verify enrichment by checking the output spec file contains descriptions - Update README to document only the two streaming-related tests - All tests pass: - TestE2E_Enrich_Help: PASS - TestE2E_Enrich_MissingArgs: PASS - TestE2E_Enrich_NonExistentFile: PASS - TestE2E_Enrich_NoStreamFlag: SKIP (requires config) - TestE2E_Enrich_WithStreaming: PASS (with config) / SKIP (without config) Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(provider): satisfy goconst in anthropic and custom providers Agent-Logs-Url: https://github.com/spencercjh/spec-forge/sessions/be20b0d1-0dc2-45c4-adc4-b2bec21a9a95 Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com> * fix(review): address PR review feedback for P4.1 streaming - Change EnrichOptions.Stream to *bool (tri-state) to avoid backwards compatibility issues with zero value of bool (false). Now nil means use default (true), explicit false/true overrides. - Add error return when LLM providers return empty choices to prevent silent enrichment failures (OpenAI, Anthropic, Ollama, Custom). - Fix comment in batch.go to reflect lowercase prefix values. - Remove stale TestE2E_Enrich_WithLocalConfig entry from README table. - Make E2E test assertion language-agnostic (check for non-empty descriptions instead of specific Chinese text). Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(review): address additional PR review feedback for P4.1 streaming Review feedback addressed: - CustomProvider: use p.name instead of hardcoded CustomProviderName in error messages - StreamWriter: optimize bytes.Contains to bytes.IndexByte for hot path - E2E tests: handle enrich.enabled=false config, pass --custom-api-key-env for custom provider - Design doc: fix StreamWriter file path, update Stream to *bool type, correct prefix casing Fixes review comments #11, #12, #13, #16, #17 on PR #57 Signed-off-by: caijiahao <caijh@inesa.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(review): address third round of Copilot review feedback for P4.1 Review feedback addressed: - factory.go: use AnthropicProviderName and CustomProviderName constants instead of string literals - design doc: update EnrichOptions summary to reflect actual API (Stream *bool, Writer io.Writer) - design doc: fix data flow example to use lowercase prefix [api] instead of [API] - stream_writer.go: validate WithFlushThreshold to use DefaultFlushThreshold for zero/negative values - e2e_enrich_test.go: pass --timeout flag when configured in E2E config Fixes review comments #21-26 on PR #57 Signed-off-by: caijiahao <caijh@inesa.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(review): address fourth round of Copilot review feedback Review feedback addressed: - e2e_enrich_test.go: change Enabled to *bool to distinguish between "not set" (nil, run tests) and "explicitly false" (skip tests) - README.md: update skip message path to match actual output - batch.go: add Flush() call after each LLM call to ensure buffered streaming output is visible for short responses Note: #30 (streaming to os.Stdout) is intentional design for real-time terminal feedback and not changed. Fixes review comments #27-29 on PR #57 Signed-off-by: caijiahao <caijh@inesa.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(review): address fifth round of Copilot review feedback Review feedback addressed: - e2e_enrich_test.go: match API key env var logic to CLI behavior (OPENAI_API_KEY, ANTHROPIC_API_KEY, or LLM_API_KEY for custom) - e2e_enrich_test.go: use YAML parsing for provider-agnostic assertions instead of language-specific substring matching - design doc: update output example to show actual raw JSON streaming behavior instead of hypothetical human-readable text Note: #32 (streaming writes to os.Stdout) is intentional design for real-time terminal feedback - already addressed in #30 response. Fixes review comments #31, #33, #34 on PR #57 Signed-off-by: caijiahao <caijh@inesa.com> Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(enricher,stream_writer): improve error handling and metrics updates Signed-off-by: spencercjh <spencercjh@gmail.com> * feat(batch): add streaming support and improve processing logic Signed-off-by: spencercjh <spencercjh@gmail.com> * fix(review): address Copilot review feedback - streaming, concurrency, docs - Remove redundant atomic ops in StreamWriter, use mutex-only sync - Handle short writes in flushLocked() - Change final Flush error to warning (streaming is ancillary) - Streaming mode processes batches sequentially for readable output - --no-stream enables concurrent processing (--concurrency applies) - Update flag descriptions, config comments, design doc, CLAUDE.md Signed-off-by: spencercjh <spencercjh@gmail.com> --------- Signed-off-by: spencercjh <spencercjh@gmail.com> Signed-off-by: caijiahao <caijh@inesa.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com>
Summary
e2e_test.gointo framework-specific test files for better organizationTest Plan
🤖 Generated with Claude Code