-
Notifications
You must be signed in to change notification settings - Fork 5k
Feature/enhanced error injection and mysql payment service #2705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
nmillardint
wants to merge
75
commits into
open-telemetry:main
Choose a base branch
from
intergral:feature/enhanced-error-injection-mysql-payment
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Feature/enhanced error injection and mysql payment service #2705
nmillardint
wants to merge
75
commits into
open-telemetry:main
from
intergral:feature/enhanced-error-injection-mysql-payment
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(cf): add cf quote service demo
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(quote-cf): enable Docker image push to GitHub Container Registry
…ging 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
chore(arm): add FR arm/amd supported images
… comprehensive endpoints - Add MySQL database container with health checks and initialization - Implement complete quote generation with customer and service data - Add new CF endpoints: email, update, remove old quotes, and reporting - Enhance load testing with ColdFusion-specific user scenarios - Configure FusionReactor APM for enhanced monitoring - Add proper error handling and logging throughout CF application 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…into feature/otel-cf-updates
chore(quote): increase jvm memory
chore(quote): add health route
chore(quote): fix getquote.cfm shipping
chore(quote): fix getquote.cfm shipping
…ations ### Major Changes: - **Database Migration**: Complete migration from SQLite to MySQL - Updated all ColdFusion code to use MySQL-compatible syntax - Converted SQLite-specific functions (RANDOM() → RAND(), DATE() → CURDATE(), etc.) - Fixed string concatenation (|| → CONCAT()) - Updated CAST functions (AS TEXT → AS CHAR) - **Performance Testing**: Enhanced slow query simulation - Replaced problematic recursive CTEs with mathematical operations - Added 5% slow query path (~15 seconds) for observability testing - 95% fast query path maintains ~0.16 second response times - **Container Optimization**: - Removed hardcoded JVM heap limits for Kubernetes compatibility - Added no-cache build option for local development - Auto-scaling heap based on container resource limits - **Database Configuration**: - Added MySQL JDBC driver and connection configuration - Updated datasource settings in Application.cfc - Cleaned up SQL files (removed SQLite artifacts) ### Files Modified: - Application.cfc: MySQL datasource configuration - *.cfm: All endpoints updated for MySQL compatibility - Dockerfile: JVM auto-scaling heap configuration - docker-compose.yml: Removed build caching for development - SQL cleanup: Kept only init_quotes.sql for MySQL ### Testing Results: - All endpoints working: health, debug, getquote, emailquote, updatequote, removeoldquotes, report - Performance testing: 4% slow queries averaging 15+ seconds - Fast queries: 96% averaging 0.16 seconds - Ready for Kubernetes deployment with auto-scaling JVM 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(quote-cf): migrate from SQLite to MySQL with performance optimiz…
- Migrate from SQLite to MySQL database system - Add comprehensive sample data (50 customers, 50 services, 30 quotes, 50 quote items) - Implement database initialization in Application.cfc with proper error handling - Update Docker configuration to use MySQL container - Remove SQLite dependencies and database files - Add health checks and connection validation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(quote-cf): complete MySQL migration with enhanced schema and data
- Optimize slow query execution from 45+ seconds to 4-6 seconds - Add 30-second MySQL connection/socket timeouts to prevent hangs - Reduce Lucee request timeout from 10 minutes to 5 minutes - Persist FusionReactor configuration with 4-second slow thresholds - Add comprehensive test script for performance validation - Maintain 5% slow query probability for realistic observability demos - Ensure system stability under 200+ consecutive requests 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(quote-cf): optimize removeoldquotes performance and add monitoring
- Create mysql_admin datasource with 2-minute timeouts for admin operations - Update onApplicationStart() to use mysql_admin for DROP/CREATE/INSERT operations - Regular mysql datasource retains 30-second timeouts for user queries - Prevents application startup timeouts on slow demo clusters - Ensures reliable database initialization while maintaining query performance 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…eouts fix(quote-cf): add separate admin datasource for database initialization
- Move database initialization from Application.cfc to MySQL Docker initialization - Mount SQL initialization script to /docker-entrypoint-initdb.d in docker-compose - Update init_quotes.sql with 10x larger dataset (50 customers, 50 services, 30 quotes, 50 quote_items) - Remove persistent MySQL data volume to ensure fresh data on each startup - Clean up Application.cfc by removing admin datasource and database initialization logic - Reduce quote service load in load generator to improve demo performance - Fix demo cluster timeout issues by eliminating database initialization delays 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(quote-cf): migrate database initialization from ColdFusion to MySQL
- Double wait times for WebsiteUser (1-10s → 2-20s) and CFStorefrontUser (15-30s → 30-60s) to halve throughput - Reduce emailquote.cfm error rate from ~15% to ~7.5% by changing error condition from dur > 620 to dur > 635 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
feat(load-generator): reduce load throughput and error rates
…rvice This commit adds comprehensive sporadic error behavior across OTEL services and integrates MySQL database with the payment service for transaction logging and database error simulation. Changes to Feature Flags (demo.flagd.json): - Enable productCatalogFailure: "on" - Product catalog errors - Enable imageSlowLoad: "5sec" - Slow loading images in frontend - Enable adManualGc: "on" - GC pauses in ad service - Enable adHighCpu: "on" - CPU spikes in ad service - Enable adFailure: "on" - 10% ad service failures - Enable kafkaQueueProblems: "on" - Message queue lag - Enable cartFailure: "on" - Cart service errors - Increase paymentFailure: 10% → 25% - More frequent payment errors - Add paymentDatabaseError: "15%" - New database error injection flag Payment Service MySQL Integration: - Add mysql2 dependency for database connectivity - Create database.js module with connection pooling - Add payment_transactions table schema for transaction logging - Update charge.js to store all payment attempts in MySQL - Implement sporadic database error injection (timeouts, deadlocks, etc.) - Add MySQL environment variables to docker-compose.yml - Ensure payment service depends on MySQL health check Error Types Simulated: - Connection timeouts (ETIMEDOUT) - Connection refused (ECONNREFUSED) - Lock wait timeouts (ER_LOCK_WAIT_TIMEOUT) - Deadlock detection (ER_LOCK_DEADLOCK) Observability Benefits: - Database operations visible in OTEL traces - Realistic error scenarios for demo purposes - Transaction audit logging in MySQL - Multi-layered chaos: service errors, performance issues, resource contention, messaging delays 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Add package-lock.json with mysql2 dependencies for payment service - Fix docker-compose.yml invalid no_cache option in quote service build These changes are required for the payment service MySQL integration to build correctly.
- Add type conversion for units and nanos from gRPC Long to JavaScript Number - Fixes database insertion error when storing payment transactions - Handles both Long objects and primitive number types ci: add GitHub Actions workflow for payment service builds - Add build-payment.yml workflow for automated Docker image builds - Triggers on changes to src/payment/** directory - Follows same pattern as other service workflows (ad, recommendation, etc.) Tested: Successfully processed 5 test payments and verified transactions stored correctly in MySQL database. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
docs-update-required
Requires documentation update
helm-update-required
Requires an update to the Helm chart when released
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Enhanced error injection across OTEL services and added MySQL integration to payment service for transaction logging and database error simulation.
Changes
Testing
✅ Successfully tested with 5 payment transactions
✅ All transactions stored correctly in MySQL database
✅ Database error injection working (15% rate)