Merge pull request #1013 from devmasalati/feat/852-853-854-855

fixed issue

Author: Emmzyemms

.changeset/852-redis-session-sharing.md

@@ -0,0 +1,37 @@
+---
+"clipsproject": minor
+---
+
+Implement Redis-based session sharing across serverless instances
+
+- Add Redis client manager with connection pooling and health monitoring
+- Implement automatic fallback to in-memory storage when Redis is unavailable
+- Add health check endpoint at `/api/health/redis` for monitoring
+- Configure connection pool with retry logic and automatic reconnection
+- Add comprehensive error handling and logging throughout Redis operations
+- Document Redis configuration and deployment for serverless environments
+
+**New Features:**
+- Redis connection pooling for optimal performance
+- Automatic health checks every 30 seconds
+- Graceful fallback to in-memory storage
+- Reconnection with exponential backoff
+- Pool metrics and connection statistics
+
+**New Environment Variables:**
+- `REDIS_MAX_RETRIES` - Max command retries (default: 3)
+- `REDIS_RETRY_DELAY` - Delay between retries in ms (default: 1000)
+- `REDIS_CONNECT_TIMEOUT` - Connection timeout in ms (default: 10000)
+- `REDIS_COMMAND_TIMEOUT` - Command timeout in ms (default: 5000)
+- `REDIS_KEEP_ALIVE` - TCP keep-alive interval in ms (default: 30000)
+- `REDIS_MAX_RECONNECT_ATTEMPTS` - Max reconnection attempts (default: 10)
+- `REDIS_HEALTH_CHECK_INTERVAL` - Health check interval in ms (default: 30000)
+
+**Breaking Changes:**
+- None - backward compatible with existing in-memory implementation
+
+**Benefits:**
+- Resolves state inconsistency across serverless instances
+- Improves job state reliability in distributed environments
+- Enables proper SSE stream updates across all instances
+- Production-ready with automatic failover capabilities

.changeset/853-database-connection-pooling.md

@@ -0,0 +1,24 @@
+---
+"clipsproject": minor
+---
+
+Add database connection pooling for Prisma
+
+- Configure Prisma connection pool settings with environment variables
+- Add connection pool monitoring and health check endpoint at `/api/health/database`
+- Implement connection timeout handling middleware
+- Add connection pool metrics to logging (configurable)
+- Create load testing script for connection pool validation
+- Add comprehensive test coverage for pooling functionality
+- Document configuration and best practices in `docs/DATABASE_CONNECTION_POOLING.md`
+
+**New Environment Variables:**
+- `DATABASE_POOL_SIZE` - Connection pool size (default: 10)
+- `DATABASE_CONNECTION_TIMEOUT` - Query timeout in ms (default: 10000)
+- `DATABASE_POOL_IDLE_TIMEOUT` - Idle timeout in ms (default: 30000)
+- `DATABASE_LOG_POOL_METRICS` - Enable detailed metrics logging (default: false)
+- `DATABASE_SLOW_QUERY_THRESHOLD` - Slow query threshold in ms (default: 1000)
+
+**Breaking Changes:**
+- Requires `@prisma/client` and `prisma` packages to be installed
+- Run `npm install` to add Prisma dependencies

.env.example

@@ -32,8 +32,30 @@ AWS_SECRET_ACCESS_KEY=
 
 # Redis connection used by the job repository (redis:// or rediss://)
 # Example: redis://:password@hostname:6379
+# For TLS: rediss://:password@hostname:6380
 REDIS_URL=
 
+# Redis connection pooling and retry configuration (optional)
+REDIS_MAX_RETRIES=3
+REDIS_RETRY_DELAY=1000
+REDIS_CONNECT_TIMEOUT=10000
+REDIS_COMMAND_TIMEOUT=5000
+REDIS_KEEP_ALIVE=30000
+REDIS_MAX_RECONNECT_ATTEMPTS=10
+REDIS_HEALTH_CHECK_INTERVAL=30000
+
+# Database (PostgreSQL with connection pooling)
+# Connection pooling is configured via URL parameters or environment variables
+# Example with connection pooling: postgresql://user:pass@localhost:5432/clips?connection_limit=10&pool_timeout=10
+DATABASE_URL=postgresql://user:password@localhost:5432/clips
+
+# Connection pool configuration (optional - overrides URL parameters)
+DATABASE_POOL_SIZE=10
+DATABASE_CONNECTION_TIMEOUT=10000
+DATABASE_POOL_IDLE_TIMEOUT=30000
+DATABASE_LOG_POOL_METRICS=false
+DATABASE_SLOW_QUERY_THRESHOLD=1000
+
 # Optional local dev settings
 CI=false
 NODE_ENV=development