Webby- 2G Website Compressor

Production-oriented split proxy for constrained networks.

• Client proxy listens on port 8080 and accepts browser traffic.
• Backend proxy listens on port 9000, fetches origin content, optimizes/compresses HTTP responses, and tunnels HTTPS via CONNECT.
• HTTPS is tunneled end-to-end (no MITM/decryption).

Run

npm install

# terminal 1
npm run backend

# terminal 2
npm run client

Browser proxy settings:

• HTTP proxy: 127.0.0.1:8080
• HTTPS proxy: 127.0.0.1:8080

Timeouts And Shutdown

The runtime now uses configurable production-safe values.

• HTTP_REQUEST_TIMEOUT_MS (backend HTTP origin request timeout, default 60000)
• BACKEND_REQUEST_TIMEOUT_MS (client->backend HTTP timeout, default 60000)
• CONNECT_IDLE_TIMEOUT_MS (backend CONNECT tunnel idle timeout, default 600000)
• CONNECT_HANDSHAKE_TIMEOUT_MS (client CONNECT handshake timeout, default 60000)
• SHUTDOWN_GRACE_MS (grace period before forced close on SIGINT/SIGTERM, default 5000)

Both servers now perform idempotent graceful shutdown and force-close lingering sockets after the grace period.

Layout

• backend-server.js backend proxy server
• client-proxy.js client-side proxy server
• middleware/ optimization/caching/compression pipeline
• tests/ all test scripts
• utils/ utility helpers

Quick Validation

node tests/test-both-fixes.js

Expected:

• HTTP response includes content-encoding: gzip
• CONNECT returns 200 Connection Established -days 365 -sha256

Per-Domain Optimization Control

Whitelist specific domains for optimization:

// config.js
filtering: {
  domainWhitelist: ['example.com', 'news.com'],  // Only optimize these
  domainBlacklist: ['cdn.example.com']           // Skip these
}

Compression Level Tuning

For faster compression (lower latency) on relay node, reduce Brotli quality:

// config.js
compression: {
  brotli: { quality: 8 }  // 1-11, lower = faster but larger
}

Cache Management

// Clear all cache on startup or periodically
// In proxy-server.js
const { clearAllCache } = require('./middleware/cache');
clearAllCache();

Troubleshooting

HTTPS Interception Not Working

1. Verify CA certificate installed on device:

   • Android: Settings -> Security -> Certificate Management (should show "2G Website Compressor")
   • iOS: Settings -> General -> About -> Certificate Trust Settings (enable certificate)

2. Check proxy is accessible:

   curl -x http://localhost:8888 https://example.com

3. Check firewall allows traffic on port 8888:

   sudo ufw allow 8888  # Linux

Proxy Causing Website Breakage

1. Temporarily disable optimization for specific domain in config.js:

   filtering: { domainBlacklist: ['problematic-site.com'] }

2. Enable debug logging to identify which middleware fails:

   DEBUG=true npm start

3. Report issue with domain name (may need to adjust HTML/JS minification)

Compression Making Things Slower

If 2G device is heavily CPU-constrained:

• Reduce Brotli quality (faster compression):

  brotli: { quality: 6 }  // Faster, slightly less compression

• Disable image optimization (most CPU-intensive):

  image: { enabled: false }

Cache Growing Too Large

Edit cache size limit:

cache: {
  maxSize: 50  // Reduced from 100 MB
}

Or manually clear:

rm -rf ./cache/*

Performance Expectations

Typical Website Compression

Site	Original	Compressed	Reduction	2G Time Saved
News homepage	2.5 MB	320 KB	87%	~120 sec
E-commerce product	1.8 MB	210 KB	88%	~90 sec
Blog post	450 KB	42 KB	91%	~30 sec
Social media feed	3.2 MB	280 KB	91%	~140 sec

2G Network Assumptions: 115 Kbps, 400ms latency

Relay Node Requirements

• Bandwidth: ~1 Mbps (handles proxying, not optimization overhead)
• CPU: 1 core (Node.js lightweight, minimal per-request optimization)
• Memory: 512 MB (cache + connections)
• Storage: 100+ MB (response caching)

Contributing

Areas for contribution:

• HTTP/2 multiplexing support (faster sequential asset loading)
• WebSocket delta-sync (reload only changed content)
• Progressive image loading (placeholder -> preview -> full)
• Video transcoding (HLS to minimal bitrate)
• Distributed proxy (load balancing across relay nodes)
• Better Socks proxy support

License

MIT

References

• Opera Mini Architecture Paper
• Google's Mobile Optimization Guide
• Brotli Compression Algorithm
• WebP Image Format
• 2G Network Performance

Support

For issues, feature requests, or questions:

1. Check troubleshooting section above
2. Enable debug logging to diagnose issues
3. Test with unit tests to verify middleware components
4. Check config.js for optimization toggles

---