[Feature Request] Implement Caching Layer

[vincentkoc]

Description

Add caching layer for frequently accessed data to improve performance and reduce API calls.

Data to Cache

• Project list
• Workspace information
• Prompt templates
• Trace statistics
• API responses (configurable)

Implementation Options

1. node-cache (in-memory, simple)
2. Redis (external, scalable)
3. lru-cache (memory-efficient)

Tasks

• Choose caching solution (node-cache recommended for simplicity)
• Implement cache layer utility
• Add caching to project list endpoint
• Add caching to workspace info
• Add caching to prompt operations
• Configure TTL (Time To Live) per cache type
• Add cache invalidation logic
• Add cache hit/miss metrics
• Configure via CLI/env vars
• Add tests
• Document usage

Example Implementation

import NodeCache from 'node-cache';

const cache = new NodeCache({
  stdTTL: 300, // 5 minutes default
  checkperiod: 60
});

// Usage
const cacheKey = 'projects-list';
const cached = cache.get(cacheKey);
if (cached) return cached;

const data = await fetchFromAPI();
cache.set(cacheKey, data, 300);
return data;

Configuration

• Enable/disable caching
• TTL per cache type
• Max cache size
• Cache statistics endpoint

Benefits

• Reduced API calls
• Faster response times
• Better user experience
• Reduced load on Opik API

Cache Invalidation

• Time-based (TTL)
• Manual invalidation
• Event-based (on mutations)