@banana.inc/cacheman-s3

TypeScript-first AWS S3 caching library for Node.JS and cache engine for cacheman.

Features

🚀 High Performance: Optimized for S3 operations with modern AWS SDK v3
🔒 Type-Safe: Full TypeScript support with comprehensive type definitions
🛡️ Secure: Supports IAM roles, encryption, and custom endpoints
⏰ TTL Support: Automatic expiration with lazy cleanup
🔍 Scanning: Basic scan operations for cache inspection
📊 Monitoring: Built-in health checks
🌐 AWS Integration: Full AWS SDK v3 compatibility with LocalStack support
📞 Callback API: Traditional Node.js callback patterns with TypeScript typing
🎯 Generic Support: Type-safe caching for any data structure
📁 Hierarchical Keys: Native support for slash-separated cache keys creating S3 object paths

Installation

npm install @banana.inc/cacheman-s3

Quick Start

TypeScript

import { S3Store } from '@banana.inc/cacheman-s3';

interface User {
  id: number;
  name: string;
  email: string;
  preferences: {
    theme: 'light' | 'dark';
    notifications: boolean;
  };
}

const cache = new S3Store<User>({
  bucket: 'my-cache-bucket',
  region: 'us-east-1'
});

// Set a typed value
cache.set('user:123', {
  id: 123,
  name: 'John Doe',
  email: '[email protected]',
  preferences: {
    theme: 'dark',
    notifications: true
  }
}, 3600, (error) => {
  if (error) throw error;
  console.log('User cached for 1 hour');
  
  // Get the typed value
  cache.get('user:123', (error, user) => {
    if (error) throw error;
    if (user) {
      // TypeScript knows user is of type User | null
      console.log(`Welcome ${user.name}!`);
      console.log(`Theme: ${user.preferences.theme}`);
    }
  });
});

JavaScript

const { S3Store } = require('@banana.inc/cacheman-s3');

const cache = new S3Store({
  bucket: 'my-cache-bucket',
  region: 'us-east-1'
});

cache.set('user:123', { name: 'John', age: 30 }, 3600, function(err) {
  if (err) throw err;
  
  cache.get('user:123', function(err, user) {
    if (err) throw err;
    console.log('User:', user); // { name: 'John', age: 30 }
  });
});

Usage with Cacheman

TypeScript

import Cacheman from 'cacheman';
import { S3Store } from '@banana.inc/cacheman-s3';

interface CacheData {
  id: string;
  data: any;
  timestamp: number;
}

const cache = new Cacheman<CacheData>('users', {
  engine: S3Store,
  bucket: 'my-cache-bucket',
  region: 'us-east-1',
  ttl: 3600 // 1 hour default TTL
});

// Type-safe operations
cache.set('profile:123', {
  id: 'profile:123',
  data: { name: 'John', role: 'admin' },
  timestamp: Date.now()
}, (error) => {
  if (error) throw error;
  
  cache.get('profile:123', (error, data) => {
    if (error) throw error;
    if (data) {
      console.log(`Profile loaded: ${data.data.name}`);
    }
  });
});

Configuration

Basic Configuration

import { S3Store, S3StoreOptions } from '@banana.inc/cacheman-s3';

const options: S3StoreOptions = {
  // Required
  bucket: 'my-cache-bucket',
  
  // AWS Configuration
  region: 'us-east-1',           // Default: 'us-east-1'
  accessKeyId: 'AKIA...',        // Use IAM roles when possible
  secretAccessKey: 'xxx',
  sessionToken: 'xxx',           // For temporary credentials
  
  // Cache Configuration
  prefix: 'cache:',              // Default: 'cacheman:'
  defaultTtl: 3600,              // Default TTL in seconds
  
  // S3 Specific
  storageClass: 'STANDARD',      // S3 storage class
  serverSideEncryption: 'AES256', // Encryption at rest
  
  // Performance
  maxRetries: 3,                 // AWS SDK retries
  httpTimeout: 30000             // Request timeout (ms)
};

const cache = new S3Store(options);

Advanced Configuration with Types

interface CacheConfig extends S3StoreOptions {
  customOption?: string;
}

const createCache = <T>(config: CacheConfig): S3Store<T> => {
  return new S3Store<T>({
    bucket: config.bucket,
    region: config.region || 'us-east-1',
    prefix: config.prefix || 'app:',
    defaultTtl: config.defaultTtl || 3600,
    storageClass: 'INTELLIGENT_TIERING',
    serverSideEncryption: 'AES256'
  });
};

// Type-safe cache creation
const userCache = createCache<User>({
  bucket: 'user-cache-bucket',
  prefix: 'users:'
});

API Reference

Constructor

new S3Store<T>(options: S3StoreOptions): S3Store<T>

Creates a new type-safe S3Store instance.

Methods

cache.set()

set(key: string, value: T, ttl?: number, callback?: SetCallback<T>): void
set(key: string, value: T, callback?: SetCallback<T>): void

Store a typed value in the cache.

interface Product {
  id: string;
  name: string;
  price: number;
}

const productCache = new S3Store<Product>({ bucket: 'products' });

productCache.set('product:123', {
  id: '123',
  name: 'Laptop',
  price: 999.99
}, 7200, (error) => {
  if (error) throw error;
  console.log('Product cached for 2 hours');
});

cache.get()

get(key: string, callback: GetCallback<T>): void

Retrieve a typed value from the cache.

productCache.get('product:123', (error, product) => {
  if (error) throw error;
  if (product) {
    // TypeScript knows product is Product | null
    console.log(`${product.name}: $${product.price}`);
  }
});

cache.del()

del(key: string, callback?: DeleteCallback): void

Delete a value from the cache.

productCache.del('product:123', (error) => {
  if (error) throw error;
  console.log('Product removed from cache');
});

cache.clear()

clear(callback?: ClearCallback): void

Clear all cached values with the configured prefix.

productCache.clear((error) => {
  if (error) throw error;
  console.log('All products cleared from cache');
});

cache.scan()

scan(pattern?: string, limit?: number, callback?: ScanCallback<T>): void
scan(pattern?: string, callback?: ScanCallback<T>): void
scan(callback: ScanCallback<T>): void

Scan cache entries with optional prefix matching.

productCache.scan('product', 100, (error, result) => {
  if (error) throw error;
  
  console.log(`Found ${result.entries.length} products`);
  result.entries.forEach(({ key, data }) => {
    // data is typed as Product
    console.log(`${key}: ${data.name} - $${data.price}`);
  });
});

cache.healthCheck()

healthCheck(callback?: HealthCallback): void

Perform a health check on the S3 connection.

cache.healthCheck((error, status) => {
  if (error) throw error;
  
  console.log('Health Status:', status);
  // {
  //   status: 'healthy',
  //   bucket: 'my-cache-bucket',
  //   region: 'us-east-1',
  //   sdkVersion: 'v3'
  // }
});

Type Definitions

Core Interfaces

// Store options
interface S3StoreOptions {
  bucket: string;
  region?: string;
  accessKeyId?: string;
  secretAccessKey?: string;
  sessionToken?: string;
  prefix?: string;
  defaultTtl?: number;
  storageClass?: 'STANDARD' | 'REDUCED_REDUNDANCY' | 'STANDARD_IA' | 'ONEZONE_IA' | 'INTELLIGENT_TIERING' | 'GLACIER' | 'DEEP_ARCHIVE';
  serverSideEncryption?: 'AES256' | 'aws:kms';
  maxRetries?: number;
  httpTimeout?: number;
}

// Scan result
interface ScanResult<T> {
  cursor: number | string;
  entries: Array<{
    key: string;
    data: T;
  }>;
}

// Health status
interface HealthStatus {
  status: 'healthy' | 'unhealthy';
  bucket: string;
  region: string;
  sdkVersion: string;
  error?: string;
}

// Callback types
type GetCallback<T> = (error: Error | null, result?: T | null) => void;
type SetCallback<T> = (error: Error | null, result?: T) => void;
type DeleteCallback = (error: Error | null) => void;
type ClearCallback = (error: Error | null) => void;
type ScanCallback<T> = (error: Error | null, result?: ScanResult<T>) => void;
type HealthCallback = (error: Error | null, result?: HealthStatus) => void;

Error Types

// Base error class
class S3StoreError extends Error {
  code: string;
  statusCode?: number;
  originalError?: Error;
}

// Specific error types
class ConfigurationError extends S3StoreError {}
class S3OperationError extends S3StoreError {}
class SerializationError extends S3StoreError {}
class TTLError extends S3StoreError {}

Advanced Usage

Hierarchical Cache Keys

S3Store supports hierarchical cache keys using forward slashes, which are preserved as S3 object paths:

const cache = new S3Store<any>({
  bucket: 'my-cache-bucket',
  prefix: 'app:'
});

// These create nested S3 object paths
cache.set('users/123/profile', { name: 'John' }, (error) => {
  // Creates S3 object: app:users/123/profile
});

cache.set('products/electronics/laptops/456', { name: 'MacBook' }, (error) => {
  // Creates S3 object: app:products/electronics/laptops/456
});

cache.set('api/v1/cache/session/abc123', { userId: 789 }, (error) => {
  // Creates S3 object: app:api/v1/cache/session/abc123
});

// Retrieve using the same hierarchical key
cache.get('users/123/profile', (error, profile) => {
  if (profile) {
    console.log('User profile:', profile);
  }
});

This allows for:

Organized Data: Logical grouping of related cache entries
S3 Console Navigation: Browse cache structure in AWS S3 console
Prefix-based Operations: Efficient scanning and clearing of key groups
Natural Hierarchies: Mirror your application's data structure

Generic Type Constraints

// Define strict interfaces
interface BaseEntity {
  id: string;
  createdAt: string;
  updatedAt: string;
}

interface User extends BaseEntity {
  name: string;
  email: string;
  role: 'admin' | 'user' | 'guest';
}

interface Product extends BaseEntity {
  name: string;
  price: number;
  category: string;
  inStock: boolean;
}

// Create type-safe caches
const userCache = new S3Store<User>({
  bucket: 'user-cache',
  prefix: 'users:'
});

const productCache = new S3Store<Product>({
  bucket: 'product-cache',
  prefix: 'products:'
});

// Type-safe operations
userCache.set('user:123', {
  id: '123',
  name: 'John Doe',
  email: '[email protected]',
  role: 'admin', // TypeScript ensures valid role
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString()
}, (error) => {
  // Handle result
});

Utility Functions

import { S3Store, isValidTTL, isDefined } from '@banana.inc/cacheman-s3';

// Type-safe cache wrapper
class TypedCache<T extends { id: string }> {
  private cache: S3Store<T>;

  constructor(options: S3StoreOptions) {
    this.cache = new S3Store<T>(options);
  }

  async setEntity(entity: T, ttl: number = 3600): Promise<void> {
    return new Promise((resolve, reject) => {
      this.cache.set(entity.id, entity, ttl, (error) => {
        if (error) reject(error);
        else resolve();
      });
    });
  }

  async getEntity(id: string): Promise<T | null> {
    return new Promise((resolve, reject) => {
      this.cache.get(id, (error, entity) => {
        if (error) reject(error);
        else resolve(entity || null);
      });
    });
  }
}

// Usage
const userCache = new TypedCache<User>({
  bucket: 'users',
  prefix: 'user:'
});

// Async/await usage
try {
  await userCache.setEntity({
    id: '123',
    name: 'John',
    email: '[email protected]',
    role: 'admin',
    createdAt: new Date().toISOString(),
    updatedAt: new Date().toISOString()
  });

  const user = await userCache.getEntity('123');
  if (user) {
    console.log(`User: ${user.name}`);
  }
} catch (error) {
  console.error('Cache operation failed:', error);
}

AWS IAM Permissions

Minimum required IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-cache-bucket",
        "arn:aws:s3:::my-cache-bucket/*"
      ]
    }
  ]
}

Development

Setup

# Install dependencies
npm install

# Build TypeScript
npm run build

# Run tests
npm test

# Run unit tests only
npm run test:unit

# Run integration tests (requires AWS credentials)
npm run test:integration

# Type checking
npm run typecheck

# Linting
npm run lint

# Coverage
npm run coverage

LocalStack Integration Testing

This package supports testing with LocalStack, which provides a local AWS cloud stack for development and testing.

Prerequisites

Docker installed and running
Docker Compose (optional, for easier management)

Quick Start with LocalStack

# Option 1: Using npm scripts (recommended)
npm run test:integration

# Option 2: Manual setup
npm run localstack:start
npm run localstack:setup  # Creates S3 bucket
npm run test:integration
npm run localstack:stop

Docker Compose Method

# Start LocalStack using Docker Compose
npm run localstack:start

# Run tests
npm run test:integration

# Stop LocalStack
npm run localstack:stop

Manual Docker Method

# Start LocalStack container
docker run --rm -d -p 4566:4566 --name localstack-s3-test localstack/localstack:3.0

# Wait for LocalStack to be ready
curl --retry 10 --retry-delay 1 --retry-connrefused http://localhost:4566/health

# Create S3 bucket for testing
aws --endpoint-url=http://localhost:4566 s3 mb s3://test-bucket

# Run integration tests
LOCALSTACK_ENDPOINT=http://localhost:4566 \
S3_TEST_BUCKET=test-bucket \
AWS_ACCESS_KEY_ID=test \
AWS_SECRET_ACCESS_KEY=test \
AWS_REGION=us-east-1 \
npm run test:integration

# Cleanup
docker stop localstack-s3-test

Using LocalStack in Your Code

import { S3Store } from '@banana.inc/cacheman-s3';

// Configure S3Store for LocalStack
const cache = new S3Store({
  bucket: 'test-bucket',
  region: 'us-east-1',
  endpoint: 'http://localhost:4566',      // LocalStack endpoint
  forcePathStyle: true,                   // Required for LocalStack
  accessKeyId: 'test',                    // Any value works
  secretAccessKey: 'test'                 // Any value works
});

// Use normally
cache.set('key', { data: 'value' }, (error) => {
  if (error) throw error;
  console.log('Cached successfully with LocalStack!');
});

LocalStack Configuration

The package automatically detects LocalStack when the endpoint option is provided:

const localstackOptions = {
  bucket: 'my-bucket',
  endpoint: 'http://localhost:4566',
  forcePathStyle: true,  // Automatically set to true for LocalStack
  accessKeyId: 'test',
  secretAccessKey: 'test',
  region: 'us-east-1'
};

const cache = new S3Store(localstackOptions);

Benefits of LocalStack Testing

No AWS Costs: Test locally without incurring S3 charges
Fast Feedback: No network latency to AWS
Isolation: Tests don't affect production resources
CI/CD Friendly: Easy to integrate in GitHub Actions
Offline Development: Work without internet connection

TypeScript Compilation

# Watch mode for development
npm run build:watch

# Clean build
npm run clean && npm run build

License

MIT

Contributing

Fork the repository
Create your feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add some amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Development Guidelines

Write TypeScript-first code with proper type definitions
Ensure test coverage for new features
Follow the existing code style (enforced by ESLint)
Update documentation for API changes
Add type definitions for all public APIs

Support

Related Projects

cacheman - Caching library for Node.js
AWS SDK for JavaScript v3 - AWS SDK for JavaScript

Name		Name	Last commit message	Last commit date
Latest commit History 19 Commits
.github/workflows		.github/workflows
scripts		scripts
src		src
test		test
.eslintrc.js		.eslintrc.js
.gitignore		.gitignore
.npmignore		.npmignore
LICENSE		LICENSE
README.md		README.md
docker-compose.localstack.yml		docker-compose.localstack.yml
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json

License

dragonxsx/cacheman-s3

Folders and files

Latest commit

History

Repository files navigation

@banana.inc/cacheman-s3

Features

Installation

Quick Start

TypeScript

JavaScript

Usage with Cacheman

TypeScript

Configuration

Basic Configuration

Advanced Configuration with Types

API Reference

Constructor

Methods

cache.set()

cache.get()

cache.del()

cache.clear()

cache.scan()

cache.healthCheck()

Type Definitions

Core Interfaces

Error Types

Advanced Usage

Hierarchical Cache Keys

Generic Type Constraints

Utility Functions

AWS IAM Permissions

Development

Setup

LocalStack Integration Testing

Prerequisites

Quick Start with LocalStack

Docker Compose Method

Manual Docker Method

Using LocalStack in Your Code

LocalStack Configuration

Benefits of LocalStack Testing

TypeScript Compilation

License

Contributing

Development Guidelines

Support

Related Projects

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 4

Packages 0

Languages

Packages