Cache & Stampede-Proof Storage

Katal v3 provides a high-throughput, leak-free caching engine with built-in Single-Flight Dog-Piling / Cache Stampede Protection, $O(1)$ LRU eviction, and pluggable storage backends (In-Memory, Redis, Memcached).


1. Quick Start

Katal boots with an ultra-fast in-memory cache by default, requiring zero configuration:

import { Application } from "katal";

const app = new Application({ port: 3000 });

// 1. Primitive Get / Set with TTL (in seconds)
await app.cacheSet("site:announcement", "Scheduled maintenance at midnight", 3600);
const announcement = await app.cacheGet<string>("site:announcement");

// 2. Read-Through with Single-Flight Stampede Protection
const userDashboard = await app.cacheRemember(
    "dashboard:user:42",
    async () => {
        // Expensive database aggregation or analytics query
        return await computeAnalyticsForUser("42");
    },
    600 // TTL: 10 minutes
);

// 3. Delete cache entry
await app.cacheDelete("dashboard:user:42");

2. Single-Flight Cache Stampede (Dog-Piling) Protection

The Problem

When a high-traffic cache key expires (e.g. 5,000 requests/sec for a trending product page), every incoming request simultaneously encounters a cache miss. In traditional frameworks, all 5,000 requests trigger parallel database queries, causing severe CPU spikes, database connection pool exhaustion, and catastrophic cascading outages.

The Katal Solution

Katal's cacheRemember implements in-flight promise deduplication. When a cache miss occurs, only the first request executes the resolver function. All subsequent concurrent requests for that key join the active promise and resolve with the same computed value once completed:

Concurrent Requests              Katal In-Flight Resolver             Database
───────────────────              ────────────────────────             ────────
Request 1 (Miss) ──────────────►  Executes Resolver() ──────────────►  Query Runs
Request 2 (Miss) ──────────────►  Awaits Active Promise
Request 3 (Miss) ──────────────►  Awaits Active Promise
Request 4 (Miss) ──────────────►  Awaits Active Promise
                                              │                              │
                                  Store in Cache & Return ◄──────────────────┘
                                              │
                      All 4 Requests Receive Identical Result

Usage Example

class ProductDetailsController extends Controller {
    constructor(private readonly db: DatabaseConnection) {
        super();
    }

    async handle(ctx: RequestContext) {
        const productId = ctx.params.id;

        // Even under 10,000 req/sec, this DB query runs exactly ONCE on cache miss!
        const product = await this.app.cacheRemember(
            `products:${productId}`,
            async () => {
                return await this.db.query(
                    "SELECT * FROM products WHERE id = $1",
                    [productId]
                );
            },
            300 // 5 minutes
        );

        return this.success(product);
    }
}

3. High-Performance $O(1)$ LRU Memory Store

The default MemoryStore is built using a Doubly Linked List + Hash Map structure that guarantees $O(1)$ reads, writes, and evictions without memory leaks:

  • Strict Capacity Bound: Drops the least recently used (LRU) entry when maxSize is reached.
  • Non-Blocking Background Sweep: Automatically purges expired TTL nodes every 60 seconds using an unreferenced timer (unref()), preventing worker process hang on shutdown.
  • Lazy Eviction on Read: Expired keys accessed between sweeps are pruned immediately upon retrieval.

Configuring Memory Store Options

import { MemoryStore } from "katal";

// Configure custom cache parameters
const memoryStore = new MemoryStore({
    maxSize: 10_000,          // Maximum number of cached items (default: 5000)
    cleanupIntervalMs: 30_000, // Sweep interval in ms (default: 60,000)
});

app.useCacheStore(memoryStore);

4. Distributed Redis Cache Store Adapter

For horizontally scaled production clusters, implement the CacheStore contract with Redis:

import type { CacheStore } from "katal";
import Redis from "ioredis"; // Or Bun's native Redis client

export class RedisCacheStore implements CacheStore {
    constructor(private readonly redis: Redis) {}

    async get<T>(key: string): Promise<T | null> {
        const raw = await this.redis.get(key);
        if (!raw) return null;
        try {
            return JSON.parse(raw) as T;
        } catch {
            return raw as unknown as T;
        }
    }

    async set<T>(key: string, value: T, ttlSeconds?: number): Promise<void> {
        const payload = JSON.stringify(value);
        if (ttlSeconds && ttlSeconds > 0) {
            await this.redis.set(key, payload, "EX", ttlSeconds);
        } else {
            await this.redis.set(key, payload);
        }
    }

    async delete(key: string): Promise<void> {
        await this.redis.del(key);
    }

    async clear(): Promise<void> {
        await this.redis.flushdb();
    }

    async has(key: string): Promise<boolean> {
        const exists = await this.redis.exists(key);
        return exists === 1;
    }
}

// Register with application
const redis = new Redis(process.env.REDIS_URL ?? "redis://localhost:6379");
app.useCacheStore(new RedisCacheStore(redis));

5. API Reference

Method Signature Description
app.cacheGet<T>(key) (key: string) => Promise<T \| null> Reads key. Returns null if expired or missing.
app.cacheSet<T>(key, value, ttl?) (key: string, value: T, ttlSeconds?: number) => Promise<void> Sets key with optional TTL in seconds.
app.cacheDelete(key) (key: string) => Promise<void> Removes key from storage.
app.cacheRemember<T>(key, resolver, ttl?) (key: string, resolver: () => Promise<T>, ttl?: number) => Promise<T> Read-through with single-flight mutex protection.
app.useCacheStore(store) (store: CacheStore) => this Swaps the active cache engine.
app.getCacheStore() () => CacheStore Returns the active cache store instance.

6. Access via Dependency Injection

Inject CacheStore directly into domain services or controllers using the "cache" DI token:

import { Controller } from "katal";
import type { CacheStore } from "katal";

export class OrderService {
    constructor(private readonly cache: CacheStore) {}

    async getOrderTotals(orderId: string) {
        const cacheKey = `order:totals:${orderId}`;
        const cached = await this.cache.get<number>(cacheKey);
        if (cached !== null) return cached;

        const totals = await calculateOrder(orderId);
        await this.cache.set(cacheKey, totals, 60);
        return totals;
    }
}

// Register in IoC container
app.singleton(OrderService, (c) => new OrderService(c.resolve("cache")));

See Also