Resilience & Testing

Circuit Breaker

Prevents thundering herd when a backend goes down. When Redis or Postgres becomes unreachable, every extension that uses the breaker fails fast instead of queueing up timeouts, and fire-and-forget operations (heartbeats, relay flushes, cursor broadcasts) are skipped entirely.

Three states:

• healthy - everything works, requests go through
• broken - too many failures, requests fail fast via CircuitBrokenError
• probing - one request is allowed through to test if the backend is back

Setup

// src/lib/server/breaker.js
import { createCircuitBreaker } from 'svelte-adapter-uws-extensions/breaker';

export const breaker = createCircuitBreaker({
  failureThreshold: 5,
  resetTimeout: 30000,
  onStateChange: (from, to) => console.log(`circuit: ${from} -> ${to}`)
});

Pass the same breaker to all extensions that share a backend:

import { breaker } from './breaker.js';

export const bus = createPubSubBus(redis, { breaker });
export const presence = createPresence(redis, { breaker, key: 'id' });
export const replay = createReplay(redis, { breaker });
export const limiter = createRateLimit(redis, { points: 10, interval: 1000, breaker });

Failures from any extension contribute to the same breaker. When one trips it, all others fail fast.

Options

API

How extensions use it

Awaited operations (join, consume, publish) call guard() before the Redis/Postgres call, success() after, and failure() in the catch block. When the circuit is broken, guard() throws CircuitBrokenError and the operation never reaches the backend.

Fire-and-forget operations (heartbeat refresh, relay flush, cursor broadcast) check isHealthy and skip entirely when the circuit is not healthy. This prevents piling up commands on a dead connection.

Per-extension degradation behavior

Note that rate limiting is fail-closed: when the breaker is open, requests are blocked, not allowed through. This is a deliberate safety choice.

Error handling

import { CircuitBrokenError } from 'svelte-adapter-uws-extensions/breaker';

try {
  await replay.publish(platform, 'chat', 'msg', data);
} catch (err) {
  if (err instanceof CircuitBrokenError) {
    // Backend is down -- degrade gracefully
    platform.publish('chat', 'msg', data); // local-only delivery
  }
}

Notifying clients of degradation

When Redis pub/sub fails, live streams on other replicas stop receiving updates. Connected clients keep showing stale data with no indication that the stream is degraded. Use the onStateChange callback to publish a system-level event so clients can surface this:

import { createCircuitBreaker } from 'svelte-adapter-uws-extensions/breaker';

let distributed; // the bus.wrap(platform) reference

export const breaker = createCircuitBreaker({
  failureThreshold: 5,
  resetTimeout: 30000,
  onStateChange: (from, to) => {
    if (!distributed) return;
    if (to === 'broken') {
      // Local-only publish - Redis is down, but local clients still receive it
      distributed.publish('__system', 'degraded', { reason: 'backend unavailable' });
    } else if (from === 'broken' && to === 'healthy') {
      distributed.publish('__system', 'recovered', null);
    }
  }
});

On the client side, subscribe to __system and show a banner when the degraded event fires. On recovered, dismiss the banner and refetch stale data.

Graceful Shutdown

All clients listen for the sveltekit:shutdown event and disconnect cleanly by default. You can disable this with autoShutdown: false and manage the lifecycle yourself.

// Manual shutdown
await redis.quit();
await pg.end();
presence.destroy();
cursors.destroy();
lobby.destroy();
breaker.destroy();

Call destroy() on any extension that runs background timers (presence heartbeats, cursor throttle timers, circuit breaker reset timers, Postgres cleanup intervals). The client quit()/end() methods close the underlying connections.

Testing

Tests use in-memory mocks for Redis and Postgres - no running services needed.

npm test

Since the extensions match the core plugin APIs, you can swap between in-memory plugins (for tests and single-instance dev) and Redis/Postgres extensions (for production) by changing the import path. Structure your code so the extension instance is created in a single file (src/lib/server/replay.js, etc.) and imported everywhere else. To switch backends, change that one file.

Testing your own extension-consuming code

The svelte-adapter-uws-extensions/testing entry point exports the same in-memory mocks used by the extensions’ own test suite. Use them to test your code without running Redis or Postgres:

import { mockRedisClient, mockPlatform, mockWs } from 'svelte-adapter-uws-extensions/testing';
import { createPresence } from 'svelte-adapter-uws-extensions/redis/presence';
import { createRateLimit } from 'svelte-adapter-uws-extensions/redis/ratelimit';
import { describe, it, expect } from 'vitest';

describe('presence', () => {
  it('tracks users across topics', async () => {
    const client = mockRedisClient();
    const platform = mockPlatform();
    const presence = createPresence(client, { key: 'id' });

    const ws = mockWs({ id: 'user-1', name: 'Alice' });
    await presence.join(ws, 'room:lobby', platform);

    expect(await presence.count('room:lobby')).toBe(1);
    expect(platform.published.some(p => p.event === 'join')).toBe(true);

    presence.destroy();
  });
});

describe('rate limiting', () => {
  it('blocks after exhausting points', async () => {
    const client = mockRedisClient();
    const limiter = createRateLimit(client, { points: 3, interval: 10000 });
    const ws = mockWs({ remoteAddress: '1.2.3.4' });

    for (let i = 0; i < 3; i++) {
      expect((await limiter.consume(ws)).allowed).toBe(true);
    }
    expect((await limiter.consume(ws)).allowed).toBe(false);
  });
});

Available mocks

The circuit breaker (createCircuitBreaker()) is pure logic with no I/O — use it directly in tests, no mock needed.