Anthropic Outages

AI System Design chooses Availability > Consistency as UX is higher value then error, and there are almost no critical data loss.

However, Anthropic currently has neither - with outages happening almost every day.

Their infrastructure maturity is severely lagging behind its aggressive product shipping velocity.

And here are some ideas from what i was able to source online.

Backpressure & Admission Control

Continuous batching mechanism is failing under heavy traffic, causing GPU inference workers to crash instead of degrading gracefully

❌ Unbounded Acceptance - Blindly enqueues requests into continuous batching queues, causing GPU workers to run out of VRAM (OOM) and crash during traffic spikes.

@app.post("/v1/messages")
async def generate(request: Request):
    # Accepts request regardless of GPU KV-cache state
    # Worker hits OOM during context allocation -> crashes with HTTP 500
    await batch_queue.put(request)
    return await gpu_worker.process(request)

✅ VRAM-Aware Backpressure - Checks KV-cache VRAM saturation at the ingress boundary and fails fast with HTTP 429 before touching worker queues.

@app.post("/v1/messages")
async def generate(request: Request):
    # Fail-fast backpressure at the gateway boundary
    if kv_cache_monitor.get_usage() > 0.90:
        raise HTTPException(status_code=429, detail="Inference saturated. Retry-After: 5s")
    
    await batch_queue.put(request)
    return await gpu_worker.process(request)

Shared B2B/B2C Gateway

A shared, monolithic API gateway means that traffic spikes from new consumer features (like their new "Import Memory" update) create thread starvation that takes down their enterprise API pipelines simultaneously

❌ Shared Thread/Connection Pool - Consumer feature spikes (e.g., "Import Memory") consume the shared pool, starving enterprise API requests.

# Shared thread/connection pool across all surfaces
shared_gateway_pool = ConnectionPool(max_size=1000)

@app.post("/route")
async def handle_request(req: Request):
    # Consumer traffic burst starves Enterprise threads -> Global Outage
    async with shared_gateway_pool.acquire():
        return await dispatch_to_model(req)

✅ Bulkhead Architecture - Isolates connection pools so consumer load can never starve enterprise API bandwidth.

# Isolated capacity per domain
consumer_pool = ConnectionPool(max_size=600)
enterprise_pool = ConnectionPool(max_size=400)

@app.post("/route")
async def handle_request(req: Request):
    pool = enterprise_pool if req.is_enterprise_api else consumer_pool
    
    # Exhaustion in consumer_pool throws 429 for web users only;
    # enterprise_pool remains 100% available
    async with pool.acquire():
        return await dispatch_to_model(req)

Subagents self-multiplication

Recently a major bug in Claude Code's sub-agent system caused agents to multiply exponentially in an infinite loop, consuming massive amounts of tokens and crashing the system.

❌ Unbounded Sub-Agent Spawning - Sub-agent tool invocation loops spawn child agents endlessly, saturating batch queues and crashing workers.

def run_agent(task):
    # No bounds on depth or total spawned sub-agents
    for subtask in task.plan_subtasks():
        run_agent(subtask)  # Can loop infinitely if context caching degrades

✅ Bounded Depth & Budget Breaker - Enforces recursion depth and total token budgets per agentic run.

def run_agent(task, depth=0, max_depth=3, total_tokens=0):
    if depth >= max_depth or total_tokens > BUDGET_LIMIT:
        raise CircuitBreakerError("Sub-agent recursion or token budget exceeded")
    
    for subtask in task.plan_subtasks():
        run_agent(subtask, depth=depth + 1, total_tokens=total_tokens + task.tokens_used)