Headroom

Headroom is a local context-compression layer

It sits between an Agent <> LLM, shrinking model input & maintaining originals. (60–95% for JSON, ~15–20% for coding agents)

It checks each request for volatile bits like timestamps and UUIDs that would break the provider's cache, then guesses the content type and hands it to the right compressor - one for JSON, one for code, one for prose and logs.

Compression loses detail, so the original is saved on disk and the model gets a headroom_retrieve tool to ask for it.

Only new text is compressed; everything already in the conversation stays untouched, so prompt caching keeps working and nothing is dropped.

It trims the model's replies too - shorter answers, less thinking on routine turns. The block in this session is that happening live.

I has saved me whole $1 already haha.

Core concepts

1. The pipeline. Every request flows through three stages:

• CacheAligner - detects volatile content (timestamps, UUIDs) that would bust the provider's KV-cache prefix; warns, never rewrites.
• ContentRouter - sniffs content type (uses Magika/ONNX, with heuristic fallback) and dispatches to the right compressor.
• Compressors - three specialists: SmartCrusher (JSON: arrays of dicts, nested objects), CodeCompressor (AST-aware, ~8 languages), and Kompress-v2-base (their HuggingFace ML model for prose/logs, trained on agent traces).

2. CCR - reversibility. Compression is lossy, so originals are cached locally (headroom/ccr/) and the LLM gets a headroom_retrieve tool. If it needs the full data, it asks - same answers, fraction of tokens up front.

3. Live-zone compression. Only new bytes (latest tool output, newest turn) get compressed; the frozen conversation prefix stays byte-identical so provider prompt caching still hits. History is never dropped.

4. Output shaping. Beyond shrinking input, the proxy trims what the model writes back: appends terseness instructions to the system prompt and dials down thinking effort on routine turns (resuming after a file read). You're seeing it live - the block in this session is Headroom injecting that.

The compressors

Compressor	Input	Core idea
SmartCrusher	JSON arrays/objects	Keep anchors (errors, first/last, query-relevant rows), drop the redundant middle. Lossless tabular compaction (csv-schema / markdown-kv) when it saves ≥15%. Output is always valid items from the original — no wrapper text.
CodeCompressor	Py, JS/TS, Go, Rust, Java, C/C++	Parse to AST, keep imports + signatures + types + error handlers, rank functions by importance, collapse low-value bodies. Output always re-parses. (LongCodeZip-style.)
Kompress-v2-base	prose, logs, tool text	ModernBERT model trained on agent traces scores tokens; low-information tokens pruned. Runs locally via ONNX.
Search/LogCompressor	grep results, build output	Heuristic: dedup repeated hits, keep failures + summary lines.

Examples

Token counts are illustrative; shapes match what each compressor actually emits.

1. JSON — SmartCrusher

One GitHub API call returning 80 issues.

Before (~14,200 tokens):

[
  {"id": 1001, "state": "open", "title": "widget crashes on empty input",
   "user": {"login": "user3", "id": 5003}, "labels": [{"name": "bug"}],
   "comments": 38, "created_at": "2026-07-02T10:00:00Z", "url": "https://..."},
  ... 79 more rows, same shape ...
]

After (~410 tokens):

[
  {"id": 1001, "state": "open", "title": "widget crashes on empty input", "comments": 38},
  {"id": 1042, "state": "open", "title": "widget crashes on unicode", "comments": 31},
  {"id": 1007, "state": "closed", "title": "crash on null flag", "comments": 29},
  {"_ccr_dropped": "<<ccr:9a5019e3 77_rows_offloaded>>"}
]

Schema detected, anchor rows kept (errors, first/last, query-relevant), the repetitive middle offloaded. If the model needs row 54, it calls headroom_retrieve and gets it back.

2. Code — CodeCompressor

A 300-line Python service file read into context.

Before:

import httpx
from .models import Invoice

def sync_invoices(client: httpx.Client, since: str) -> list[Invoice]:
    """Fetch and normalize invoices modified after `since`."""
    page = 1
    results = []
    while True:
        resp = client.get("/invoices", params={"since": since, "page": page})
        resp.raise_for_status()
        batch = resp.json()["items"]
        # ... 40 more lines of pagination, retries, field mapping ...
    return results

After:

import httpx
from .models import Invoice

def sync_invoices(client: httpx.Client, since: str) -> list[Invoice]:
    """Fetch and normalize invoices modified after `since`."""
    page = 1
    results = []
    # [43 lines omitted; calls: client.get, resp.raise_for_status, resp.json]
    pass

Imports, signature, types and docstring first line survive; the body is truncated at whole-statement boundaries, with an omission comment listing what the hidden code calls. Output is guaranteed to re-parse.

3. Build logs — LogCompressor

A pytest run: 240 lines, 2 failures.

Before:

tests/test_auth.py::test_login PASSED
tests/test_auth.py::test_logout PASSED
... 200 more PASSED lines ...
tests/test_billing.py::test_refund FAILED
    def test_refund():
>       assert refund(order).status == "done"
E       AttributeError: 'NoneType' object has no attribute 'status'
tests/test_billing.py::test_invoice FAILED
E       ValueError: currency mismatch: USD != EUR
WARNING db.pool: connection retry 1
WARNING db.pool: connection retry 2
... 30 more identical warnings ...
==== 2 failed, 238 passed in 41.2s ====

After:

tests/test_billing.py::test_refund FAILED
    def test_refund():
>       assert refund(order).status == "done"
E       AttributeError: 'NoneType' object has no attribute 'status'
tests/test_billing.py::test_invoice FAILED
E       ValueError: currency mismatch: USD != EUR
WARNING db.pool: connection retry 1  [x32 similar]
==== 2 failed, 238 passed in 41.2s ====

Errors and full stack traces score highest and are kept verbatim (blank lines inside tracebacks survive); repeated warnings are deduped conservatively; passing noise is dropped. The summary line always stays.