In the landscape of building applications based on Language Models (LLMs), RAG (Retrieval-Augmented Generation) systems, and AI Agents, we clash daily with an invisible yet extremely costly enemy: the "Token Tax".

Feeding an LLM with raw data — such as giant JSON arrays, endless system logs, redundant code, or dirty HTML code — not only drives up API costs, but quickly eats through the context window and degrades the quality of the model's responses.

The release of Caveman v1.3.0 answers exactly this challenge. Drawing inspiration from the open-source headroom architecture, this release introduces a content-aware compression pipeline. The underlying idea is as simple as it is powerful: every type of data the LLM has to read deserves a dedicated compression algorithm.

The best part? The update is fully additive: existing APIs remain unchanged, guaranteeing full backward compatibility.

The Heart of the System: Intelligent Routing

The real turning point of this release is centralizing compression logic. It's no longer up to the developer to decide how to treat data before sending it to the prompt.

CavemanContentRouter

It's the single entry point of the pipeline. The router dynamically analyzes incoming text, detects its nature (a JSON array, a log, a git diff, tabular text, or code) and automatically routes it to the most efficient algorithm.

To guarantee industrial-grade performance, the router integrates:

1. Skip-Set + Two-Level Result Cache: An $O(1)$ cache with a 30-minute TTL and lazy eviction to avoid re-processing identical or non-compressible content.
2. Circuit Breaker: If the system detects 3 consecutive failures, it temporarily disables compression, passing the text through in passthrough mode so as not to block the application.
3. Inflation Guard: A guardian that compares output tokens against input tokens; if for any anomaly the output were to exceed the input size, the system instantly restores the original text.

Configuration is immediate thanks to support for the CompressionProfile enum (Light, Balanced, Agent, Aggressive), enabling a single-line setup:

C#

The Specialized Compressors: How the Data Changes

Let's look in detail at how the individual modules optimize the different types of information.

1. Structured Data and Tables

1. CavemanJsonCrusher: Handles JSON arrays following two fundamental strategies. The Lossless route converts uniform arrays (up to 6 keys and 50 rows) into Markdown tables or compact CSV format (leveraging the #schema: header + RFC 4180 rows), provided the savings are at least 15%. The Lossy route (with controlled loss) leverages the BM25 algorithm to remove less relevant rows while keeping the anchors fixed (the first 30% and last 15% of the data) and detecting anomalies. Removed rows are replaced by the marker <<ccr:HASH,dropped=N/TOTAL>>.
2. CavemanCcrStore: The in-memory (thread-safe, 5-minute TTL) store that holds rows removed by JsonCrusher, indexing them via a 12-character hexadecimal prefix of the SHA-256 hash, allowing the LLM to retrieve them if needed.
3. CavemanTabularCompressor: Optimizes CSV files and markdown tables by removing empty or constant columns and sampling rows based on query relevance.

2. Logs, Development

1. CavemanLogCompressor: Analyzes logs by assigning a severity score (ERROR, WARN, INFO, DEBUG). Detects multilingual stack frames, isolates a context window of $\pm2$ lines around errors, and applies conservative deduction on warnings (e.g. converting variable numbers to N and hexadecimal addresses to ADDR).
2. CavemanSearchCompressor: Ideal for processing the output of tools like grep or ripgrep. Groups results by file, scores matches by relevance to the query, and keeps the leading and trailing segments of each file.
3. CavemanDiffCompressor: Optimized for git diff. Keeps all change lines intact (+ / -), reduces static context lines to a minimum (default 2 lines), and removes blocks of pure unchanged context.

3. Source Code and Web

1. CavemanCodeCompressor: A multilingual module (C#, Java, JS/TS, Go, Rust, Python, Ruby, SQL, Shell) that removes comments and collapses blank lines, guaranteeing an output that is a safe structural subset of the original source code.
2. CavemanHtmlExtractor: An extractor based on pure regular expressions, with no external dependencies. Strips scripts and styles, converts block elements into new lines, and decodes HTML entities to return clean text readable by the LLM.

Performance and KV-Cache Optimization

Beyond reducing the raw token count, Caveman v1.3.0 introduces advanced strategies to fully exploit the hardware and software architecture of modern LLM providers.

KV-Cache Protection with CavemanCacheAligner

One of the sneakiest problems in conversational systems is the "bust" of the KV-Cache (Key-Value Cache). If the system prompt contains volatile elements that change with every single call — such as UUIDs, ISO-8601 timestamps, JWT tokens or variable hashes — the LLM's infrastructure is forced to recompute the entire prompt from scratch, drastically increasing latency (Time-To-First-Token) and compute costs.

CavemanCacheAligner detects and isolates or normalizes these volatile tokens in system prompts, ensuring that the static parts remain aligned and ready in the AI provider's cache.

Expert tip: Always use CavemanWasteAnalyzer during development. This module analyzes prompts in a non-destructive way and estimates exactly how many tokens you're wasting due to excessive whitespace, base64 blobs, or overly heavy JSON, helping you fine-tune the compression pipeline.

Multi-Agent Architectures and Output Control

The Caveman ecosystem also grows richer on the front of managing complex conversational flows:

1. CavemanSharedContext: A compressed context store designed for interaction between multiple agents. The Put command compresses and stores information; subsequent nodes can retrieve the compressed version via Get (saving tokens on every cross-read) or request the original via Get(full:true).
2. CavemanMessageDeduplicator: Monitors message history to identify hash-based duplicates. It can distinguish a genuine "re-read" (a re-read more than 3 messages apart) from close-together polling activity, replacing duplicate text with a compact [duplicate of message #N].
3. CavemanOutputShaper: Works on the system prompt by injecting verbosity control instructions (such as SkipCeremony, NoRestatement, ConclusionsOnly, MinimumTokens). The system is idempotent, byte-stable for each chosen level, and easily removable.

Finally, extending the builder pattern via CavemanContentRouterBuilder allows granular customization of flows, introducing fluent options like .WithProseLevel() to define the degree of linguistic synthesis applied to descriptive text.

Conclusions

Caveman v1.3.0 isn't just a technical update — it represents a paradigm shift in context management for AI applications. By centralizing compression and specializing it by data type, it allows cutting live inference costs and improving model responsiveness, all with zero-configuration insertion into existing projects via the ICompressionService.CompressContentAsync interface.

Nuget Package

dotnet add package Caveman --version 1.3.0