Caveman

Product Introduction

1. Definition: Caveman is a token-efficient AI agent optimization stack and local proxy. It is a technical middleware layer designed to intercept, analyze, and compress AI provider traffic (e.g., to Claude, GPT, Gemini) before it is sent to the model, thereby reducing token consumption and cost.
2. Core Value Proposition: It exists to provide automated, verifiable cost reduction for AI-native applications and agent workflows. Its primary value is applying lossless and lossy compression techniques—like caching, routing, and converting text to images—to context and tool outputs, delivering proven savings measured in fewer input and output tokens without sacrificing output quality or correctness.

Main Features

1. Caveman Skill (Open Source): This is a prompt-level optimization that modifies an AI agent's communication style to be more concise. It works by rewriting verbose instructions and user prompts into a terse, "caveman"-like syntax (e.g., dropping articles, simplifying structure). The technology is implemented as an installable skill/plugin for agents like Claude Code, Cursor, and others, directly reducing output token count by an average of 65% while maintaining byte-for-byte accuracy for code and errors.
2. Caveman Proxy (Local Engine): A command-line tool that launches a local proxy server. It wraps existing AI agent traffic (e.g., caveman wrap claude) and applies the Caveman Engine in real-time. The engine performs recoverable context compression by recognizing patterns in logs, JSON, code, diffs, and tables. It stores the original bytes and replaces them with compressed, eligible versions before sending the context to the AI model, all without requiring a cloud account.
3. Caveman Cloud / Enterprise Gateway: This is a managed service (in development) that acts as a centralized optimization layer. It provides a single endpoint for AI traffic, automatically applying eval-gated optimizations like semantic caching, model routing (e.g., switching a task from GPT-4 to a cheaper model like Claude Haiku if evals pass), and advanced compression. It features a dashboard that attributes every dollar of AI spend by member, API key, workflow, and model using public catalog pricing.
4. Verification & Proof Plane: A core architectural feature that distinguishes inferred savings from verified savings. It uses a three-layer ladder: Inferred (modeled headroom), Replayed (fixes measured against the provider on real request traces), and Verified (provider-causal evidence signed into an immutable ledger). Savings are only "booked" and reported to the user after passing strict eval gates for exactness, structure, task success rate, and cost/latency improvement.

Problems Solved

1. Pain Point: Exponentially high and unpredictable AI API costs due to token inflation from verbose prompts, repeated system instructions, large tool outputs (logs, JSON), and inefficient model usage.
2. Target Audience: AI Engineers and ML Ops teams building production AI agents; Developers using AI-powered coding assistants (Claude Code, Cursor); Enterprise DevOps and FinOps professionals managing large-scale AI spend; Startups and indie hackers needing to optimize limited AI budgets.
3. Use Cases: Optimizing recurring agent interactions with large system prompts; Compressing massive log files or JSON API responses before sending them to an AI for analysis; Implementing automatic model routing for different task complexities (e.g., using a cheaper model for simple extraction); Gaining granular, auditable visibility into AI spend across teams and projects.

Unique Advantages

1. Differentiation: Unlike simple prompt-chaining tools or basic caching proxies, Caveman offers a full-stack "efficiency operating system" with a focus on verifiable, evidence-based savings. It goes beyond inference by replaying optimizations against real traffic and providing signed receipts. Competitors often show projected savings; Caveman's dashboard starts at $0.00 and only increments with verified, provider-causal proof.
2. Key Innovation: The "recoverable compression" engine and the "eval-gated" optimization pipeline. The engine can lossily compress structured data (like converting text logs into PNG images for the model to "see") while storing the original bytes for perfect recovery, maintaining functionality. The eval-gate system ensures any automated optimization (like model routing) is only applied after passing a battery of user-defined correctness checks, guaranteeing no degradation in output quality for the sake of savings.

Frequently Asked Questions (FAQ)

1. How does Caveman reduce AI costs without breaking my agent's functionality? Caveman uses eval-gated optimizations. Any transformation, like compression or model routing, is first tested against your own evaluation criteria (e.g., code exactness, schema validation, task pass rates). The change is only applied to live traffic if it passes these checks, ensuring functional correctness is preserved while token count is reduced.
2. Is Caveman secure? Does it see or store my prompt data? In the local Proxy mode, compression happens on your machine, and original bytes are stored locally. In the Cloud/Enterprise tier, the architecture supports a "zero data retention" mode where the control plane never receives prompt bytes, and data is not persisted. All versions emphasize that provider-reported usage is multiplied by public catalog prices; Caveman does not handle your actual API billing.
3. What is the difference between the Caveman Skill and the Caveman Proxy? The Caveman Skill is a prompt-level optimization you install directly into an AI agent (like Claude Code) to make its communication style more concise. The Caveman Proxy is a local server that sits between your existing agents and the AI provider, applying broader, context-aware compression techniques (on logs, JSON, etc.) to all traffic that passes through it.
4. What does "verified savings" mean, and how is it calculated? "Verified savings" is not an estimate. It is a dollar amount calculated from provider-causal evidence. Caveman replays an optimized version of a real API call against the actual AI provider, measures the token delta, and multiplies it by the provider's public list price. This signed result is added to the ledger. Only savings proven in this way move from "inferred" to "verified" in the dashboard.
5. Can I use Caveman with any AI model or agent framework? The open-source Caveman Skill integrates with specific agents like Claude Code. The Caveman Proxy works at the HTTP layer, so it can wrap traffic from any agent or SDK that allows you to configure a proxy endpoint (like OpenAI, Anthropic, or custom clients). The Caveman Agent SDK provides locked adapters for popular frameworks like Vercel AI SDK and LangChain.