Cohesor

Product Introduction

1. Definition: Cohesor is a neutral AI gateway and control plane, a technical middleware layer that sits between AI agents (like Claude Code, Cursor, or custom workflows) and the underlying large language models (LLMs). It functions as a unified API endpoint that manages, optimizes, and governs all outgoing AI requests.
2. Core Value Proposition: Cohesor exists to solve the exploding and unpredictable cost of running AI agents at scale. Its primary value is reducing AI model spend by 60–90% through intelligent token compression and dynamic model routing, while providing enterprise-grade governance, observability, and tool management through a single, zero-code integration point.

Main Features

1. Smart LLM Routing: A dynamic routing system that analyzes each incoming request in real-time. It uses a lightweight complexity classifier to score prompts and automatically route them to the optimal LLM based on configurable objectives: Quality, Balanced, Cheapest, or Fastest. This ensures expensive models like Claude Opus are only used for complex tasks, while simpler queries are handled by cost-effective models like Llama 70B on Groq, optimizing cost versus performance per request.
2. Lossless Token Compression: An advanced compression engine that reduces prompt size before billing. It employs multiple techniques: Stale-read supersession (removing outdated file content), context compaction (folding out stale conversation turns), semantic deduplication (collapsing near-identical text spans above a 0.97 similarity threshold), and structural rewrite (normalizing whitespace and markup). This process cuts input tokens by approximately 50% on average without degrading semantic meaning or output quality.
3. Unified MCP (Model Context Protocol) Gateway: A centralized broker for AI agent tools. It provides a single audited endpoint (api.cohesor.com/mcp) to connect to services like GitHub, Gmail, and Google Drive. The gateway manages OAuth 2.1 authentication, securely vaults credentials, enforces per-agent tool scopes (e.g., gmail:read for support agents), and logs every tool call with full traces for replay and audit.
4. Granular Spend Governance & Observability: A real-time policy enforcement and monitoring layer. It applies gateway keys, team scopes, rate limits, and hard budget caps at the network edge (<1ms). The dashboard provides live attribution of spend per user, API key, and team, with showback reports, audit logs, and full request traces that include token counts, model used, latency, and cost for every AI call.

Problems Solved

1. Pain Point: Uncontrolled and opaque AI agent spend. Teams lack visibility into which users, agents, or prompts are driving costs, and have no mechanism to enforce budgets before invoices spiral.
2. Pain Point: Inefficient LLM usage. Developers manually hardcode agents to use a single, often overpowered and expensive model for all tasks, leading to significant waste on simple operations.
3. Target Audience: Engineering & DevOps Leaders managing fleets of AI agents; FinOps and Finance Teams responsible for cloud and AI budgets; AI/ML Engineers and Developers building and deploying agentic workflows; Enterprise IT/Security teams requiring governance and audit trails for AI tool usage.
4. Use Cases: Governing spend for a team using Claude Code and Cursor IDE; optimizing a customer support chatbot that uses a mix of simple and complex queries; managing tool access and credentials for multiple internal research and ops agents; providing a single audit point for all AI activity across a large organization.

Unique Advantages

1. Differentiation: Unlike vendor-specific proxies or simple API gateways, Cohesor is model-neutral and provider-agnostic. It does not sell an LLM, eliminating lock-in and conflict of interest. It compares favorably to DIY orchestration layers by offering a production-ready, zero-code solution that adds minimal latency (~12ms p50).
2. Key Innovation: The integration of lossless token compression specifically for agentic workflows. By compressing not just prompts but also verbose tool outputs and conversation history before they are sent to the LLM, it directly attacks the largest driver of cost—input tokens—at the network layer.
3. Key Innovation: Protocol-native integration. By speaking the Anthropic and OpenAI wire protocols natively, it requires only a one-line environment variable change (ANTHROPIC_BASE_URL) to onboard existing agents using Claude Code, Cursor, or any compatible SDK, enabling immediate deployment without code rewrites.

Frequently Asked Questions (FAQ)

1. How does Cohesor reduce AI costs without sacrificing quality? Cohesor uses a two-pronged approach: first, its lossless token compression reduces the number of billed input tokens by ~50% on average. Second, its smart router sends each request to the most cost-effective model capable of handling the task's complexity, ensuring you don't overpay with a premium model for simple tasks.
2. Is Cohesor a model provider or do I still need LLM API keys? Cohesor is not a model provider. It is a routing and optimization layer. You bring your own API keys from providers like OpenAI, Anthropic, or Google. Cohesor manages them securely and uses them to route requests on your behalf, providing a single control plane.
3. What is the performance latency impact of using the Cohesor gateway? The end-to-end latency added is approximately 12 milliseconds at the 50th percentile (p50), with most overhead from the sub-millisecond policy checks. The total request journey from agent to final LLM response is about 41ms p50, making it negligible for most asynchronous agent workflows.
4. How does Cohesor handle data privacy and security? Cohesor operates on a privacy-first principle. Prompts and responses are not retained after processing and are never used for training. The system only stores usage metadata (tokens, cost, model). It also centralizes and secures MCP credentials in a vault, keeping them out of agent context windows and logs.
5. Can I use Cohesor with my existing AI agents and SDKs? Yes. Cohesor offers drop-in compatibility by supporting the standard Anthropic and OpenAI API protocols. You can point existing agents using Claude Code, Cursor, LangChain, or direct HTTP clients to Cohesor's endpoint by changing a single base URL configuration variable.