Spanlens

Product Introduction

1. Overview: Spanlens is a self-hostable, MIT-licensed observability and monitoring platform specifically engineered for Large Language Model (LLM) applications. It functions as a middleware layer that intercepts, logs, and analyzes API calls to major providers like OpenAI, Anthropic, and Google's Gemini.
2. Value: It provides developers and engineering teams with granular, real-time visibility into LLM usage, enabling precise cost control, performance optimization, and debugging of complex AI agent workflows without requiring extensive code changes.

Main Features

1. Unified Request Logging: Automatically captures every LLM API call with full metadata, including model used, token counts, exact cost, latency, and the complete request/response body. Data can be filtered, grouped, and exported for analysis.
2. Multi-Step Agent Tracing: Visualizes complex, multi-step AI agent workflows (e.g., built with LangGraph) as waterfall span trees. This allows for critical path analysis, cost attribution per step, and easy identification of latency outliers or errors within a chain of actions.
3. Proactive Cost Intelligence: Goes beyond simple tracking by offering a model recommender engine. It analyzes call patterns (e.g., classification, summarization) and suggests cheaper, suitable alternative models (like switching from gpt-4o to gpt-4o-mini), projecting potential dollar savings.

Problems Solved

1. Challenge: LLM API costs are unpredictable and can spiral due to inefficient model usage, unoptimized prompts, or faulty agent logic. Debugging multi-step AI workflows is notoriously difficult without proper tracing.
2. Audience: Engineering teams, AI product managers, and developers building production applications with OpenAI, Anthropic, or Gemini APIs who need to manage budgets, ensure reliability, and improve performance.
3. Scenario: A SaaS company notices a sudden 50% spike in their monthly OpenAI bill. Using Spanlens, they quickly identify that a new feature is incorrectly using gpt-4 for simple text classification. The model recommender suggests gpt-4o-mini, and by replaying a dataset, they confirm quality remains high while cutting costs by over 80%.

Unique Advantages

1. Vs Competitors: Unlike many SaaS-only observability tools, Spanlens is open-source and self-hostable (MIT license), giving enterprises full control over their sensitive LLM request data. Its integration requires minimal code change—often just setting a single baseURL—compared to full SDK rewrites.
2. Innovation: It combines traditional observability (logging, tracing) with AI-specific intelligence: LLM-as-judge evaluations for quality scoring, statistical anomaly detection (3σ deviations) on cost/latency, and automated PII (SSN, email) and prompt injection pattern scanning at ingestion time.

Frequently Asked Questions (FAQ)

1. How does Spanlens integrate with my existing LLM code? Spanlens uses a lightweight SDK (available for TypeScript and Python) that typically requires only changing your LLM client's API endpoint (baseURL) to point to the Spanlens proxy, which then forwards requests to OpenAI, Anthropic, or Gemini. No major code rewrites are needed.
2. Can I use Spanlens with locally hosted models like Ollama? Yes, starting from SDK v0.6.1, Spanlens supports observability for locally hosted LLMs via Ollama, allowing you to monitor cost-equivalents, latency, and usage of open-weight models alongside commercial APIs in a single dashboard.
3. What makes Spanlens different from just using the providers' own dashboards? Spanlens provides a unified view across multiple LLM providers (OpenAI, Anthropic, Gemini), offers much more granular per-request and per-user analytics, enables tracing for multi-agent workflows, and delivers proactive features like model recommendations and anomaly alerts that native dashboards lack.