Foglamp

Product Introduction

1. Definition: Foglamp is an open-source observability layer specifically designed for AI agents and applications built using the Vercel AI SDK. It is a monitoring and analytics platform that instruments Large Language Model (LLM) calls to provide deep insights into performance, cost, and behavior.
2. Core Value Proposition: Foglamp exists to make AI agents transparent and manageable by providing comprehensive LLM observability. It enables developers and teams to see and control the cost, latency, token usage, distributed traces, evaluation scores, and error rates for every generateText and streamText call, with just two lines of code integration.

Main Features

1. SDK Integration & Agent Spacing: Foglamp provides a lightweight observability SDK that directly instruments the Vercel AI SDK's core functions. By wrapping the model client, it automatically captures metrics for every LLM call without complex manual instrumentation. This creates per-agent spans, allowing you to trace the full flow of a multi-step agent—from an orchestrator to a researcher or writer—and see the latency and cost of each individual step.
2. Cost Intelligence & Token Analytics: The platform offers granular LLM cost tracking. It breaks down expenses by model (e.g., Claude Opus, Gemini 3.5 Pro), by individual agent or function, and even by customer or API key. It precisely calculates costs based on token usage and model-specific pricing, giving you a clear, real-time understanding of your AI operational spend.
3. Distributed Tracing & Prompt Replay: Foglamp implements distributed tracing for AI workflows, visualizing the entire execution path as a waterfall diagram. Crucially, it captures the exact prompt and response for every span in the trace, allowing for direct inspection of inputs and outputs to debug hallucinations, incorrect outputs, or unexpected behavior.
4. Production Evals & Quality Scoring: It includes a system for LLM evaluation (evals) to score production traffic. You can run code-based checks and use LLM judges to assess the quality and correctness of outputs, measuring metrics like "pass rate" directly on live data. This shifts quality assurance from development to production monitoring.
5. Alerting & Threshold Rules: Foglamp enables proactive monitoring with configurable alerting rules. You can set thresholds for cost spikes, latency degradation, or increases in error rates (e.g., failed LLM calls, refused requests) and receive notifications, allowing teams to react to issues before they impact end-users.

Problems Solved

1. Pain Point: "Black Box" LLM Costs and Performance. Development teams lack visibility into the real-world cost and performance of their AI agents after deployment, leading to unexpected budget overruns and difficulty diagnosing latency issues or poor-quality outputs.
2. Target Audience: This product is essential for AI/ML Engineers, Full-Stack Developers using the Vercel stack, DevOps/SRE teams responsible for production reliability, Technical Product Managers needing to forecast AI costs, and AI Agent Builders iterating on multi-step workflows.
3. Use Cases: Foglamp is critical for debugging production AI agent failures (e.g., an agent quoting incorrect policy), optimizing model selection based on cost/quality trade-offs, tracking per-customer AI spend in a SaaS product, setting performance SLAs for LLM-driven features, and validating improvements in prompt engineering or fine-tuning via production eval scores.

Unique Advantages

1. Differentiation: Unlike general Application Performance Monitoring (APM) tools (like Datadog or New Relic), Foglamp is purpose-built for the AI agent observability stack. It understands the structure of Vercel AI SDK calls, provides native cost calculation for LLM tokens, and integrates evals directly into the monitoring pipeline, offering a cohesive solution instead of requiring custom instrumentation across disparate tools.
2. Key Innovation: Its core innovation is the tight, two-line integration with the Vercel AI SDK that automatically unlocks end-to-end visibility. This "wrap and light on" approach drastically reduces the barrier to implementing comprehensive AI monitoring, moving beyond simple logging to provide correlated traces, costs, and evaluations for every single model interaction.

Frequently Asked Questions (FAQ)

1. How does Foglamp integrate with the Vercel AI SDK? Foglamp integrates by wrapping the AI SDK's model client (e.g., OpenAI, Anthropic) with its own client. You import and initialize foglamp() once, and it automatically intercepts and instruments every subsequent generateText and streamText call, sending performance data to your Foglamp dashboard without requiring changes to your existing application logic.

2. Can Foglamp track costs for multiple LLM providers like OpenAI and Anthropic? Yes. Foglamp's cost intelligence is provider-agnostic. It automatically detects the model identifier from each SDK call and applies the corresponding token-based pricing (for input/output tokens) to calculate precise costs, allowing you to compare expenses across different models and providers in a unified view.

3. What is the main difference between Foglamp and using plain logs or an APM for LLM monitoring? Traditional logs or APMs show that a request happened and maybe its latency. Foglamp provides AI-specific context: it links the exact prompt, the full generated response, the precise token count and cost, the associated eval score, and places it all within the flow of a multi-step agent. This correlation is essential for diagnosing quality issues and managing AI-specific expenses.

4. Is Foglamp only for monitoring errors, or can it help optimize performance? Foglamp is primarily for proactive optimization and management, not just error capture. By visualizing latency across agent steps and comparing cost-per-call for different models, you can identify bottlenecks and make data-driven decisions to switch to faster or cheaper models. The evaluation features also directly help you monitor and improve output quality over time.