The new Firecrawl /search

Product Introduction

1. Definition: The new Firecrawl /search is an advanced web search API endpoint (/v2/search) that integrates a proprietary AI model to generate "Search Highlights." This technology automatically extracts and returns the most query-relevant passages from web pages, replacing generic meta descriptions in search results.
2. Core Value Proposition: It exists to solve the critical problem of token inefficiency and low relevance in AI agent workflows. By providing highly targeted, context-rich excerpts instead of full-page content or vague descriptions, it enables AI agents and developers to ground their applications in accurate web data while reducing LLM processing costs by up to 10x.

Main Features

1. Search Highlights (AI-Powered Excerpts): This is the core feature. Firecrawl's trained model analyzes the content of each search result page in real-time, identifies passages that directly answer the user's query, and returns them. It works by semantically matching the query intent against page content, prioritizing text with high informational density. The technology preserves structural Markdown elements like headings, lists, and code blocks from the source when relevant.
2. Zero-Configuration Activation: Search Highlights are enabled by default on all /v2/search API calls. No additional parameters, output format changes, or scrapeOptions are required. The system falls back gracefully to a website's plain meta description if a highlight cannot be generated, ensuring result completeness.
3. Unified Response Schema: The feature maintains backward compatibility by injecting the highlighted text into the existing description (for web results) or snippet (for news results) fields. This means developers can integrate the enhanced functionality without refactoring their existing data parsing logic. The response retains all standard metadata: URL, title, position, and ranking score.

Problems Solved

1. Pain Point: Prohibitive LLM Token Costs. Traditional AI agent workflows that scrape and process entire web pages for every query incur massive and unsustainable token consumption, making real-time web-augmented applications expensive to run at scale.
2. Target Audience: AI Agent & RAG Developers, Data Scientists, and SaaS Platforms. Specifically, technical teams building retrieval-augmented generation (RAG) systems, autonomous AI agents, research tools, or any application that requires cost-effective, high-precision grounding from live web data.
3. Use Cases: Building a research assistant that fetches precise technical documentation excerpts; powering a customer support bot that needs accurate, cited answers from help centers; creating a market intelligence tool that extracts specific data points from news articles without noise.

Unique Advantages

1. Differentiation: Unlike standard search APIs (which return generic meta descriptions) or full-page scraping services (which return unstructured, token-heavy content), Firecrawl /search operates at an intermediate, intelligent layer. It provides the specificity of scraping with the efficiency and structure of an API, outperforming providers on benchmarks like SimpleQA with a 94.7% score.
2. Key Innovation: The proprietary AI model trained specifically for query-relevant passage extraction. This is not simple keyword matching or a summarization model; it is optimized to understand query intent and select the most contextually rich, answer-bearing segments from a page's full content, dramatically improving signal-to-noise ratio for downstream AI processing.

Frequently Asked Questions (FAQ)

1. How does Firecrawl Search reduce AI agent costs? By returning only the query-relevant passages (Search Highlights) instead of full page HTML, it reduces the context sent to Large Language Models by up to 90%, directly lowering token consumption and API costs per query.
2. Can I disable Firecrawl Search Highlights? Yes, you can disable highlights by setting the highlights parameter to false in your API request. This will revert to returning the website's standard meta description or snippet for each result.
3. What happens if Firecrawl cannot generate a highlight for a page? The system is designed for graceful degradation. If a highlight cannot be generated for a specific URL, the API will fall back to that page's plain meta description or snippet. This failure does not affect the highlights returned for other results in the same query.
4. Does Firecrawl Search work with JavaScript-rendered pages? Yes, the underlying infrastructure handles JavaScript rendering automatically. The highlight model processes the fully rendered DOM content, ensuring excerpts are pulled from dynamic, client-side generated text as well as static HTML.
5. What is the difference between Search Highlights and using scrapeOptions? Search Highlights modify the description/snippet field within the search results themselves. The scrapeOptions parameter is for separately fetching and returning the full, structured markdown or HTML of a page. They are complementary and can be used together.