pdf-inspector

Product Introduction

1. Definition: pdf-inspector is a high-performance, open-source PDF parsing and conversion library. Its technical category is a native-text PDF processing engine, built with a Rust core for speed and reliability.
2. Core Value Proposition: It exists to solve the problem of extracting clean, structured, and layout-aware Markdown from PDFs at high speed. Its primary value is intelligent PDF classification (text-based vs. scanned) and high-fidelity text extraction, enabling developers to embed fast, local PDF analysis into applications without relying on cloud APIs or slow, inaccurate parsers.

Main Features

1. Intelligent PDF Classification: The library automatically detects and classifies PDFs into categories like TextBased, Scanned, ImageBased, or Mixed. It provides confidence scores and per-page signals, allowing for smart routing in document processing pipelines—for example, sending only scanned pages to an OCR service.
2. Layout-Aware Text Extraction & Reading Order Reconstruction: It doesn't just extract raw text; it analyzes the positional coordinates of text blocks to reconstruct logical reading order. This is critical for accurately parsing multi-column layouts, academic papers, and newspaper-style formats, ensuring the output follows the document's intended flow.
3. Advanced Table Structure Detection: pdf-inspector uses a combination of rectangle detection, line-grid analysis, and cell alignment heuristics to identify and reconstruct table structures. It handles complex financial tables and can even track table continuations across multiple pages, outputting them as clean Markdown tables.
4. Semantic Markdown Conversion: The engine converts the extracted and structured content into clean, token-efficient Markdown. It preserves semantic elements like headings (H1, H2), lists, links, code blocks, emphasis (bold/italic), captions, and inserts page markers, making the output ideal for LLM ingestion or human-readable documentation.
5. Multi-Language Package Support: The Rust core is packaged for easy integration into popular tech stacks. It is available as @firecrawl/pdf-inspector for Node.js (includes a CLI), pdf-inspector on PyPI for Python, and pdf-inspector on crates.io for Rust, all exposing the same unified API.

Problems Solved

1. Pain Point: Inconsistent and low-quality text extraction from PDFs, where layout is destroyed, tables are rendered as garbled text, and reading order is incorrect. This leads to poor results in downstream tasks like search indexing, data analysis, or LLM processing.
2. Target Audience: Software engineers and developers building document processing pipelines, data scientists and researchers analyzing PDF corpora, and product teams needing to embed reliable PDF-to-text functionality into SaaS applications, knowledge bases, or RAG (Retrieval-Augmented Generation) systems.
3. Use Cases: Converting financial reports, research papers, and legal documents into structured data; preprocessing PDFs for LLM context windows; building internal search over document repositories; automating data entry from invoices and forms; and creating accessible text versions of PDF content.

Unique Advantages

1. Differentiation: Unlike generic PDF libraries or model-based parsers, pdf-inspector is purpose-built for speed and accuracy on native-text PDFs. Benchmarks show it outperforms alternatives like LiteParse, OpenDataLoader, and PyMuPDF4LLM in overall score, table structure (TEDS), and reading order, while being significantly faster (2.8s median vs. 13.9s for LiteParse on a 200-document corpus).
2. Key Innovation: Its modular, single-parse architecture. The PDF is parsed only once, with the raw data (text, fonts, rectangles) shared across discrete, focused stages (detection, layout, tables, Markdown). This separation of concerns makes the pipeline efficient, maintainable, and embeddable, avoiding the bloat of monolithic parsers.

Frequently Asked Questions (FAQ)

1. How does pdf-inspector handle scanned PDFs or PDFs with images? pdf-inspector specializes in native-text PDFs. It will classify pages as Scanned or ImageBased and return a list of pages_needing_ocr. It does not perform OCR itself but is designed to seamlessly integrate with OCR engines by routing only the necessary pages, optimizing processing time and cost.
2. What is the performance and memory footprint of the pdf-inspector library? Built in Rust, it is designed for high performance and a low memory footprint. The benchmark on an Apple M4 Pro shows a median processing time of 2.8 seconds for 200 diverse PDFs, making it one of the fastest local parsing engines available.
3. Can I use pdf-inspector in a web browser or serverless environment? Yes, the core Rust library can be compiled to WebAssembly (WASM), as demonstrated in the browser demo where processing happens entirely client-side. This enables deployment in edge functions, serverless platforms, and static web applications.
4. How accurate is the table extraction compared to other tools? According to the OpenDataLoader benchmark, pdf-inspector achieved a table structure (TEDS) score of 0.814, significantly higher than LiteParse (0.693) and OpenDataLoader (0.489). It uses advanced heuristics to detect cell boundaries and spans, making it highly effective for financial and data-rich tables.
5. Is pdf-inspector suitable for production use in commercial applications? Yes. It is open-source under the permissive MIT license, allowing unrestricted commercial use. It is actively maintained by Firecrawl, packaged for major platforms (npm, PyPI, crates.io), and includes TypeScript definitions and a CLI, indicating a focus on production-ready developer experience.