On-Device Field Extraction by Veryfi

Product Introduction

1. Definition: On-Device Field Detection is a new capability within the Veryfi Lens SDK, a mobile and browser-based camera SDK for document capture. It is a real-time, offline validation engine that performs optical character recognition (OCR) and data extraction directly on a user's smartphone or browser, without requiring an internet connection to a cloud server.
2. Core Value Proposition: It exists to eliminate the latency and user friction associated with post-capture cloud processing. By validating critical receipt fields (like date, total, merchant) instantly on the device at the moment of capture, it ensures data quality upfront, prevents bad submissions, and creates a seamless, instantaneous user experience for mobile receipt scanning and data capture workflows.

Main Features

1. Real-Time, On-Device Field Validation: The SDK processes the camera feed locally using optimized machine learning models. As a user frames a receipt, the technology identifies and extracts key fields in milliseconds. It can be configured to provide immediate visual feedback (e.g., green checkmarks, red alerts) if required fields are missing, illegible, or invalid, prompting a re-scan before upload.
2. Offline-First Operation: All processing for field detection and validation occurs locally on the device. This is crucial for users in low-connectivity environments (e.g., retail stores, construction sites, airplanes) and enhances data privacy, as sensitive document images never leave the device until after validation and user confirmation.
3. Seamless Integration with Cloud APIs: The on-device component acts as a gatekeeper. Once a document passes local validation, the full-resolution image and data can be sent to Veryfi's cloud-based Document Data Extraction APIs for comprehensive, deep parsing (including line-item extraction, tax calculations, and categorization). This hybrid approach combines instant feedback with high-accuracy backend processing.
4. White-Label & Customizable SDK: Veryfi Lens SDK is provided as a white-label solution, allowing developers to fully customize the camera UI, validation rules, and user guidance to match their brand. It supports advanced camera controls like auto-capture, perspective correction, and glare detection.

Problems Solved

1. Pain Point: High Rejection Rates and Poor User Experience in Post-Capture Processing. Traditional mobile OCR requires uploading an image to the cloud, waiting seconds (or minutes) for processing, only to receive an error that the receipt is blurry, missing a total, or is otherwise unprocessable, forcing the user to start over.
2. Target Audience: Mobile App Developers (iOS/Android), Product Managers in FinTech, Expense Management, Accounting Software, and Retail/Cashback platforms. Specific personas include developers integrating receipt scanning into expense apps, product leads for accounts payable automation tools, and engineers building field data capture for insurance or construction.
3. Use Cases:
   • Expense Management Apps: Ensuring employees capture valid receipts in the field before submission, reducing accounting team cleanup.
   • Accounts Payable (AP) Automation: Validating invoice totals and dates at the point of vendor intake via a mobile phone.
   • Loyalty & Cashback Programs: Instantly confirming purchase eligibility from long retail receipts at the point of sale, without network dependency.
   • Insurance Claims Processing: Validating that photos of claim documents (estimates, bills) contain all necessary data on-site.
   • KYC/KYB Onboarding: Checking the readability of ID documents (driver's licenses, passports) during mobile capture.

Unique Advantages

1. Differentiation: Unlike pure cloud-based OCR services (AWS Textract, Google Document AI) or basic mobile scanner SDKs that only handle image capture, Veryfi's solution merges robust on-device AI with its specialized, in-house foundation models for financial documents. Competitors often lack this real-time, offline validation layer, leading to the "upload-and-wait-for-failure" paradigm.
2. Key Innovation: The shift of validation logic from the server-side to the "edge" (the device) is the key innovation. By leveraging on-device ML cores (Apple's Neural Engine, Android's NNAPI), Veryfi executes its proprietary document understanding models locally. This "validate first, upload second" architecture is a fundamental change that prioritizes user experience and data quality at the source.

Frequently Asked Questions (FAQ)

1. How does on-device field extraction work without an internet connection? The Veryfi Lens SDK packages optimized, lightweight versions of its AI models directly into your mobile app or web application. These models run on the device's local processor (CPU/GPU/Neural Engine), analyzing the camera stream in real-time to detect document boundaries and extract text for validation, all without sending data to the cloud.
2. What is the accuracy of on-device field extraction compared to cloud processing? On-device field extraction is designed for speed and core field validation (date, total, merchant). For maximum accuracy on complex documents, full parsing is handled by Veryfi's cloud APIs, which use larger, more powerful models. The on-device component ensures only high-quality, valid images are sent for this deep cloud processing, improving overall system accuracy.
3. Does on-device processing improve data privacy and security? Yes, significantly. Since the initial image processing and validation happen locally, sensitive document images are never transmitted over the network unless they pass validation and the user chooses to submit them. This aligns with data sovereignty regulations and reduces the attack surface for data breaches.
4. Can I customize which fields are validated on-device? Yes. The Veryfi Lens SDK is highly configurable. Developers can define which receipt or invoice fields (e.g., total amount, date, tax, merchant name) are mandatory for validation, set data format rules (e.g., date ranges), and customize the visual feedback presented to the user within the camera view.
5. What types of documents does Veryfi's On-Device Field Detection support? While the announcement highlights receipts, the underlying Veryfi Lens SDK and AI platform supports validation and extraction for a wide range of documents including invoices, purchase orders, bank checks, bank statements, W-2s, and business cards. The on-device capability can be extended to validate key fields across these document types.