Product Introduction
- Overview: AI Scraper is an open-source Python utility designed for LLM visibility analysis, leveraging Selenium and Google Chrome to automate consumer interfaces for Perplexity and ChatGPT.
- Value: It automates prompt sampling to capture generative AI responses and their exact source citations, enabling researchers and marketers to audit AI search engine optimization (AIO/GEO).
Main Features
- Browser Automation: Utilizes Selenium to drive a real Chrome browser instance, executing CSV-based prompt lists while pacing requests to mimic human behavior.
- Citation Tracking: Captures both the generated text response and the specific URLs cited as sources by the LLM, exporting structured data to CSV or JSONL formats.
- Flexible Authentication: Supports direct user login within the automated browser window for session-based platforms like ChatGPT.
Problems Solved
- Challenge: Manually querying AI answer engines like Perplexity and ChatGPT to check brand visibility or citation sources is tedious and time-consuming.
- Audience: SEO professionals, content marketers, AI researchers, and data analysts tracking Generative Engine Optimization (GEO).
- Scenario: Running a batch of industry-specific prompts overnight to analyze how often a brand is cited by AI answer engines.
Unique Advantages
- Strengths & Limitations: Highly cost-effective and open-source (MIT license), but relies on DOM scraping which means it can break when Perplexity or ChatGPT update their frontend markup.
- Alternatives & Differentiation: Unlike official APIs that provide raw completions without consumer-facing UI context, AI Scraper audits the actual consumer web experience, though official APIs remain recommended for large-scale operations.
Frequently Asked Questions (FAQ)
- Q1? How does AI Scraper collect data from ChatGPT and Perplexity? Answer: It uses a Python script powered by Selenium to control a real Google Chrome browser, inputting prompts from a CSV file and parsing the resulting answers and citations.
- Q2? Is AI Scraper compliant with platform terms of service? Answer: The tool automates consumer web interfaces, and users are strictly responsible for checking and complying with the current Terms of Service of Perplexity and ChatGPT.
- Q3? What output formats are supported by AI Scraper? Answer: It primarily saves collected prompt results, answers, and cited source URLs to a CSV file, with optional JSONL output.