ai-goofish-monitor

Product Introduction

1. Definition: The ai-goofish-monitor is an open-source, AI-powered multi-task monitoring and analysis system specifically designed for the Xianyu (闲鱼) second-hand marketplace. Technically, it is a web application built on a Python FastAPI backend with a modern frontend, utilizing Playwright for browser automation and integrating with multimodal AI models for intelligent content analysis.
2. Core Value Proposition: It exists to solve the inefficiency of manually and continuously sifting through Xianyu's massive inventory. The system automates the discovery of desirable or high-value-for-money products through real-time or scheduled monitoring, intelligent AI filtering, and comprehensive management tools, making it essential for Xianyu power users and small-scale merchants.

Main Features

1. Web-Based Visual Management: The system provides a complete administrative UI accessible via a browser. This includes modules for task management (creation, editing, scheduling), account management (importing and rotating Xianyu login states), AI prompt editing, real-time log viewing, and browsing/searching filtered results directly from the SQLite database.
2. AI-Driven Task Creation & Analysis: Users can create monitoring tasks using natural language descriptions of their needs. The system uses configured multimodal AI models (like GPT-4V) to analyze product titles, descriptions, and downloaded images to generate filtering criteria or make direct purchase recommendations. This enables complex, nuanced filtering beyond simple keywords.
3. Concurrent Multi-Task Engine with Advanced Filtering: The core spider engine (spider_v2.py) can run multiple independent monitoring tasks concurrently. Each task supports granular filters including keywords, price ranges, "free shipping" toggle, item recency (newly published within X hours), and three-level regional selection (Province/City/District) based on Xianyu's data.
4. Notification & Alert System: Upon finding a matching item, the system can send instant alerts through multiple channels including ntfy.sh, Gotify, Bark, Telegram, WeCom (Enterprise WeChat) bots, and generic webhooks, ensuring users don't miss opportunities.
5. Account & Proxy Rotation Management: To mitigate anti-bot detection and rate limiting, the system supports managing multiple Xianyu accounts and proxy configurations. Tasks can be bound to specific accounts or use automatic rotation. It includes failure retry logic, blacklisting of failed proxies, and task-level failure thresholds with automatic pausing.

Problems Solved

1. Pain Point: The overwhelming manual effort and time required to find specific, high-quality, or underpriced items among Xianyu's constantly updating sea of listings.
2. Target Audience: Second-hand trading enthusiasts, individual sellers (geek sellers), and small-scale merchants (small business owners) on Xianyu who need to batch monitor specific product categories, brands, or models for sourcing or sniping deals.
3. Use Cases: A user looking for a specific camera model under a target price; a seller sourcing inventory for a niche product category; a collector hunting for rare items who needs 24/7 monitoring with intelligent filtering to avoid irrelevant listings.

Unique Advantages

1. Strengths & Limitations (Pros & Cons):

   • Pros:
     • Open-Source & Self-Hosted: Provides full control over data, no reliance on external SaaS services, and no usage fees beyond infrastructure and AI API costs.
     • Powerful AI Integration: Moves beyond keyword matching to understand context and visual details in listings, significantly improving filtering accuracy for complex needs.
     • Comprehensive Management UI: Lowers the technical barrier for setup and daily operation compared to pure script-based solutions.
     • Docker-First Deployment: Simplifies deployment and dependency management across different operating systems.
   • Cons:
     • Technical Overhead: Requires self-deployment, environment configuration (Python, Docker), and ongoing maintenance, which has a steeper initial learning curve than a commercial app.
     • Dependency on AI API: Operational cost and performance are tied to the chosen AI provider's pricing, latency, and reliability (e.g., OpenAI, Gemini).
     • Anti-Bot Arms Race: As a browser automation tool, its long-term effectiveness is subject to Xianyu's anti-bot measures, requiring users to responsibly manage request frequency and maintain valid login states.

2. Key Alternatives & Differentiation:

   • Manual Browsing / Xianyu App Alerts: The native Xianyu app offers basic keyword alerts. ai-goofish-monitor differentiates by offering concurrent multi-keyword monitoring, AI-powered analysis, cross-account management, and far more granular filtering (region, recency, image analysis).
   • Generic Web Scraping Frameworks (e.g., Scrapy + Selenium): These are general-purpose tools requiring significant custom development to achieve similar functionality. This project is a turnkey solution specifically tailored for Xianyu, with built-in account handling, AI integration, and a management UI.
   • Proprietary Monitoring SaaS: Commercial services might offer similar monitoring. ai-goofish-monitor differentiates through its open-source nature (no vendor lock-in, customizable), one-time setup cost vs. subscription fees, and complete data privacy as all data resides on the user's own server.

Frequently Asked Questions (FAQ)

1. Is ai-goofish-monitor free to use? Yes, the software itself is open-source and free under the MIT license. However, you are responsible for hosting costs (e.g., a VPS) and any fees associated with the AI API services (like OpenAI) that you choose to integrate.
2. Do I need programming skills to set up ai-goofish-monitor? Basic command-line and Docker knowledge is highly recommended for the smoothest setup via the provided docker-compose method. The Web UI manages most daily operations, but initial deployment requires following technical instructions.
3. How do I log into Xianyu within the monitor? The system uses a "login state" (cookies) approach, not direct password entry. You must manually log into Xianyu in your Chrome/Edge browser, then use the provided browser extension to export the login state JSON file, which you then import into the system's account management page.
4. Can using this tool get my Xianyu account banned? Any automated tool carries risk. The project includes features like proxy rotation, request pacing, and account rotation to mimic human behavior and reduce risk. However, users must configure these responsibly and avoid excessively aggressive monitoring intervals to comply with Xianyu's terms of service.
5. What AI models are supported? The system is designed to work with any OpenAI-compatible API endpoint. This includes OpenAI's own models (GPT-4, GPT-4V), as well as locally-run or third-party hosted models that provide a compatible interface, allowing for flexibility in cost and capability.