free-stockdb

Product Introduction

1. Definition: Free-stockdb is a local-first, C++-based quantitative data engine and time-series database specifically designed for the Chinese A-share market. It functions as an offline data warehouse, integrating data synchronization, cleaning, adjustment for splits/dividends, storage, and high-performance batch querying.
2. Core Value Proposition: It solves the critical bottleneck of data engineering in quantitative trading and research by providing a fast, reliable, and private local data layer. It eliminates dependency on slow, expensive, and rate-limited remote APIs for historical stock data (daily K, minute K, ETF), enabling efficient full-market backtesting and technical analysis.

Main Features

1. Incremental Synchronization & Local Caching: The engine pulls data from a configurable source (via sync_url.txt) using Zstd compression for efficient transfer. It performs incremental updates, downloading only changed data, and supports resumable transfers. All data is cached locally in ./data, allowing research and backtesting to continue offline.
2. On-the-fly Adjustment for Splits/Dividends: Instead of storing pre-adjusted data, the system stores raw market data alongside complete adjustment factors. It supports query-time calculation for forward adjustment, backward adjustment, or raw (unadjusted) prices, providing flexibility and data integrity for different analysis needs.
3. High-Performance Batch Query & Calculation Engine: Built with a custom C++ time-series engine, it supports bulk queries across thousands of securities over arbitrary date ranges. It includes a Rust-based computational core (zb.get()) that calculates 39 built-in technical indicators (like MA, MACD, KDJ, BOLL) and 5 types of indices (equal-weighted, float market cap-weighted, etc.) 3x faster than typical pandas operations.
4. Multi-Interface Access Layer: It exposes the local data through five unified protocols: a Python SDK (sync/async), an HTTP API (port 7899), Excel/WPS macros, an HTML web interface for no-code browsing, and a custom MCP (Model Context Protocol) server for direct integration with AI tools like Claude and Cursor.
5. Integrated Sector & Concept Mapping: The engine includes built-in mappings for Shenwan primary/secondary/tertiary industry classifications and over 1200 concept boards. The bk.get() function allows for millisecond-speed bidirectional queries between stock codes and their associated sectors.

Problems Solved

1. Pain Point: The impracticality of full-market, minute-level backtesting using remote APIs due to rate limits, token costs, IP blocking, and multi-day data fetching times.
2. Target Audience: Quantitative traders, algorithmic trading researchers, financial data scientists, and developers in the A-share market who require fast, reliable, and private batch data access for strategy development, backtesting, and technical indicator computation.
3. Use Cases: 1) Running a full-universe (7000+ securities) backtest on 5-minute K-line data. 2) Rapidly screening stocks based on complex multi-indicator conditions across historical data. 3) Building a private research or trading system that cannot depend on the uptime or latency of third-party data vendors. 4) Using AI coding assistants to directly query and analyze local market data via MCP.

Unique Advantages

1. Differentiation: Unlike cloud-based quant platforms or direct API wrappers, free-stockdb decouples data engineering from strategy research. It is not a data vendor but a local data management and computation engine. Users control the data source and retain a permanent, queryable local snapshot.
2. Key Innovation: The integration of a complete local data pipeline—from incremental sync and adjustment factor management to a high-performance C++ query engine and multi-protocol interface—into a single, lightweight (~2.2MB server) desktop application. The custom MCP server implementation for AI tools is a particularly forward-looking feature for modern development workflows.

Frequently Asked Questions (FAQ)

1. How does free-stockdb handle stock splits and dividends? It stores raw transaction data and separate corporate action adjustment factors. When you query data using parameters like fq='qfq' (forward adjustment), the engine calculates the adjusted prices in real-time using these factors, ensuring accuracy and flexibility.
2. Can I use my own data source with free-stockdb? Yes. The data source is configurable via the sync_url.txt file or the --source command-line argument. You can point it to your own synchronization server, a local network share, or a file directory (file:// protocol), making the engine independent of any single data provider.
3. Is free-stockdb suitable for real-time trading? No, it is primarily designed for historical data analysis, research, and backtesting. It focuses on synchronized, cleaned historical data (daily, minute, tick) rather than live market feeds.
4. What are the system requirements to run free-stockdb? It requires Windows 7+ (with macOS/Linux versions planned), approximately 20GB of disk space for full minute-level data (compressed with Zstd), and a minimum of 2GB RAM. The local server is very lightweight.
5. How do I calculate technical indicators on bulk data with free-stockdb? You use the zb.get() function from the Python SDK or HTTP API. Specify the indicator name (e.g., "macd"), security codes, date range, and parameters. The calculation is executed server-side in Rust, returning results for the entire batch in seconds, avoiding slow DataFrame iterations.