Buildermark

Product Introduction

1. Definition: Buildermark is an open-source AI code attribution and benchmarking tool designed to quantify the volume of AI-generated code within a software project. It functions as a local app container managing a Go-based server that analyzes developer workflows by cross-referencing coding agent conversation history with version control data.

2. Core Value Proposition: Buildermark provides transparency and data-driven insights into the modern development lifecycle. By accurately calculating the percentage of commits authored by AI agents versus human developers, it enables organizations to benchmark developer productivity, evaluate the efficacy of different Large Language Model (LLM) agents, and maintain an audit trail of AI contributions without compromising data privacy.

Main Features

1. Formatting-Agnostic Diff Matching: The core engine of Buildermark utilizes a sophisticated matching system that compares diffs generated during AI coding agent conversations with actual git commit diffs. This system is specifically engineered to be formatting-agnostic, ensuring that the tool remains robust even if the code undergoes automatic linting, reformatting, or minor reorganization after being generated by the agent but before being committed.

2. Automated Conversation and Commit Import: Buildermark streamlines data ingestion by automatically importing chat histories from a wide array of coding agents. It supports local imports from Virtual Machines (VMs) and containers via shared folder paths and integrates with browser extensions (Chrome, Firefox, Safari) to pull data from cloud-based agent interfaces. Simultaneously, it syncs with local git repositories to fetch relevant commit history for comparative analysis.

3. Multi-Agent Performance Benchmarking: The platform provides a centralized dashboard to compare the performance of various AI coding agents. Users can track and rate conversations manually or utilize agent-led self-rating systems. This feature allows technical leads to determine which agents (such as Claude Code, Codex, Gemini, or Cursor) deliver the highest quality code or the most significant contributions to specific modules within their codebase.

4. Local-First Privacy Architecture: Unlike SaaS-based analytics tools, Buildermark operates entirely on the user's local machine. A local Go server serves a web UI at http://localhost:55022. There are no external cloud dependencies, no mandatory user accounts, and no telemetry or analytics sent to third-party servers. The only outbound network request made is an optional check for software updates.

Problems Solved

1. Lack of AI Attribution Transparency: Engineering managers often struggle to quantify the actual impact of AI coding assistants on their codebase. Buildermark solves this by providing a concrete "percentage of code written by agents" metric, allowing for objective measurement of AI adoption and ROI.

2. Security and Intellectual Property Auditing: For industries with strict compliance requirements, knowing exactly which lines of code were generated by external LLMs is critical for IP management. Buildermark provides a verifiable trail linking agent conversations to specific lines in the production repository.

3. Target Audience:

• Engineering Managers and Tech Leads: Seeking to measure team velocity and the impact of AI tools on the development process.
• Open Source Maintainers: Who want to provide transparency regarding the use of AI in their public repositories.
• Security and Compliance Officers: Needing to audit code origins for legal or safety reasons.
• Individual Developers: Looking to optimize their own AI workflows and compare the efficiency of different LLM-powered IDEs and CLIs.

4. Use Cases:

• Agent Benchmarking: Testing whether Claude Code or Cursor produces more "commit-ready" code for a specific tech stack.
• Codebase Auditing: Generating a report for stakeholders showing that 94% of a project’s code was generated by AI agents to demonstrate the power of agentic workflows.
• Development Process Optimization: Identifying bottlenecks where AI-generated code requires excessive human refactoring.

Unique Advantages

1. Local and Open Source: Buildermark distinguishes itself from proprietary telemetry tools by being fully open source and local-first. This eliminates the risk of sensitive source code or proprietary prompt data leaking to the cloud, making it suitable for high-security environments.

2. Native OS Integration: The tool runs natively on macOS (15+), Windows (10+), and Linux. It leverages native Notification Centers to provide immediate attribution feedback as soon as a commit is made, integrating seamlessly into the developer's existing desktop environment.

3. Interoperability: Buildermark supports a diverse ecosystem of coding agents, including Claude Code CLI/Cloud, Codex CLI/Cloud, Gemini CLI, and Cursor. Its architecture is designed to be extensible, allowing for feature requests and new agent integrations through its open-source community.

4. Hybrid Workflow Support: By supporting both CLI tools and browser-based agent interfaces through extensions, Buildermark captures the full spectrum of AI interaction, whether the developer is using a terminal-based agent or a web-based chat interface.

Frequently Asked Questions (FAQ)

1. How does Buildermark calculate the percentage of AI-generated code? Buildermark matches the specific code changes (diffs) found in your coding agent's conversation history with the diffs recorded in your git commits. By identifying overlapping code blocks through a formatting-agnostic algorithm, it attributes specific lines to the agent and calculates the ratio against the total code changed in the repository.

2. Is my source code sent to Buildermark’s servers? No. Buildermark is a local-first application. All processing, matching, and data storage occur on your local machine. It does not require a cloud account, and the data never leaves your infrastructure, ensuring maximum privacy for your intellectual property.

3. Which AI coding agents are currently supported by Buildermark? Buildermark currently supports a wide range of popular agents including Claude Code (both CLI and Cloud), Codex (CLI and Cloud), Gemini CLI, and Cursor. Support for additional agents can be requested via the project's GitHub repository.

4. Can Buildermark be used for team-wide AI metrics? While the standard version is a local app for individual developers, a "Team Server" is currently in development. This will be a paid, self-hosted solution that allows organizations to aggregate Buildermark data from multiple developers to view AI code metrics across the entire enterprise.