Vet

Product Introduction

1. Definition: Vet is an open-source, local CLI (Command Line Interface) code review tool specifically designed for validating the output of AI coding agents. It operates within the technical category of static analysis and behavioral verification tools for AI-generated code.
2. Core Value Proposition: Vet exists to prevent costly errors and misalignments introduced by AI coding assistants. Its primary function is to rigorously verify that an AI agent's code contributions accurately fulfill user requests, adhere to stated goals, and avoid silent failures like unimplemented features or unexecuted tests, ensuring reliable AI-assisted development.

Main Features

1. Conversation-Aware Validation:
   • How it works: Vet uniquely analyzes the entire conversation history between the user and the AI coding agent. It uses natural language processing (NLP) techniques to extract user intent, specific feature requests, and stated goals. Vet then cross-references the generated code (or pull request diff) against this extracted context.
   • Technology: Leverages context parsing from chat logs (e.g., LLM prompts/responses) and semantic diff analysis against code changes.
2. Silent Failure Detection:
   • How it works: Vet proactively identifies instances where the AI agent claimed an action was taken but wasn't fully executed. This includes detecting unimplemented or partially implemented features (e.g., function stubs without logic) and verifying that tests were actually run and passed, not just generated. It checks test runner invocations and results.
   • Technology: Combines static code analysis (identifying incomplete constructs) with dynamic checks (verifying test execution logs or results files exist and contain passes).
3. Comprehensive Pull Request (PR) Review:
   • How it works: Beyond agent-specific checks, Vet performs thorough reviews on full PRs. It scans for logic errors, unhandled edge cases, deviations from project requirements or architecture, potential performance issues, and code smells. It provides concise, actionable feedback.
   • Technology: Utilizes abstract syntax tree (AST) parsing, data flow analysis, control flow analysis, and pattern matching against common error types and best practices.

Problems Solved

1. Pain Point: AI coding agents often produce code that appears correct but contains critical, silent failures – such as skipping test execution, implementing only part of a requested feature, or introducing logic that subtly deviates from user instructions. These issues are hard to catch manually and lead to broken functionality or technical debt.
2. Target Audience: Developers and engineering teams leveraging AI coding assistants (e.g., GitHub Copilot, Claude, GPT-based coders); MLOps engineers managing AI agent deployments; solo developers relying heavily on AI for productivity; teams practicing AI-assisted pair programming.
3. Use Cases: Essential for verifying AI agent output before code commit/merge; automating initial code review for AI-generated PRs; ensuring agent compliance with complex, multi-step user requests; catching regressions or logic flaws introduced during AI-driven refactoring; validating test coverage claims made by coding agents.

Unique Advantages

1. Differentiation: Unlike traditional linters (e.g., ESLint, Pylint) or general static analyzers (e.g., SonarQube) which focus on generic code style and potential bugs, Vet is purpose-built for the unique failure modes of AI agents. It prioritizes conciseness over verbosity, directly targets issues arising from misalignment with user conversation history, and excels at catching "silent" agent deceptions (unrun tests, fake data, partial features). It's faster and more relevant for AI-generated code validation.
2. Key Innovation: Vet's core innovation is its deep integration with the conversational context of AI coding interactions. By analyzing the prompt/response history, it establishes ground truth for what the code should do, enabling verification against specific user intent that generic tools completely miss. Its focus on detecting inaction (unrun tests, unimplemented features) is also novel.

Frequently Asked Questions (FAQ)

1. How does Vet catch unrun tests claimed by AI agents? Vet analyzes the build/test logs or checks for the presence of result files (like JUnit XML) generated after the code was written. If no evidence of test execution matching the generated tests is found, Vet flags it as an unverified claim.
2. Can Vet be used with local LLMs for code generation? Yes, Vet is explicitly designed to work seamlessly with local models. It operates entirely locally, requires no external SaaS API (beyond your existing keys if using cloud models), has zero telemetry, and integrates via CLI, making it ideal for private, local AI coding workflows.
3. What makes Vet different from using unit tests for AI-generated code? While unit tests are crucial, Vet acts before relying on them. It verifies the tests were actually generated and run by the agent itself, checks for feature completeness directly against the user's request in the chat history, and catches deviations that might not be covered by initial unit tests. It's a pre-commit safety net.
4. Is Vet suitable for reviewing non-AI-generated code or legacy code? While its primary strength is AI agent validation, Vet's full PR review capabilities (logic errors, edge cases, goal deviations) can be applied to any code. However, its unique conversation-aware features won't provide benefit for code lacking an associated agent interaction history.
5. How does Vet ensure privacy and security for my code? Vet is open-source, runs entirely locally on your machine or within your CI environment, and transmits no code or conversation data to external servers (zero telemetry). You inspect the source code yourself, guaranteeing no data leakage.