Merlin by Encord

Product Introduction

1. Definition: Merlin is an agentic intelligence layer and conversational AI agent natively embedded into the Encord platform. It represents a new category of AI data infrastructure management tool that operates via natural language prompts and integrates directly into existing development environments.
2. Core Value Proposition: Merlin exists to eliminate the manual overhead and fragmented workflows of AI data preparation, enabling teams to manage their entire data lifecycle—from project setup to model optimization—through a single conversation. Its core value is "getting the data right" faster by embedding agentic intelligence into the tools data scientists and ML engineers already use.

Main Features

1. Build: Merlin transforms a natural language prompt or document into a fully configured Encord project. It uses agentic intelligence to automatically define a label schema, customize the labeling interface for specific tasks (e.g., image segmentation, object detection), and set up multi-stage review workflows. This feature is powered by its integration with the Model Context Protocol (MCP), allowing it to execute complex setup tasks from a simple intent expressed in platforms like Claude or Codex.
2. Observe: This feature provides on-demand, real-time data and project analytics. Instead of running manual queries or analysis scripts, users can ask Merlin in plain language for specific metrics, coverage gaps, or quality reports. Merlin processes the request against the Encord platform's data, delivering instant insights that would traditionally require a dedicated analysis pass, thus enabling continuous monitoring of data health.
3. Optimize: Merlin acts as a diagnostic agent that identifies performance bottlenecks linked to data quality. By analyzing model performance metrics, it pinpoints problematic data points, labeling errors, or workflow inefficiencies within the Encord platform. It then guides the user directly to the issue in the Encord UI, facilitating a closed-loop system for data correction and model improvement.

Problems Solved

1. Pain Point: The traditional AI data lifecycle is manual, slow, and siloed. It involves tedious setup from blank slates, delayed manual analysis for understanding data status, and a disjointed process for diagnosing and fixing data-related model issues. This creates long iteration cycles and inefficiencies in AI development.
2. Target Audience: The product targets Machine Learning Engineers, Data Scientists, AI/ML Team Leads, and Data Operations Managers who are responsible for the data infrastructure underpinning computer vision and multimodal AI models. It is designed for teams already using annotation platforms like Encord and seeking to accelerate their development loops.
3. Use Cases: Essential scenarios include rapidly bootstrapping a new computer vision annotation project for autonomous vehicle data; instantly checking the annotation coverage and quality metrics for an ongoing surgical video analysis dataset; and diagnosing why an image classification model is underperforming by using Merlin to find and fix specific labeling inconsistencies in the training data.

Unique Advantages

1. Differentiation: Unlike traditional data labeling platforms that require extensive manual configuration and separate analytics tools, Merlin is the first to natively embed an agentic, conversation-driven layer across the entire data lifecycle (Build, Observe, Optimize). It differentiates by moving beyond a static interface to become an active, intelligent partner accessible through external, preferred developer tools via MCP, rather than being confined solely to its own web UI.
2. Key Innovation: The key innovation is the integration of agentic intelligence via the Model Context Protocol (MCP) directly into a comprehensive data infrastructure platform. This allows Merlin to perform complex, multi-step operations—like project creation or root-cause analysis for model performance—autonomously from within agentic coding platforms (Claude, Codex) and soon, collaboration hubs like Slack. This represents a paradigm shift from "tool-use" to "agent-collaboration" in AI data management.

Frequently Asked Questions (FAQ)

1. What is an "agentic intelligence layer" in the context of AI data tools? An agentic intelligence layer like Merlin is an AI system that can understand high-level goals, reason about tasks, and take autonomous actions within a software platform to achieve them. For data tools, this means it can manage projects, analyze data, and recommend optimizations through conversation, acting as an intelligent agent on behalf of the user.

2. How does Merlin integrate with existing workflows and what platforms does it support? Merlin integrates via the Model Context Protocol (MCP), allowing it to be accessed and controlled directly from within agentic coding platforms. At launch, it supports Claude, OpenAI Codex, and all other platforms compatible with MCP. Integration with Slack and other communication tools is planned for the near future, enabling team-based management of data infrastructure.

3. What types of AI projects and data modalities can Merlin help manage? Merlin is designed to support projects across multiple data modalities including images, video, 3D, LiDAR, audio, and text, as these are core to the Encord platform. It can assist in setting up and managing projects for computer vision tasks like image segmentation, object detection, and tracking, as well as for medical imaging (DICOM/NIfTI) and other domain-specific annotation workflows.

4. Is Merlin a replacement for human data scientists or annotators? No, Merlin is an augmentation tool, not a replacement. It accelerates and automates administrative, setup, and analytical tasks that are traditionally manual and time-consuming. This frees human experts to focus on higher-value work like complex labeling, model architecture design, and strategic decision-making, while Merlin handles the infrastructure management loop.

5. How does Merlin help improve model performance specifically? Merlin improves model performance by directly linking model metrics to underlying data issues. Through its "Optimize" feature, it can analyze your model's failure points, trace them back to specific data problems (e.g., inconsistent labels, poor coverage of edge cases), and guide you to the exact location in your Encord dataset to make corrections, thereby closing the data-model feedback loop efficiently.