Dictation API by AssemblyAI

Product Introduction

1. Definition: The Dictation API by AssemblyAI is a specialized, cloud-based Speech-to-Text (STT) API designed for the specific task of converting short-form spoken audio into polished, ready-to-use text. It is a hybrid AI system that combines a state-of-the-art speech recognition model (Universal-3.5 Pro) with a Large Language Model (LLM) post-processing step within a single, synchronous API call.
2. Core Value Proposition: It exists to eliminate the post-processing gap between raw speech transcription and usable text. Unlike standard transcription APIs that output verbatim transcripts, the Dictation API delivers "finished text in one call," removing filler words, correcting false starts, and applying custom formatting instructions, thereby directly integrating voice-to-text functionality into end-user applications without requiring additional development for text cleanup.

Main Features

1. Single-Call Finished Text: The API's core architecture executes a two-stage process in one request. First, the Universal-3.5 Pro model transcribes the audio with high accuracy. Immediately, an integrated LLM rewrites the raw transcript based on provided instructions (e.g., "strip filler words," "format as a clinical note," "apply formal tone"). This eliminates the need for developers to chain separate transcription and text-processing services.
2. Ultra-Low Latency Performance: Engineered for real-time interaction, the API boasts a p50 latency of 134 milliseconds for typical short clips. This sub-second response time is critical for user experience in applications like hotkey dictation, where text must appear in a field almost instantly after the user stops speaking, preventing the perception of lag.
3. Context-Aware Customization: The API accepts per-request parameters for keyterms (ensuring proper spelling of names, brands, or technical terms) and formatting_instructions (a text prompt guiding the LLM's output style). This allows a single integration to power different text formats across an application, from casual Slack replies to structured CRM entries or HIPAA-compliant clinical notes.
4. Multilingual & Code-Switching Support: It natively supports 19 languages (including English, Spanish, Hindi, Japanese, Arabic) with automatic language detection. A key technical capability is native code-switching, where the model can accurately transcribe and process speech that fluidly mixes multiple languages within a single utterance without any prior configuration.

Problems Solved

1. Pain Point: The inefficiency and poor user experience of using raw, unedited speech transcripts. Verbatim output from standard STT APIs is filled with "ums," "ahs," repetitions, and grammatical errors, requiring manual editing or additional AI processing before the text is usable, which breaks workflow continuity.
2. Target Audience: Product Managers & Developers at SaaS companies building features like in-app voice notes, voice-powered chatbots, or dictation tools. Clinical/Healthcare IT Developers integrating dictation into Electronic Health Record (EHR) systems. Support & CRM Platform Engineers automating note-taking for agents and account executives.
3. Use Cases: Hotkey Dictation: A user holds a key, speaks, and releases to get clean text in any desktop or web application field. AI Agent Voice Prompting: Users speak naturally to an AI assistant, and the API delivers a grammatically perfect prompt. Clinical Scribing: Doctors dictate patient notes, receiving a formatted, spell-checked (via keyterms for drug names) note ready for the chart. Multilingual Customer Support: Support agents dictate replies in their native language, with the API returning professionally formatted responses in the company's required style.

Unique Advantages

1. Differentiation: Versus generic Speech-to-Text APIs (like Google Speech-to-Text or Amazon Transcribe), the Dictation API is not a transcript engine but a "finished text" engine. Versus DIY solutions that combine a transcription API with a separate LLM call (e.g., using OpenAI's Whisper + GPT), it offers a simplified, lower-latency, cost-predictable endpoint ($0.62/hr all-in) without managing multiple services and token costs.
2. Key Innovation: The tight, optimized integration of a best-in-class speech recognition model (Universal-3.5 Pro, trained for short-form audio) with a purpose-tuned LLM rewrite step, delivered as a single, low-latency API call. This "transcribe-and-rewrite" pipeline, configurable per request, is a packaged innovation that specifically addresses the dictation-to-text workflow gap.

Frequently Asked Questions (FAQ)

1. What is the difference between AssemblyAI's Dictation API and its Speech-to-Text API? The standard Speech-to-Text API provides a verbatim, time-stamped transcript of spoken audio, ideal for analytics, captioning, or where every utterance matters. The Dictation API is designed for text creation, outputting cleaned, formatted, and finalized text ready for immediate use, making it superior for note-taking, messaging, and content drafting.
2. How accurate is the Dictation API for medical terminology and names? Built on the high-accuracy Universal-3.5 Pro model and enhanced with the keyterms parameter, it excels at specialized vocabulary. For medical dictation, you can pass drug names, provider names, and ICD codes as keyterms, significantly improving accuracy and ensuring correct spelling in the final output, which is crucial for HIPAA-compliant clinical documentation.
3. Can I use the Dictation API for real-time, streaming dictation? No, the Dictation API is a synchronous API for short, pre-recorded audio clips (like a sentence or paragraph). For real-time, continuous speech-to-text as a user speaks, you would use AssemblyAI's Realtime Speech-to-Text API, which streams a live transcript but does not include the integrated LLM rewrite for finished text.
4. Is the Dictation API cost-effective compared to using separate transcription and LLM services? Yes, at $0.62 per audio hour (billed per second), the price includes both high-accuracy transcription and the LLM rewrite. This simplifies cost forecasting compared to paying for separate transcription service hours and unpredictable LLM token consumption, which can vary greatly based on transcript length and rewrite complexity.
5. How do I ensure the formatted output matches my application's specific style guide? You control the output via the formatting_instructions parameter in each API request. This is a natural language prompt where you can specify rules like "remove all filler words," "format as a bulleted list," "use a formal tone," or "structure as: [Date]: [Summary]." This allows dynamic formatting per use case within your app.