Skip to main content
Welcome to this lesson on getting started with speech in Azure. This guide covers the fundamentals of Azure AI Speech: recognizing spoken language (speech-to-text), converting text into natural-sounding audio (text-to-speech), and building voice-capable AI agents that communicate with users in real time. We begin with speech recognition. Speech recognition enables an application to listen to spoken words and convert them into text the computer can process. Microsoft Foundry exposes three primary ways to work with Azure AI Speech recognition: All three use the same Azure AI Speech recognition engine—Foundry supplies the tooling and UIs to try, configure, and integrate the service. Transcription configuration options When transcribing audio, configure recognition to match your scenario. The most common options are: The diagram below illustrates a transcription UI with these controls and a sample transcript, showing how audio becomes text.
A presentation slide titled "Implementing Speech Recognition – In Code" showing an Azure AI Speech SDK transcription UI with a sample transcript, playback waveform, and parameter controls. The right side lists transcription options like language selection, speaker diarization, language identification, and output format.
Key UI controls to notice: language selection, speaker diarization, language identification, and output format. Transcribing audio with the Speech SDK (Python) Below is a concise example that uses the Azure Speech SDK for Python to transcribe a single audio file. It demonstrates the minimal steps required to authenticate, point to an audio source, and run a recognition request.
Steps in the example
  1. Import the Azure Speech SDK to access the SDK classes.
  2. Create a SpeechConfig with subscription and endpoint so the SDK can authenticate and target the correct service.
  3. Create an AudioConfig pointing to the audio source (here, record.wav).
  4. Construct a SpeechRecognizer that ties together speech and audio configurations.
  5. Call recognize_once_async() to perform a single recognition and read the result from result.text.
Continuous recognition (microphone) For interactive or streaming scenarios—playgrounds, live transcription, and voice agents—use continuous recognition and event handlers. The sample below shows a typical continuous-recognition setup using the default system microphone.
Store keys and endpoints securely (for example, in environment variables or a secret store) and never check them into source control.
Speech synthesis (text-to-speech) Speech synthesis converts text into spoken audio. You can use the Playground to experiment with voices, speaking rate, pitch, and SSML, or integrate synthesis into an app with the Speech SDK. You can also connect synthesis to an agent so model responses are spoken back to users. The UI below shows a speech synthesis demo: voice selection, tuning sliders (pitch, rate, volume), and a preview panel for the output text “Hello, welcome to Microsoft Foundry!”
A screenshot of a speech synthesis demo titled "Implementing Speech Synthesis – In Code" showing voice selection, tuning sliders (pitch, rate, volume) and a preview panel. The preview text reads "Hello, welcome to Microsoft Foundry!" and the right side shows controls for customizing voice and output format.
Important synthesis controls:
  • Voice selection: choose neural voices with different accents and speaking styles.
  • Speaking rate, pitch, volume: tune delivery without changing the text.
  • Output format and sample rate: pick the audio format appropriate for playback or storage.
  • SSML (Speech Synthesis Markup Language): control pauses, emphasis, pronunciation, and speaking style.
A simple Python example that plays synthesized speech to the system default audio output:
Combining recognition and synthesis with generative AI By combining speech recognition, a generative AI model, and speech synthesis, you can create a continuous voice-enabled agent:
  1. User speaks (live audio delivered to the agent).
  2. Speech-to-text converts audio into text.
  3. A generative AI model reasons and produces a text response (optionally using tools or external knowledge).
  4. The response text is converted to speech and played back.
  5. The loop continues for a natural conversational experience.
This flow supports multi-turn conversations so users and agents can speak back and forth. Creating speech-capable agents in Foundry Microsoft Foundry enables you to create and configure voice-first agents. The “Let’s talk” start session experience shows a simple entry point for a speech session.
A webpage UI titled "Creating a Speech‑Capable Agent" showing sample settings on the left (e.g., Customer service, GPT Realtime) and a large central panel with a purple circle, the text "Let's talk," and a "Start session" button.
Speech services can be used inside Foundry or as standalone Azure resources. From a Foundry resource in the Azure portal you can find service endpoints and keys for speech-to-text, text-to-speech, translation, and other speech APIs.
A screenshot of the Microsoft Azure portal open to a Foundry resource’s "Keys and Endpoint" page, showing masked API keys, service endpoints, and region settings. The left navigation pane lists Azure services and resource management options.
Inside Foundry you can view deployments and services (for example, deployed models and speech services). The Deployments dashboard shows deployed models such as gpt-4.1-mini.
A screenshot of the Microsoft Foundry / Azure AI "Deployments" dashboard listing deployed models (text-embedding-3-small, gpt-4.1-mini, gpt-5.4-mini). The gpt-4.1-mini row is selected, showing its details and action buttons (Open in playground, Edit, Delete) and options to deploy new models.
The Foundry Playgrounds provide an accessible way to experiment with speech-to-text, text-to-speech, translation, and other speech tools—supporting real-time transcription, language detection, speaker diarization, post-refinement, and drag-and-drop audio.
A screenshot of the Microsoft Foundry / Azure Speech-to-Text playground showing transcription settings on the left and a transcript panel on the right that reads, "Hey, hello, Can you hear me? This is a lecture on artificial intelligence." A waveform player and uploaded audio file controls are visible along the bottom.
The playground also displays code snippets (for example, live streaming and continuous recognition) that you can adapt into your application code. Text-to-speech in the playground supports SSML and expressive features—whispering, shouting, sighs, and emotions—so you can craft nuanced spoken responses. The screenshot below reiterates the playground’s combined transcription and audio controls.
A screenshot of the Microsoft Foundry / Azure Speech-to-Text playground showing transcription settings on the left and a transcript panel on the right that reads, "Hey, hello, Can you hear me? This is a lecture on artificial intelligence." A waveform player and uploaded audio file controls are visible along the bottom.
Creating a voice-first agent To build a real-time speech agent in Foundry, go to Agents and create a new agent, selecting a voice-first experience. Agent configuration options include auto language detection, microphone settings, and attaching tools such as a speech MCP server or Blob Storage.
A screenshot of a web UI (Microsoft Foundry / Azure AI) on the Agents page with a modal dialog open to "Create an agent," showing the agent name field partly filled. The background lists an existing agent ("ai901") with status "Running."
You can add tools from the Foundry catalog (for example, Azure Speech MCP Server or Azure Text to Speech) when your agent needs external services. The tools catalog shows available speech tools and other integrations.
A screenshot of the Microsoft/Azure Foundry web UI showing a "Select a tool" catalog modal with various speech tools (Azure Speech MCP Server, Azure Text to Speech, IBM Watson, etc.). The background shows the agent configuration and sidebar navigation for Agents, Deployments, Tools, Knowledge, and other settings.
Once the agent is running, users can have live spoken dialogs: the agent listens, transcribes, reasons with a model, and replies with synthesized speech. The example below shows a travel assistant voice session that responds to spoken requests with travel recommendations.
A screenshot of the Microsoft Foundry/Azure AI Agents playground showing an "sst-tts" chat session with travel recommendations for China. The UI also displays model and voice-mode settings, a speech input/output configuration panel, and left-side navigation for agents and tools.
As sessions run, Foundry captures transcripts and retains conversational context so the agent can continue multi-turn dialogs—delivering a natural, interactive voice experience. Security and privacy considerations
Audio can contain personal or sensitive information. Ensure you follow your organization’s compliance and data retention policies, mask or redact PII where necessary, and configure logging and retention in line with regulations.
Next steps and references
  • If you want to experiment: use the Foundry Playgrounds for quick prototyping and to see sample code.
  • For integration: use the Speech SDK to add recognition and synthesis into applications (desktop, web, mobile).
  • For agents: create voice-first agents in Foundry and attach speech tools from the catalog.
Links and references With these concepts, code samples, and UI examples you can start experimenting with speech in Azure: try the Playground for exploration, use the Speech SDK for application integration, and connect speech to agents for real-time voice experiences.

Watch Video