Understanding Speaker Diarization and Speaker Recognition in Azure AI Speech

When several people join a Teams meeting from the same room and speak into one PC’s microphone, recordings and transcripts may attribute all their remarks to the Teams participant associated with that PC. After being asked how to distinguish who spoke in the room, I looked into how far Microsoft services can address the problem.

This article explains the distinction between speaker diarization in Azure AI Speech and speaker recognition that identifies individuals. It also covers architectures for post-processing recordings and for assigning names during a meeting with Teams Rooms.

To compare Azure AI Speech and GPT-4o using an audio file, try the speaker diarization demo.

Contents

• Bottom line
• What Azure AI Speech can do
• Speaker diarization and speaker recognition are different features
• Diarizing recordings with Azure AI Speech
• What happens during overlapping speech
• gpt-4o-transcribe-diarize as an alternative
• Legacy Speaker Recognition has been retired
• Inferring speaker names with an LLM
• For Teams meeting rooms, start with Teams Rooms
• Choosing an approach for this scenario
• Summary

Bottom line

It helps to divide this requirement into two stages:

1. Separate remarks by speaker: Use diarization in Azure AI Speech
2. Assign names to the separated speakers: Use People Recognition in Teams Rooms, or infer likely names from the transcript with Azure OpenAI

Diarization in Azure AI Speech divides audio captured by the same microphone into anonymous speaker IDs such as Guest-1 and Guest-2. It does not identify a person from their voice.

For recorded audio processed with Azure OpenAI, gpt-4o-transcribe-diarize is another option that includes anonymous speaker diarization. As of August 21, 2026, however, it is in Preview and does not provide the same capabilities as Azure AI Speech for high-volume batch processing or Custom Speech.

Azure AI Speaker Recognition previously supported matching people against enrolled voiceprints. According to the official Azure update, the feature was retired on September 30, 2025. As of August 21, 2026, Microsoft has not announced a successor Azure speaker identification API for general application use.

For this meeting-room scenario, Microsoft Teams Rooms provides People Recognition. When users enroll voice profiles in Teams and recognition is enabled for the room, speaker names can appear in live transcripts, post-meeting Recap, and Copilot. If you are setting up a new meeting room, evaluating Teams Rooms before adding a custom post-processing pipeline is the natural starting point.

What Azure AI Speech can do

Azure AI Speech provides capabilities for applications that work with audio. You can access them through the Speech SDK, Speech CLI, and REST APIs. Some capabilities can run in the cloud or in a container.

Speech to text includes real-time transcription for streaming audio, fast transcription for processing recordings quickly, and batch transcription for asynchronously processing large volumes of recordings.

For post-processing meeting-room recordings, fast transcription and batch transcription are the main options. Real-time diarization is also available when a custom application can capture the meeting’s audio stream.

Speaker diarization and speaker recognition are different features

Speaker diarization and speaker recognition may sound similar, but they answer different questions.

The Speech to text Transparency Note describes diarization as distinguishing voices even in a single-channel recording and tagging each transcript segment with labels such as Guest1 and Guest2.

Azure AI Speech alone can therefore produce a result like this:

Guest-1: Let's begin the release go/no-go review.
Guest-2: All tests have been completed.
Guest-1: Tanaka, please update us on monitoring.
Guest-3: The error rate is within the normal range.

At this point, the identity of Guest-1 is still unknown. The service groups remarks by voice characteristics, but it does not compare those voices with organizational users or names.

Also, SpeechRecognizer in the Speech SDK is the class that converts audio to text. The retired feature that identified individuals was called Speaker Recognition. They are distinct despite the similar names.

Diarizing recordings with Azure AI Speech

Post-processing a recorded Teams meeting typically follows this workflow:

1. Prepare the Teams video or audio recording in a format supported by Speech to text
2. Enable diarization in fast transcription or batch transcription
3. Store each segment’s start time, end time, text, and speaker ID
4. If needed, send the transcript and meeting context to Azure OpenAI to infer candidate speaker names
5. Have a person review low-confidence sections before adding them to the meeting minutes

The real-time diarization quickstart uses ConversationTranscriber from the Speech SDK to retrieve a speaker ID from each recognition result. The ID is a generic identifier assigned within that conversation.

Recording quality affects both transcription and speaker diarization. Distance from the microphone, overlapping speech, room echo, HVAC noise, and similar-sounding voices can all cause errors. Enabling diarization does not guarantee that every remark will be separated correctly. Test with the actual meeting room, number of participants, seating positions, and microphone before processing production recordings.

Recordings and transcripts contain not only what people said but also voice data associated with individuals. Define participant notification, consent, retention periods, access permissions, and deletion procedures before collecting this data.

What happens during overlapping speech

Azure AI Speech estimates who spoke and when from a single-channel recording. When two voices reach the same microphone at the same time, however, the source audio has no independent channels from which to reconstruct both remarks perfectly in every case.

The Speech to text Transparency Note identifies overlapping speech as a factor that reduces transcription accuracy. In practice, possible failure modes include:

• Only the louder speaker is transcribed, and words from the other speaker are omitted
• Remarks from two people are merged into one segment and assigned to one speaker ID
• The speaker ID changes mid-utterance, splitting it into short segments
• The text is correct, but a word or segment is assigned to the wrong speaker

These are possible symptoms when accuracy declines during overlapping speech, not fixed behaviors that always occur. Microsoft does not guarantee that overlapping voices will always be separated into distinct transcripts. Evaluate diarization quality with WDER (Word Diarization Error Rate) and transcription quality separately with WER (Word Error Rate).

With a single meeting-room microphone, changing the model alone cannot recover voices already mixed at the input. For meetings where overlapping speech matters, create test audio that reflects actual seating positions and volume differences. Measure WER in overlapping sections, speaker-assignment accuracy, and the number of omitted remarks separately. If the recording equipment can preserve multiple channels, using channel information provides additional separation cues.

gpt-4o-transcribe-diarize as an alternative

Azure OpenAI’s gpt-4o-transcribe-diarize model transcribes audio directly and adds speaker labels. It is an alternative entry point that performs transcription and anonymous speaker diarization in one model, not a pipeline that first runs diarization in Azure AI Speech and then sends the text to a general-purpose LLM.

The guide to choosing an Azure speech recognition technology explains that recorded audio can be processed through /audio/transcriptions and that gpt-4o-transcribe-diarize includes speaker diarization. In the Azure OpenAI model catalog, model version 2025-10-15 is in Preview as of August 21, 2026, and audio files are limited to 25 MB per request. Check available regions and deployment types in advance.

Do not confuse gpt-4o-transcribe-diarize with an LLM inferring speaker names from context. The speaker labels generated by this model are distinct from a later Azure OpenAI step that uses the participant list and conversation content to infer candidate names. Whichever transcription path you choose, assigning names requires downstream inference and human review.

Microsoft also does not guarantee that gpt-4o-transcribe-diarize will always separate overlapping speech perfectly. Process the same meeting recording with both services, split the evaluation into normal and overlapping sections, and compare WER, speaker assignment, latency, and cost. Because the model is in Preview, a production adoption plan must also account for model updates, regional availability, quotas, and API changes.

Legacy Speaker Recognition has been retired

Azure AI Speaker Recognition included a Speaker Identification API for identifying a speaker from a set of enrolled candidates and a Speaker Verification API for checking whether a speaker was the person they claimed to be.

The Azure update announcing the retirement of Azure AI Speaker Recognition lists September 30, 2025, as the retirement date. Earlier Limited Access guidance said that access to the feature required registration, but the service has now passed its retirement date rather than merely closing to new applications.

The official retirement notice does not identify a general-purpose Azure API with equivalent capabilities as a replacement. Diarization can distinguish anonymous speakers, but it cannot replace matching a voice against an enrolled voiceprint to identify a person.

Within the Microsoft product portfolio, there are two practical options:

• For a Teams meeting room, use Speaker Recognition in Teams Rooms or BYOD Rooms
• For a custom application, use Azure OpenAI to infer names from the content after diarization, with final confirmation by a person

The second option is not voiceprint authentication. Do not use it as evidence for identity verification, access control, or fraud detection.

Inferring speaker names with an LLM

Meeting content sometimes includes clues that can identify a likely speaker:

• A self-introduction such as, “I’m Sato from the infrastructure team”
• A response immediately after someone says, “Tanaka, please give us the test results”
• The participant list and assigned roles from the meeting invitation
• The presenter assigned in the agenda
• Known ownership information in a remark such as, “the pull request I created”

Given this context, Azure OpenAI can infer that Guest-2 is likely Tanaka. Instead of replacing speaker IDs with names directly, return structured evidence and confidence with each candidate.

{
  "speakerId": "Guest-2",
  "candidateName": "Tanaka",
  "confidence": "high",
  "evidence": [
    "Guest-2 responded immediately after Guest-1 asked Tanaka for the test results",
    "The participant list assigns Tanaka to testing"
  ],
  "needsReview": false
}

Set clear boundaries for the inference step:

In the prompt, limit candidates to people included in the meeting invitation, require unknown when there is no explicit evidence, and request the remarks that support each inference. Because names are involved, minimize the scope and retention of both inputs and outputs.

This approach works in meetings where speakers introduce themselves or address one another by name. For short remarks such as “Yes” or “Understood,” an LLM has no reliable evidence either. LLM inference is post-processing that can supplement diarization; it is not biometric authentication and does not replace speaker recognition.

For Teams meeting rooms, start with Teams Rooms

For scenarios where several people join a Teams meeting from one room, Microsoft provides People Recognition in Microsoft Teams Rooms.

By default, Teams Rooms can distinguish people in the room as Speaker 1, Speaker 2, and so on. When Voice Recognition is enabled and users enroll voice profiles in the Teams desktop app, enrolled participants can be identified by name. Participants without enrolled profiles remain labeled as Speaker X.

The official Intelligent Speaker requirements specify a Teams Rooms Pro license for Teams Rooms and a Teams Premium or Copilot license for the room host in BYOD Rooms. People who will be identified must enroll voice profiles and be invited to the scheduled meeting. Japanese is among the languages that support Voice Recognition in live meeting transcripts.

Teams Rooms on Windows and Android support Voice Recognition. Face Recognition is limited to Teams Rooms on Windows paired with a supported camera. Because this scenario concerns identifying speakers, evaluate Voice Recognition first.

For rooms where recognition accuracy is especially important, a Teams-certified Intelligent Speaker is another option. Microsoft has expanded Speaker Recognition to all Teams Rooms devices, but notes that certified Intelligent Speakers with multiple microphones may provide better audio quality and recognition accuracy.

Choosing an approach for this scenario

Use the following priorities for this requirement:

For a small trial in an existing environment, run diarization on a real meeting recording collected with participant consent and determine how many speakers it can separate reliably. Then measure the accuracy and unknown rate of LLM-based inference using the participant list and agenda.

For ongoing Teams meeting-room use, validating Voice Recognition with Teams Rooms or BYOD Rooms provides a consistent path through in-meeting display, transcripts, Recap, and Copilot rather than continuing to tune and maintain a custom pipeline.

Summary

Diarization in Azure AI Speech can divide remarks captured by one microphone into anonymous speaker IDs, but it cannot provide names by itself. During overlapping speech, voices mix in the same channel, so evaluations must account for omitted or merged remarks and incorrect speaker assignments, with overlapping sections measured separately.

For recorded audio, Azure OpenAI’s gpt-4o-transcribe-diarize can perform transcription and speaker diarization in one step. As of August 21, 2026, it is in Preview and has a 25 MB per-request limit. It is not a direct upgrade from Azure AI Speech, so compare accuracy, latency, cost, and operational fit on the same real recordings.

The legacy Azure AI Speaker Recognition service, which matched voices against enrolled voice profiles, was retired on September 30, 2025. Microsoft has not announced a successor Azure API for general application use. Azure OpenAI can infer likely names from self-introductions, direct forms of address, participant lists, and agendas, but this is content-based reasoning rather than identity verification. Return evidence and confidence, use unknown when the evidence is insufficient, and include human review.

For this Teams meeting-room scenario, People Recognition in Teams Rooms is Microsoft’s most direct solution. With Teams Rooms Pro, enrolled voice profiles, and the appropriate meeting policies, remarks from the room can flow into transcripts, Recap, and Copilot with speaker names. When reusing an existing PC, BYOD Rooms with a Teams Premium or Copilot license is another option to evaluate.

References

• What is Azure Speech?
• Quickstart: Create real-time diarization
• Use cases for Speech to text
• Choose an Azure speech recognition and generation technology
• Foundry Models sold by Azure: Audio models
• Azure OpenAI image, audio, and video REST API reference
• Azure AI Speaker Recognition retirement
• Speaker Recognition REST API archive
• Manage voice recognition for Intelligent Speaker
• Enable People Recognition on Teams Rooms