Make agents speak with unique avatars using the Voice Live API

Microsoft Foundry's Voice Live API is an API for creating agents that converse using voice, handling speech recognition, response generation, and speech synthesis together. You can also choose the voice used to speak the responses and the avatar video, enabling uses such as the following:

• Use Microsoft's provided standard avatars and voice models as-is
• Create your own avatar from a single photo (photo avatar)
• Create your own voice model from recorded audio (personal voice)
• Create your own avatar from a recorded video (custom video avatar)

In this article, we first organize the avatar patterns, then explain how to prepare and configure standard avatars and custom photo avatars, and finally introduce the voice used for custom video avatars (avatar-sync voice). Voice configuration is independent of the avatar, so Personal Voice is covered in a separate article.

This article is the sixth installment in the Voice Live feature overview series. Information on the prerequisite SDK 1.2.0 and API 2026-07-15 (GA) is consolidated in the introductory article.

List of articles in the series (SDK 1.2.0 / API 2026-07-15)

1. Changes in Voice Live SDK 1.2.0 and API 2026-07-15 GA
   Classification of untyped features in the SDK
2. Handling pauses in conversations with the Voice Live API
   Speaking first, waiting for a hesitation, interrupting, and speaking during wait times
3. Improving the quality of Voice Live audio input
   Choosing noise suppression, echo cancellation, and Live-Reference AEC
4. Using MCP server tools with Voice Live
   Differences from function calling and parallel tool calls
5. Calling Foundry agents as tools with Voice Live
   How to use the chat-supervisor pattern and choose the model in the preceding stage
6. Create a “voice of your own” from 30 seconds of audio and have a Microsoft Foundry agent speak
   From applying for Personal Voice to implementation
7. Choosing between Voice Live API avatars (this article)
   Configuration and implementation for standard, custom photo, and custom video avatars
8. Receiving Voice Live avatar video over WebSocket
   Differences from WebRTC and video formats

Extra edition: Connecting to Voice Live’s WebRTC audio connection with .NET’s SIPSorcery
A record of comparing the SDP line by line

The code used for verification is published here.

Four avatar patterns

The overall picture is as follows.

The choices are as follows.

• When you just want to get it running, use a standard avatar. Since no application is required, you can try it the same day.
• When you want to use your own face, use a custom photo avatar.A single photo is sufficient
• When you want the avatar to perform specified movements with its entire body, use a custom video avatar. However, the filming cost is considerably higher.

Preparation

You can get set up by following the official quick start from creating the resource through getting your first voice conversation running in C#. If you are using an avatar, check the supported regions and access request requirements below before creating the resource.

Supported Regions

The avatar featurecannot be used in every region. The supported regions also vary by use case, so if you do not check before creating the resource, you will have to recreate it later.

There is a note stating that capacity is limited in francecentral.

Furthermore, it does not match the supported regions for Personal Voice either. If you want the voice and avatar to be provided by a single resource, the following six regions support both: eastus / eastus2 / southeastasia / swedencentral / westeurope / westus2.

For the latest list, check the “Voice synthesis avatar” tab in the Speech Service regions table.

Requesting access to custom photo avatars (Limited Access)

Standard avatars are available without an application. Custom photo avatars are provided through the Limited Access feature, where access is restricted based on eligibility and terms of use, so you must apply first. As you can tell when you actually use it, the results are reasonably faithful, so this application process may have been adopted partly in consideration of potential misuse, such as deepfakes.

1. Open the Limited Access registration form
2. Fill out and submit the form using an email address associated with your organization

The official Limited Access page lists who is eligible as follows:

Only customers managed by Microsoft, meaning those who are working directly with Microsoft account teams, are eligible for access.

The official FAQ also provides the review criteria and the email address requirements.

After applying, you will be contacted regarding the review within 5 to 10 business days.
Use an email address associated with your organization in the registration form. Registration forms submitted using a personal email address will be rejected.

As a note, email addresses such as Gmail cannot be used as the contact address. In Microsoft terms, you must use an email address associated with an organizational account. Forms submitted using a personal email address are likely to be rejected.

Using a standard avatar

Standard video avatar (full body)

A full-body character provided by Microsoft.There are Rowan, Celine, Nia, Malik, Harry, Lisa, Roli, Max, Meg, and others, with each character having multiple styles (business, casual, formal, etc.).

Specify using character and style.

The video avatar's full body is positioned within a wide frame, so unless you specify crop to cut it out, the video will consist mostly of the background. In the example above, an area near the center of the 1920x1080 image (560–1360) is cropped out.

There are three constraints to be aware of when using standard video avatars with Voice Live.

When incorporating standard avatars into a product that will be used over the long term, you need to consider the first and third points.

Standard photo avatars

These are head-only avatars. There are 34 available,and each has a designated base model (vasa-1 or vasa-2).

Here, leave customized set to false. No application is required, so if you only want to verify that the avatars work in Voice Live, this is the quickest place to start.

Note that all standard photo avatars have a style of N/A, and you cannot specify a style as you can with video avatars.

Create a custom photo avatar

Create using Speech in Microsoft Foundry

The creation location is Microsoft Foundry, not Speech Studio.

1. Sign in to Microsoft Foundry and confirm that the [New Foundry] toggle is ON
Open the resource project for creating an avatar and select [Build] from the top menu
Select [Services] in the left pane and then select [Speech] from the services displayed
3. 
   
4. Open the [Custom models] tab and press [Create] in the [Customizations] list
   
5. In [Select a model], choose [Azure Speech - Text to Speech Avatar], select [Photo Avatar] under [Type], enter an avatar name, and press [Next]
   
6. Under [Data], add an image (described below)
7. For a real person, register a consent video under [Avatar Talent Registration] (described below)
8. Under [Review], check the content, select the approval checkbox, and press [Submit]. VASA-1 was displayed as the base model
   
9. Once the job status in the [Customizations] list changes to [Succeeded], you can preview the avatar (the list on the screen in Step 4)

How to provide images

In [Select Data Source] in Step 6, you can choose from the following two options:

• [Create with image]: Upload a photo (.jpg / .jpeg / .png) or select a photo already uploaded to Foundry. New photos can be added through [Upload Data]
• [Create with AI] (called "Create with AI" in the English UI): Specify the age, gender, style, and other details, describe the desired character in text, and have the GenAI image model generate it. The consent process is skipped (the generated avatar image can be checked on the details screen after successful creation)

Images must meet the following requirements:

• Because photo avatars only handle the head, images framed from the shoulders up are preferable
• The face should look like a real person or a virtual human. Anime-style features, such as extremely large eyes, are not supported
• Avoid flashy accessories or jewelry
• The entire head must be visible and facing forward
• No shadows or hidden areas

Consent video for live-action cases

If you are creating an avatar from a photo of a real person, that person's consent is required. Specifically, upload

a video in which the consent statement is read aloud. Microsoft will verify whether the read-aloud content matches the prescribed script and whether the face in the video and the person in the photo are the same person. This video is also required if you use your own face.

A sample consent statement is located in the verbal-statement-all-locales.txt file in the Azure-Samples/cognitive-services-speech-sdk repository.

The video must meet the following requirements:

In Step 7, [Register Avatar Talent], select an already registered speaker or register a new one via [Add Avatar Talent]. Once registered, a speaker can be reused when creating the next avatar.

In [Add Avatar Talent], enter the language spoken, avatar talent name, and company name, in the same language as spoken in the video.Once entered, the consent statement to be read aloud is displayed on the screen.

Avatar settings

Differences between the configuration items for photo avatar and video avatar

Even for the same "avatar," the fields to specify are considerably different. A video avatar sample cannot be reused as-is because it will not fit, so here are the differences.

video.resolution is particularly easy to get wrong. Even if you specify 1920x1080, it does not come through as specified and comes back with the aspect ratio of the original image (a vertically oriented headshot).

Specify the avatar with session.update

The Voice Live API exchanges events over WebSocket. After connecting and opening the session, the client sends session.update to pass the session configuration. Specify the avatar in the avatar property of the session object.

For a custom photo avatar, it looks like the following. Set type to photo-avatar and model to the base model, vasa-1.

Compared with the example of a standard video avatar, the difference is that type and model are specified, but style and crop are not.

If you send this session.update and the server returns session.updated, the settings have been applied. The avatar video is delivered as a WebRTC media stream when output_protocol is webrtc, or as a response.video.delta event when it is websocket. The difference in how to receive it will be covered in the next session (Part 7).

Specifying customized

When specifying a custom avatar,specify true for customized.

Standard characters and custom avatars have separate namespaces. If you send a custom avatar name without customized, the service searches among the standard characters and cannot find it. In this case, an empty error with all fields set to null is returned. Because the message is empty, the cause cannot be determined, resulting in a situation where “an error occurs even though the avatar name is correct.” Conversely, if you specify a standard character name while customized: true remains set, the same thing happens because that name does not exist in the custom namespace.

Therefore, on the implementation side, automatically determining customized based on whether the specified name is included in the list of standard characters allows users to avoid having to be aware of the flag.

When using SDK 1.2.0

In SDK 1.2.0, all photo avatar settings can be specified using types.

StandardTalkingHeads is a set of standard character names and directly implements the "determine customized from the name" described above. Since the number of standard characters has increased to 34, this set must also be updated to match.

Adjust the size and position in the scene

As a setting exclusive to the photo avatar, you can adjust how it appears in the scene.

The detailed behavior of zoom will be described later in "Actual-device testing."

Combining with voice

Voice is also a configuration item within the same session. Since it is a key separate from avatar, the two can be configured independently. When combining them, simply write both.

You can also make combinations such as applying your own voice to the standard avatar.

Custom video avatar voice (avatar-sync voice)

Custom video avatars have an avatar-sync voice that uses a voice learned from recordings made when filming the avatar.It is created from a character with the same appearance and voice. It cannot be used with photo avatar.

Since the voice is taken from the avatar side, do not specify a name for voice; specify only the base model. Enter the avatar name in avatar.character.

Real-device testing

Scene zoom

When I tried scene zoom values of 0.5 / 1.0 / 1.2 / 2.0, values below 1 reduced the size as expected, but values above 1 were indistinguishable from 1.0. The Speech real-time avatar documentation also gives the range as “0–1,” which matches its actual appearance. It seems that zoom only works in the direction of reducing the size.
If you want a closer composition, rather than using zoom, it may be better to use the original photo’s composition for a photo avatar, assuming the source image is already a bust-up and without using crop.

The custom video avatar encountered several issues before it could be used on a real device.

First, when I set the voice in SDK 1.2.0, the avatar name was sent as the model name. The AzureAvatarSyncVoice constructor accepts a base model (PersonalVoiceModels), but because PersonalVoiceModels can be implicitly converted from a string, the code still compiles even when an avatar name is passed. The sample initially failed for the same reason, so I changed it to pass the base model.

Next is the resource in which to deploy the avatar. A custom video avatar must be deployed in the same Foundry resource used to call Voice Live, and an avatar in a separate resource cannot be used. If the training and deployment regions differ, copy it using Copy to. If the destination region does not support voice sync, the voice will disappear.

There were also restrictions on the model’s region. If it fails with invalid_model: Model gpt-4o is not supported in this region, the region does not support gpt-4o. In the Voice Live region table, canadacentral / canadaeast / southeastasia / westcentralus do not support gpt-4o, so switch to something such as gpt-4.1.

Finally, there are the costs. Unlike a photo avatar, a custom video avatar continues to incur hosting charges while it is deployed. Deployment and deletion are available only through the portal, so delete it immediately after testing. As a cost guide (as of September 2026), hosting costs ¥95.6/hour, real-time avatar sessions cost a separate ¥95.6/minute, and training costs ¥2,389.8/compute hour.
In my testing, training took 30.3 compute hours (approximately 10 hours in practice). In addition to the filming costs, the training and maintenance costs are also considerably higher than for a photo avatar.

Summary

• Voice Live API avatars have four patterns: standard video, standard photo, custom photo, and custom video. Only the two custom patterns require an application.
• Standard photo avatars can be used without applying, but a base model is specified for each character
• Some styles of Jeff and Lisa will be discontinued in December 2026
• Avatars are available only in certain regions, and the Japan region is excluded
• Custom photo avatars are created through Foundry Speech (Custom models), and no deployment operation is required
• On the Voice Live side, specify the avatar in session.update, which can be combined with voice
• When specifying a self-created avatar, set customized to true
• Custom video avatars must be deployed to the same resource, and charges continue while they are deployed

The voice side is covered in a separate article.

In the next installment (7th), we will discuss whether to receive avatar video over WebSocket or WebRTC.

Reference links

• Supported standard text-to-speech avatars
• Table of Speech Service regions (voice synthesis avatars)
• FAQ for limited-access features
• How to create a custom photo avatar
• Limited Access (Text to speech)
• Limited Access registration form
• Sample using an avatar with the Voice Live API
• TakahiroMiyaura/VoiceLiveAPISamples