EMO Expressive Motion for Vocal Avatars

How to Create Vocal Avatars from Any Portrait and Audio with EMO

2 min

Have you ever wondered what it would be like to see your favorite portrait come to life and sing or speak any language or style? Well, now you can, thanks to a new AI framework called EMO, which stands for Expressive Motion for Vocal Avatars.

What is EMO?

EMO is a new method that can generate realistic and expressive portrait videos from a single reference image and a vocal audio input, such as singing or talking. EMO can handle various head poses, facial expressions, and lip movements, as well as different languages, genres, and durations of audio.

EMO Vocal Avatars

How does EMO work?

EMO works by first extracting the audio features and the facial landmarks from the input audio and image, respectively. Then, it uses a neural network to learn the mapping between the audio features and the facial landmarks and generates a sequence of facial landmarks that are synchronized with the audio. Finally, it uses another neural network to synthesize the portrait video from the facial landmarks and the reference image.

EMO Vocal Avatars

EMO can produce stunning results for a wide range of portraits, including paintings, historical figures, celebrities, cartoons, and even AI-generated characters. EMO can also animate portraits from different cultures and languages, such as Mandarin, Japanese, Korean, Cantonese, and English. EMO can also handle fast-paced and complex audio inputs, such as rap songs and Shakespearean monologues.

remember the video of the woman walking in Tokyo streets from Sora? In this EMO-generated video, she is singing a song by Dua Lipa that looks so realistic.

EMO is not only a fun and creative way to bring portraits to life, but also a potential tool for various applications, such as education, entertainment, art, and communication. Imagine being able to learn from the portraits of famous people, watch them perform in different languages and styles, or even interact with them in real-time.

How to access EMO?

If you want to learn more about EMO, you can check out the original paper and the demo videos on the project website. You can also try it out yourself when the code becomes available to the public on the official Github project. EMO is a breakthrough in vocal avatars generation and a glimpse into the future of AI and human expression.

Join Guidady AI Mail List

Subscribe to our mailing list and get interesting stuff and updates to your email inbox.

Thank you for subscribing.

Something went wrong.

Like it? Share with your friends!



Your email address will not be published. Required fields are marked *


I am an IT engineer, content creator, and proud father with a passion for innovation and excellence. In both my personal and professional life, I strive for excellence and am committed to finding innovative solutions to complex problems.
Choose A Format
Personality quiz
Series of questions that intends to reveal something about the personality
Trivia quiz
Series of questions with right and wrong answers that intends to check knowledge
Voting to make decisions or determine opinions
Formatted Text with Embeds and Visuals
The Classic Internet Listicles
The Classic Internet Countdowns
Open List
Submit your own item and vote up for the best submission
Ranked List
Upvote or downvote to decide the best list item
Upload your own images to make custom memes
Youtube and Vimeo Embeds
Soundcloud or Mixcloud Embeds
Photo or GIF
GIF format