DeepFaceLab Avatar tutorial

Sep 24th, 2019 by DFBlue

Iperov fixed and added back the Avatar model to DeepFaceLab on August 24th 2019.

Avatar or puppet is a way to transform the facial expressions of the source person onto a destination person.

This was great news for the deepfake community since we were missing easy to use tooling for it. The source person is usually a voice actor and the destination person is usually a celebrity. You’ve probably seen this method used in the Jordan Peele / Obama video.

It’s the second category of face deepfaking, the first one being face swap. We have a separate tutorial on the face swap models in DeepFaceLab. Read that first if you don’t have familiarity with DeepFaceLab since it goes over setting up and configuring the software.

Great. But how do we use the Avatar model?

Quick overview

This is the general process for creating an Avatar deepfake:

Once we’ve familiarized ourselves with the process, let’s begin!

Source and destination video requirements

The naming convention for source and destination is a bit confusing when it comes to the Avatar model, but just remember that the SOURCE is the CELEBRITY.

The Avatar model requires input videos in a square aspect ratio. This is different from the other models which have no such requirement. But don’t worry, it’s easy to crop our existing videos to square using ffmpeg or a video editing tool. See ffmpeg.org for more info.

Cropping to square using ffmpeg

 ffmpeg -i input.mp4 -filter:v 'crop=ih/1:ih' -c:v libx264 -c:a copy output.mp4

Extracting frames and faces

Once our source (data_src) and destination (data_dst) videos are in the workspace folder, we can start extraction. The extraction of frames and faces from the videos is also slightly different from the other face swap models.

Here are the steps we need to follow:

Training the Avatar model

After extraction, we will train the Avatar model. Avatar has a 2 stage training process, but both stages are started by running 6) train AVATAR.

Stage 1

Obama avatar training stage 1
Obama avatar training stage 1

Don’t worry that column 3 and 5 in the preview window stay grey, they will be trained during stage 2.

Stage 2

Obama avatar training stage 2
Obama avatar training stage 2

Stage 0

We can run both stage 1 and 2 together by running stage 0. However, the batch size will be limited by the batch size for stage 2, which is significantly lower than what is possible for stage 1.

Converting the final video

Thankfully, converting is the same as the other models in DeepFaceLab.

After running those two scripts we should see a result.mp4 file in our workspace folder.

🎉 Congratulations, you just trained an Avatar/puppet model!

Troubleshooting


Follow us on Twitter or Reddit to keep up with everything that is going on in the world of deepfakes. We aim to provide publications, tools, and services to further the ethical creation, detection, and awareness of deepfakes and digital forgery.

Think Blue.


Follow us on Twitter @dfblue or Reddit u/deepfakeblue to keep up with everything that is going on in the world of deepfakes. Think Blue.