We have witnessed the gradual exploitation of new techniques to create deepfakes, digitally intervened audiovisual pieces, with results ranging from the rustic and comic, to dangerous reproductions that can hardly be distinguished as fake.
On this occasion, we will review a tool that allows editing clips with this technique, obtaining good results. You can also try it directly from the web, although only with short clips.
This is Wav2Lip, a tool available in GitHub as part of a research paper entitled “A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild”.
With Wav2Lip, video clips can be synchronized with an external voice source with high precision. It can work with any identity, language and voice, even accepting computer-generated animations and synthetic voices.
This utility can be exploited from different levels of use, since complete training codes, inference codes and previously trained models are available. Despite how complex this may seem to those who don’t master those skills, there is the possibility of performing some tests from the web. At the Wav2Lip site you can perform some tests with the clips available there or by uploading your own video and voice clips, limited in this case to fragments of up to 20 seconds.
Using this tool you can get interesting results, such as this montage of scenes from well-known movies with the song All Star by Smash Mouth.
As you can see, this tool has indeed the ability to adapt to different scenes, with really good results.
The use of these tools is subject to the individual ethical responsibility of the person in charge. As long as it is not used for malicious purposes, such as defamation or disinformation, this utility can provide an interesting audiovisual recreation.