Amazon tests an Artificial Intelligence that adapts actors' lips to dubbing: repercussions in the Industry

Amazon is incorporating a new artificial intelligence tool into its audiovisual content with a specific objective: to improve the synchronization between dubbed voices and the lip movements of actors. The technology is not limited to generating or replacing audio, but digitally modifies certain parts of the face so that the apparent pronunciation matches the language of the dubbing.

The company has already begun testing this system in Maxton Hall, one of its best-known German productions. In this first phase, the technology is being used to adapt the series to English. Amazon has presented the project as a way to facilitate access to international productions and achieve better integration into the viewer experience.

The operation starts from the original image of the production. Artificial intelligence analyzes the movements of the actors and modifies the mouth area and the lower part of the face to make it match the words pronounced in the new audio track.

In this way, the viewer not only hears a different voice, but can also perceive lip movements adapted to the language they are listening to. The goal is to reduce one of the most obvious differences in traditional dubbing: that the duration and articulation of words do not always match the original mouth movements.

Amazon has not yet publicly shown a demonstration of this tool, so it is not possible to evaluate the final result in detail. The company has also not yet specified when it might extend its use to other productions or languages.

This initiative joins other artificial intelligence projects that Amazon is developing for its video platform. The company has already experimented with AI-assisted dubbing systems for movies and series and has also used generative tools in certain content.

The strategy is not limited to dubbing. Prime Video has incorporated AI-based functions to facilitate the consumption of its series and movies, although some of these tools have also received criticism for the errors generated. The company has also explored the production of content partially created using generative technology.

In the case of dubbing, one of the main debates revolves around the role of voice professionals. Associations and unions in the sector have expressed their concern about the use of synthetic voices and the conditions related to the use of actors' recordings to develop artificial intelligence models.

Although Amazon is now incorporating this feature into its productions, the modification of facial movements to adapt content to other languages is not a completely new technology. Companies specialized in audiovisual production have been developing tools of this type for years and similar systems have also been used in video games.

The technology also poses a challenge related to the realism of digitally generated or modified faces. Small alterations in facial expression can be perceptible to the viewer, especially when they affect an area as important for communication as the mouth.

For now, Amazon's project is in an initial phase. Its evolution will depend both on the quality of the results and on the acceptance of viewers and the conditions that the company establishes with dubbing professionals.

The arrival of these tools opens a new debate in the audiovisual industry. If artificial intelligence can automatically adapt actors' movements to different languages, the localization process could gain speed and reduce certain production costs. At the same time, it raises questions about the role that dubbing professionals will play in this new scenario.

One of the main points of discussion is related to voice rights. Unions and professional associations have demanded guarantees to prevent actors' recordings from being used to train artificial intelligence systems without specific authorization or without adequate compensation.

The debate is not limited to the replacement of voices. It also affects the possibility that an interpretation can be digitally reproduced or modified after recording. For professionals in the sector, establishing clear limits on the use of their voices, their image, and their interpretations has become one of the central issues in the face of the expansion of these technologies.

Amazon is not the only company exploring these possibilities, so the development of these systems could end up influencing the way films and series are dubbed and distributed internationally. The evolution of technology will predictably be accompanied by a debate on regulation, intellectual property, and labor conditions in the audiovisual sector.

Add ElConstitucional.es as a preferred Google source for free.

Stay informed about all the latest breaking news with the best information. Against disinformation, for democracy and social rights.