[PhD defence] 27/03/2025 - Jarod Ducret: "Translating expressive speech" (UPR LIA)

Mr Jarod DUCRET will publicly defend his thesis entitled "Translating expressive speech" on Thursday 27 March 2025.

Laboratory

UPR 4128 LIA - Avignon Computing Laboratory

Composition of the jury

Mr Yannick ESTEVE	Avignon University	Thesis supervisor
Mr Anthony LARCHER	LIUM	Rapporteur
Mr Damien LOLIVE	IRISA	Rapporteur
Mr Loïc BARRAULT	META-AI	Examiner
Mr Fethi BOUGARES	LIUM	Examiner
Mrs Marie TAHON	LIUM	Examiner
Marcely ZANON-BOITO	NAVER LABS Europe	Examiner
Mr Titouan PARCOLLET	Samsung AI	Thesis co-supervisor
Mr Laurent PILATI		Guest

Summary

This thesis explores the preservation of expressiveness in speech-to-speech translation (S2ST), without recourse to text as an intermediate representation. The aim is to develop a system capable of transferring not only the linguistic content but also the emotional and expressive characteristics of the source utterance to the target language. The approach developed has two components. Firstly, the use of discrete speech units, extracted from self-supervised models, to efficiently capture phonetic content. Secondly, a multilingual emotion encoder, with the aim of extracting language-independent expressive features. These representations are then integrated into the speech synthesis process in order to condition its generation.

Keywords translation, machine learning, speech synthesis