Analyse des Représentations Latentes des Modèles de Text-To-Speech Neuronaux pour le Contrôle de la Synthèse Audio-Visuelle Expressive

Analysis of Latent Representations of Neural Text-To-Speech Models for Expressive Audio-Visual Synthesis

• Auteur / Autrice : Martin Lenglet
• Direction : Gérard Bailly, Olivier Perrotin
• Type : Thèse de doctorat
• Discipline(s) : Signal, image, paroles, télécoms
• Date : Soutenance le 12/12/2023
• Etablissement(s) : Université Grenoble Alpes
• Ecole(s) doctorale(s) : École doctorale électronique, électrotechnique, automatique, traitement du signal (Grenoble ; 199.-....)
• Partenaire(s) de recherche : Laboratoire : Grenoble Images parole signal automatique (2007-....)
• Jury : Président / Présidente : Didier Schwab
• Jury : Examinateurs / Examinatrices : Gustav Eje Henter
• Jury : Rapporteurs / Rapporteuses : Marie Tahon, Simon King

Mots clés

Mots clés contrôlés

Synthèse automatique de la parole

Apprentissage profond

Réseaux neuronaux (informatique)

Synthèse de la voix

Mots clés libres

Synthèse vocale

Parole expressive

Agent conversationnel

Expliquabilité IA

Réseau de Neurones Profond

Expressive speech

Conversational agent

Speech synthesis

Explainable AI

Deep Neural Network

Résumé

In recent years, deep neural architectures display groundbreaking performances in various speech processing area, including Text-To-Speech (TTS). Models have grown bigger, including more layers and millions of trainable parameters to achieve almost natural synthesis, at the expense of interpretability of computed intermediate representations, called embeddings. However, statistical learning performed by these neural models constitutes a valuable source of information about language. This presentation aims at openning this "black box" to explore intermediate embeddings computed by state-of-the-art TTS models. By identifying phonetic and acoustic features in model representations, the proposed methods help understanding how neural TTS are able to organize speech information on an unsupervised manner and provide new insights on phonetic regularities captured by statistical learning on massive data that are beyond human expertise. This work open the route toward designing more careful control architectures for neural TTS, without the need for additional data nor training process. These results led us to propose an auxiliary module for expressive synthesis called Local Style Tokens (LST), which models local variations in prosody with respect to the type of embeddings to bias.