[Thesis defence] 11/12/2025 – Nicolas André: «Real and hyper-complex representations and activation functions in neural networks for signal processing» (UPR LIA)

Mr Nicolas ANDRÉ will publicly defend his thesis entitled: «Real and hyper-complex representations and activation functions in neural networks for signal processing», supervised by Mr Mohamed MORCHID, on Thursday 11 December 2025.

Laboratory

UPR 4128 LIA - Avignon Computing Laboratory

Composition of the jury

Mr Mohamed MORCHID	Avignon University	Thesis supervisor
Mr Javier RAMIREZ RODRIGUEZ	Autonomous Metropolitan University	Rapporteur
Ms Irina ILLINA	University of Lorraine	Rapporteur
Mr Patrice BELLOT	Aix-Marseille University	Examiner
Mr Majed HADDAD	Avignon University	Guest

Summary

The research presented in this thesis focuses on the design and theoretical analysis of neural architectures in machine learning, using a resolutely mathematical approach. It aims to improve the efficiency and approximation capacity of neural networks while reducing their parametric complexity. A significant part of this research concerns activation functions, which are central to the learning process. In particular, this thesis explores the creation of new functions with original structural properties capable of reducing the number of parameters without loss of performance, while maintaining formal rigour in their definition and analysis. The first part of the study focuses on real neural networks. It proposes new architectures based on existing Rational Activation Functions (RAF), parametric activation functions capable of reproducing the behaviour of a linear layer with ReLU while offering greater flexibility. This framework has led to the introduction of Exponential-Based RAFs, which combine exponential and rational components to enhance approximation capabilities and significantly reduce the number of parameters. Experiments conducted on classic classification and regression tasks have confirmed the superiority of these models, which achieve better performance while using much more compact architectures. A second focus of this work concerns quaternion neural networks, a family of hyper-complex models offering representations particularly suited to multidimensional data, such as in signal processing, computer vision or language. Activation functions in this context pose unique challenges due to the non-commutative structure of quaternions. This thesis proposes RAFs and Component-Specific RAFs (CSRAFs) adapted to this framework, accompanied by mathematical justifications and experiments demonstrating their ability to outperform conventional split approaches. More recently, an efficient method for calculating the exponential of quaternion matrices has also been developed, paving the way for new activation functions and further reduction in computational resources. In parallel with this theoretical work, this thesis also focuses on automatic language processing. A first article looked at the classification capabilities of Transformer models applied to highly noisy telephone conversations. It studied the detection of themes and sub-themes as well as the influence of their interconnections on learning, demonstrating the robustness of these architectures in degraded acoustic conditions. All of these contributions (whether new activation functions for real or quaternionic networks, hyper-complex matrix calculation methods, or applications to language processing) have a common goal: to design more compact, mathematically robust and efficient models capable of meeting the growing needs of modern machine learning, while opening up application prospects in various fields such as vision, audio and signal processing.

Keywords : Neuron, Quaternion, Language, Activation function, Machine learning