Khoder Ahmad: Contribution to supervised representation learning :algorithms and applications.

Zuzendariak_Directores: Abdelmalik Moujahid Moujahid- Fadi Dornaika.

2021_05_31, 10:30   Sala Ada Lovelace aretoa.

Abstract

In this thesis, we focus on supervised learning methods for pattern categorization. In this context, it remains a major challenge to establish efficient relationships between the discriminant properties of the extracted features and the inter-class sparsity structure.

Our first attempt to address this problem was to develop a method called ”Robust Discriminant Analysis with Feature Selection and Inter-class Sparsity” (RDA FSIS). This method performs feature selection and extraction simultaneously. The targeted projection transformation fo-cuses on the most discriminative original features while guaranteeing that the extracted (or transformed) features belonging to the same class share a common sparse structure, which contributes to small intra-class distances.

In a further study on this approach, some improvements have been introduced in terms of the optimization criterion and the applied opti-mization process. In fact, we proposed an improved version of the original RDA FSIS called ”Enhanced Discriminant Analysis with Class Sparsity using Gradient Method” (EDA CS). The basic improvement is twofold: on the first hand, in the alternating optimization, we update the linear transformation and tune it with the gradient descent method, resulting in a more efficient and less complex solution than the closed form adopted in RDA FSIS. On the other hand, the method could be used as a fine-tuning technique for many feature extraction methods. The main feature of this approach lies in the fact that it is a gradient descent based refinement applied to a closed form solution. This makes it suitable for combining several extraction methods and can thus improve the performance of the classification process.

In accordance with the above methods, we proposed a hybrid linear feature extraction scheme called ”feature extraction using gradient de-scent with hybrid initialization” (FE GD HI). This method, based on a unified criterion, was able to take advantage of several powerful linear discriminant methods. The linear transformation is computed using a de-scent gradient method. The strength of this approach is that it is generic in the sense that it allows fine tuning of the hybrid solution provided by different methods.

Finally, we proposed a new efficient ensemble learning approach that aims to estimate an improved data representation. The proposed method is called ”ICS Based Ensemble Learning for Image Classification” (EM ICS). Instead of using multiple classifiers on the transformed features, we aim to estimate multiple extracted feature subsets. These were obtained by multiple learned linear embeddings. Multiple feature subsets were used to estimate the transformations, which were ranked using multiple feature selection techniques. The derived extracted feature subsets were concate-nated into a single data representation vector with strong discriminative properties.

Experiments conducted on various benchmark datasets ranging from face images, handwritten digit images, object images to text datasets showed promising results that outperformed the existing state-of-the-art and competing methods.

Keywords: Machine Learning, Pattern Classification, Discriminant Embedding, Manifold learning, Linear Embedding, Image Categorization, Supervised Learning, Hybrid Embedding, Hybrid Initialization, Computer Vision, Ensemble Learning, Fine Tuning, Gradient Descent Optimization.