Unified Framework for Matrix-Variate Linear Discriminant Analysis

The linear discriminant analysis (LDA) is a feature extractor used in classification of high-dimensional data in a wide range of applications. In the classification of matrix-variate data, LDA can be used on the vectorized representation of the data in the commonly called one-dimensional LDA (1DLDA) approach. However, 1DLDA leads to overwhelming computational complexity and ignores the inherent structure of the data in many applications. A more recent two-dimensional LDA (2DLDA) approach avoids the initial vectorization step, and leads to a computationally efficient matrix-variate feature extractor formulation. Yet, the theoretical foundations and Bayes optimality of the existing 2DLDA formulations are mostly unclear. This work provides a theoretical framework to develop and analyze the 2DLDA methods, and unify the 2DLDA and 1DLDA formulations. This thesis utilizes the general framework of separable transformations underlying common 2D image compression techniques, and provides a similar framework for separable 2D feature extraction. First, the necessary and sufficient conditions for separability of the 1DLDA operator into the row-wise and column-wise operators of 2DLDA are found. It is shown that an additional condition is required to provide the computational advantages of a completely matrix- variate training procedure similar to the existing 2DLDA methods. Based on the resulting data model, a separable matrix-variate LDA formulation is developed which provides both the Bayes optimality of 1DLDA and the computational and structural advantages of 2DLDA methods. Furthermore, it is shown that the block-wise feature selection approach used by the majority of 2DLDA methods generally results in unnecessary restrictions and loss of discriminatory information. In contrast, the feature selection in the proposed framework is based on sorting of the features based on a discriminance score. Finally, a simple criterion is provided to decide the required number of extracted features in most practical problems. Experimental results using face image databases indicate that assuming basic preprocessing steps, the proposed 2DLDA feature extractor provides the best classification performance if there are only moderate pose and expression variations in the face images. Although this thesis mainly focuses on the design of 2DLDA methods for matrix-variate face images, the derived framework is mainly extensible to the design of multi-linear LDA methods for higher-order tensor-variate data in different applications.

Department dc:contributor.department

Electrical and Computer Engineering

Year dc:date.issued

2015

Author dc:creator

• Mahanta, Mohammad Shahin

Advisor dc:contributor.advisor

• Plataniotis, Konstantinos N

• 2DLDA
• face recognition
• feature extraction
• Kronecker separability
• linear discriminant analysis
• matrix-variate data

Handle dc:identifier.uri

http://hdl.handle.net/1807/71605

OAI identifier oai:identifier

oai:utoronto.scholaris.ca:1807/71605