Dimensionality Reduction and Feature Selection using a Mixed-norm Penalty Function

Dimensionality reduction, which is the process of mapping high-dimension patterns to lower dimension subspaces, is a key issues in enhancing the processing efficiency of high dimensional data such as hyperspectral images. Dimensionality reduction has been widely discussed in the areas of data mining, image processing, pattern recognition, etc. Because in most situations, many of the dimensions are redundant or unnecessary for the tasks of interest, removing those dimensionality will produce more efficient computation while maintaining the original performance. Dimensionality reduction also reduces the measurement and storage requirements, reduces training and utilization times and it defies the curse of dimensionality to improve classification performance. Feature selection, the process of constructing and selecting the subsets of features that are useful to build a good predictor is of interest for many years. Before Kohavi and John published a special issue on feature selection in 1997, usually no more than 40 features are studied. Ever since then, people started looking at problems with hundreds to tens of thousands of features. Like dimensionality reduction, feature selection reduces the measurement and storage requirements, reduces training and utilization times, and it facilitates data visualization and data understanding. In this work, popular methods for dimensionality reduction and feature selection, such as vector space method, penalty function and support vector machine (SVM) are reviewed and compared. A novel penalty function called the mixed-norm penalty function is proposed. It minimizes the 1-norm of the weight vector while keeping the 2-norm constant. Both dimensionality reduction and feature selection in this work are realized via artificial neural networks (ANNs). Together with Bi-level optimization (BLO) technique, the mixed-norm penalty establishes great performance for both the synthetic data and hyperspectral images.

Author dc:creator

• Zeng, Huiwen

Advisors dc:contributor.advisor

• Carl T. Kelley, Committee Member
• H. Joel Trussell, Committee Chair
• Wesley Snyder, Committee Member
• Arne A. Nilsson, Committee Member

• dimensionality reduction
• feature selection
• neural networks
• machine learning
• penalty function
• mixed-norm penalty function

Statement dc:rights

• I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to NC State University or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report.

Dc Identifier Other

etd-12082005-154733

OAI record oai:identifier

https://repository.lib.ncsu.edu/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Arepository.lib.ncsu.edu%3A1840.16%2F4989