Kernel Machines are Not Black Boxes - On the Interpretability of Kernel-based Nonparametric Models

Kernel-based nonparametric models are often perceived as uninterpretable black boxes. This notion is however questionable; permutation-based variable importance, for example, is one approach for interpreting various nonparametric models. Unfortunately, besides being computationally intensive, permutation-based variable importance is merely a predictor importance measure for supervised regression analysis, and is inapplicable for unsupervised models. This thesis overcomes these issues by using the gradient of kernel-based nonparametric functions to interpret kernel-based models. In particular, the gradient of a kernel regression function can produce variable importance scores with significant computational advantage over permutation-based variable importance. Gradient information can further reveal predictor-response association structure through dimension reduction and graphical analysis; a property not enjoyed by variable importance scores. Most importantly, kernel methods embody a family of supervised and unsupervised learning techniques, and gradient information can be used to interpret not only supervised kernel models, but also unsupervised ones. This fact is demonstrated by using gradient information to interpret the co-dependence structures, as discovered by kernel canonical correlation analysis, between two variable sets.

Department dc:contributor.department

Dalla Lana School of Public Health

Year dc:date.issued

2014

Author dc:creator

• Chang, Billy Heung Wing

Advisor dc:contributor.advisor

• Rafal, Kustra

• Canonical Correlation Analysis
• Dimension Reduction
• Kernel Machines
• Nonparametric Regression
• Variable Importance

Handle dc:identifier.uri

http://hdl.handle.net/1807/68403

OAI identifier oai:identifier

oai:utoronto.scholaris.ca:1807/68403