Interpreting decision boundaries of deep neural networks
Date
2019-12
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Stellenbosch : Stellenbosch University
Abstract
ENGLISH ABSTRACT: As deep learning methods are becoming the front runner among machine learning
techniques, the importance of interpreting and understanding these methods
grows. Deep neural networks are known for their highly competitive prediction
accuracies, but also infamously for their “black box” properties when
it comes to their decision making process. Tree-based models on the other end
of the spectrum, are highly interpretable models, but lack the predictive power
with certain complex datasets. The proposed solution of this thesis is to combine
these two methods and obtain the predictive accuracy from the complex
learner, but also the explainability from the interpretable learner. The suggested
method is a continuation of the work done by the Google Brain Team in
their paper Distilling a Neural Network Into a Soft Decision Tree (Frosst and
Hinton, 2017). Frosst and Hinton (2017) argue that the reason why it is difficult
to understand how a neural network model comes to a particular decision,
is due to the learner being reliant on distributed hierarchical representations.
If the knowledge gained by the deep learner were to be transferred to a model
based on hierarchical decisions instead, interpretability would be much easier.
Their proposed solution is to use a “deep neural network to train a soft decision
tree that mimics the input-output function discovered by the neural network”.
This thesis tries to expand upon this by using generative models (Goodfellow
et al., 2016), in particular VAEs (variational autoencoders), to generate
additional data from the training data distribution. This synthetic data can
then be labelled by the complex learner we wish to approximate. By artificially
growing our training set, we can overcome the statistical inefficiencies of
decision trees and improve model accuracy.
Description
Thesis (MCom)--Stellenbosch University, 2019.
Keywords
Neural networks (Computer science), Deep learning, Machine learning -- Decision making, Decision trees, Prediction (Logic), UCTD, Generative models, Interpretability