Browsing by Author "Wessels, Zander"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    Interpreting decision boundaries of deep neural networks
    (Stellenbosch : Stellenbosch University, 2019-12) Wessels, Zander; Lamont, M. M. C.; Reid, Stuart; Stellenbosch University. Faculty of Economic and Management Sciences. Dept. of Statistics and Actuarial Science.
    ENGLISH ABSTRACT: As deep learning methods are becoming the front runner among machine learning techniques, the importance of interpreting and understanding these methods grows. Deep neural networks are known for their highly competitive prediction accuracies, but also infamously for their “black box” properties when it comes to their decision making process. Tree-based models on the other end of the spectrum, are highly interpretable models, but lack the predictive power with certain complex datasets. The proposed solution of this thesis is to combine these two methods and obtain the predictive accuracy from the complex learner, but also the explainability from the interpretable learner. The suggested method is a continuation of the work done by the Google Brain Team in their paper Distilling a Neural Network Into a Soft Decision Tree (Frosst and Hinton, 2017). Frosst and Hinton (2017) argue that the reason why it is difficult to understand how a neural network model comes to a particular decision, is due to the learner being reliant on distributed hierarchical representations. If the knowledge gained by the deep learner were to be transferred to a model based on hierarchical decisions instead, interpretability would be much easier. Their proposed solution is to use a “deep neural network to train a soft decision tree that mimics the input-output function discovered by the neural network”. This thesis tries to expand upon this by using generative models (Goodfellow et al., 2016), in particular VAEs (variational autoencoders), to generate additional data from the training data distribution. This synthetic data can then be labelled by the complex learner we wish to approximate. By artificially growing our training set, we can overcome the statistical inefficiencies of decision trees and improve model accuracy.