Browsing by Author "Magangane, Luyolo"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    Link prediction in knowledge graphs using latent feature modelling and neural tensor factorisation
    (Stellenbosch : Stellenbosch University, 2020-12) Magangane, Luyolo; Brink, Willie; Stellenbosch University. Faculty of Science. Dept. of Mathematical Sciences. Division Applied Mathematics.
    ENGLISH ABSTRACT: Reasoning over knowledge expressed in natural language is a problem at the forefront of artificial intelligence. Question answering is one of the core tasks of this problem, and is concerned with giving machines the capability of generating an answer given a question, by mimicking the reasoning behaviour of humans. Relational learning, in combination with information retrieval, has been explored as a framework for solving this problem. Knowledge graphs (KGs) are used to represent facts about multiple domains as entities (nodes) and relations (edges), and the resource description framework formalism, subject-predicate-object, is used to encode these facts. Link prediction then powers knowledge discovery by scoring possible relationships between entities. This thesis explores latent feature modelling using tensor factorisation as an approach to link prediction. Tensor decompositions are an attractive approach as relational domains are usually high-dimensional and sparse, a setting where factorisation methods have shown very good results. Previous approaches have focused on shallow models that can scale to large datasets, and recently deep models have been applied, specifically neural tensor factorisation models, as these models are more expressive and automatically learn the most useful latent features for entities and relations. In this work we introduce training algorithm optimisations to the neural tensor network (NTN) and HypER neural tensor factorisation models. We make use of the TensorFlow reimplementation of NTNs and apply early stopping, adaptive moment estimation and hyperparameter optimisation using random search. We see improvements in both cost and accuracy over the baseline NTN reimplementation, using standard link prediction benchmark datasets WordNet and Freebase. We then apply optimisations to the HypER model training algorithm. We begin with compensating for covariate shift caused by hypernetworks, using batch normalisation, and propose HypER+. We see similar performance to the HypER baseline on the WN18 dataset, and see significant improvement using the FB15k dataset. We extend our optimisation by initialising entity and relation embeddings using pretrained word vectors from the GloVe language model. We see marginal improvements over the baseline using the WN18RR and FB15k-237 datasets. Our results establish HypER+ as a state-of-the-art model in latent feature modelling based link prediction.