Investigating the evolution of modularity in neural networks
| dc.contributor.advisor | Van den Heever, David Jacobus | en_ZA |
| dc.contributor.advisor | Du Plessis, Stefan | en_ZA |
| dc.contributor.author | Werle van der Merwe, Andreas | en_ZA |
| dc.contributor.other | Stellenbosch University. Faculty of Engineering. Dept. of Mechanical and Mechatronic Engineering. | en_ZA |
| dc.date.accessioned | 2020-02-26T11:01:40Z | |
| dc.date.accessioned | 2020-04-28T12:22:49Z | |
| dc.date.available | 2020-02-26T11:01:40Z | |
| dc.date.available | 2020-04-28T12:22:49Z | |
| dc.date.issued | 2020-03 | |
| dc.description | Thesis (MEng)--Stellenbosch University, 2020. | en_ZA |
| dc.description.abstract | ENGLISH ABSTRACT: Neural networks are not inherently interpretable as a direct consequence of their operating principle and the high dimensional opacity of their internal computations. The neural network interpretability problem is detrimental to reliability, meaningful human-AI interaction and the ethics of deployment. The problem can be approached from the perspective of neural modularity which frames a modular network as one that contains any number of disjoint subnetworks and identifies an interpretable modular network as one that groups its internal representations within such subnetworks in an explanative, task specific way. This study aims to investigate how neural modularity evolves and how it can benefit interpretability. HyperNEAT under connectivity constraints is the chosen neuroevolutionary method, and the following key points of research are investigated with respect to the evolution of neural modularity: general substrates, a variety of connection cost and novel input competition constraints, HyperNEAT modifications based on CPPN disjoints, and the interaction between lifetime and evolutionary learning with neuron nomination given the inclusion of a training phase. The results indicate that the connectivity constraints successfully promote the evolution of neural modularity across a variety of tasks on a general substrate and show that the novel input competition constraints are competitive with the established connection costs as a means of driving the evolution of neural modularity. The HyperNEAT modifications based on CPPN disjoints did not benefit the evolution of neural modularity. Investigating the interaction between lifetime and evolutionary learning with neuron nomination links greater concurrency between the processes that determine a network’s form and function with higher levels of evolved neural modularity for the connection cost constraints. The interpretability assessment shows that while the evolved networks’ interpretable qualities are task dependent, two connectivity constraints deliver statistically different functional module overlap distributions. This study highlights new possibilities for future research and contributes to the knowledge basis on evolving neural modularity by showing that input competition constraints are competitive with connection cost constraints, by examining how the interaction between lifetime and evolutionary learning influences the evolution of neural modularity as well as looking at the interpretable qualities of the evolved networks. | en_ZA |
| dc.description.abstract | AFRIKAANSE OPSOMMING: Neurale netwerke is nie inherent interpreteerbaar nie as 'n gevolg van hulle werkbeginsel en die hoë dimensionele ondeursigtigheid van hulle interne berekeninge. Die neurale netwerk interpreteerbaarheidsprobleem is nadelig tot betroubaarheid, betekenisvolle mens-AI-interaksie en die etiek van ontplooiing. Die probleem kan benader word vanuit die perspektief van neurale modulariteit. 'n Modulêre netwerk bevat 'n aantal afwykende subnetwerke. 'n Interpreteerbare modulêre netwerk is 'n neurale netwerk met interne voorstellings wat binne sulke afwykende subnetwerke op 'n verklarende en taakspesifieke manier groepeer is. Hierdie studie ondersoek hoe neurale modulariteit ontwikkel en hoe dit interpreteerbaarheid kan bevoordeel. HyperNEAT met konnektiwiteitsbeperkings is die gekose neuro-evolusionêre metode en die volgende navorsing sleutelpunte word ondersoek met betrekking tot die evolusie van neurale modulariteit: (1) algemene raamwerke, (2) 'n verskeidenheid van verbindingskoste en nuwe insetkompetisiebeperkings, (3) HyperNEAT-modifikasies gebaseer op CPPNafwykings, en (4) die interaksie tussen lewenslange en evolusionêre leer met neuronenominasie gegewe die insluiting van 'n opleidingsfase. Die resultate dui daarop dat die konnektiwiteitsbeperkings die evolusie van neurale modulariteit oor 'n verskeidenheid take op 'n algemene raamwerk suksesvol bevorder en toon dat die nuwe insetkompetisiebeperkings mededingend is met die vasgestelde verbindingskoste as 'n manier om die evolusie van neurale modulariteit te dryf. Die HyperNEAT-modifikasies gebaseer op CPPN-afwykings het nie die ontwikkeling van neurale modulariteit bevoordeel nie. Die ondersoek van interaksie tussen leeftyd en evolusionêre leer met neuronenominasie dui aan dat hoër neurale modulariteits vlakke met verbindingskoste veroorsaak word 'n groter mate van samewerking tussen die prosesse wat die vorm en funksie van 'n netwerk bepaal. Die interpretasie evaluering toon aan dat alhoewel die ontwikkelde netwerke se interpreteerbare eienskappe taakafhanklik is, die funksie-oorvleuel van twee konnektiwiteitsbeperkings oorvleuel is statisties verskillend. Hierdie studie belig nuwe moontlikhede vir toekomstige navorsing en dra by tot die kennisbasis oor die ontwikkeling van neurale modulariteit deur aan te toon dat insetkompetisiebeperkings mededingend is met verbindingskoste beperkings, deur te ondersoek hoe die interaksie tussen leeftyd en evolusionêre leer die ontwikkeling van neurale modulariteit beïnvloed, sowel as die ondersoek van die interpreteerbare eienskappe van die ontwikkelde netwerke. | af_ZA |
| dc.format.extent | Masters | en_ZA |
| dc.format.extent | xiv, 142 leaves : illustrations (some color) | |
| dc.identifier.uri | http://hdl.handle.net/10019.1/108161 | |
| dc.language.iso | en | en_ZA |
| dc.publisher | Stellenbosch : Stellenbosch University | en_ZA |
| dc.rights.holder | Stellenbosch University | en_ZA |
| dc.subject | Evolutionary computation | en_ZA |
| dc.subject | Artificial neural networks | en_ZA |
| dc.subject | Human-computer interaction | en_ZA |
| dc.subject | Neural modularity | en_ZA |
| dc.subject | HyperNEAT | en_ZA |
| dc.subject | UCTD | en_ZA |
| dc.subject | Modular programming | en_ZA |
| dc.subject | Neural networks (Computer Science) | en_ZA |
| dc.title | Investigating the evolution of modularity in neural networks | en_ZA |
| dc.type | Thesis | en_ZA |