Using transformers to assign ICD codes to medical notes

Simple item page
dc.contributor.advisorVan der Merwe, Brink
dc.contributor.authorDreyer, Andrei Michael
dc.contributor.otherStellenbosch University. Faculty of Science. Dept. of Computer Science.en_ZA
dc.date.accessioned2023-03-04T05:58:51Z
dc.date.accessioned2023-05-18T07:03:37Z
dc.date.available2023-03-04T05:58:51Z
dc.date.available2023-05-18T07:03:37Z
dc.date.issued2023-03en_ZA
dc.descriptionThesis (MSc)--Stellenbosch University, 2023.en_ZA
dc.description.abstractENGLISH ABSTRACT: International Classification of Disease (ICD) coding plays a significant role in classifying morbidity and mortality rates. Currently, ICD codes are assigned to a patient’s medical record by hand by medical practitioners or specialist clinical coders. This practice is prone to errors, and training skilled clinical coders requires time and human resources. Automatic prediction of ICD codes can help alleviate this burden. In this research, we look at transformer-based architectures for predicting ICD codes. Firstly, we expand the size of an XLNet model with label-wise attention to determine whether an increase in model size leads to a better performing model. We also look at using two transformer-based architectures that are specifically designed to handle long input sequences and compare the results from these architectures based on our best-performing XLNet model. Lastly, we look at the use of different attention mechanisms with our XLNet model to determine which attention mechanism works the best. We found the following three things: an increase in model size does lead to better results, XLNet performs better than the architectures designed for longer sequence lengths, and the label-wise attention used by our XLNet model performs better than the other attention mechanisms. en_ZA
dc.description.abstractAFRIKAANS OPSOMMING: Internasionale Klassifikasie van Siektes (ICD)-kodering speel ’n beduidende rol in die klassifikasie van morbiditeit en sterftesyfers. Tans word ICD-kodes tot ’n pasi¨ent se mediese rekord met die hand deur mediese praktisyns of spesialis kliniese kodeerders toegeken. Hierdie praktyk is geneig tot foute, en opleiding van geskoolde kliniese kodeerders verg tyd en menslike hulpbronne. Outomatiese voorspelling van ICD-kodes kan help om hierdie las te verlig.In hierdie navorsing kyk ons na transformator-gebaseerde argitekture vir die voorspelling van ICDkodes. Eerstens brei ons die grootte van ’n XLNet-model uit met etiketwyse aandag om te bepaal of ’n toename in modelgrootte lei tot ’n beter presterende model. Ons kyk ook na die gebruik van twee transformator-gebaseerde argitekture wat spesifiek ontwerp is om lang invoerreekse te hanteer en vergelyk die resultate van hierdie argitekture gebaseer op ons beste presterende XLNet-model. Laastens kyk ons na die gebruik van verskillende aandagmeganismes met ons XLNet-model om te bepaal watter aandagmeganisme die beste werk. Ons het die volgende drie dinge gevind: ’n toename in modelgrootte lei wel tot beter resultate, XLNet presteer beter as die argitekture wat ontwerp is vir langer reekslengtes, en die etiketgewyse aandag wat deur ons XLNet-model gebruik word, presteer beter as die ander aandagmeganismes.af_ZA
dc.description.versionMastersen_ZA
dc.format.extentxi, 84 pages : illustrationsen_ZA
dc.identifier.urihttp://hdl.handle.net/10019.1/127086
dc.language.isoen_ZAen_ZA
dc.language.isoen_ZAen_ZA
dc.publisherStellenbosch : Stellenbosch Universityen_ZA
dc.rights.holderStellenbosch Universityen_ZA
dc.subject.lcshMedical careen_ZA
dc.subject.lcshMedical records -- Data processingen_ZA
dc.subject.lcshDiseases -- Classificationen_ZA
dc.titleUsing transformers to assign ICD codes to medical notesen_ZA
dc.typeThesisen_ZA
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
dreyer_transformers_2023.pdf
Size:
1.54 MB
Format:
Adobe Portable Document Format
Description:
Download
Collections
Masters Degrees (Computer Science)