The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
Date
2015-03
Authors
Du Toit, Jacqueline
Hattingh, Retha
Pitcher, Richard
Journal Title
Journal ISSN
Volume Title
Publisher
Springer Verlag
Abstract
Background
Speech recognition (SR) technology, the process whereby spoken words are converted to
digital text, has been used in radiology reporting since 1981. It was initially anticipated that
SR would dominate radiology reporting, with claims of up to 99% accuracy, reduced
turnaround times and significant cost savings. However, expectations have not yet been
realised. The limited data available suggest SR reports have significantly higher levels of
inaccuracy than traditional dictation transcription (DT) reports, as well as incurring greater
aggregate costs.
There has been little work on the clinical significance of such errorshowever, and little is
known of the impact of reporter seniority on the generation of errors, or the influence of
system familiarity on reducing error rates.
Furthermore, there have been conflicting findings on the accuracy of SR amongst users with
English as first- and second-language respectively.
Methods
The aim of the study was to compare the accuracy of SR and DT reports in a resource-limited
setting. The first 300 SR and the first 300 DT reports generated during March 2010 were
retrieved from the hospital’s PACS, and reviewed by a single observer. Text errors were
identified, and then classified as either clinically significant or insignificant based on their
potential impact on patient management. In addition, a follow-up analysis was conducted
exactly 4 years later.
Results
Of the original 300 SR reports analysed, 25.6% contained errors, with 9.6% being clinically
significant. Only 9.3% of the DT reports contained errors, 2.3% having potential clinical
impact. Both the overall difference in SR and DT error rates, and the difference in ‘clinically
significant’ error rates (9.6% vs. 2.3%) were statistically significant. In the follow-up study,
the overall SR error rate was strikingly similar at 24.3%, 6% being clinically significant.
Radiologists with second-language English were more likely to generate reports containing
errors, but level of seniority had no bearing.
Conclusion
SR technology consistently increased inaccuracies in Tygerberg Hospital (TBH) radiology
reports, thereby potentially compromising patient care. Awareness of increased error rates in
SR reports, particularly amongst those transcribing in a second-language, is important for
effective implementation of SR in a multilingual healthcare environment.
Description
Publication of this article was funded by the Stellenbosch University Open Access Fund.
Keywords
Speech recognition, Transcriptionist, Error rate, Radiology reporting
Citation
Du Toit, J., Hattingh, R., & Pitcher, R. 2015. The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital. BMC Medical Imaging, 15(8), doi.org/10.1186/s12880-015-0048-1.