Efficient Decoding of High-order Hidden Markov Models

Simple item page
dc.contributor.advisorDu Preez, J. A.
dc.contributor.authorEngelbrecht, Herman A.en_ZA
dc.contributor.otherUniversity of Stellenbosch. Faculty of Engineering. Dept. of Electrical and Electronic Engineering.
dc.date.accessioned2008-03-26T11:56:12Zen_ZA
dc.date.accessioned2010-06-01T08:12:20Z
dc.date.available2008-03-26T11:56:12Zen_ZA
dc.date.available2010-06-01T08:12:20Z
dc.date.issued2007-12
dc.descriptionThesis (PhD (Electrical and Electronic Engineering))--University of Stellenbosch, 2007.
dc.description.abstractMost speech recognition and language identification engines are based on hidden Markov models (HMMs). Higher-order HMMs are known to be more powerful than first-order HMMs, but have not been widely used because of their complexity and computational demands. The main objective of this dissertation was to develop a more time-efficient method of decoding high-order HMMs than the standard Viterbi decoding algorithm currently in use. We proposed, implemented and evaluated two decoders based on the Forward-Backward Search (FBS) paradigm, which incorporate information obtained from low-order HMMs. The first decoder is based on time-synchronous Viterbi-beam decoding where we wish to base our state pruning on the complete observation sequence. The second decoder is based on time-asynchronous A* search. The choice of heuristic is critical to the A* search algorithms and a novel, task-independent heuristic function is presented. The experimental results show that both these proposed decoders result in more time-efficient decoding of the fully-connected, high-order HMMs that were investigated. Three significant facts have been uncovered. The first is that conventional forward Viterbi-beam decoding of high-order HMMs is not as computationally expensive as is commonly thought. The second (and somewhat surprising) fact is that backward decoding of conventional, high-order left-context HMMs is significantly more expensive than the conventional forward decoding. By developing the right-context HMM, we showed that the backward decoding of a mathematically equivalent right-context HMM is as expensive as the forward decoding of the left-context HMM. The third fact is that the use of information obtained from low-order HMMs significantly reduces the computational expense of decoding high-order HMMs. The comparison of the two new decoders indicate that the FBS-Viterbi-beam decoder is more time-efficient than the A* decoder. The FBS-Viterbi-beam decoder is not only simpler to implement, it also requires less memory than the A* decoder. We suspect that the broader research community regards the Viterbi-beam algorithm as the most efficient method of decoding HMMs. We hope that the research presented in this dissertation will result in renewed investigation into decoding algorithms that are applicable to high-order HMMs.en_ZA
dc.format.extent984190 bytesen_ZA
dc.format.mimetypeapplication/pdfen_ZA
dc.identifier.urihttp://hdl.handle.net/10019.1/1095
dc.language.isoenen_ZA
dc.publisherStellenbosch : University of Stellenbosch
dc.rights.holderUniversity of Stellenbosch
dc.subjectHidden Markov Modelen_ZA
dc.subjectDecodingen_ZA
dc.subjectHigh-orderen_ZA
dc.subjectTheses -- Electrical and electronic engineeringen_ZA
dc.subjectDissertations -- Electrical and electronic engineeringen-ZA
dc.subject.lcshAutomatic speech recognitionen_ZA
dc.subject.otherElectrical and Electronic Engineeringen_ZA
dc.titleEfficient Decoding of High-order Hidden Markov Modelsen_ZA
dc.typeThesisen_ZA
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
engelbrecht-ha-2007.pdf
Size:
961.12 KB
Format:
Adobe Portable Document Format
Description:
Filename corrected
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
1.72 KB
Format:
Plain Text
Description:
Download
Collections
Doctoral Degrees (Electrical and Electronic Engineering)