Sixth International Conference on Spoken Language Processing
We present a novel approach towards a multimodal analysis of natural speech and handwriting input for entering mathematical expressions into a computer. It utilizes an integrated, multilevel probabilistic architecture with a joint semantic and two distinct syntactic models describing speech and script properties, respectively. Compared to classical multistage solutions our single-stage strategy benefits from an implicit transfer of higher level contextual information into the lower level segmentation and pattern recognition processes involved. For visualization and postprocessing purposes, a transformation into Adobe® FrameMaker® documents is performed. Fully spoken or handwritten realistic formulas were examined, yielding a structural recognition accuracy of 61.1 % for speech (speaker independent) and 83.3 % for handwriting (writer dependent).
Bibliographic reference. Hunsinger, Jörg / Lang, Manfred (2000): "A single-stage top-down probabilistic approach towards understanding spoken and handwritten mathematical formulas", In ICSLP-2000, vol.4, 386-389.