Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

A Single-Stage Top-Down Probabilistic Approach Towards Understanding Spoken and Handwritten Mathematical Formulas

Jörg Hunsinger, Manfred Lang

Institute for Human-Machine Communication, Technical University of Munich, Germany

We present a novel approach towards a multimodal analysis of natural speech and handwriting input for entering mathematical expressions into a computer. It utilizes an integrated, multilevel probabilistic architecture with a joint semantic and two distinct syntactic models describing speech and script properties, respectively. Compared to classical multistage solutions our single-stage strategy benefits from an implicit transfer of higher level contextual information into the lower level segmentation and pattern recognition processes involved. For visualization and postprocessing purposes, a transformation into Adobe® FrameMaker® documents is performed. Fully spoken or handwritten realistic formulas were examined, yielding a structural recognition accuracy of 61.1 % for speech (speaker independent) and 83.3 % for handwriting (writer dependent).

Full Paper

Bibliographic reference.  Hunsinger, Jörg / Lang, Manfred (2000): "A single-stage top-down probabilistic approach towards understanding spoken and handwritten mathematical formulas", In ICSLP-2000, vol.4, 386-389.