ISCA Archive SLPAT 2016
ISCA Archive SLPAT 2016

The Effect of Semantic Difference on Non-expert Judgments of Simplified Sentences

Sven Anderson, S. Rebecca Thomas, Ki Won Kwon, Wayne Zhang

Advances in text simplification depend on reliable judgments of sentence difficulty. The ability of untrained native English speakers to judge sentence difficulty in the presence of variation in semantic similarity is examined using cloze tests and a forced-choice comparison task. Judgments from participants in web-based experiments demonstrate ability to assess sentence difficulty of professionally leveled sentence pairs with 84% accuracy. The comparison task results suggest that participants’ ability to judge comparative sentence difficulty is inversely related to semantic similarity; that is, contrary to our intuition, speakers appear more accurate at judging sentence difficulty for sentences that are dissimilar than for those that are similar.


doi: 10.21437/SLPAT.2016-10

Cite as: Anderson, S., Thomas, S.R., Kwon, K.W., Zhang, W. (2016) The Effect of Semantic Difference on Non-expert Judgments of Simplified Sentences. Proc. 7th Workshop on Speech and Language Processing for Assistive Technologies (SLPAT 2016), 56-62, doi: 10.21437/SLPAT.2016-10

@inproceedings{anderson16_slpat,
  author={Sven Anderson and S. Rebecca Thomas and Ki Won Kwon and Wayne Zhang},
  title={{The Effect of Semantic Difference on Non-expert Judgments of Simplified Sentences}},
  year=2016,
  booktitle={Proc. 7th Workshop on Speech and Language Processing for Assistive Technologies (SLPAT 2016)},
  pages={56--62},
  doi={10.21437/SLPAT.2016-10}
}