Nearly all discourse is multi-modal, because those who communicate are never single-channel receptors. Much of linguistic analysis has, however, focused on text out of context. To treat mono-modal transcriptions as if they were the whole of the discourse is to miss the essence and motivation of discourse's structuring from the outset. No. Discourse is cognitive and processual. And dialogue is constituted of responses among cognitive agents. Realistic analysis means asking how agents who are using dialogue do it: What are the facilitation mechanisms, both overt and covert, for multi-modal comprehension and production of discourse? Dialogue takes place within a time frame and involves the exchange not of tokens or texts, but of-meaning? What is meaning, that it can be exchanged among cognitive agents? What is language, that it contributes to the meaning exchanged? Description of dialogue in the language of a formal system suited for multi-modality cognitive processing (Candelaria de Ram 1990B, 199IS) is able to use its capacities to render multi-modality as multi-sort justification. Multi-modal concepts coalesce into cognispaces in which reasoning can be done and through which actions can be enacted. Exchange of meaning among interlocutors requires further setting out properties linking the ongoing cognition of the interlocutors. Besides the properties permitting sensory grounding of linguistic tokens laid out earlier, causal consequent operators are needed that can link across times. They enrich the vocabulary of available grammar tools, so that grounded, process, domain grammars can be composed to describe the structure of multi-modal dialogue realistically.
Cite as: Ram, S.C.d. (1991) Cognitive integration of multi-modal comprehension and response in discourse: dialogue pragmasemantics proposed. Proc. 2nd VENACO Workshop - The Structure of Multimodal Dialogue, 137-142
@inproceedings{ram91b_smmd,
author={Sylvia Candelaria de Ram},
title={{Cognitive integration of multi-modal comprehension and response in discourse: dialogue pragmasemantics proposed}},
year=1991,
booktitle={Proc. 2nd VENACO Workshop - The Structure of Multimodal Dialogue},
pages={137--142}
}