ISCA Archive IberSPEECH 2022
ISCA Archive IberSPEECH 2022

Bridging the Semantic Gap with Affective Acoustic Scene Analysis: an Information Retrieval-based Approach

Clara Luis-Mingueza, Esther Rituerto-González, Carmen Peláez-Moreno

Human emotions induce physiological and physical changes in the body and can ultimately influence our actions. Their study belongs to the field of Affective Computing, to improve human-computer interaction tasks. Defining an ’affective acoustic scene’ as an acoustic environment that can induce specific emotions, in this work we aim to characterize acoustic scenes that elicit affective states regarding the acoustic events occurring and the available acoustic information. This is achieved by generating emotion embeddings to define the ’affective acoustic fingerprint’ of such affective acoustic scenes. We use YAMNet, an acoustic events’ classifier trained in Audioset to classify acoustic events in the WEMAC Audiovisual stimuli dataset. Each video in this dataset is labelled by crowd-sourcing with the categorical emotion it induces. Thus we determine the relevance of the detected acoustic events that induce each emotion by performing an affective acoustic mapping, creating interpretable acoustic fingerprints of such emotions, by means of the well-known information-retrieval-based TF-IDF algorithm. This paper intends to shed light on the path to the definition of emotional acoustic embeddings.


doi: 10.21437/IberSPEECH.2022-19

Cite as: Luis-Mingueza, C., Rituerto-González, E., Peláez-Moreno, C. (2022) Bridging the Semantic Gap with Affective Acoustic Scene Analysis: an Information Retrieval-based Approach . Proc. IberSPEECH 2022, 91-95, doi: 10.21437/IberSPEECH.2022-19

@inproceedings{luismingueza22_iberspeech,
  author={Clara Luis-Mingueza and Esther Rituerto-González and Carmen Peláez-Moreno},
  title={{Bridging the Semantic Gap with Affective Acoustic Scene Analysis: an Information Retrieval-based Approach }},
  year=2022,
  booktitle={Proc. IberSPEECH 2022},
  pages={91--95},
  doi={10.21437/IberSPEECH.2022-19}
}