14thAnnual Conference of the International Speech Communication Association

Lyon, France
August 25-29, 2013

Inferring Actor Communities from Videos

Sumit Negi (1), Ramnath Balasubramanyan (2), Santanu Chaudhury (3)

(1) IBM Research India, India
(2) Carnegie Mellon University, USA
(3) IIT Delhi, India

In recent years there has been a growing interest in inferring social relations amongst actors in a video using audiovisual features, co-appearance features or both. The discovered relations between actors have been used for identifying leading roles, detecting rival communities in a movie plot etc. In this paper we propose an unsupervised method which uses the video's transcript and closed caption information for discovering actor communities (group of actors or characters in a film that share a common perspective/viewpoint on an issue) from videos. The method proposed groups together actors using a topic model based approach, which jointly models actoractor interaction (two actors interact when they share the same scene) and the topics associated with their conversations/dialogs. This joint modeling approach shows encouraging results compared to existing methods.

Full Paper

Bibliographic reference.  Negi, Sumit / Balasubramanyan, Ramnath / Chaudhury, Santanu (2013): "Inferring actor communities from videos", In INTERSPEECH-2013, 3751-3755.