Recognizing signals of social attitude in interacting with Ambient Conversational Systems

Ambient Intelligence systems require a natural and personalized experience in interacting with services provided by the environment. In this view, the interaction may happen either in a pervasive way, through a combination of devices embedded in the environment, or using a conversational interface acting as an environment concierge. In the latter case, the interface can be embodied in a conversational agent able to involve users in a human-like conversation and to establish a social relation with them. Developing such an Ambient Conversational System (ACS) requires a model of the user that considers not only the cognitive ingredients of his mental state, but also extra-rational factors such as affect, engagement, attitudes. This paper describes a multimodal framework for recognizing the social attitude of users during the interaction with an embodied agent in a smart environment. In particular, we started from the analysis and annotation of advisory dialogs between humans and then we used the annotated corpus to build a framework for recognizing the social attitude in multimodal dialogs with an ACS. Results of the study show an acceptable performance of the framework in recognizing and monitoring the social attitude during the dialog with an ACS. We also compared results of the analysis of human-human interactions with respect to the human-ACS interaction and, even if the level of initiative of subjects during the dialog was lower in this latter modality, the difference in the average number of social moves was not significant, thus showing that subjects probably were keen to establish a social relation with the conversational agent.