Towards social pattern characterization in egocentric photo-streams
share
Following the increasingly popular trend of social interaction analysis in egocentric vision, this manuscript proposes a new pipeline for automatic social pattern characterization of a wearable photo-camera user, relying on visual analysis of captured egocentric photos. The proposed framework consists of three major steps. The first step is dedicated to social interaction detection where the impact of several social signals is explored. Detected social events are inspected in the second step for categorization into different social meetings. These two steps act at event-level where each potential social event is modeled as a multi-dimensional time-series, whose dimensions correspond to a set of relevant features for each task, and LSTM is employed for time-series classification. The last step of the framework corresponds to the social pattern characterization of the user, where recurrences of the same person across the whole set of social events of the user are clustered to achieve a comprehensive understanding of the diversity and frequency of the social relations of the user. Experimental evaluation over a dataset acquired by a user wearing a photo-camera during a month demonstrates the relevance of the considered features for social interaction analysis, and show promising results on the task of social pattern characterization from egocentric photo-streams.
READ FULL TEXT