ChaLearn LAP Large Scale Signer Independent Isolated Sign Language Recognition Challenge: Design, Results and Future Research

by   Ozge Mercanoglu Sincan, et al.

The performances of Sign Language Recognition (SLR) systems have improved considerably in recent years. However, several open challenges still need to be solved to allow SLR to be useful in practice. The research in the field is in its infancy in regards to the robustness of the models to a large diversity of signs and signers, and to fairness of the models to performers from different demographics. This work summarises the ChaLearn LAP Large Scale Signer Independent Isolated SLR Challenge, organised at CVPR 2021 with the goal of overcoming some of the aforementioned challenges. We analyse and discuss the challenge design, top winning solutions and suggestions for future research. The challenge attracted 132 participants in the RGB track and 59 in the RGB+Depth track, receiving more than 1.5K submissions in total. Participants were evaluated using a new large-scale multi-modal Turkish Sign Language (AUTSL) dataset, consisting of 226 sign labels and 36,302 isolated sign video samples performed by 43 different signers. Winning teams achieved more than 96 recognition rate, and their approaches benefited from pose/hand/face estimation, transfer learning, external data, fusion/ensemble of modalities and different strategies to model spatio-temporal information. However, methods still fail to distinguish among very similar signs, in particular those sharing similar hand trajectories.


page 4

page 7

page 8


Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition

Sign language recognition using computational models is a challenging pr...

AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methods

Sign language recognition is a challenging problem where signs are ident...

Visual Recognition of Isolated Swedish Sign Language Signs

We present a method for recognition of isolated Swedish Sign Language si...

Visual Methods for Sign Language Recognition: A Modality-Based Review

Sign language visual recognition from continuous multi-modal streams is ...

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Vision-based sign language recognition aims at helping the hearing-impai...

Evaluation Of Hidden Markov Models Using Deep CNN Features In Isolated Sign Recognition

Isolated sign recognition from video streams is a challenging problem du...

Transferring Cross-domain Knowledge for Video Sign Language Recognition

Word-level sign language recognition (WSLR) is a fundamental task in sig...

Please sign up or login with your details

Forgot password? Click here to reset