Design a smart platform translating Arabic sign language to English language

Abstract

Sign language is the only means of communication for deaf and hearing-disabled people in their communities. It uses body language and gestures, such as hand shapes and facial expressions, to convey a message. It is important to note that sign language is specific to the region; that is, Arabic sign language (ArSL) is different from English sign language. Therefore, this research proposes a way to improve the translation of ArSL using a new artificial intelligence (AI) architecture. Specifically, a convolutional neural network (CNN) based on fine-tuning of the SSD-ResNet50 V1 FPN is applied to build a real-time ArSL recognition and translation system with fast and accurate results. The proposed AI architecture can provide translation of sign language in real-time to enhance communication in the deaf community. We achieved an average F-score of 86% and an average accuracy of 94%.