Faster R-CNN implementation for hand sign recognition of the Indonesian sign language system (SIBI)

Abstract

The Indonesian sign language system (SIBI) is the authorized sign system in Indonesia that the deaf society uses to convey in Indonesian. However, its use still needs to be expanded and more widespread in the community, causing difficulties in communication for hard-of-hearing people. The product of deep learning technologies such as faster region-based convolutional neural network (Faster R-CNN) in object recognition has the potential to help improve communication between deaf people and the general public. This research will implement the Faster R-CNN algorithm with three different residual network (ResNet) architectures (50, 101, and 152) for SIBI recognition. The comparison of the faster R-CNN algorithm with different architectures is also conducted to identify the best architecture for SIBI recognition, and the results are evaluated using accuracy, precision, recall, and F1-score metrics from confusion matrix calculation and execution time. Faster R-CNN model with ResNet-50 architecture showed the best and most efficient performance with accuracy, recall, precision, and F1-score metrics of 96.15%, 95%, 93%, and 94%, respectively, and an execution time of 36.84 seconds in the testing process compared to models with ResNet-101 and ResNet-152 architectures.