SCRS Proceedings of International Conference of Undergraduate Students
Real-time recognition framework for Indian Sign Language using fine-tuned convolutional neural networks
Authors: Rajat Soni, Anshul Vijay, Aakash Khandelwal, Radhika Vijay, Vipin Yadav and Deepak Bhatia
Publishing Date: 09-01-2023
ISBN: 978-81-95502-01-1
Abstract
Sign language is a very crucial aspect in the lives of those who cannot speak or listen to those around them. People with these disabilities have difficulty communicating with the outside world and feel left behind. Much research is ongoing to create a better way of communicating for these people. This work establishes interaction between hearing or speech impaired with the world by recognizing the 33-hand pose and gestures of Indian Sign Language (ISL). This framework can recognize alphabets and numbers in real-time and also generate gestures in real-time for the given alphabets and numbers. The fine-tuned Convolutional Neural Network (CNN) model is explored for the recognition of alphabets and numbers in real-time. A GUI is developed for an easy-to-use interface and immediate visual feedback. Data acquisition software is also developed to create a database. A database of 74,200 images of 33 static signs is captured and used in this work. The results are evaluated on different CNN architectures and learning rates. Accuracy, precision, recall, and F-score are used as performance metrics. The proposed work accomplished the most noteworthy training precision of 99.97% and a validation accuracy of 99.59%.
Keywords
Deep Learning, Sign language, CNN, Data acquisition.
Cite as
Rajat Soni, Anshul Vijay, Aakash Khandelwal, Radhika Vijay, Vipin Yadav and Deepak Bhatia, "Real-time recognition framework for Indian Sign Language using fine-tuned convolutional neural networks", In: Prashant Singh Rana, Deepak Bhatia and Himanshu Arora (eds), SCRS Proceedings of International Conference of Undergraduate Students, SCRS, India, 2023, pp. 95-106. https://doi.org/10.52458/978-81-95502-01-1-10