Data Science and Intelligent Computing Techniques
Machine Learning Based Framework for Human Action Detection
Authors: Srividya M S and Anala M R
Publishing Date: 20-12-2023
ISBN: 978-81-955020-2-8
Abstract
Understanding human actions has been an important area of computer vision based deep learning domain. Several landmark extraction frameworks like media pipe and open Pose are used to extract the landmark coordinates from the body. The proposed work leverages open-source body landmark extraction and then trains a deep learning model on custom dataset created. The proposed work classifies the human body actions into blank face, yawn, namaste, punch and kick actions. The dataset creation phase involved recording of actions corresponding to every class and flattening them into a data frame. The dataset was later trained on a machine learning pipeline with machine learning algorithms like logistic regression, ridge classifier, random forest, and gradient boosting classifier. The algorithm with best accuracy was taken for real time usage. The landmark extraction model i.e., Mediapipe was used both in creation of dataset and execution of model in real time. The deep learning model was evaluated and validated based on several evaluation metrics like accuracy, confusion matrix, confidence score and recall score. The work proposed computationally efficient way of detecting the actions performed by the subject on camera by leveraging deep learning methods and mediapipe perception model for landmark extraction.
Keywords
Deep learning, Landmark, Machine learning, mediapipe, confusion matrix
Cite as
Srividya M S and Anala M R, "Machine Learning Based Framework for Human Action Detection", In: Satyasai Jagannath Nanda and Rajendra Prasad Yadav (eds), Data Science and Intelligent Computing Techniques, SCRS, India, 2023, pp. 849-857. https://doi.org/10.56155/978-81-955020-2-8-72