Implementasi Mediapipe Holistic dengan Artificial Neural Network untuk Pengenalan Kata dalam Sistem Isyarat Bahasa Indonesia
Total View This Week0
Total View This Week0
Institusion
Institut Teknologi Telkom Purwokerto
Author
Ahmad, Fayyadh
Subject
T Technology (General)
Datestamp
2024-10-10 03:27:42
Abstract :
Communication is an important part of human social life because it involves the exchange of messages both verbal and non-verbal. For individuals with hearing and speech impairment, the challenges in communicating can be overcome through sign language. This research aims to develop a word recognition model in the Indonesian Sign Language System (SIBI) to help communicate between individuals with hearing impairment and speaking to normal. Although a number of previous studies focused on the recognition of alphabetic letters in the Indonesia Sign Language (BISINDO) and SIBI, there are still shortcomings in the understanding of vocabulary in both sign language systems. The word recognition method used involved video image data of sign language that was converted into image image and then extracted its features through Mediapipe, then converted to tabular data with hundreds of features and 21 labels and then classified with one of the Artificial Neural Network (ANN) architectures named Multi Layer Perceptron. (MLP). The results of this study show that MLP using 8 layer parameters, 1072 neurons (225-200-175-150-125- 100-75-22), ReLU function in hidden layer, learning rate 0.001, 16 batch size, 100 epochs, adam optimizer, and sparse categorical crossentropy function obtained an accuracy score of 97.95% and a loss of 0.0598. However, the model experienced a decrease in performance when predicting new video data indicating overfitting. This is shown from a model that got an average accuracy score of 58%. Keywords: Word Recognition, Indonesian Sign Language System, Mediapipe, Artficial Neural Network
Communication is an important part of human social life because it involves the exchange of messages both verbal and non-verbal. For individuals with hearing and speech impairment, the challenges in communicating can be overcome through sign language. This research aims to develop a word recognition model in the Indonesian Sign Language System (SIBI) to help communicate between individuals with hearing impairment and speaking to normal. Although a number of previous studies focused on the recognition of alphabetic letters in the Indonesia Sign Language (BISINDO) and SIBI, there are still shortcomings in the understanding of vocabulary in both sign language systems. The word recognition method used involved video image data of sign language that was converted into image image and then extracted its features through Mediapipe, then converted to tabular data with hundreds of features and 21 labels and then classified with one of the Artificial Neural Network (ANN) architectures named Multi Layer Perceptron. (MLP). The results of this study show that MLP using 8 layer parameters, 1072 neurons (225-200-175-150-125- 100-75-22), ReLU function in hidden layer, learning rate 0.001, 16 batch size, 100 epochs, adam optimizer, and sparse categorical crossentropy function obtained an accuracy score of 97.95% and a loss of 0.0598. However, the model experienced a decrease in performance when predicting new video data indicating overfitting. This is shown from a model that got an average accuracy score of 58%. Keywords: Word Recognition, Indonesian Sign Language System, Mediapipe, Artficial Neural Network
Download
book
BibTex
Latex, Jabref
cloud_download
Original Resource
url resource Institution
