Electronic Glove: A Teaching AID for the Hearing Impaired

Received Oct 28, 2017 Revised Feb 2, 2018 Accepted Feb 9, 2018 Learning how to speak in order to communicate with others is part of growing up. Like a normal person, deaf and mutes also need to learn how to connect to the world they live in. For this purpose, an Electronic Glove or E-Glovewas developed as a teaching aid for the hearing impaired particularly children. E-Glove makes use ofthe American Sign Language (ASL) asthe basis for recognizing hand gestures. It was designed using flex sensors and an accelerometer to detect the degree of bend made by the fingers as well asa movement of the hand. E-Glove transmits the data received from the sensors wirelessly to a computer and then displays the letter or basic word that correspondsto a gesture made by the individual wearing it. E-Glove provides a simple, accurate, reliable, cheap, speedy gesture recognition and userfriendlyteaching aid for the instructors that are teaching sign language to the deaf and mute community. Keyword:


INTRODUCTION
Communication is very important in all areas of life. Its significance is considered by many people to be as important as breathing. It aids in the process of sharing information and also improves the relationship with other people. Unfortunately, not all are blessed with the ability to talk and hear. There are some who cannot talk nor hear what others are saying, in fact,the World Federation of the Deaf published a report that 72 million deaf-mute people [1] make up the world's population.
In order to communicate with the society, researchers studied how to recognize hand gestures and there are two well-known approaches; image processing [2] and the data glove. The data glove approach has less computational time and faster response in real time [3] applications which makes it more advantageous than image processing technique. As a result of these advantages, there has been quite a reasonable amount of research work in building the data glove using different input components like flex sensor [1]- [9], accelerometer [4], surface electromyography [4] andMicro-Electro-Mechanical Systems (MEMS) sensor [10]. While all of these data gloves can recognize the American Sign Language (ASL) alphabet, few have designed it to recognize ASL sign words. No one has also looked into using the data glove in Deaf Education.
Hearing-impaired children who didn't learnto speakalso go to schools that teach them howto communicate with other people. In educating deaf and mute children, there are three great approaches that can be successfully used: the bilingual-bicultural (BiBi) approach which makes use of ASL; the auditoryverbal approach that teaches the English Language through residual hearing and speech instead of sign language; and total communication that combines auditory and visual communication for instruction. These techniques has under gone waves of popularity, but this doesn't mean though that one is better than the other. Hearing-impaired children are different and each needs a different type of technique to help them with their development of language and communication. However, ASL, which wasthe first language ever taught in the BiBi approach, is said to be the native language for them and is historically used in Deaf Education. Keeping in mind the use of BiBi approach in teaching hearing-impaired children, technology can be integrated across the curriculum to assist professionals and teachers in Deaf Educationand make learning more interesting. Technology has been digitizing classrooms through digital learning tools that have increased student's engagement and motivation towards learning.
This paper explored on the development of a data glove as a digital learning tool for teaching hearing-impaired children. The data glove was named Electronic Glove or E-Glove. It was developed using a general-purpose microcontroller board for processing input coming from the combination of flex sensors and accelerometer. The use of the accelerometer only for recognizing ASL sign words is what separates the E-Glove from other existing data gloves.

RESEARCH METHOD
The primary goal in the development of E-Glove is showing the recognized ASL letters and words through a software program installed in a computer. It shall facilitate in teaching hearing-impaired children through teachers showing the proper hand gestures and then letting the students perform it using the glove. Figure 1 represents the block diagram of E-Glove. Five flex sensor and one accelerometer were connected to the microcontroller that contained the functionality of the device. UHF modules were used to wirelessly connect the E-Glove to the computer. Any letter or word recognized using the E-Glovewill be shown and read aloud on the computer.  Figure 2 shows the connection of the different components used in the development of E-Glove.Once the microcontroller detects the sign language, it will send the letter or word that corresponds to the hand gesture wirelessly into the computer using UHF (transmitter). The UHF (receiver) connected to the UART acts as a bridge between the microcontroller and the computer. The computer will then display the letters or words on the screen and also convert it into speech. a. Arduino Mega b. Flex sensor c. Accelerometer d. UHF Module e. Software Program

Arduino mega microcontroller
Arduino microcontroller is a long-established general purpose microcontroller [11] used for developing do-it-yourself electronic projects and numerous embedded systems. Arduino microcontrollers are physically programmable circuit boards that can be programmed using an Arduino IDE (Integrated Development Environment) [12] which is available in all portals [13] that deal with a product. The IDE is also used tocompile and upload the written source code to the Arduino board. Arduino is both an open source [14] hardware and software.
There are many types of Arduino microcontroller board. We used the Arduino Mega for E-Glove because it has lots of digital input/output pins and analog input pins. This board is very convenient for projects that require a lot of digital input and output. The Arduino Mega has the following specifications: a. contained the whole functionality of the device. Its absence won't make the device work. It holds the code for the sensors as well as the database of the device. It also converts the analog signals coming from the sensor into digital in order for the computer to read the data.
The flex sensors were connected to the analog pins A0-A4 while the accelerometer used A5 of the Arduino microcontroller. The first transceiver which served as a transmitter was connected to the Rx and Tx communication port of the Arduino microcontroller.

Flex sensors
Flex sensors act as variable resistors mainly used to detect bending or flexing. The resistance of this kind of sensors changes when bent. They are made up of carbon on a strip of plastic wherein resistance gets higher and higher while bending in one direction. Flex sensors differ in length but most havea resistance ranging from about 10 kilo-ohms to 35 kilo-ohms.
In the E-Glove, the degree of bend of the flex sensor was compared to the mapped valuesin the code to recognize aparticular sign language. Five pieces of flex sensors were placed on each finger. Their values are being recorded and compared with the values that represent a particular sign language. If the value matches, it will be shown in the software program.

Accelerometer
The accelerometer is an inertial sensorthatis dynamically capable of sensing in a vast range. Accelerometers are used to measure acceleration forces in one, two, or three orthogonal axes depending on its application to a certain device. For the E-Glove to detect hand movements when doing a certain ASL sign word, the accelerometer was used. The value read from the accelerometer was compared to the value saved in the software program to determine if a certain word corresponds to the hand movement.

UHF module
UHF module was used in the project as a bridge of the E-Glove and the computer for them to communicate wirelessly. The UHF module connected to the E-Glovesends the data to the computer. The purpose of making the device wireless is to make the user comfortable and not to stay on just one area of the room.
Specifically, UHF-EX was used in the E-Glove. It is a wireless UART transceiver with 100 milliwatts (mW) radio frequency output. Two UHF-EXwere used in E-Glovewhich served different purposes. The first one was used as a transmitter connected to the Arduino microcontroller while the second one was used as a receiver connected to the computer. The radio frequency power of UHF-EX allowed a useful control range of more than 500 meters while unobstructed line of sight range can reach 700 meters.

Software program
The data coming from the two sensors attached to the E-Glove were wirelessly sent to the computer and a software program was developed to receive this data and show the corresponding sign language equivalent. Moreover, the equivalent letter or word was also read aloud using a text-to-speech function.
The software program was developed using Microsoft Visual C# Express. Microsoft Visual C# Express is a high-level programming language that is intended for building a variety of application using the .NET Framework. This programming language issimple, powerful, type-safe and objected-oriented. The continuous development of this language enables rapid application development while retaining the simplicity and elegance of a C based programming language.

American sign language (ASL) alphabet and sign words
ASL is a natural language that serves as the first language taught in the BiBi approach of deaf education before the English language. One of the first things that a normal person should learn is spelling. In deaf education, the ASL alphabet is being taught to hearing-impaired children for them to spell words and most especially names.  Figure 3 represents the ASL alphabet from North Star Teacher Resources. ASL sign words were also taught to hearing-impaired children especially those important words that they need in everyday conversation like Know, You, Good, Fine, Understand, Thank you, Again, Me, Hi and Bad.

RESULTS AND ANALYSIS 3.1. The E-Glove
The E-Glove shown in Figure 4 was sewn with flex sensors in each finger. The accelerometer was positioned on top of the hand. Both flex sensors and accelerometer were connected in the Arduino microcontroller which was placed inside a box that is wrapped around the forearm. The E-Glove sends the data wirelessly to the computer. This feature is not exhibited on most of the previous data gloves [2], [4], [6], [8]- [10] but proven beneficial to other designs [1], [3], [5], [7]. The wireless feature of E-Glove should encourage a healthy classroom interaction in which the teacher can let the students use the E-Glove in their seat and perform a particular hand gesture that may serve as a form of recitation in Deaf Education.

Sensor values
Output data were directly obtained from data glove and each sensor produced different resistance value depending on the hand gesture. Signals produced by the sensors were translated by the Arduino microcontroller into digital form and through serial communication, the data were transferred to the Arduino software.
As seen in Figure 5, the sensors located at the fingers haddifferent readings depending on the sign language. Using the Arduino software, the sensor values were mapped to get the specific letter or word. The letter or the word which corresponds to the values was sent to the software program. These letters or words were then shown in the graphical user interface and at the same time, it was read aloud by the system. Figure 6 shows the reading of accelerometer in the Arduino software. On each columns were the readings of x, y, and z-axis respectively. The accelerometer was also mapped according to the movement of the hand.

Software program
The software program developed in Visual C# is shown in Figure 7. It was designed to be userfriendly and attractive for children. There were two, large and small, textboxes. The large text box is where the accumulated words or letters show up while the smaller text box shows the last letter or word recognized by the software program. Buttons were labeled concisely and clearly. The "Start" button enables the software program to show the corresponding letter or word on the text boxes based on the gesture exhibited by the user. The "Back Space" button deletes the last character in the large text box while the "Clear" button removes all the characters in the large text box. The "Stop" button stops the program to show the letter or word recognized by the E-Glove. The design of the GUI enabled children in Deaf Education to spell out words shown in the large text box, something that the previous data gloves lack since they can only show the last recognized letter [1], [4]. Moreover, the GUI design is forgiving because it lets the user to correct misspelled words using the "Backspace" button.The accumulation of letter in the large text box will enable the teacher to evaluate and determine the number of correct gestures the student has made.
The letters and words recognized were also converted into speech for teachers to check if the gesture was right even without looking on the GUI. Additionally, the size of the characters was large enough to be seen by the students.  ISSN: 2088-8708

Testing of ASL alphabet and sign words
Each of the letters in the ASL alphabetwas tested by doing the gestures once to check the accuracy of the device. The Table 1 shows that on the second and third trial, one letter was not recognized. Calculating for the rate of accuracy, E-Glove has 98% accuracy in detecting the letters. The simple formula below was used in determining the rate of accuracy of E-Glove in recognizing letters in the ASL alphabet: The device was able to recognize all the letters on most of the trials but there were undetected letters due to conflicts in the reading because the mapped values of the flex sensors are closely related.When the conflict in reading happened, the E-Glove did not show any letters on the software program and the user had to repeat the gesture again. This implies that E-Glove is reliable because the software program will not show incorrect letters. The same with the ASL alphabet, simple words were also tested. Accelerometer with gyroscope was used in the detection of words. Table 2 shows the number of recognized words for every trial. The E-Glove was not able to recognize all the words for the second and third trial. One wordwas missed.Using the simple formula for accuracy, testing the device achieved an accuracy of 98%.
The device was only able to detect words with the right hand orientation or position fed in the software program.The recognition of words also exhibited the behavior of E-Glove when recognizing letters in which no words will be shown if gesture was undetected. This does not only to make E-Glove reliable in recognizing gestures for words but also let the users topractice the right orientation or positions of the hand when doing the gestures. The acceptable rate of accuracy achieved by the E-Glove suggests that the method of recognizing letters in which only one type of sensor was used compared to multiple sensors [1], [4] is sufficient to perform its function. Since fewer data will be read and processed, the performance of the device in terms of analyzing the gesture will be improved unlike the used of multiple sensors by previous studies where recognition speed was traded off to accuracy. Moreover, the number of flex sensors used was also reduced compared to other gloves which will make E-Glove cheaper to produce.
In terms of recognizing ASL sign words, E-Glove was also able to show that with the use of a single accelerometer, an acceptable rate of accuracy can also be achieved. The accelerometer from previous studieswas used in combination with flex sensor [1], [4] to only recognize letters.

CONCLUSION
In deaf education using the BiBi approach, children learn American Sign Languageas a first language. They learn ASL as the primary way of communicating with other people. E-Glove works as an 2297 automated translator and converts sign language directly into a vocal and textual format using flex sensors and accelerometer. Considering the accuracy it exhibited, the reduction of sensors to improve the gesture recognition speed and the cheaper productioncost, it can very well be implemented as a teaching aid for Deaf Education. Furthermore, the design of the GUI has an increased aesthetic appeal for children. This educational tool for hearing-impaired children would encourage a healthy classroom interaction. The E-Glove can further be improved by adding more ASL sign words from the ASL dictionary to its pool of words. The software program can also be developed in such a way that it can construct sentences from words detected.