SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

ISSN 2063-5346 SMART GLOVE TRANSLATES SIGN

LANGUAGE IN REAL TIME

Divya P S [1], Dinesh Kumar T R [2], Jalaja S [3], Bharathraj M [4]

, Mohamed Anas S [5] , Karthick L [6]

Article History: Received: 01.02.2023 Revised: 07.03.2023 Accepted: 10.04.2023

Abstract
India's population is disproportionately made up of dumb and deaf people. To translate ASL
into speech, the system is therefore creating a glove-based device. Only via speech can we
convey our thoughts or propagate a message, yet a person with a disability finds it challenging
to engage in typical social interactions. To overcome this communication barrier, we are
developing a glove that transforms sign language into text and then voice. The basic system is
divided into 2 parts recognition of sign language and conversion to text, followed by voice. To
carry out this whole procedure, we have trained and tested the dataset for that. The fundamental
operation of hand gloves that make up the sign language glove have IR sensors that can detect
how the fingers flex and an accelerometer that can detect how the hand moves. The Node MCU
receives data from the sensors, converts it to binary signals, compares them to the recorded
value for sign detection, and displays the results as text on the mobile display. Deaf or hard-of-
hearing people connect with society through sign languages that are challenging for non-deaf
or hard-of-hearing people to understand. Communication between deaf-mutes and non-mutes
has therefore always been difficult. There are challenging tasks all throughout the world. Those
who are deaf or dumb should be able to express themselves better. Since then, they have
improved their ability to blend in with their environment.
KEYWORDS: IR Sensors , NodeMCU ,Gesture Recognition, Sign Languages,
Accelerometer.
[email protected] [1], [email protected] [2], [email protected]
[3]
, [email protected].[4], , [email protected] [5],
[email protected][6]
1,2,3
Assistant Professor, Department of Electronics and Communication Engineering
(Vel Tech High Tech Dr. Rangarajan Dr. Sakunthala Engineering College, Chennai, India)
4,5,6
UG Student, Department of Electronics and Communication Engineering
(Vel Tech High Tech Dr. Rangarajan Dr. Sakunthala Engineering College, Chennai, India)

DOI:10.31838/ecb/2023.12.s1-B.224

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2258

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

1. INTRODUCTION 2. PROPOSED SYSTEM

Around the globe, there are about 70 A glove that instantaneously translates
million deaf-mute people, according to American sign language into voice and text
statistics from the World Federation of the has been a project of ours.. The fingertips
Deaf and the WHO. Around 32 million. of the glove are equipped with small IR
youngsters that are deaf out of a total of 360 sensors. Through accelerometer and IR
million people. The majority of those with sensor, these sensors can detect finger
speech and hearing impairments are unable location and motions. Our project is
to read or write in standard languages. To completely wireless and includes hand sign
communicate with others, deaf and mute symbols and voice tips for user convenient.
people utilise sign language (SL), which is
their native tongue. In SL, gestures—which
incorporate the use of finger gestures, hand
motions, and facial expressions—are used
primarily to convey message rather than
voice. Real-time communication between
those who have hearing and speech
impairments is the major goal of this
initiative. The only means of conveying
thoughts is through communication, Fig 2.1: block diagram
although some individuals are not as
fortunate as us. This shouldn't limit them in Popular microcontroller board called the
any way. Therefore, it is crucial to create a Node MCU is based on the ESP8266 Wi-Fi
model that is effective and satisfies the module. It is a great option for IoT
needs of those who are deaf or have trouble applications because it has built-in Wi-Fi
speaking or hearing so that they may capability and utilises the Arduino IDE to
communicate effectively. We therefore be programmed.. The accelerometer is used
mostly communicate via hand gestures, to measure the acceleration and tilt of the
which are connected to a mobile application glove, while the IR sensor can be utilised to
that outputs text and speech commands. detect hand movements. You would need to
Speech-impaired people's verbal create a database of sign language motions
communication is constrained, whereas and their accompanying meanings before
hearing-impaired people's inability to you could put the system into use. The
understand what others are saying is a glove would then require calibration to
constraint. We have created a model that make sure the accelerometer and IR sensor
accepts gestures as input and outputs voice are correctly detecting hand movements.
and display in order to better The Node MCU can then be configured to
communication between them. The person read the sensor data and compare it to.
with hearing loss is able to see but not hear.
whereas a person with speech impairments 3. Existing Method
can communicate using hand gestures and These gloves often contain sensors that can
speak while still understanding through track the movements of the wearer's hands
hearing. We combine hardware and and fingers, which may then be translated
software to create the technologies we use. into spoken or written language using
The hardware will process user gesture software and machine learning techniques.
input in the microcontroller and output The "SignAloud" glove, created by two
speech and display as a result. University of Washington undergraduate
students, is an illustration of such a smart
glove. The wearer's hand movements are
detected by sensors in the glove, which

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2259

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

wirelessly sends the information to a connect the data to the gestures made, they
computer where machine learning used the Arduino IDE.Arduino IDE's space
techniques are used to convert the hand and speed restrictions are a downside. Their
movements into spoken words. Another system is confined to a small number of
illustration is the "GestureTek" glove, gestures. Flex sensors have another
which recognises and understands sign drawback since they alter in resistance
language using a mix of sensors, motion value with heavy use. Because of this, the
tracking, and machine learning. The glove likelihood Reduces by a certain proportion
can instantly Text or spoken language that of getting the right output.
is signed.. can be used to communicate AbhinandanDas [2] used flex sensors and
between hearing and deaf people. In gyro sensors for finger and hand
general, real-time sign language translation movement, respectively, in his study as a
capabilities in smart gloves represent an different strategy. He did this by using a
exciting advancement in the field of gyro sensor for hand movement and a flex
assistive technology and have the potential sensor on each finger. As a result, the
to significantly enhance accessibility and dataset of gestures rose along with the
communication for the deaf and hard-of- number of gesture combinations. Xbee
hearing communities. transceivers are used for data transmission
to ensure smooth data flow. They classified
4. RELATED WORK the activities and hand motions into the sign
4.1 LITERATURE SURVEY .To categorise quickly many types of
Numerous studies and investigations have gestures, it was necessary to use this robust
been conducted in the area of smart processor. According to the method they
communication for those who are deaf or designed, a user's movements could be an
hard of hearing. According to which, sign alphabet or a number, for which suitable
language is the most efficient medium, classification is required. The data is
other from writing and sketching, for a handled quickly thanks to this strong Intel
smooth communication between normal processor. A microprocessor processed the
people and these specifically challenged data when it was received, and the output
persons. In this field, numerous patent was displayed in accordance with the text
applications have been made with diverse mapping to the sign language. The Grove
approaches to solve this problem. In his LCD displays the text that was generated.
development of a sign language/text system The Grove-buzzer sensor further converts it
for the hearing and speech impaired, to voice. The main flaw with this entire
DalalAbdullah [1] took a certain approach. setup is that it can only produce 200 words.
In which he put up the concept of creating The use of alphabets and numbers, which
a two-part electric gadget, one of which significantly slows down communication,
would serve as a transmitter and the other is another drawback of this method.
as a receiver. The Arabic sign language can Adarsh Ghimire's [3] technology, which
now be translated into speech or text thanks is based on a desktop programmer and uses
to this method. The concept was a machine learning algorithm, produced yet
straightforward: attach flex sensors to each another remarkable result. Data collection
finger and connect them to an Arduino. from the embedded flex sensors in each
This microcontroller gathers data from finger, coupled with the accelerometer and
flexible sensors and transmits it using an gyroscope from each hand, is the first step
RF transmitter. In contrast, the Arduino of the procedure .The data is fed into a
Mega receives the output from the machine learning system to forecast how
transmitter and delivers it to the RF the user's gesture would be received by a
receiver, which then processes it to produce speaker or an output screen, respectively.
the output for the LCD and speakers. To The machine learning model's accuracy

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2260

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

ranges from 75 to 80%, and it can store 350 Test the integrated system: After the
datasets for each alphabet. This system's NodeMCU board and mobile software were
accuracy and precision are much higher connected, we tested the system to make
than those of the other suggested systems. sure everything operated as it should. To
However, a disadvantage of the correlation test the system's ability to identify and
plot was that it often left the results unclear interpret different hand signs, this will be
because the algorithm had trouble necessary. Once the system is functioning
understanding some alphabets because of as anticipated, we want to further refine and
how similarly the fingers were bent. optimise it. This might entail enhancing the
hand sign recognition algorithm's precision,
lowering energy usage, or increasing the
5. METHODOLOGY system's usability. Overall, setting up a
Collect your materials: We'll need an hand sign recognition system can be a
accelerometer, IR sensor, NodeMCU (or a challenging process that includes hardware
similar microcontroller board) and a mobile assembly, software development, and
app with the required code to construct this testing. Yet, by adhering to a clear
system. Put the components together: methodology and spending the time to test
Connect the accelerometer and IR sensor to and improve the system, we may create a
the NodeMCU device. To make sure the dependable and efficient tool for
connection is correct, we must adhere to the communicating using hand gestures.
directions for our particular sensors.
Write the code: We created the code to 6. COMPONENTS
decipher the data coming from the
accelerometer and IR sensor. This code These are the Components are
should be able to identify the particular  NodeMCU
hand signals we're looking for to detect and  IR sensor
convert them into digital signals that can be  7805 voltage regulator
electronically transmitted to the mobile  9v battery
app. Additionally, we made sure the code  MPU6050
could be compiled, published, and work
with the NodeMCU. The code was tested 6.1 NodeMCU
after it was created to make sure it functions
as intended. To do this, the IR sensor and
accelerometer will be put through its paces
to make sure they are accurately sensing the
hand signals and transmitting them to the
NodeMCU board. Additionally, we
examined the wireless link between the
NodeMCU and the mobile software.
Integrate with the mobile app: We FIGURE :6.1 [NodeMCU ESP8266]
combined the two systems, assuming that
the mobile app already possesses the
Wi-Fi connection is incorporated into
required code for deciphering the signals
NodeMCU, making it simple to connect to
sent by the NodeMCU board. This will
the internet and communicate with other
require making sure the mobile app can
devices. General Purpose Input/Output
receive the signals sent by the NodeMCU
(GPIO) Pins: The NodeMCU contains a
board and convert them into the proper text
number of GPIO (General Purpose
and voice commands.
Input/Output) pins that can be used to
connect to sensors, switches, LEDs, and

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2261

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

other devices. NodeMCU also features infrared light that it emits. Remote control:
analogue input pins that can be used to IR sensors can be used to receive signals
measure analogue signals like temperature, from a remote control device and translate
light intensity, and sound. PWM Output: them into instructions for an electronic
NodeMCU offers PWM (Pulse Width device, such as a television or DVD player.
Modulation) output, which can be used to IR sensors can measure the infrared
regulate a variety of devices, including the radiation emitted by light sources like
brightness of LEDs and the speed of LEDs and incandescent bulbs to detect the
motors. NodeMCU enables over-the-air presence of light. In applications including
(OTA) updates, allowing you to upgrade remote sensing, wireless communication,
the firmware of the device without and data transmission between devices, IR
physically connecting to it. These are only sensors can be utilised to send and receive
a few of the NodeMCU features that may data.
be applied to applications. You could also
be able to leverage other platform
capabilities and functions depending on the
needs of your project. I2C (Inter-Integrated
Circuit) communication, a widely used
protocol for interacting with sensors and
other devices, is supported by NodeMCU.
SPI Communication: Another widely used
protocol for interacting with sensors and FIGURE: 6.2 [IR Sensor]
other devices is SPI (Serial Peripheral
Interface), which NodeMCU also supports. 6.3 7805 VOLTAGE REGULATOR
NodeMCU can be used to build a web A consistent output voltage of 5 volts is
server that enables remote device control produced by the 7805 by regulating an
and data viewing from any location with an input voltage that can range from 7 to 35
internet connection. MQTT: NodeMCU is volts. This makes sure that electrical
compatible with the MQTT (Message equipment get a consistent power supply
Queuing Telemetry Transport) protocol, and helps shield them from voltage swings.
which enables device communication via
the internet in Internet of Things projects. Current limiting: The 7805 contains an
inbuilt circuit that limits the amount of
current it can give without damaging the
6.2 IR SENSOR device. By preventing overloading, this
helps safeguard the regulator and the
The two primary tasks that IR sensors are
electronic devices it is attached to.
capable of are object detection and
temperature sensing. In order to identify the
existence of an object, infrared sensors
gather up the infrared radiation that the
object emits or reflects. IR sensors are
advantageous for applications including
proximity sensing, motion detection, and
obstacle detection because of this
capabilities. IR sensors, on the other hand,
are valuable in a variety of applications,
including medical diagnostics, thermal FIGURE: 6.3 [7805 Voltage Regulator]
imaging, and temperature monitoring in The 7805 may produce a lot of heat,
industrial operations. They may measure an especially when operating at high currents.
object's temperature by detecting the The regulator is intended to dissipate heat

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2262

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

through a heat sink or other cooling detects changes in capacitance. A proof

mechanism to prevent overheating.Reverse mass suspended by beams from a silicon
polarity protection: The 7805 features built- substrate makes up the MEMS structure.
in reverse polarity protection that guards The proof mass suffers an acceleration or
against harm to the regulator and attached angular velocity force when the sensor is
devices in the event that the input voltage is moved or rotated, which causes it to deflect
applied in reverse. and alter the capacitance between the proof
mass and the electrodes. An analog-to-
digital converter amplifies and transforms
6.4 9V BATTERY the capacitance change into a voltage
Little electronic equipment: Portable signal, which is then translated into a digital
radios, smoke detectors, and remote signal (ADC).The MPU6050 also features
controls are just a few examples of the a digital motion processor (DMP), which
compact electronics that are frequently can carry out intricate computations on the
powered by 9-volt batteries. unprocessed sensor data to produce outputs
Microcontrollers are also powered by a 9- that are more useful, such quaternion
volt battery, which can be used with many values, for example. This can be used to
microcontroller boards including the figure out how the sensor is oriented in
Arduino and Raspberry Pi. In applications space. Moreover, the DMP is capable of
that call for mobility, this makes them complex motion processing tasks and
transportable and simple to utilise. Lighting gesture recognition. The I2C (Inter-
up a few LEDs only needs a small amount Integrated Circuit) protocol, which enables
of power, and a 9-volt battery is more than two- wire serial communication between
capable of supplying it. LEDs (Light the microcontroller and the sensor, can be
Emitting Diodes) require relatively little used to interface with the MPU6050. The
voltage and current to operate.cells in MPU6050 library, for example, can be used
succession are used to create primary to quickly receive the raw sensor data from
lithium kinds. the device and carry out the necessary
computations to provide useful data.

7. PROJECT IMAGES
TOP VIEW

FIGURE: 6.4 [9V BATTERY]

6.5 MPU6050 x 1

FIGURE : 7.1 FIGURE : 7.2

HAND SIGN OF A will be exposed
A in audio format
FIGURE: 6.5 [MPU6050 x 1] Also the user
The microelectromechanical system
(MEMS) sensor structure, which is housed
on a tiny silicon chip, is how the MPU6050

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2263

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

SIDE VIEW sign language identification

Miniaturization: At the moment, smart
gloves are very large and can be difficult to
wear for extended periods of time.
Upcoming developments in wearable
technology and material science may result
in the creation of smaller, lighter smart
gloves that are more comfortable to wear.
Increased language support: Although
American Sign Language (ASL) can be
translated using current technology, there
are numerous additional sign languages in
use throughout the world. Future
technological developments might result in
FIGURE : 7.3 FIGURE : 7.4 the creation of smart gloves that can
HAND SIGN OF C will be exposed understand and translate a variety of sign
C in audio format languages. Connectivity with other devices:
Also the user Smart gloves that can translate sign
language could be connected to other
gadgets like tablets or smartphones for
8. CONCLUSION easier accessibility and communication.
Those who are blind, deaf, or dumb can Real-time feedback: Future smart gloves
converse with one another using this could translate sign language as well as
method. Those who are dumb communicate give the user immediate feedback on how
in a language that is difficult for blind and accurately they are signing. This could help
illiterate people to grasp. To aid the deaf, the user become more fluent in the
the sign language is also translated into language.
written form. These words are displayed on
a computer screen. Those who are deaf
REFERENCES
must be healed. We’ll utilizing it to watch
how the blind and deaf move their hands. [1] Authors: Shihab Shahriar Hazari,
The system translates hand motions into Asaduzzaman, Lamia Alam, and
text, which is subsequently translated into Nasim Al Goni wrote "Designing a
voice. A provision has been included to the sign language translation system
text system so that people can still read and utilising kinect motion sensor device."
comprehend what the other person is [2] [Authors: M. G. Kumar, M. K. Gurjar,
attempting to say even if they are unable to and M.S. Singh, "American sign
hear the sound emitted owing to those language translating glove using flex
limitations. sensor," Imperial Journal of
Interdisciplinary Research, vol. 2, no.
9. Future Scope 6, 2016.
[3] Authors: A. Z. Shukor, M. F. Miskon,
Enhanced accuracy: The accuracy rate of M. H. lamaluddin, F. bin Ali@Ibrahim,
the most recent state-of-the-art technology M. F. Asyraf, and M. B. bin Bahar, "A
for translating sign language using smart new data glove approach for Malaysian
gloves is about 98%. Nonetheless, there is sign language detection," Procedia
always potential for improvement, and Computer Science, vol. 76. 2015.
upcoming developments in artificial [4] Authors: H. Ikeda, H. Sagawa, T.
intelligence and machine learning may
Sakiyama, M. Ohki. and Y. Kaneko,
contribute to enhancing the precision of

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2264

SMART GLOVE TRANSLATES SIGN LANGUAGE IN REAL TIME Section A-Research paper

"Sign language editing apparatus," US Technology, Volume 2 Issue 5, pp:

Patent 5,990,878. 336-341, 1 May 2013, ISSN: 2277-
[5] Authors: Madhuri, Y.; Anitha, G.; 1581
Anburajan, M., “Vision-based sign [8] Authors: Laura Dipietro, Angelo M.
language translation device," Senior, member IEEE, “A Survey of
Information Communication and Glove Based System and their
Embedded Systems (ICICES), 2013 applications,” IEEE transaction on
International Conference on, vol., no., System, Man, and Cybernetics-Part C:
pp.565,568, 21-22 Feb. 2013, DOI: Application and Review, vol. 38, no. 4,
10.1109/ICICES.2013.65083 2008.
[6] Authors: Gunasekaran K., [9] Authors: Sidney Fels and Geoffrey
Manikandan. R., “Sign Language to Hinton, “Glove Talk II,” Department
Speech Translation System Using PIC of Computer Science of University of
Microcontroller”, International Journal Toronto, Toronto, 6 November 1992
of Engineering and Technology [10] Authors: KuoChue Neo, Audiogram
(IJET), Vol 5 No 2, pp 1024-1028, and Wan MohdYusofRahiman Wan
Apr-May 2013, ISSN: 0975-4024 Abdul Aziz, ―Development of Sign
[7] Authors: Anbarasi Rajamohan, Signal Translation System Based on
Hemavathy R., Dhanalakshmi M., Altera ‘s FPGA DE2 Board‖,
“Deaf-Mute Communication International Journal of Human-
Interpreter”, International Journal of Computer Interaction (IJHCI), Vol. 2,
Scientific Engineering and Issue 3, pp. 101-114, 2011.

Eur. Chem. Bull. 2023,12(Special Issue 1, Part-B), 2258-2265 2265