Voice Based Email System For Visually Impaired

Chapter 1

INTRODUCTION
With a boom in internet technologies, the communication has become a lot easier. Internet is considered as
a vault of innovation, technologies and information. Numerous networking and social media sites. The most
conventional way of online communication is email. Nowadays everybody is connected to the internet. It is
an inseparable part of life. It contains all the information of individuals and day to day history[17].
Communication and interaction are possible mainly through the internet. Out of many technologies email is
the most common way of communication for business and educational perspective. Although not all use net
and have access. This is due to lack of facilities, knowledge and money. The users should have vision to see
and read the screen. For the physically and visually challenged people net is like a useless and unfamilier
thing. However there are technologies that do not provide full accuracy and efficiency to the impaired
people to use the internet. As communication is needed in everyone’s life, net facilities should be available
for everyone. That’s why this project is based on voice email which will help impaired people to
communicate[16].

A voice mail system is a computer based system that allows users and subscribers to exchange messages
without typing. This systems are designed to convert a caller’s recorded audio message into text and then it
will be sent to recipient. It is mainly useful for blind people, as every official messages are only sent through
mail they cannot test the message so this application helps them a lot. A voice mail system is also called as
voice bank.

This system acts an application which contains accessible user interface to select, play, and manage
messages; a delivery method to either play or deliver the message otherwise.

Normal gmail does not contain the voice recording option. In this project designing record option and the
recorded voice is converted to text and sent to the particular mail. Now a days people are very busy, so they
are interested in recording a message and sending it instead of typing.

This type of application helps people in a way of time consuming and also perfect message will be received.
This application is a user system interaction in which the system interacts with the the user like it asks the
user to enter the message and the entered message is displayed and also the system also asks to enter the
recipient mail id and then it sends when the user says to send the message[18] .

The voice activation detection is used to convert our audio into text. The system also speaks i.e. response to
the user accordingly.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

This is done by using answer activation detection. The application consists of user interface that has a 2
text boxes, one is used for entering the message and the other is used for entering the recipient mail id. It
consists of two buttons, one is to clear or edit the message and the other is to send the message.

At finally after entering the system repeats everything what we have spoken. By this we can confirm that
the exact message is texted or not.

1.1 Voice Based Application

In the proposed system, a desktop application is to be developed that can be used by people with various
visual imparities, to access emails easily and efficiently. All the existing voice based email systems, provide
their own user developed email services and do not incorporate the use of Google’s Gmail.

So, considering this here, the intention is to develop the application by linking it with the Gmail Client,
thereby giving users an additional advantage.

Hardware
Desktop PC Speakers, Mic
Requirements

Software Jupyter
Python, Djyngo Requirements Notebook,

Speech SDK Pycharm

Module
Speech to text Text to speech

Send, Receive
Mails

Figure1.1: Block Diagram of proposed system[1].

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

1.2 Structure of Report

Chapter 1: This chapter deals with the introduction of the project.

Chapter 2: This chapter describes the literature survey done on the concerned topic.

Chapter 3: This chapter describes the methodology used in the project..

Chapter 4: This chapter deals with the specifications of software components as well as work completed and
expected results.

Chapter 5: This chapter addresses the referred papers and there information.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

Chapter 2

LITERATURE SURVEY
The literature survey is a systematic method of identifying, evaluating and interpreting the work
produced by the researchers, scholars and practitioners. The literature survey basically helped in getting
acquainted with the various techniques through which our voice based conversation could be made to work
efficiently. We looked at various tools that have been developed already for the visually impaired people.
Apart from this, the need and challenges in real time conversion of speech to text were also studied. With
the help of various research paper, we also looked upon the rate of communication that could be increased
and the process of communication between impaired and normal ones could be made more efficient.

2.1 Literature Review

In this paper the proposed system uses interactive voice response in order to interact with the users and
there is no use of keywords. This done by the use of text to speech and automated speech recognition . This
system is to give user voice control mail rather than touch control. This uses interactive voice response in
order to interact with the user. It makes the system realistic and natural way to import the message . When
the user interact with the system it will automatically generate the voices to do the actions . In this system
only mouse click operations are performed to initiate the operations.[1]

The present review aims to high light accessible features and apps along with usages, including
educational, and access to smart phones as assistive technology for visual impairment and blindness. These
have been significant access recent developments in mobile technology that incorporates computer
technology relating to electronic information , and touch screen accessibility[2] .

This paper presents an email system working on the voice controlling principle like interactive voice
response(IVR) which enables PC to associate with people using voice, and automatic speech recognition
(ASR) also known as speech to text converter. It is technology that allows human beings to use their voices
to speak with a computer interface in a way that it resembles normal human conversation. Voice message
engineering encourages blind individuals to get to email and other interactive media elements and other
working frame work. Here author depict the voice message design utilized by blind individuals to get to
email and media elements of working frame work effectively and productively. This design will likewise
decrease intellectual burden taken by oblivious in regards to recall and type characters utilizing console[3].

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

In this paper the proposed work aims to develop an application that recognizes the user voice and
performs the given command by making use of speech to text for email composing and text to speech for
reading emails. It makes use of google web kit application programming interface. This application is
developed to ease the process of email writing not only for visually impaired people but for everyone.So
additional skills required for type writing will not be necessary any more. The proposed system aims to
develop a mechanism which converts speech to text for email composing and converts text to speech for
reading emails[4].

In this paper we have seen the vision application which is its enterit aimed to aid and improve the
visually impaired. The application is made user friendly audience centric and voice activated making it a
blissfull experience for users. Furthermore the improvement like creating more mature voice recognisation
to ease difficulties[5] .

This paper helps in understanding various technologies such as text to speech converter, speech to text
converter, barille touch and eyes free text entry and also provides comparision and different types of
applications currently present. These accessibility applications provide accessibility for messaging, calling,
text entry, and helping with identifying medicines. Touch screen mobiles are inaccessible for visually
impaired users due to the absence of any physical key and insufficient accessibility features[6].

This paper shows developing a smart watch that enables user to send an email any intended recipient
using speech recognition. The user shall press the dial of the watch so as to start the email application. The
watch generates a feedback indicating the user that the app has been opened. The user then dictates the name
of the intended recipient and the content of the email. The mail is then sent by the command send. This
system will not let user to make use of keyboard instead will work on speech recognition[7].

In the proposed system it helps in understanding the way blind user uses a mobile device that is
necessary to design accessible gestures that can be utilized visually impaired in their daily life. The current
screen readers is IOS and Android can be enhanced to include multimodal gesture that are natural to users.
Gestures is designed to include personification factors such as using speech recognition, touch and motion
sensing gestures instead of touch gesture only. This is to minimize the visually impaired smart phone user
from performing too much taps to perform the desired task[8].

In this paper the client targeted is people who are visually impaired. The page aims in creating the app
with features which can help make life of visually impaired people easier by knowing whats around them by
using the app. It is seen that that all the applications similar to this app need to capture image and then return
what object it is. But this application does not need to take an image and it only scan object and then tells the

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

user which object it is. That way the visually impaired user doesn’t have to detect images from every now
and then[9].

This paper explains to extensively covers the concept of web based communication system. The
proposed system is really very useful for blinds and handicapped people to easily access mails and thus
reduce reliance of those people on the normal people real time composition of external composition texture
mail through audio input is done. Keywords are used to trigger mail related operations[10].

Earlier blind people does not send email using the system. The multitude of email types along with the
ability setting enables their use in nomatic daily context but their emails are not useful in all types of people
such as blind people. Audio based email are normally preferrable for blind people. They easily respond to
audio instructions in this system. There is a chance for availability of this audio based emails to blind
people[11].

In this paper we have seen the intelligent eye android mobile application. The application targets
visually impaired people and provides them with a set of useful features such as light detection, object
detection, object recognition and bank note detection. It has a user friendly interface customized for blind
people, where the friendly interface read out loud so that the user can clearly hear them. Result showed that
application correctly detected objects, bank notes, and light levels[12].

The architecture of the proposed system of e-commerce web application interpreted with the speech to
text is depicted in the system. The system describes the implementation details of prototype application. The
next is to incorporate the text to speech service in the application. So that in the case the application is not
able to service users using a generated voice, complete purchase of the user and confirm and cancel
transaction. This application will adverse the accessibility problem of e-commerce website to some
extent[13].

In proposed the system that relies on a voice command based system unlike the existing mail system.
The complete system is primarily based on speech to text commands. Once using this system the application
will be prompting the user to speak specific commands to avail respective services and if the user wants to
access the respective services the user needs to speak that command. This application makes use of IMAP
(Internet Message Access Protocol). It is an Internet standard protocol used by email clients to retrieve
clients to retrieve email messages from a mail server over a TCP/IP connection. The Main activity Screen
will be the First screen to be displayed on start of the app. This screen waits for the user to press the button
so that the system will start accepting voice commands. And this is a full sized button so they can press
anywhere on the screen. Then using Voice commands users can send, read emails[14].

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

It enables visually impaired people to use many interactive and learning devices. Various studies on
assistive technology have suggested that it is necessary to understand visually impaired users needs, strength,
weakness. The use of assistive technology has effect on visually impaired people self esteem and social
condition and perceptions. Assistive technology is valuable to enable active participation, learning and
economic and social development for visually impaired people in work place[15].

The paper have proposed an email system which can be accessed easily by blind people. The use of
Speech to Text convertor, Text to speech convertor and Viterbi Algorithm are taken into consideration.

The algorithmic rule works with the technique that the system detects the foremost acceptable word once the
user spells it so matches the word that's guessed with the particular word that's pronounced. The user needs
to register to the website when they visit the site for the first time. This system reduces some drawbacks of
the existing system. The drawback of this system is that the efficiency of the Viterbi algorithm decreases as
the number of errors increases as well as it requires more space[16].

In this paper a speech interface system in mobile application combined with object and distance will
help the visually impaired community to navigate to there destination only with the use of smart phone. The
project is intended to develop a mobile assistant application based on the android platform that enables to
classify image through voice narration which helps to recognize a object or person. The application is able to
detect the distance between obstacles. All functions can be triggered using voice recognition that able to
understand local languages[17].

2.2 Motivation

As nowadays communication plays a very major role in persons day to day life, it may through the
online mode or through offline mode. For the visually impaired people it is very difficult to communicate
through the online mode as the screen cannot be seen by them. Someone has to help them, to make the
communication easier for the visually impaired, and become independent this system is used. This system
makes use of speech to text and text to speech converter. Using this system will make them independent and
their life a little bit easier.

2.3 Need(scope) of the project

For people who can see, e-mailing is not a big deal, but for people who are not blessed with gift of
vision it postures a key concern because of its intersection with many vocational responsibilities. This voice
based email system has great application as it is used by blind people as they can understand where they are.
E.g. whenever cursor moves to any icon on the website say register it will sound like “Register button”.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

There are many screen readers available. But people had to remember the mouse clicks. Rather, this project
will reduce this problem. This system focuses more on user friendliness of all types of persons including
regular persons, visually impaired people.

2.4 Problem statement

Blind people does not send email using the system.The multitude of email types along with the ability
setting enables their use in nomadic daily contexts. But these emails are not useful in all types of people
such as blind people they can’t send the email. Nowadays people use email for discussing, informing all the
important information through mail. It has become the daily used service for professional as well as personal
use.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

Chapter 3

METHODOLOGY
1. Speech- to-Text Converter Speech-to-text converter recognizes the speech, analyzes the sounds you make
by filtering what you say, then digitizes it to a format it can read. The recognized text can be saved in a file.
Using Python code in Pycharm platforms are used here to develop this. Our speech to-text system directly
obtains and converts speech to text. Speech recognition systems can be divided into several blocks: feature
extraction, acoustic models database which is created based on the training data, dictionary, language model
and the speech recognition algorithm.

2. Text-to-Speech Converter Using speech synthesis techniques, it converts text to voice output. It is
employed by the blind to concentrate to written material; it’s now used extensively to convey financial data,
e-mail messages, and other information through the telephone for everybody. Text-to-speech is also used on
devices such as portable GPS units to announce street names when giving directions.

3. Voice based Email Application The user can send emails, listen to what they have written and also
receive emails and listen to them with voice commands. In Email, the application makes use of the SMTP
protocol for sending emails and POP3 protocol for receiving emails. SMTP is the reliable protocol to send
emails and it works in a simple way that the SMTP server passes on the email messages quickly. POP3 is
used to receive emails. The POP3 server stores the email and on request the emails are displayed. The same
is implemented in our application, that on the request by the user the emails are downloaded. Speech
recognition, also known as automatic speech recognition (ASR), computer speech recognition, or speech-to-
text, is a capability which enables a program to process human speech into a written format.

The Simple Mail Transfer Protocol (SMTP) is an internet standard communication

protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send
and receive mail messages. User-level email clients typically use SMTP only for sending messages to a mail
server for relaying, and typically submit outgoing email to the mail server on port 587 or 465 per RFC 8314.

Post Office Protocol 3, or POP3, is the most commonly used protocol for receiving email over the
internet. This standard protocol, which most email servers and their clients support, is used to receive emails
from a remote server and send to a local client.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

New user/Already existing

Incorrect
Login Invalid login/Password

Correct

Inbox

Check mail Compose mail

Exit

Figure 3.1: Flowchart of the proposed system[1]

Figure.3.1: explains the flow of the application:

 The user has to register themselves by creating their account.

 It will prompt for Email id and Password, following which, the authentication will take place.

 For correct credentials entered the user’s registration process will be successfully completed.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

 The user can log into their mail systems.

 Check ,Mails and compose them.

 Here, the Speech-to-text & Text-to-speech modules Come handy.

 System reads out mails, the sender’s name, the subject and the main body.

 Similarly, users can compose emails with the help of the Speech Recognition module or directly
send audio attachments with the recorder provided by the application.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually Impaired

CHAPTER 4

WORK COMPLETED AND EXPECTED RESULTS

4.1 Software Used

DJANGO

Django a high is level python web framework that encourages rapid development and clean, pragmatic
design. Built by experienced developers, it takes care of much of the hassle of web development, so we can
focus on writing our app without needing to reinvent the wheel. Its a free and open source.Django is an
extremely popular and fully featured server-side web framework, written in Python. This module shows you
why Django is one of the most popular web server frameworks, how to set up a development environment,
and how to start using it to create your own web applications.

PYCHARM

Python is a dedicated python Integrated Development Environment(IDE) providing a wide range of

essential tools for python developers, tightly integrated to create a convenient environment for productive
python web, data science development. PyCharm is a hybrid-platform developed by JetBrains as an IDE for
Python. It is commonly used for Python application development. Some of the unicorn organizations such as
Twitter, Facebook, Amazon, and Pinterest use PyCharm as their Python IDE! It supports two versions: v2.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

4.2 Work done

Figure 4.2.1 Shows the output of sign up page created

Sign up page the user has to sign up to this. The new user can create the new id by entering the valid
email id and the new password and confirming the password. The already existed user can directly login.
Use of same email id for multiple account is not enabled. The passwords must match at the time of login.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

Figure 4.2 .2Shows the output for login page.

At the time of the login the user must enter the same email id that he/she entered during the sign up, and
passwords should also be the same. If the user is new sign up option is enabled exactly below the submit
button.

Figure 4.2.3 Shows the output when the wrong password is entered

If the user give the correct email and password that creates a miss match in the entered data. The page
will give the pop up of wrong password. The user has to check for the entered fields once again.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

Figure 4.2.4 Shows the output when correct password is entered.

In this the back end data recorder is taken as admin. The data is in encrypted form. When the email and
password entered is correct the login is successful. The user can then listen to the subject, sender, body of
the mail. The user can even compose the mail by telling the receivers mail id and the subject and the body of
the mail.

Figure 4.2.5 Shows the composing of the mail

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

Figure 4.2.6: Inbox

After composing the mail the inbox can be seen where the received messages can be read. The user can
hear the mail received with the receivers mail id.

4.3 EXPECTED RESULTS

1. It provides a voice based mailing service where they could read and send mail on their own, without any
guidance.

2. Here the users have to use certain keywords which will perform certain actions for e.g. Read, Send,
Compose Mail, Address Book etc.

3. The email system also notifies the user about the messages received.

4. This EMAIL system can be used by a blind person to access mails easily and efficiently. Thus reliance of
visually impaired on other people for their activities related to mail can be reduced.

5. System has eliminated all these concepts and overcome all difficulties faced by the visually impaired.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

6. It uses a speech recognition application which provides an efficient voice input method for mailing devices
for blind. It is also useful for handicapped and illiterate people.

7. In future, we attempt to make the system keyboard free and fully voice based. So it's easy for the visually
impaired people to access the services.

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

REFERENCES
[1] Sunny Kumar, Yogitha R, Aishwarya “Voice email based on SMTP for physically handicapped”30 June
2021 Proceedings of the Fifth International Conference on Intelligent Computing and Control Systems
(ICICCS 2021) IEEE Xplore Part Number: CFP21K74-ART; ISBN: 978-0-7381-1327-2

[2] Suraj Singh Sengam , Souvik Manna ,Covodonga Bascaran “Smart phones Based Assistive Technology”
27 November 2021 Clinical Optometry 2021:13.

[3] Paulus A.Tiwari , Pratiksha Zodwan , Harsha P.Nimkar, Trishna Rotke, Priya G.Wanjari, Umesh Saarth
“A Review on voice based email system for blind” 09 June 2020.Proceedings of the Fifth International
Conference on Inventive Computation Technologies (ICICT-2020) IEEE Xplore Part Number:CFP20F70-
ART; ISBN:978-1-7281-4685-0

[4] Sherly Noel “Human computer interaction (HCI) based smart voice email(Vmail) application assistant
for visually impaired users(VIU)” 06 October 2020.Proceedings of the Third International Conference on
Smart Systems and Inventive Technology (ICSSIT 2020) IEEE Xplore Part Number: CFP20P17-ART;
ISBN: 978-1-7281-5821-1

[5] Shrikesh Suresh , Akhilaa “Android For The Visually Impaired”2020 IEEE International Conference for
Innovation in Technology (INOCON) Bengaluru, India. Nov 6-8, 2020.

[6] Aishwaraya A. Sagale , Anagha Chaudhari “Accessibility feature in mobile phone for visually impaired
user” 25 April 2018 Fourth International Conference on Computing Communication Control and
Automation (ICCUBEA)

[7] Prajakta Chavan, Devesh Jain, Pradnya Savant, Zeba Shaik “Voice Based Email System” International
Journal of Scientific & Engineering Research Volume 9, Issue 2, February-2018. ISSN 2229-5518

[8] NOR Azman Ismail, Yohgamalar W.G, Mohammad Sah Salam “Gesture Design For Visually
Impaired”International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN: 2278-
3075, Volume-8 Issue-8 June, 2019

[9] Mohammad Amanat Khan , Shishiv, Shahariar Rashid Fahim , Fairnz Maesha Halsid , Tanjila Farah
“Eye Assistant Using Mobile Application To Help Visually Impaired”1st International Conference on
Advances in Science, Engineering and Robotics Technology 2019 (ICASERT 2019)

Dept. of ECE, MITE, 2021- Page

 Voice Based Email System For Visually

[10] K M Sanghavi , Radhika, Payal Kumnt, Ankita Katariya , Ruchika Dudhediya “Communication Media
For Blind Based On Voice” International Journal of Advanced Trends in Computer Science and Engineering,
9(1), January – February 2019, 182 – 185

[11] Carmel Mary Belinda M.J, Rupavathy.N, Mahalakshmi N R “Voice Based Text Mail System for
Visually Impaired”1 July 2018 International Journal of Engineering &Technology, 7 (1.7) (2018) 132-136

[12] Milius Awad , Jad Haddad , Mohammad Malli, Tarek Mahmond, Elias Yaacous “Intelligent Eye: A
Mobile Application For Assisting Blind People”International Journal of Scientific Research in Science and
Technology, 4, 2018, pp. 188-193.

[13] Mandeep Singh Kandhari, Farhana Zukerine, Haruna Isah “A Voice Controlled E-Commerce Web
Application” International Research Journal of Engineering and Technology (IRJET) - Volume 05, Issue 04,
April-2018, pp. 2398-2400.

[14] Milan Badigar, Nikita Dias, Jemima Dias and Mario Pinto, “Voice Based Email Application For
Visually Impaired”. International Journal of Science Technology & Engineering (IJSTE) - Volume 4, Issue
12, June 2018, pp. 166-170

[15] Mahfuz Asharay, Pradeed Ray, Najamul Hasan, Md Rashatul Hasan, Lundy Lewiz“Application Of
Information Communication Technology For Visually Impaired People”: January 2017 International Journal
of Disability Management Volume 11, e6, pp. 1–18.

[16] Bishal Kalita and Santosh Kumar Mahto, “Voice Based Email for Blind People”. International Journal
of Engineering Science and Computing (IJESC) - Volume 9, Issue 10, October-2019, pp. 23789-23799.

[17]Abdul Majid Norkhalid, Masyura Ahmad Faudzi, Azimah Abdul Ghapar, Fiza Abdul Rahim “Mobile
Application: Mobile Assistance for Visually Impaired People – Speech Interface System (SIS)”2020 8th
International Conference on Information Technology and Multimedia (ICIMU).

[18] Akif Khan, Shah Khusro, Badam Niazi, Jamil Ahmad, Iftikhar Alam and Inayat Khan, "Tetra Mail: A
usable email client for blind people". Universal Access in the Information Society-04 September 2018.

Dept. of ECE, MITE, 2021- Page