Vol-9 Issue-6 2023 IJARIIE-ISSN(O)-2395-4396

Virtual assistant for the visually blind people

Shivkumar Dhabade SKNSITS Lonavla

Dipak Jawkar SKNSITS Lonavla

Tejas Magar SKNSITS Lonavla

Onkar Jadhav SKNSITS Lonavla

Abstract
This paper addresses the unique challenges faced by visually blind individuals by providing a virtual assistant to
aid their daily lives. This innovative system employs speech recognition and synthesis, object recognition, navigation
assistance, and information retrieval capabilities. Users can interact with the assistant through natural language
commands, enabling access to real-time information, navigation support, and object recognition, which describes the
user's surroundings. The "Virtual Assistance for Visually blind people" project aims to create a software solution that
empowers visually impaired individuals by providing them with a virtual assistant capable of aiding them in their daily
lives.

Keywords— Visually impaired; Voice control; automate website;blind people; voice based email

I. INTRODUCTION
The advancements in the field artificial intelligence has given rise to various virtual assistants such as
Siri in iPhone, Google Assistant, Microsoft Cortona, etc., Despite this advancement there is very little done
to implement these technology to help the blind community. The objective of this project is to help the blind
people in various day to day activities such as – identifying the physical objects in front of them, giving the
description about the various objects by scanning the QR code, information about various hospitals, and
allowing them to give input to all functionalities through voice.
The "Virtual Assistant for Visually Impaired Individuals" is an innovative Python project designed to
empower and improve the daily lives of people with visual impairments. This project leverages the power of
modern technology, including speech recognition, natural language processing, and machine learning, to
create a virtual assistant that provides assistance and enhances accessibility for the visually impaired
community.
Visual impairment poses significant challenges for individuals in their daily activities, from navigating
their surroundings to accessing information from various sources.
This project aims to bridge the accessibility gap by developing a sophisticated virtual assistant that can
help users with tasks such as reading text, identifying objects, navigating environments, and providing
information on a wide range of topics. In a world increasingly defined by technology, the "Virtual Assistant for
Visually Impaired Individuals" represents a pioneering Python project with a profound social impact. Visual
impairment, while presenting unique challenges, should not be a barrier to accessing information, interacting
with the environment, and achieving a sense of independence. This innovative virtual assistant project is an
embodiment of the principle of inclusivity, designed to empower individuals with visual impairments and
offer them newfound opportunities for autonomy and accessibility.
Living with visual impairment can be a daunting journey, as it often entails navigating a predominantly
visual world through alternative means. Tasks that many take for granted, such as reading printed materials,
recognizing objects, or simply moving about safely, can be a significant challenge for those with limited or no
sight. The Virtual Assistant for Visually Impaired Individuals seeks to level the playing field, providing a
technologically advanced solution that assists, informs, and empowers users in numerous aspects of their
daily lives.

21968 ijariie.com 377

Vol-9 Issue-6 2023 IJARIIE-ISSN(O)-2395-4396

II. LITERATURE SURVEY

We have studied many IEEE papers for the purpose of this project.
IEEE paper by Vinayak Iyer states that they built a application that works based on the voice commands
given by the user. They built the application in which user first interacts with the main menu once the
computer is turned on. The main menu shows some web modules which performs some specific tasks and
each module has TTS and STT ability. Every time the beep is generated for speech
recognition. Any voice command given to the system is confirmed by repeating it. And they have given
three web modules i.e. Google, email, Wikipedia.

IEEE paper by Md. Rakibuz Sultan states that they built a application named ABYS always by user
side for visuallyblind people which takes voice command as input using microphone then SRE converts
speech to text. The output ofSRE is fed to CPM which is core of the project this CPMprocesses the
input and calls the appropriate module for execution. And at the end speech synthesizer displays output.

IEEE paper by A. Karthik states that their project information or message is read by the digital camera.
Using one dimensional signal system scans the image vertically and horizontally to find right zone to apply
OCR recognition. After recognizing the text then it is converted to the audio.

IEEE paper by Prince Bose tells us that their project they used speech recognition to convert speech to
text. This text is sent to central processor which determines the nature of command and calls the relevant
script.

III. SYSTEM OVERVIEW AND DESIGN

The user logs into the system using voice based authentication in which he has to input the username
and password then after validation he is navigated to the main menu in which user will be informed about all
the available options like send email, delete mail, inbox, WhatsApp, archives, logout, etc. The user is asked
what to do with the help of inbuilt voice assistant. And user has to tell the task he wanted to do.

Figure 1: System Architecture

Figure 1 is a representation of the system architecture of our software. The user accesses the software

21968 ijariie.com 378

Vol-9 Issue-6 2023 IJARIIE-ISSN(O)-2395-4396

using the web interface where the speech to text (STT) module converts the voice input to text. The user is
then presented with the main menu where they have three options to choose from and decide which website
they want to browse. Accordingly, the module is invoked with its corresponding speech to text modules.

IV. METHODOLOGY
Step 1: User Authentication and Authorization System:
Design and implement a secure user authentication system, including role-based access control, to ensure
authorized access to the application.

Step 2: Interface Selection and Data Collection:

Develop a module allowing users to enter the username and password once the module asks for it. The
fields should be filled once the user speaks accordingly.

Step 3: Data Processing and Analysis:

Design algorithms and methods for accepting the user data, login based on the collected data, and voice-
based navigationand tasks.
Integrate tools for or APIs for the applications that are included in the project.

Step 4: User Interface Development:

Create the Simple UI for the project which should be easily understood to the viewer.

Step 5: Security Enhancement and Threat Analysis:

Integrate the face detection system for the security purpose. And the user should be quickly logged out
once the unauthorized access is noticed.

Figure 2: Main Menu

21968 ijariie.com 379

Vol-9 Issue-6 2023 IJARIIE-ISSN(O)-2395-4396

V. RESULTS
The development and implementation of the "Virtual Assistant for Visually Blind People" project have
yielded several significant results and outcomes. These results represent the successful realization of the
project's objectives and functionalities. Here are some key results:

Speech Recognition and Natural Language Processing (NLP):

The virtual assistant effectively recognizes user speech and converts it into text, enabling seamless
communication.

NLP algorithms interpret user commands, facilitating naturalconversations and responses.

Text-to-Speech (TTS) Synthesis:

The TTS system converts digital text into high-quality, natural sounding speech, facilitating the reading of
various types of content.

Voice Commands and Accessibility:

The virtual assistant offers a user-friendly voice commandinterface, making it accessible and easy to
use.
It can be deployed on various accessible devices, including smartphones and wearable technology.

VI. APPLICATION
Virtual Assistant for the visually blind people acts as a support to the visually disabled people to
access the email and WhatsApp on any browser as our project is browser independent. They can access the
internet using their speech and then can navigate the website using voice commands. The software will read
out the content of the inbox to the user thus making it more accessible. This feature will not only help the
visually blind but also allow other people to access the internet with ease and eliminate the use of hardware
devices like the keyboard.
The assistant will also help to read emails and avoid human errors. In fact this project will help blind
people to access the email just like the normal people but using the voice commands which will save time as
well as efforts.

VII. CONCLUSION
In this paper, a modular solution is presented to improve web-based accessibility for the visually blind
people. The virtual assistant is operating system independent and also allow user to input data using keyboard
also. Through speech to text and text to speech interfaces, the user can communicate with the system. The
system design is presented and methodology of the email module is current under implementation.

The virtual assistant provides an easy way to access any website for the visually impaired. It
eliminates the need toremember complex keyboard commands or the use of screen readers. The assistant is
not only a great way to interact with thewebsites but also an effective way to do so.

VIII. FUTURE ENHANCEMENT

Currently the application only accepts commands in English language so in future this can be
improved by allowing different language options.
In this project many web modules can be integrated likeFacebook, Wikipedia, YouTube etc.

21968 ijariie.com 380

Vol-9 Issue-6 2023 IJARIIE-ISSN(O)-2395-4396

VI. REFERENCES

[1] Virtual Assistant For Visually Impaired, Vinayak Iyer, Sahil Sheth
https://ieeexplore.ieee.org/document/9137874.

[2] ABYS (Always by Your Side): A Virtual Assistant forVisually Impaired Persons, By Authors: Md.
RakibuzSultan, Md Moinul Hoque 18 December 2019 to 20 December 2019
https://ieeexplore.ieee.org/document/9038603.

[3] Voice Assistant For Visually Blind People, By A.Karthik, V. Karthick Raja https://sci-
hub.se/10.1109/IC3IoT.2018.8668188.

[4] Satyabhama Institute Of Technology And Science, ByAuthors: Neeraj Jayaram, Akash S
https://sist.sathyabama.ac.in/sist_naac/documents/1.3. 4/1822-b.e-cse-batchno-10.pdf.

[5] Digital Assistant For The Blind People https://sci-hub.se/10.1109/I2CT.2017.8226327.

[6] Vision-Cap: Virtual Eye And Assistant For Blind, By BITS Pilani Dubai
https://www.jamesdysonaward.org/enAE/2019/project/visioncap-virtual-eye-assistant-
forblind/#:~:text=What%20it%20does-
,VisionCap%20is%20a%20Smart%20AI%20Based%
20Virtual%20Eye%20%26%20Assistant%20for,in%2 0using%20basic%20smartphone%20features.

21968 ijariie.com 381