Pir Mehr Ali Shah

Arid Agriculture University Rawalpindi

University Institute of Information Technology

FYP I – Comprehensive
Survey Report of Your FYP
Project

Project Title: SpeakSign

Supervisor: Miss Bushra Hamid Project No:______

Sr. No. Registration No. Student Name

1 21-ARID-481 M ZAIN ALI KHAN
2 21-ARID-454 M AWAIS SALEEM
3 21-ARID-515 SYED ASNEEF ALI

a. Project Complexity:
The development of the SpeakSigns app involves a moderate to high complexity level due to the
integration of various technical components, each serving distinct functions that enhance the user
experience. Below is a comprehensive breakdown of the complexities associated with each key module
of the application:

1. Daily Sign Language Challenges

 Complexity Level: Moderate

 Description: This feature offers users a daily word or phrase in sign language, paired with a video or image
demonstrating the sign. The backend system is designed to dynamically update the challenge each day,
keeping the content fresh and engaging. This daily interaction encourages users to consistently practice their
sign language skills and helps establish a routine.
 Key Technical Components:
o Backend Media Storage: A robust storage system is essential for hosting the various multimedia
content associated with each daily challenge. This storage must be scalable to accommodate a
growing library of sign language videos and images.
o Push Notifications: To maximize user engagement, push notifications will remind users about the
daily challenge. This requires the implementation of a reliable notification service that ensures timely
reminders without overwhelming users.
o Progress Tracking: Users will have the ability to track their progress over time. This feature
necessitates a database to log user interactions, challenges completed, and improvements observed,
fostering a sense of accomplishment.
 Challenges:
o Seamless Daily Updates: Ensuring that the daily sign language challenges are updated without lags
or delays is critical for maintaining user interest. The system needs to handle the scheduling and
execution of these updates efficiently.
o User Engagement: Maintaining user engagement through gamification strategies, such as rewards
for completing challenges or streaks for consecutive days of practice, is essential. Developing a
balanced approach that motivates users while preventing burnout is a key consideration.

2. Custom Sign Language Keyboard

UIIT/BS/FYP/Form-02 Page 1 of 11
  Complexity Level: High
 Description: This feature allows users to install a custom keyboard that includes sign language symbols or
videos. Users can utilize this keyboard for communication within the app or as an input method in other
applications, facilitating seamless interaction.
 Key Technical Components:
o Custom Keyboard Development: Creating a custom keyboard compatible with both Android and
iOS platforms requires expertise in mobile app development and adherence to platform-specific
guidelines.
o Integration of Video and Text Input: The keyboard should support various modes of
communication, including both video and text-based input. This will require careful design and
implementation to ensure that users can easily switch between input methods.
o Interoperability with Messaging Platforms: Smooth integration with popular messaging and social
media platforms is crucial for enhancing user experience. This involves working with APIs of these
platforms to allow users to share signs or messages seamlessly.
 Challenges:
o Cross-Platform Compatibility: Achieving consistent performance and functionality across different
devices and operating systems presents a significant challenge. Extensive testing will be required to
ensure that the keyboard works flawlessly on various screen sizes and orientations.
o User Interface Design: Creating an intuitive and efficient user interface for the custom keyboard is
paramount. Users should be able to quickly look up signs and use them in their conversations without
feeling overwhelmed by the layout.

3. Text-to-Sign Conversion

 Complexity Level: High

 Description: This innovative feature allows users to input text, which the app then converts into
corresponding sign language videos. This real-time translation will help users effectively bridge the gap
between written language and sign language, promoting better communication.
 Key Technical Components:
o Text Recognition and Mapping: The development of an efficient algorithm that recognizes and
maps text to the appropriate sign language videos is vital. This will involve creating a comprehensive
database of signs and their corresponding text entries.
o Scalable Video Retrieval System: To ensure quick access to sign language videos during
conversion, a scalable and efficient retrieval system must be implemented. This involves optimizing
the backend infrastructure for fast querying and retrieval.
 Challenges:
o Building a Comprehensive Dictionary: Developing and maintaining an extensive dictionary of
signs that includes various contexts and nuances is a considerable challenge. Continuous updates and
user feedback will be necessary to keep the dictionary accurate and relevant.
o Ensuring Fast Conversion: Providing fast conversion times is critical for user satisfaction. Users
expect real-time results; thus, the application must be optimized for speed while handling complex
queries.

4. Overall Project Challenges

 Accessibility and Usability: Ensuring the app is accessible and user-friendly for diverse users, including
both tech-savvy individuals and beginners, is crucial. This involves rigorous user testing and feedback
collection to refine the app's design and functionality.
 Performance Optimization: As the app's content library grows, maintaining optimal performance becomes
increasingly important. The development team must implement efficient algorithms and manage data storage
effectively to prevent slowdowns.
 Cross-Device Experience: Delivering a smooth user experience across various devices and operating
systems is essential for user retention. This requires thorough testing on different devices to identify and
resolve potential issues, ensuring that all users enjoy a consistent experience.

UIIT/BS/FYP/Form-02 Page 2 of 11
 b. Technological Aspects:
The SpeakSigns app is designed as an Android-only solution, leveraging the Android ecosystem’s full
potential to deliver an innovative and interactive experience for users looking to learn and practice sign
language. This section explores the technologies used for front-end development, back-end
infrastructure, data storage, media handling, and security, focusing on scalability, performance, and
accessibility.

1. Frontend Technologies

The frontend of SpeakSigns is critical to ensuring users have a seamless experience when interacting with the app.
It must not only be visually appealing but also intuitive and easy to navigate.

 Programming Language: Kotlin

Kotlin is the preferred language for Android development due to its concise syntax, null safety, and modern
programming features. It is fully interoperable with Java and reduces the potential for bugs, especially null
pointer exceptions, which can occur frequently in Java. Kotlin’s concise code also leads to faster
development and better readability.
o Advantages of Kotlin over Java:
 Null Safety: Prevents common null pointer exceptions.
 Coroutines: Facilitates asynchronous programming, which is critical for handling tasks such
as loading videos or interacting with the database without blocking the main thread.
 Less Boilerplate Code: Kotlin reduces the amount of code, making it easier to maintain.
 Development Environment: Android Studio
Android Studio is the official Integrated Development Environment (IDE) for Android app development. It
includes powerful tools for coding, testing, debugging, and UI design. Android Studio provides the following
advantages:
o Layout Editor: Visual editor for building responsive user interfaces, including drag-and-drop
support for UI components.
o Emulator: Allows testing on various device configurations and Android versions.
o Performance Profiler: Helps track app performance, including CPU usage, memory allocation, and
network traffic.
 User Interface Design
The app’s user interface (UI) is designed using Material Design guidelines, which are the standard for
Android applications. Material Design offers a set of rules for colors, shapes, interactions, and layout to
ensure a consistent, modern, and user-friendly experience.
o Key UI Components:
 ConstraintLayout: This is the most flexible layout system for creating responsive UIs that
adapt to different screen sizes. The layout will be optimized for both portrait and landscape
orientations.
 Custom Animations: Animations will be used for transitions between different activities or
to provide feedback for user actions. Android’s MotionLayout and View animations will
ensure a smooth and engaging experience.
 Interactive Buttons: Buttons for features such as “Start Challenge” or “Convert Text” will be
large, clearly labeled, and easy to interact with, ensuring usability across different devices and
user abilities.

2. Backend Technologies

The backend will manage user data, media files (such as sign language videos), and app logic. The backend also
handles real-time interactions, such as fetching daily challenges and converting text into corresponding sign
language videos.

UIIT/BS/FYP/Form-02 Page 3 of 11
  Firebase as a Backend

Firebase is chosen for its seamless integration with Android, scalability, and wide range of services that can be used
to manage user authentication, data storage, and notifications. Firebase’s serverless architecture allows for fast
development while providing high performance and low latency for real-time updates.

o Firebase Authentication: This service handles user sign-up, login, and authentication. It supports
various authentication methods including email/password, Google, and social logins. Firebase also
ensures secure user management, maintaining privacy and protecting user data.
o Firebase Realtime Database or Firestore: Firestore will be used to store user data such as progress
tracking, daily challenges, and other app-related data. It offers a scalable and flexible NoSQL
database, which ensures that data can be retrieved in real time without significant delay.
 Key Benefits of Firestore:
 Real-time Updates: Changes made in the database are instantly reflected in the app,
which is important for features like daily challenges.
 Offline Support: Firestore enables data synchronization even when the device is
offline. Once the device regains network access, it syncs the data seamlessly.
 Structured Queries: The ability to query and filter data based on different criteria,
such as fetching daily challenges or user statistics.
o Firebase Storage: Firebase Storage will handle the uploading and retrieval of sign language videos.
Given that the app will feature a large collection of videos, Firebase’s storage solution is ideal for
hosting and delivering this content efficiently.
 Advantages:
 Scalability: Firebase Storage is scalable to handle large volumes of video files.
 Security Rules: Built-in security rules ensure that only authenticated users can access
or upload media, maintaining the integrity of the app.
 CDN Integration: Firebase integrates with a global CDN (Content Delivery Network)
to ensure fast delivery of videos to users worldwide.

3. Data Storage and Management

The SpeakSigns app relies heavily on efficient data storage and management for user data, sign language
dictionaries, and video files. The data must be stored securely and retrievable in real time.

 Firestore Database:

Firestore will be the primary storage for user-related data, such as progress tracking, daily challenges, and user
preferences. Firestore’s NoSQL structure allows for flexible data models, making it easier to adapt as the app
evolves with new features or challenges.

o Data Management Strategy:

 User Profiles: Each user will have a unique profile that stores information such as their
progress in learning sign language, completed challenges, and preferences for daily
reminders.
 Sign Language Dictionary: This will be stored as structured data, where each sign is
associated with a corresponding video and metadata (such as the word, difficulty level, and
usage frequency).

4. Media Handling

UIIT/BS/FYP/Form-02 Page 4 of 11
Since the app involves heavy media usage, specifically videos of sign language, efficient media handling is crucial
to ensure the best user experience.

 ExoPlayer for Video Playback

ExoPlayer is an open-source media player for Android that provides an advanced framework for handling video and
audio playback. It supports a variety of media formats and allows for customization, making it perfect for streaming
sign language videos.

o Key Features:
 Adaptive Bitrate Streaming: Videos will adjust their quality based on the user’s internet
connection, ensuring smooth playback even in low-bandwidth conditions.
 Caching: ExoPlayer can cache media locally on the device, allowing users to replay videos
without needing to reload them each time.
o Optimization:
 Videos will be compressed using efficient codecs like H.264 to reduce file sizes while
maintaining high quality.
 Preloading: Videos for daily challenges or frequently accessed signs can be preloaded,
reducing loading times and improving user experience.

5. Notification System

To keep users engaged, the app will include a robust notification system that reminds users about daily challenges,
updates, or new sign language content.

 Firebase Cloud Messaging (FCM)

FCM will be used to send push notifications to users. Notifications are customizable and can be set up based on user
preferences. For example, users can receive reminders to complete their daily challenges or alerts when new content
is added.

o Personalized Notifications: Notifications will be tailored based on user progress, ensuring that users
receive relevant updates. For instance, if a user is consistently practicing a particular set of signs, they
may receive content recommendations or tips on improving their proficiency in that area.
o Scheduling Notifications: The app will allow users to schedule when they want to receive
notifications, ensuring they engage with the app at a time most convenient for them.

6. Security

Ensuring the security and privacy of user data is a top priority for SpeakSigns. Several layers of security will be
implemented to protect sensitive information such as user credentials, progress data, and personal settings.

 Authentication and Authorization: Firebase Authentication ensures that only verified users can access the
app’s features. Additional security measures include multi-factor authentication (MFA) for enhanced account
protection.
 Encryption: All communication between the app and Firebase servers will be encrypted using HTTPS to
prevent interception of sensitive data. Stored data, such as user profiles, will also be encrypted both at rest
and in transit.
 Compliance with GDPR: The app will comply with data privacy regulations such as the General Data
Protection Regulation (GDPR), ensuring that users have control over their data and can request data
deletion if desired.

7. Accessibility Features

UIIT/BS/FYP/Form-02 Page 5 of 11
Accessibility is a core component of SpeakSigns, ensuring that the app can be used by individuals with different
abilities.

 Voice Instructions: For users with visual impairments or those who prefer auditory guidance, voice
instructions will guide them through app functionalities.
 Simple Navigation: The app will offer a simplified interface with large buttons, clear labels, and easily
accessible menus, making it easy for users of all ages and technical skills to navigate the app.

c. Potential Impact on Society:

The SpeakSigns app has the potential to significantly impact society by enhancing communication for
individuals who are deaf or hard of hearing, promoting inclusivity, raising awareness about sign
language, and fostering connections between different communities. This section delves into various
dimensions of its impact on individuals, educational institutions, and society at large.

1. Enhancing Communication for the Deaf and Hard of Hearing Community

One of the primary goals of the SpeakSigns app is to bridge the communication gap between the hearing and deaf
communities. By providing a platform that enables users to learn and practice sign language, the app promotes better
understanding and communication.

 Empowerment through Language : Language is a fundamental tool for expression and connection. By
learning sign language through SpeakSigns, users can gain a greater sense of empowerment, enabling them
to communicate more effectively in various contexts, such as at work, in social settings, or during emergency
situations. This empowerment can lead to increased confidence and independence among users.
 Facilitating Everyday Interactions : The app will equip users with essential signs for daily interactions,
such as greetings, asking for help, or expressing needs. By familiarizing users with commonly used signs, the
app can facilitate smoother communication in public spaces, reducing misunderstandings and fostering more
inclusive environments.
 Real-time Practice and Challenges : The app's daily challenges and interactive features will encourage
users to practice their skills regularly, which can help them retain and enhance their sign language abilities.
As users become more proficient, they will be better positioned to engage with deaf individuals, thus
enriching their lives and experiences.

2. Promoting Inclusivity and Awareness

The SpeakSigns app serves not only as a learning tool but also as a platform to promote inclusivity and awareness
about the deaf and hard of hearing community.

 Educational Resource for the General Public : The app can be an invaluable resource for hearing
individuals interested in learning about sign language and the culture of the deaf community. By increasing
the number of individuals who can communicate using sign language, the app fosters a more inclusive
society.
 Workshops and Community Engagement : The app can facilitate community workshops and events where
users can come together to practice sign language and engage with the deaf community. These events can
help break down barriers and encourage friendships, understanding, and respect among participants.
 Collaboration with Educational Institutions : Partnering with schools and universities can enhance the
app's reach. By integrating sign language learning into educational curriculums, SpeakSigns can raise
awareness and foster respect for the deaf community among younger generations. This proactive approach
can help eliminate stigma and promote a culture of acceptance.
 Social Media Campaigns and Content Creation : The app can also include features for users to create and
share content related to sign language, such as videos of themselves performing signs or discussing their
experiences learning sign language. By leveraging social media platforms, SpeakSigns can create viral
campaigns that increase visibility for the deaf community and promote understanding among a broader
audience.

UIIT/BS/FYP/Form-02 Page 6 of 11
3. Addressing Mental Health and Social Isolation

Many individuals who are deaf or hard of hearing experience social isolation due to communication barriers. The
SpeakSigns app aims to mitigate these feelings of isolation by providing opportunities for social engagement.

 Creating Connections : The app’s community features can help users connect with others who are learning
sign language or who are part of the deaf community. These connections can lead to friendships, support
networks, and opportunities for collaboration on projects, thus combating social isolation.
 Mental Health Support : The app can include resources related to mental health awareness and provide
access to counseling services tailored for individuals who may experience anxiety or depression due to
communication difficulties. By addressing these mental health concerns, SpeakSigns can promote overall
well-being among its users.
 Empowering Caregivers and Family Members : The app can serve as a valuable resource for the families
and caregivers of deaf individuals. By learning sign language, caregivers can communicate more effectively,
providing emotional support and understanding for their loved ones. This empowerment can lead to stronger
family bonds and better care for individuals with hearing impairments.

4. Economic Impact and Workforce Inclusion

The SpeakSigns app has the potential to impact the economy by promoting workforce inclusion for individuals who
are deaf or hard of hearing.

 Increasing Employment Opportunities : By equipping users with sign language skills, SpeakSigns can
help them access a broader range of job opportunities. Employers who prioritize inclusivity can benefit from
hiring skilled individuals who can communicate effectively with both deaf and hearing coworkers.
 Training and Skill Development : The app can also feature resources for professional development,
including sign language interpretation skills or vocational training specific to industries with high demand
for bilingual employees. This focus on skill development can lead to higher employability rates for deaf
individuals and promote a more diverse workforce.
 Encouraging Inclusive Business Practices : The app can encourage businesses to adopt inclusive practices,
such as providing sign language interpreters or training staff in basic sign language. As companies become
more aware of the importance of inclusivity, they can attract a wider customer base and improve their public
image.

5. Long-term Societal Change

The ultimate vision for the SpeakSigns app extends beyond individual benefits. It aims to contribute to long-term
societal change regarding perceptions and interactions with the deaf community.

 Changing Perceptions : By promoting awareness and understanding of sign language, SpeakSigns can
contribute to changing societal perceptions of individuals with hearing impairments. As more people learn
and use sign language, the stigma associated with deafness can diminish, leading to a more accepting and
understanding society.
 Advocacy and Policy Change : The app can serve as a platform for advocacy, encouraging users to engage
in conversations about the rights and needs of the deaf community. By raising awareness about the barriers
faced by deaf individuals, the app can contribute to policy changes that promote accessibility and inclusivity
in various sectors, including education, healthcare, and employment.
 Global Reach and Cultural Exchange : As the app expands globally, it can facilitate cultural exchange
between different sign languages and deaf cultures. Users can learn about various sign languages, fostering a
deeper appreciation for the diversity within the deaf community worldwide. This global perspective can
enhance cross-cultural understanding and collaboration, contributing to a more interconnected and
supportive global community.

d. Benchmarking:

UIIT/BS/FYP/Form-02 Page 7 of 11
1. Literature Review: Current Problems in Sign Language Learning

The need for effective sign language learning tools is underscored by several studies that point to communication
barriers between the deaf and hearing communities. Traditional learning methods, such as in-person classes and
printed materials, can be cumbersome, time-consuming, and costly. Below are key findings from literature regarding
the current state of sign language learning tools:

 Accessibility Issues: Many individuals interested in learning sign language face geographical and financial
barriers to accessing quality educational resources.
 Static Learning Methods: Most applications utilize static content (videos, text) without interactivity, which
can lead to disengagement and poor retention of learned material.
 Limited Feedback Mechanisms: Existing applications often lack real-time feedback on user performance,
making it challenging for learners to assess their proficiency effectively.
 Insufficient Language Coverage: Many sign language apps focus primarily on American Sign Language
(ASL) while neglecting other regional sign languages, limiting their audience.
 Lack of Community Engagement: The absence of interactive and social elements can hinder users'
motivation to practice and learn effectively.

2. Competitor Analysis

Feature/Attribute The ASL SpeakSign

SignSchool Lingvano Marlee Signs
App

Video Tutorials ✔ ✗ ✔ ✔ ✔

✔ ✗
ASL Dictionary ✔ ✔ ✔

✗ ✗ ✗
Celebrity Content ✔ ✗

✗ ✗ ✗
Flashcards ✔ ✔

✗ ✔ ✗ ✔
Vocabulary Lessons ✔

✗ ✗ ✗ ✗
Live Practice ✗

✗ ✗ ✔ ✔
Interactive Lessons ✔

✗ ✗ ✗ ✗
Quizzes ✔

✗ ✗ ✔ ✗
Daily Reminders ✔

ASL to Text/Speech ✗ ✗ ✗ ✗
✔
Conversion
✗ ✗ ✔
Simple Lessons ✔ ✔

✗ ✗ ✗ ✔
Everyday Phrases ✔

UIIT/BS/FYP/Form-02 Page 8 of 11
3. Identifying the Gaps

The comparison highlights several gaps in current sign language applications:

 Real-Time Feedback Mechanisms: Existing apps primarily offer passive learning experiences without real-
time feedback, making it hard for users to correct mistakes. SpeakSigns will leverage AI technology for
gesture recognition to provide instant feedback on users' sign accuracy.
 Limited Coverage of Sign Languages: The overwhelming focus on ASL restricts access for non-English-
speaking audiences. SpeakSigns will address this by supporting multiple sign languages, making it a
globally relevant tool.
 Engagement through Static Content: Current apps often rely on unengaging methods like videos or
flashcards. SpeakSigns will incorporate gamification and interactive learning modules to enhance user
engagement and retention.
 Community Support and Engagement: Many apps lack community features, which are crucial for
motivation and practice. SpeakSigns will include social elements, such as forums and challenges, to promote
interaction and learning among users.
 Absence of Immersive Learning Experiences: Current solutions rarely utilize immersive technologies like
augmented reality (AR) to enhance learning. SpeakSigns will incorporate AR elements, allowing users to
practice in simulated environments, improving the learning experience.

e. Project Features List:

The SpeakSigns app is designed to revolutionize sign language learning through innovative features that
enhance user engagement, accessibility, and interactivity. Below is a comprehensive list of features that will
be integrated into the app:

1. Multi-Language Support

 Diverse Sign Languages: Support for multiple sign languages (e.g., ASL, BSL, LSF, etc.) to cater to a
global audience.
 Localized Content: Content tailored to specific regions and cultures to ensure relevance and accuracy in
learning.

2. Interactive Learning Modules

 Video Tutorials: High-quality video lessons demonstrating various signs and their contextual uses.
 Gesture Recognition: AI-driven technology that analyzes user gestures and provides real-time feedback on
accuracy.
 Gamified Learning: Interactive games and quizzes to reinforce learning and make practice enjoyable.

3. Community Engagement

 Social Forums: Platforms for users to connect, share experiences, and practice signing with each other.
 Challenges and Competitions: Regular challenges to motivate users and allow them to showcase their
skills.

4. Augmented Reality Integration

 AR Practice Mode: Immersive AR features allowing users to practice signing in simulated environments.
 Virtual Sign Language Conversations: Engage with virtual avatars in AR to practice real-life
conversations in sign language.

UIIT/BS/FYP/Form-02 Page 9 of 11
5. Personalization Features

 User Profiles: Customizable profiles that track progress, achievements, and learning preferences.
 Personalized Learning Pathways: Adaptive learning paths that adjust based on user performance and
interests.

6. Resources and Tools

 Sign Language Dictionary: Comprehensive dictionary with videos demonstrating each sign and its
variations.
 Phrasebook: Common phrases and expressions to aid in daily communication.
 Cultural Context Lessons: Insights into the cultural significance and context behind various signs and
gestures.

7. Progress Tracking and Analytics

 Learning Analytics: Insights into user performance, strengths, and areas for improvement.
 Milestone Tracking: Celebrate user achievements with badges and rewards for completing lessons and
challenges.

8. Offline Access

 Downloadable Content: Ability to download lessons and resources for offline access, ensuring learning is
possible anytime and anywhere.

9. Two-Way Communication

 Speech-to-Sign Translation: Functionality that converts spoken language into sign language, facilitating
communication between hearing and deaf individuals.
 Sign-to-Speech Translation: Functionality that converts sign language into spoken language, enhancing
inclusivity.

10. Accessibility Features

 User-Friendly Interface: Intuitive design that accommodates users of all ages and technical abilities.
 Text-to-Speech: Voice narration for instructions and content, aiding users who may have difficulties with
reading.

UIIT/BS/FYP/Form-02 Page 10 of 11
 FYP Project Report Evaluation: (For Official use only)

Criteria Good Normal Inferior

Project Complexity
Technological Aspect
Potential Impact on Society
Benchmarking
Project Features

Suggestions/Remarks:

Name of Examiner Date Signature

Definition of Terms:

i. Project Complexity: The project complexity is referred to as the degree of significant

contribution that a group of students will put in the design and development of project,
spanning over two academic semesters. Secondly, determine if the domain of the project
marks the standard of complexity required from a bachelor’s student degree, as this
project will determine the skills they learnt throughout the degree.

ii. Technological Aspects: Technological aspects of the project means tools/technologies and
language(s) used to develop it.

iii. Potential Impact on Society: Determine how much impact the product could have in its
stated strategy for a society or community/focused group.

iv. Benchmarking: The proposed project should be compared with existing similar type of
works. A comparison table is more helpful for comparative view, listing features of existing
works and proposed project.

v. Project Features: Verify that the features mentioned are complete and significant enough
for an FYP project.

UIIT/BS/FYP/Form-02 Page 11 of 11