This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
Journal of Intelligent and Fuzzy Systems, Jun 14, 2023
Sign language recognition is a significant cross-modal way to fill the communication gap between ... more Sign language recognition is a significant cross-modal way to fill the communication gap between deaf and hearing people. Automatic Sign Language Recognition (ASLR) translates sign language gestures into text and spoken words. Several researchers are focusing either on manual gestures or non-manual gestures separately; a rare focus is on concurrent recognition of manual and non-manual gestures. Facial expression and other body movements can improve the accuracy rate, as well as enhance signs’ exact meaning. The current paper proposes a Multimodal –Sign Language Recognition (MM-SLR) framework to recognize non-manual features based on facial expressions along with manual gestures in Spatio temporal domain representing hand movements in ASLR. Our proposed architecture has three modules, first, a modified architecture of YOLOv5 is defined to extract faces and hands from videos as two Regions of Interest. Second, refined C3D architecture is used to extract features from the hand region and the face region, further, feature concatenation of both modalities is applied. Lastly, LSTM network is used to get spatial-temporal descriptors and attention-based sequential modules for gesture classification. To validate the proposed framework we used three publically available datasets RWTH-PHONIX-WEATHER-2014T, SILFA and PkSLMNM. Experimental results show that the above-mentioned MM-SLR framework outperformed on all datasets.
Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It... more Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It creates, collaborates and innovates to engage students in hands-on projects and develop a large range of skills. In other words makerspace is place to make choice, create and connect. Community based workshops and DIY (Do It Yourself) projects are promoting the practice of peer production and fabrication. Existent science and multimedia applications are done by tinkering, the idea of university maker space is infantile and young but getting well-liked in global world. Universities are working a lot in terms to make their undergraduates and graduating engineers able to harness innovation and creativeness to stay competent and applicable in market or industry with increasingly growing requirements. This paper delves into a case study where multimedia lab is converted into a Makerspace SMMS (Space to Make Multimedia Systems) by assigning real time projects, related to the curriculum needs, and making them able to identify problem, construct model, become skilled at to apply, rework ideas, and share acquaintance with others. Here teacher acts as a “Maker Teacher” who designs the concepts of projects and decides the moves. Teacher is also redesigning their labs into a makerspace sloping multimedia lab. Engineering project ideas and prototype of real world scenarios are also part of the multimedia makerspace. In present study the students are engaged in inquiring models, building codes and architectural blueprints and their processes by hacking and decoding the older versions and understanding the prototypes of the real time projects. Drawing upon observations, videos, interviews and other documentation, author concludes with a qualitative discussion and results of the connotation of the findings for this embryonic and emergent field. Outcomes of applied makerspace in no doubt effective and producing better results to blend digital and physical technologies to work better in practical and theoretical state of affairs.
Sign language recognition is a significant cross-modal way to fill the communication gap between ... more Sign language recognition is a significant cross-modal way to fill the communication gap between deaf and hearing people. Automatic Sign Language Recognition (ASLR) translates sign language gestures into text and spoken words. Several researchers are focusing either on manual gestures or non-manual gestures separately; a rare focus is on concurrent recognition of manual and non-manual gestures. Facial expression and other body movements can improve the accuracy rate, as well as enhance signs’ exact meaning. The current paper proposes a Multimodal –Sign Language Recognition (MM-SLR) framework to recognize non-manual features based on facial expressions along with manual gestures in Spatio temporal domain representing hand movements in ASLR. Our proposed architecture has three modules, first, a modified architecture of YOLOv5 is defined to extract faces and hands from videos as two Regions of Interest. Second, refined C3D architecture is used to extract features from the hand region a...
It is a challenging task to interpret sign language automatically, as it comprises high-level vis... more It is a challenging task to interpret sign language automatically, as it comprises high-level vision features to accurately understand and interpret the meaning of the signer or vice versa. In the current study, we automatically distinguish hand signs and classify seven basic gestures representing symbolic emotions or expressions like happy, sad, neutral, disgust, scared, anger, and surprise. Convolutional Neural Network is a famous method for classifications using vision-based deep learning; here in the current study, proposed transfer learning using a well-known architecture of VGG16 to speed up the convergence and improve accuracy by using pre-trained weights. We obtained a high accuracy of 99.98% of the proposed architecture with a minimal and low-quality data set of 455 images collected by 65 individuals for seven hand gesture classes. Further, compared the performance of VGG16 architecture with two different optimizers, SGD, and Adam, along with some more architectures of Alex...
Sign language fills the communication gap for people with hearing and speaking ailments. It inclu... more Sign language fills the communication gap for people with hearing and speaking ailments. It includes both visual modalities, manual gestures consisting of movements of hands, and non-manual gestures incorporating body movements including head, facial expressions, eyes, shoulder shrugging, etc. Previously both gestures have been detected; identifying separately may have better accuracy, but much communicational information is lost. A proper sign language mechanism is needed to detect manual and non-manual gestures to convey the appropriate detailed message to others. Our novel proposed system contributes as Sign Language Action Transformer Network (SLATN), localizing hand, body, and facial gestures in video sequences. Here we are expending a Transformer-style structural design as a “base network” to extract features from a spatiotemporal domain. The model impulsively learns to track individual persons and their action context in multiple frames. Furthermore, a “head network” emphasizes hand movement and facial expression simultaneously, which is often crucial to understanding sign language, using its attention mechanism for creating tight bounding boxes around classified gestures. The model’s work is later compared with the traditional identification methods of activity recognition. It not only works faster but achieves better accuracy as well. The model achieves overall 82.66% testing accuracy with a very considerable performance of computation with 94.13 Giga-Floating Point Operations per Second (G-FLOPS). Another contribution is a newly created dataset of Pakistan Sign Language for Manual and Non-Manual (PkSLMNM) gestures.
It is in human nature to keep certain details intimate. This confidential information, if made pu... more It is in human nature to keep certain details intimate. This confidential information, if made public, can result in annoyance, humiliation, and even devastating loss. The latter one holds true for individuals as well as organizations. In the case of ubiquitous environment, system adapts to the context of user's actions and intentions and the collection of data becomes pervasive. Thus, privacy turns out to be a vital aspect. This study aims to determine privacy awareness among people in ubiquitous environment. For this purpose, a survey was conducted. This survey was based on a questionnaire. The results show that people consider themselves quite aware, but, more concerned with privacy issues in ubiquitous environment. Also, a significant number of people admit to not taking privacy measures. The analysis was carried out on the basis of current status of participants in the university, namely, Undergraduate students, Graduate students, and Faculty members.
Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It... more Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It creates, collaborates and innovates to engage students in hands-on projects and develop a large range of skills. In other words makerspace is place to make choice, create and connect. Community based workshops and DIY (Do It Yourself) projects are promoting the practice of peer production and fabrication. Existent science and multimedia applications are done by tinkering, the idea of university maker space is infantile and young but getting well-liked in global world. Universities are working a lot in terms to make their undergraduates and graduating engineers able to harness innovation and creativeness to stay competent and applicable in market or industry with increasingly growing requirements. This paper delves into a case study where multimedia lab is converted into a Makerspace SMMS (Space to Make Multimedia Systems) by assigning real time projects, related to the curriculum needs, and making them able to identify problem, construct model, become skilled at to apply, rework ideas, and share acquaintance with others. Here teacher acts as a “Maker Teacher” who designs the concepts of projects and decides the moves. Teacher is also redesigning their labs into a makerspace sloping multimedia lab. Engineering project ideas and prototype of real world scenarios are also part of the multimedia makerspace. In present study the students are engaged in inquiring models, building codes and architectural blueprints and their processes by hacking and decoding the older versions and understanding the prototypes of the real time projects. Drawing upon observations, videos, interviews and other documentation, author concludes with a qualitative discussion and results of the connotation of the findings for this embryonic and emergent field. Outcomes of applied makerspace in no doubt effective and producing better results to blend digital and physical technologies to work better in practical and theoretical state of affairs.
The use of Bluetooth Low Energy (BLE) beacons is becoming widespread for context-aware learning e... more The use of Bluetooth Low Energy (BLE) beacons is becoming widespread for context-aware learning environments. This technology can be used in indoor and outdoor settings such as museums, shops, home and schools in order to identify the location of learners and their contexts. Specifically, in the constructivist view of learning, learners can use such sensing technology for constructing knowledge and experiencing learning at any time, anywhere. However, exploration of such technology has been limited in the constructivist context-aware ubiquitous learning (U-learning) literature. For this purpose, the authors evaluated two android-based U-learning applications: one was an outdoor learning environment in a garden, the other was about locating books in a library. The applications were implemented and tested with school and university students. The qualitative results are presented in this article, and show how such sensing technology is promising for both pedagogical environments.
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
Journal of Intelligent and Fuzzy Systems, Jun 14, 2023
Sign language recognition is a significant cross-modal way to fill the communication gap between ... more Sign language recognition is a significant cross-modal way to fill the communication gap between deaf and hearing people. Automatic Sign Language Recognition (ASLR) translates sign language gestures into text and spoken words. Several researchers are focusing either on manual gestures or non-manual gestures separately; a rare focus is on concurrent recognition of manual and non-manual gestures. Facial expression and other body movements can improve the accuracy rate, as well as enhance signs’ exact meaning. The current paper proposes a Multimodal –Sign Language Recognition (MM-SLR) framework to recognize non-manual features based on facial expressions along with manual gestures in Spatio temporal domain representing hand movements in ASLR. Our proposed architecture has three modules, first, a modified architecture of YOLOv5 is defined to extract faces and hands from videos as two Regions of Interest. Second, refined C3D architecture is used to extract features from the hand region and the face region, further, feature concatenation of both modalities is applied. Lastly, LSTM network is used to get spatial-temporal descriptors and attention-based sequential modules for gesture classification. To validate the proposed framework we used three publically available datasets RWTH-PHONIX-WEATHER-2014T, SILFA and PkSLMNM. Experimental results show that the above-mentioned MM-SLR framework outperformed on all datasets.
Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It... more Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It creates, collaborates and innovates to engage students in hands-on projects and develop a large range of skills. In other words makerspace is place to make choice, create and connect. Community based workshops and DIY (Do It Yourself) projects are promoting the practice of peer production and fabrication. Existent science and multimedia applications are done by tinkering, the idea of university maker space is infantile and young but getting well-liked in global world. Universities are working a lot in terms to make their undergraduates and graduating engineers able to harness innovation and creativeness to stay competent and applicable in market or industry with increasingly growing requirements. This paper delves into a case study where multimedia lab is converted into a Makerspace SMMS (Space to Make Multimedia Systems) by assigning real time projects, related to the curriculum needs, and making them able to identify problem, construct model, become skilled at to apply, rework ideas, and share acquaintance with others. Here teacher acts as a “Maker Teacher” who designs the concepts of projects and decides the moves. Teacher is also redesigning their labs into a makerspace sloping multimedia lab. Engineering project ideas and prototype of real world scenarios are also part of the multimedia makerspace. In present study the students are engaged in inquiring models, building codes and architectural blueprints and their processes by hacking and decoding the older versions and understanding the prototypes of the real time projects. Drawing upon observations, videos, interviews and other documentation, author concludes with a qualitative discussion and results of the connotation of the findings for this embryonic and emergent field. Outcomes of applied makerspace in no doubt effective and producing better results to blend digital and physical technologies to work better in practical and theoretical state of affairs.
Sign language recognition is a significant cross-modal way to fill the communication gap between ... more Sign language recognition is a significant cross-modal way to fill the communication gap between deaf and hearing people. Automatic Sign Language Recognition (ASLR) translates sign language gestures into text and spoken words. Several researchers are focusing either on manual gestures or non-manual gestures separately; a rare focus is on concurrent recognition of manual and non-manual gestures. Facial expression and other body movements can improve the accuracy rate, as well as enhance signs’ exact meaning. The current paper proposes a Multimodal –Sign Language Recognition (MM-SLR) framework to recognize non-manual features based on facial expressions along with manual gestures in Spatio temporal domain representing hand movements in ASLR. Our proposed architecture has three modules, first, a modified architecture of YOLOv5 is defined to extract faces and hands from videos as two Regions of Interest. Second, refined C3D architecture is used to extract features from the hand region a...
It is a challenging task to interpret sign language automatically, as it comprises high-level vis... more It is a challenging task to interpret sign language automatically, as it comprises high-level vision features to accurately understand and interpret the meaning of the signer or vice versa. In the current study, we automatically distinguish hand signs and classify seven basic gestures representing symbolic emotions or expressions like happy, sad, neutral, disgust, scared, anger, and surprise. Convolutional Neural Network is a famous method for classifications using vision-based deep learning; here in the current study, proposed transfer learning using a well-known architecture of VGG16 to speed up the convergence and improve accuracy by using pre-trained weights. We obtained a high accuracy of 99.98% of the proposed architecture with a minimal and low-quality data set of 455 images collected by 65 individuals for seven hand gesture classes. Further, compared the performance of VGG16 architecture with two different optimizers, SGD, and Adam, along with some more architectures of Alex...
Sign language fills the communication gap for people with hearing and speaking ailments. It inclu... more Sign language fills the communication gap for people with hearing and speaking ailments. It includes both visual modalities, manual gestures consisting of movements of hands, and non-manual gestures incorporating body movements including head, facial expressions, eyes, shoulder shrugging, etc. Previously both gestures have been detected; identifying separately may have better accuracy, but much communicational information is lost. A proper sign language mechanism is needed to detect manual and non-manual gestures to convey the appropriate detailed message to others. Our novel proposed system contributes as Sign Language Action Transformer Network (SLATN), localizing hand, body, and facial gestures in video sequences. Here we are expending a Transformer-style structural design as a “base network” to extract features from a spatiotemporal domain. The model impulsively learns to track individual persons and their action context in multiple frames. Furthermore, a “head network” emphasizes hand movement and facial expression simultaneously, which is often crucial to understanding sign language, using its attention mechanism for creating tight bounding boxes around classified gestures. The model’s work is later compared with the traditional identification methods of activity recognition. It not only works faster but achieves better accuracy as well. The model achieves overall 82.66% testing accuracy with a very considerable performance of computation with 94.13 Giga-Floating Point Operations per Second (G-FLOPS). Another contribution is a newly created dataset of Pakistan Sign Language for Manual and Non-Manual (PkSLMNM) gestures.
It is in human nature to keep certain details intimate. This confidential information, if made pu... more It is in human nature to keep certain details intimate. This confidential information, if made public, can result in annoyance, humiliation, and even devastating loss. The latter one holds true for individuals as well as organizations. In the case of ubiquitous environment, system adapts to the context of user's actions and intentions and the collection of data becomes pervasive. Thus, privacy turns out to be a vital aspect. This study aims to determine privacy awareness among people in ubiquitous environment. For this purpose, a survey was conducted. This survey was based on a questionnaire. The results show that people consider themselves quite aware, but, more concerned with privacy issues in ubiquitous environment. Also, a significant number of people admit to not taking privacy measures. The analysis was carried out on the basis of current status of participants in the university, namely, Undergraduate students, Graduate students, and Faculty members.
Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It... more Maker space is a powerful form of renovated learning which ethos is based on invents to learn. It creates, collaborates and innovates to engage students in hands-on projects and develop a large range of skills. In other words makerspace is place to make choice, create and connect. Community based workshops and DIY (Do It Yourself) projects are promoting the practice of peer production and fabrication. Existent science and multimedia applications are done by tinkering, the idea of university maker space is infantile and young but getting well-liked in global world. Universities are working a lot in terms to make their undergraduates and graduating engineers able to harness innovation and creativeness to stay competent and applicable in market or industry with increasingly growing requirements. This paper delves into a case study where multimedia lab is converted into a Makerspace SMMS (Space to Make Multimedia Systems) by assigning real time projects, related to the curriculum needs, and making them able to identify problem, construct model, become skilled at to apply, rework ideas, and share acquaintance with others. Here teacher acts as a “Maker Teacher” who designs the concepts of projects and decides the moves. Teacher is also redesigning their labs into a makerspace sloping multimedia lab. Engineering project ideas and prototype of real world scenarios are also part of the multimedia makerspace. In present study the students are engaged in inquiring models, building codes and architectural blueprints and their processes by hacking and decoding the older versions and understanding the prototypes of the real time projects. Drawing upon observations, videos, interviews and other documentation, author concludes with a qualitative discussion and results of the connotation of the findings for this embryonic and emergent field. Outcomes of applied makerspace in no doubt effective and producing better results to blend digital and physical technologies to work better in practical and theoretical state of affairs.
The use of Bluetooth Low Energy (BLE) beacons is becoming widespread for context-aware learning e... more The use of Bluetooth Low Energy (BLE) beacons is becoming widespread for context-aware learning environments. This technology can be used in indoor and outdoor settings such as museums, shops, home and schools in order to identify the location of learners and their contexts. Specifically, in the constructivist view of learning, learners can use such sensing technology for constructing knowledge and experiencing learning at any time, anywhere. However, exploration of such technology has been limited in the constructivist context-aware ubiquitous learning (U-learning) literature. For this purpose, the authors evaluated two android-based U-learning applications: one was an outdoor learning environment in a garden, the other was about locating books in a library. The applications were implemented and tested with school and university students. The qualitative results are presented in this article, and show how such sensing technology is promising for both pedagogical environments.
Uploads
Papers by Sameena Javaid