Pbi. University Innovates Technology to Transform Spoken Punjabi into Indian Sign Language, Empowering the Hearing-Impaired
Patiala, August 16
In a groundbreaking step toward inclusive communication, Punjabi University, Patiala, has developed an innovative system that converts spoken Punjabi into Indian Sign Language (ISL) in real time. This transformative technology, supported by funding from the Department of Science and Technology (DST), Government of India, aims to bridge the communication gap between hearing-impaired and non-hearing-impaired individuals, promoting greater accessibility and inclusion.
Led by Dr. Williamjeet Singh, Assistant Professor in the Department of Computer Science and Engineering, and supported by Dr. Amandeep Singh, Research Project Assistant, the project integrates advanced speech recognition, natural language processing, and synthetic animation to translate spoken Punjabi into ISL gestures.
watch……Punjabi University Professor Dr. Satinder Kumar Secures ICSSR Research Project Grant
Dr. Williamjeet Singh explained that the system is designed to empower the hearing-impaired community by facilitating seamless communication in critical areas such as education, healthcare, and public services. He noted that communication is fundamental to human interaction, enabling the exchange of emotions, ideas, and information. While speech is the primary mode for most, hearing /
-impaired individuals rely on sign language, which uses hand movements and facial expressions. However, the lack of a universal sign language and limited ISL knowledge among the general population often create barriers, leading to exclusion and restricted opportunities for the hearing-impaired.
The newly developed system addresses these challenges by leveraging advanced technologies. It uses the Hamburg Notation System (HamNoSys) to represent sign gestures and the Signing Gesture Markup Language (SiGML) to generate animated 3D sign sequences. The system incorporates an ISL dictionary that includes both manual (hand gestures) and non-manual (facial expressions and body movements) features, ensuring accurate and natural sign representation.
Dr. Amandeep Singh elaborated that the system employs two approaches for sign language representation: synthetic animation and video-based sign presentation. A novel optimization technique enhances the accuracy of converting spoken words into structured ISL scripts. Rigorous testing has confirmed the system’s efficiency, accuracy, and usability, making it a reliable platform for real-time communication. Available as a Progressive Web Application, the system supports multilingual speech input and is accessible on both web and mobile platforms. Beyond daily communication, it serves as an educational tool, enabling users to learn new signs and enhance their communication skills.
Dr. Amandeep Singh further stated that by enabling real-time conversion of spoken Punjabi to ISL, the system enhances accessibility in key sectors like education, healthcare, and public services. The goal is to empower hearing-impaired individuals to fully participate in society. The research team plans to expand the system to support additional languages and develop a dedicated mobile app to reach a broader audience.
Vice Chancellor Dr. Jagdeep Singh congratulated the research team, emphasizing that this innovation has the potential to transform the lives of millions of hearing-impaired individuals in India and beyond. He highlighted that the project showcases the transformative power of technology, as implemented at Punjabi University, in addressing societal challenges. By converting spoken words into visual signs, this system brings the world closer to the hearing-impaired, amplifying their needs and aspirations through the language of signs.
