Students at RNS Institute of Technology, within the IEEE Bangalore Section of India, will be working on their project “Real Time Sign Language Conversion to Audio and Text Form for the Hearing Impaired" throughout 2013.
The objective of this EPICS project is to design and develop a cost-effective gadget that can convert sign language used by people with hearing and speech impairments to a Word document. Upon successful completion of converting the sign language to a text document, the Word document would be converted to audio. This gadget has been conceived to provide quick information exchange between people with hearing and speech impairments and those around them who may not know sign language. Ultimately, the device can be viewed as a translator of sign language.
University students from the RNS Institute of Technology student branch will collaborate with students from Sri Aurobindo Memorial School in Bangalore, Karnataka. They will aid in collecting images of hand gestures from children at RV Integrated School for the Hearing Impaired, which is the nonprofit collaborator for this project.
In order to capture the gestures onto the gadget, the project group will utilize a web camera. This webcam will be connected to an FPGA where further processing can be performed. The FPGA would act as the real-time image processor to analyze the image data and to comprehend the gestures made. This could be later converted to text and displayed or converted to speech and sound. The kinds of pictures the webcam would need to take photos of for processing include: extraction of hand from background, orientation of hand, which fingers are open, and implementing gestures for common-day activities. The device would collect these common gestures to better recognize them.
Portability is also planned for the prototype. The final project is envisioned to be a handheld device that helps the user with hearing and speech impairments to communicate immediately in real time. The device will be interfaced with a computer and used for reporting and documentation by the user, in addition to audio that gives the user speech. The device hopes to help people with hearing and speech impairments in everyday situations more easily, such as in the medical field, at police stations, and with job opportunities for clients in offices. The device can also be useful in public places such as parks, museums, bus and rail stations, banks, post offices, and other government offices.