Students RNS Institute of Technology, within the IEEE Bangalore Section of India, will be working on their project, “Real Time Sign Language Conversion to Audio and Text Form for the Hearing Impaired," throughout 2013.
The objective of this EPICS project is to design and develop a cost effective gadget which can convert sign language used by hearing and speech impaired people to a word document. On successful completion of converting the sign language to a text document, the word document would be converted to audio. This gadget has been conceived to provide quick information exchange between a hearing and speech impaired people and those around them who may not know sign language. Ultimately, the device can be viewed as a translator of sign language.
University students from the RNS Institute of Technology student branch will collaborate with students from Sri Aurobindo Memorial School in Bangalore, Karnataka. They will aid in collecting images of hand gestures from children at RV Integrated School for the Hearing Impaired, which is the non-profit collaborator for this project.
In order to capture the gestures on to the gadget, the project group will utilize a web camera. This web cam will be connected to an FPGA where further processing can be performed. The FPGA would act as the real time image processor to analyze the image data and to comprehend the gestures made. This could be later converted to text and displayed or converted to speech and sound. The kinds of pictures the webcam would need to take photos of for processing include: extraction of hand from background, orientation of hand which fingers are open, and implementing gestures for common day activities. The device would collect these common gestures to better recognize them.
Portability is also planned for the prototype. The final project is envisioned to be a hand held device which helps the hearing and speech impaired user communicate immediately in real time. The device will be interfaced with a computer and be used for reporting and documentation by the user, in addition to audio which gives the user speech. The device hopes to help the hearing and speech impaired in everyday situations more easily, such as in the medical field, at police stations, and with job opportunities for clients in offices. The device can also be useful in public places such as parks, museums, bus and rail stations, banks, post offices, and other government offices.