Student used AI to create program that actually translates sign language to English in real time
Topics: Artificial Intelligence, Technology, Science, India, Viral
Emily Brown
Featured Image Credit: Priyanjali Gupta
Topics: Artificial Intelligence, Technology, Science, India, Viral
An engineering student looking into 'inclusive technology' successfully used AI to create a program which translates sign language into English.
The advancements made in artificial intelligence can be considered terrifying in some instances, but then you hear about the kind of work being done by people like Priyanjali Gupta.
It was February 2021, when Gupta's mom asked her to 'do something' with her skillset when she started studying computer science, specializing in data science, at the Vellore Institute of Technology in Tamil Nadu, India.
Advert
Speaking to Interesting Engineering, Gupta admitted her mom had 'taunted' her - but added: "It made me contemplate what I could do with my knowledge and skillset. One fine day, amid conversations with Alexa, the idea of inclusive technology struck me. That triggered a set of plans."
One year later, Gupta created an AI model which could translate the signs and symbols she made for sign language into English sounds.
If you're familiar with the tech, you'll understand how Gupta's model harnessed Tensorflow object detection API, making use of transfer learning through a different, pre-trained model.
If you're not so familiar with the technical side of things, just know that it was an impressive achievement.
Gupta credited a video by data scientist Nicholas Renotte on Real-Time Sign language Detection as the inspiration for her model, and said: “The dataset is manually made with a computer webcam and given annotations.
"The model, for now, is trained on single frames. To detect videos, the model has to be trained on multiple frames for which I’m likely to use LSTM (Long-Short Term Memory)."
Speaking in 2022, Gupta pointed out she was still 'learning' about the tech, but news of her creation went viral when she shared it online.
She was flooded with praise for her work, with one person writing: "Hi. This post of yours is reaching the world through other people who are sharing it.
"You will get many negative comments from regressive thinkers. But remember that there are more positive thinkers out there. Keep on chugging along. You're doing great.
"Join minds with others who share this passion, include deaf people, get their inputs, see if you can apply them, and keep on keeping on. Kudos to you. This is how we change the world."
Gupta said at the time that researchers and developers were 'trying their best to find a solution that can be implemented', but she believes the first step is to 'normalize sign languages and other modes of communication with the specially-abled and work on bridging the communication gap'.
Gupta's LinkedIn page now describes her as a 'self-starter with a keen interest in development and research', who is 'looking for opportunities to apply [her] technical knowledge to build stuff to solve today's problems the world is facing'.