These Students Built A Glove That Translates Sign Language Into English

A pair of undergraduates at the University of Washington made a glove that translates gestures in American Sign Language (ASL) into English and speaks it via speakers. The SignAloud glove won them a $10,000 Lemelson-MIT Student Prize and international attention. Now they’re figuring out how to refine their gadget for social good…and make sure they pass their college exams.

The SignAloud glove captures ASL gestures with sensors that measure everything from XYZ coordinates to the way individual fingers flex or bend. That sensor data is sent via Bluetooth to a nearby computer and fed into coding algorithms that categorize the gestures, which are translated into English and then audibly spoken via speaker. But co-creator Navid Azodi emphasizes that SignAloud is still very much in a prototype phase.

“Keep in mind, we have by no means captured the entire language and we’re nowhere near that. [ASL] is more than just words and phrases, and we know that. It has complex grammar structures. What we eventually want to get is for SignAloud to categorize a majority of the language,” says Azodi.

The two creators, Azodi and fellow sophomore Thomas Pryor, planned and built SignAloud in their dorms and the school’s CoMotion MakerSpace. After a rough first version built on cardboard and string, they put together the current working prototype for about a hundred bucks in parts.

“Many of the sign language translation devices already out there are not practical for everyday use. Some use video input, while others have sensors that cover the user’s entire arm or body,” says Pryor in a UW press release.

Innovating past existing limitations is what earned the team the $10,000 award, which Azodi isn’t sure how they will spend. The money could help pay their tuition or perhaps even get them a nice dinner, Azodi jokes, but refining their gadget is top priority. Both are motivated by SignAloud’s possibilities, both as an input device and an empowerment tool.

“Communication is a fundamental human right,” says Azodi, whose personal experience helped inform the tool. “When I was young, I didn’t actually speak for about seven years of my life. I was seven years old when I finally started to speak. I had that communication barrier for a good amount of time.”

Azodi and Pryor met when they were freshmen. They shared a desire to give back to the world using their engineering talents and considerable experience. Azodi has been a systems intern for NASA, a technology lead for UW Information Technology, and a campus representative for Apple; Pryor has had internships at Raytheon and Boeing, is an undergraduate researcher in the Composite Structures Laboratory, and is a software lead on the Husky Robotics Team.

Before building the next version of SignAloud, Azodi and Pryor are going to consider the critiques they received as news of their grant win went global. Azodi laments that the device was portrayed as a tool that would speak for the speechless.

“[SignAloud] is not something that will speak for them. They do have a voice and they do speak. Honestly, they speak so much more fluidly in dynamic and beautiful ways using their bodies,” says Azodi. “This is more of a tool to break down those barriers between the physical language of ASL and English or Spanish.”

SignAloud could even help translate ASL into other languages, or be used as a pure input device from ASL into a computer, says Azodi. Even in its current state, SignAloud could also work as a teaching tool for folks who want to learn ASL, as it could give students instant feedback on what they were actually signing.

Considering the international feedback they they’ve received, Azodi is determined to make sure the team does its homework to learn the deep syntax of ASL, something they will likely recruit an expert to help them with. Azodi and Pryor knew that using a gadget to translate ASL would be a challenge, but they didn’t anticipate the difficulty they would encounter when it came to tracking the nuances of language. And the critiques they received after SignAloud was publicized by the press astonished them.

“None of my previous experience can compare to being put out into the national spotlight and having people all over the world throwing their critiques and views on it. It opened my eyes to so much more,” says Azodi. “That learning process was one of, if not the most humbling experience. People called it a miracle device, and it’s not. It’s not about helping people; they don’t need help. It’s about how technology can be accessible and inclusive. It’s just a means of building bridges and breaking down barriers.”

Not all the attention the duo received was negative: Thanks to the media coverage, Azodi and Pryor are facing a wealth of partnership and investment offers. But they aren’t rushing headlong into anything. After all, as 19-year-old college sophomores, they still have to worry about normal things. They’ll make progress on SignAloud, but have to work around their exam study schedule.

“The opportunities for something like this are unbelievable, I can’t even categorize where this can go. What is the best option for us using what we know and what we can do?” says Azodi. “I won’t give you a definite answer. We’re just a couple of college students. I just had midterms and we’re worried about how to get in our majors.”