While there are some adaptive designs that aim to help those who are deaf or hard-of-hearing (DHH), most of them rely too heavily on subtitles and audio, losing out on physical gestures and facial expressions. Some products are designed to convert sign language to text, some AI robots are meant to replace hearing dogs, and then there’s even a wearable device that translates sign language into speech. Awarded by James Dyson with Japan’s highest award, a team of designers from Digital Nature Group developed a new solution for bridging communication gaps between hearing and hard-of-hearing folks called See-Through Captions.
Developed for those who are deaf or experience varying levels of hearing loss, See-Through Captions was designed by a team of hearing and deaf individuals and tested in real-life situations to ensure its effectiveness. See-Through Captions is essentially a transparent projector that converts audio to subtitles and displays those conversations on its two-way screen.
Since most pre-existing related products only focus on translating audio to subtitles, Digital Nature Group gave See-Through Captions a transparent screen to guarantee that users’ facial expressions and physical gestures aren’t missed. In addition to the incorporation of physical expression, Digital Nature Group improved its automatic speech recognition (ASR) technology to optimize real-time captioning and ensure an accurate relay of communication.
The See-Through Captions product comes in two different physical forms, stationary and portable, so the ASR technology can be applied across different interactions. In developing See-Through Captions, Digital Nature Group demonstrated different prototypes in a museum setting. In the National Museum of Emerging Science and Innovation, See-Through Captions was used in its stationary form at the museum’s front desk as well as its portable form on guided tours.
Following its demonstration, Digital Nature Group decided some improvements could be made to the size and amount of text displayed on the portable product. Overall, See-Through Captions is a simple, understated solution for the DHH community that incorporates facial expression as well as accurate ASR technology to bridge communication gaps between DHH and hearing individuals.
According to Yanko Design