Communication tool verbalizes thoughts for the speech impaired
Students in an upper-level computer software engineering class at the University at Buffalo helped to solve a real-world problem—and restore a sense of independence to persons with speech and motor disabilities—by designing augmentation communication devices.
The students produced UB Talker, a laptop computer with a touch-screen interface and synthetic voice that helps its users communicate (it comes in models for both adults and children). Currently the device is being brought to market with the assistance of the UB Office of Science, Technology Transfer and Economic Outreach (STOR).
The ongoing project began in March 2002 when senior students were asked by Kris Schindler and Michael Buckley, lecturers in the Department of Computer Science and Engineering in the UB School of Engineering and Applied Sciences, to design a speech-enhanced, computer-aided device that would allow David Jauch, a 43-year-old nursing home resident who had suffered a stroke twenty years previously, to communicate.
“David can’t speak—he’s in a wheelchair and has very limited motor skills,” says Schindler. “Mentally, he’s no different than you or I—it’s just very hard for him to communicate. He had a sheet of paper and communicated by pointing out letters and letter groups. It was frustrating and very time consuming.”
Now David readily taps out his thoughts on the Talker screen, able to express a range of emotions and comments, from conversational insights to wry jests.
Child Talker
During the first year of the assignment in 2002, 120 students formed fourteen design teams. At the end of that year, Buckley and Schindler took the best ideas from the class and asked the students who generated them to join the newly formed UB Technology Group to continue work on the UB Talker and develop other socially relevant projects for local not-for-profit organizations in need of technology.
“There were still too many good ideas to let the project drop, so we went to the Center for Handicapped Children in Cheektowaga and approached them about doing a child version of the UB Talker. They allowed us to use them as a testing ground,” says Buckley.
“In the spring of 2003, we reassigned the project with new requirements to meet the needs of children with cerebral palsy, both those with and without reading skills. Special requirements were written to include children with visual impairment. Students responded with interest, commitment, intensity, and philanthropy,” says Buckley.
Dale O’Toole, from the Center for Handicapped Children, recently started using the UB Talker on a multiply challenged child who is nonverbal and has some visual difficulties. “She was very tuned in to it and was really making an effort to use the Talker to communicate.”
Helene Kershner, lecturer and assistant chair of the Department of Computer Science and Engineering, also taught the software engineering class, working with project teams during the semester the child Talker was developed.
Talker advantages
Both the adult and child versions of the UB Talker combine commercial-grade technology updated for ease of use and better voice synthesis, Buckley explains.
Moreover, he adds, the Talker is much more affordable than speech augmentation devices currently on the market costing between $800 and $10,000 that may be of poor quality.
“They aren’t suitable for use in any environment, and programming a commercial device is extremely time-consuming,” Buckley points out.
Even worse, says Schindler, a user like David couldn’t program one—a therapist, often with a busy workload, would have to do it. David, however, can program the UB Talker.
Unlike the commercial products, the UB Talker features phrase prediction—words and phrases that are used frequently are automatically stored in the computer—and it is time sensitive. If David “wants to go to lunch” and begins typing that phrase, a list of phrases appears on the screen before he has completed the task, and he can finish the thought with one or two clicks or touches. Entire phrases are stored in the computer according to the time of day they are most likely to be used, or they can be programmed according to time-sensitive needs, eliminating tedious and repetitious typing.
“If you’re talking about food at eight in the morning, it knows you’re talking about breakfast,” says Buckley.
Conversational mode
One of the most unique and useful features of the device is its story, or lecture, mode, which allows users to participate in more natural, give-and-take conversations. When David was testing the device, he could input questions, comments, or conversation topics into the laptop before the UB students arrived, and play those comments either one at a time—and wait for their response—or all at once.
“If he knew we were coming and wanted to tell us there were certain things that were not working with the Talker, or certain things that needed to be changed, he could put these phrases in the lecture mode,” says Buckley. “When we arrived he wouldn’t have to construct the phrases.” One of the goals for the child Talker, he adds, is for a user to be able to “speak” a phrase in three clicks or less. “Three clicks and you’ve got lunch,” says Buckley.
In fact, Buckley relates, on the day that David first received the UB Talker, he called Buckley at home later that night.
“I couldn’t be there when they delivered the computer to David. Late that night the phone rings and there’s this robotic voice on the other end of the phone talking to me. David had it programmed in the lecture mode. He said, ‘I’m new at using the device so it’s going to take me a little time to (do this).’ He hadn’t spoken on the phone in twenty years—and he calls me,” says Buckley.
The UB Talker, he says, restores a sense of freedom and independence to its users. “It’s a quality-of-life issue,” he explains. “It restores relationships.”
