Harnessing AI to Help Kids Speak

Kids with speech and language disorders can easily go unnoticed, leaving them without the therapy they need—a problem exacerbated by the COVID-19 pandemic and a nationwide shortage of speech-language pathologists.

But now, that problem is being addressed.

The University at Buffalo, with a five-year, $20 million grant from the National Science Foundation, will establish a national institute that develops artificial intelligence systems to identify and assist children between the ages of 3 and 10 with communication disorders.

“The University at Buffalo is leading New York and our nation in developing major innovations in education technology,” said U.S. Sen. Chuck Schumer. “This major federal investment will help bring technology into the classroom to ensure all children receive the help they need with speech and language processing challenges—all thanks to the research of Buffalo scientists.”

The National AI Institute for Exceptional Education at UB will focus on the more than 3.4 million children nationwide who require speech and language services under the Individuals with Disabilities Education Act. They are currently served by fewer than 61,000 speech language pathologists.

“There simply aren’t enough speech-language pathologists in the United States and, as a result, children are not receiving life-changing interventions soon enough,” said principal investigator Venu Govindaraju, SUNY Distinguished Professor in the Department of Computer Science and Engineering and vice president of research and economic development at UB. “Our multidisciplinary team will create advanced artificial intelligence systems that address this critical problem, allowing for earlier diagnoses and tailored interventions that close educational gaps and create more inclusive learning environments.”

Specifically, it will develop two advanced AI solutions: the AI Screener and the AI Orchestrator.

The AI Screener will observe children in the classroom, collecting samples of their speech, facial expressions, gestures and other data. It will create weekly summaries that catalogue each child’s vocabulary, pronunciation, video snippets and more. These summaries will help teachers monitor their students’ speech and language-processing abilities and, if needed, suggest a formal evaluation with a speech-language pathologist.

This is critical because, typically, the earlier speech and language concerns are addressed, the greater the likelihood children will excel academically and socio-emotionally.

The AI Orchestrator is an app that will help speech-language pathologists, most of whom have caseloads so large they cannot provide individualized care, by recommending personalized content tailored to students’ needs.

The institute will consist of more than 30 researchers from nine universities including UB: the University of Illinois Urbana-Champaign; Stanford University; the University of Washington; Cornell University; the University of Nevada, Reno; the University of Texas at El Paso; Penn State University; and the University of Oregon.

Govindaraju and UB engineering researchers Jinjun Xiong and Srirangaraj Setlur will coordinate the team’s work, while Letitia Thomas, assistant dean for diversity in the engineering school, will lead the broadening participation and diversity, equity and inclusion sections of the grant.

The award builds upon a rich history of AI research at UB that dates back decades to when computer scientists, including Govindaraju, developed a handwriting-recognition system that has saved the U.S. Postal Service hundreds of millions of dollars.

More recently, UB has made strategic investments to enhance the university’s national leadership in AI, including forming the Institute for Artificial Intelligence and Data Science and the Center for Information Integrity.

In addition to Govindaraju, Xiong and Setlur, the institute will include 13 UB researchers representing the School of Engineering and Applied Sciences, the College of Arts and Sciences and the Graduate School of Education.