Empire AI at UB

An Interactive Multimodal Language Model for Children

UB researchers will utilize Empire AI to build an agentic, multimodal large language model to perform real-time, interactive speech-language therapies for children with speech and language challenges. This model will be used by the National AI Institute for Exceptional Education and is designed to plan, deliver, and adapt interventions in real-time based on the child’s responses and scene dynamics.

Using speech, text, and visual cues to support personalized education and developmental assessment

This project, part of the National AI Institute for Exceptional Education, is developing an AI-powered screening tool that can interpret multiple types of information—such as speech, text, and visual cues—to support identification of children’s learning and developmental needs. By training a large language model on a rich, multimodal dataset, the system is designed to recognize patterns in communication and behavior that traditional assessments may not easily detect. The goal is to support educators and specialists in tailoring interventions and strategies for each child, with the potential to inform approaches to assessment and personalized learning.