February 11, 2019
Professor Laura-Ann Petitto and the multidisciplinary team of the innovative language learning tool and research platform, called the RAVE (Robot, Avatar, thermal Enhanced language learning tool) has released a National Science Foundation (NSF) outcomes report to the general public.
The RAVE prototype makes available multiple components of human language in socially interactive and contingent conversational ways to young infants during critical periods of human learning and brain development.
The multidisciplinary research team especially targets periods in early child development whereupon minimal language input (or, lack of sufficient language input), and other states of extreme language deprivation, can have devastating deleterious impact on language, reading, and academic success spanning the lifetime.
They bring together science from multiple disciplines to explore the potential for technologies such as functional near infrared spectroscopy neuroimaging (fNIRS; measures neural activity underlying human cognition), thermal infrared imaging (measures human emotions), robotics, and avatars to positively impact children’s language learning.
The team’s broader impact objectives are ultimately to allow early learning gains for populations of deaf and hearing children who would otherwise be at a lifelong disadvantage, and also, more immediately, to advance involvement of underrepresented groups in STEM, propel interdisciplinary student scholarship, and train young deaf scientists in the advancement of scientific knowledge with transformative translational significance for all society.
Learn more about the NSF RAVE Revolutionary Learning Tool Prototype and how the system works.
Accomplishments
Professor Petitto and the RAVE team have have successfully built a novel prototype of an Artificial Intelligence Machine consisting of two Artificial Agents (robot+avatar) that engages babies ages 6 to 12 months to promote early language learning, especially deaf and hearing infants with minimal or no early-life language exposure.
The RAVE prototype also involves advancement of new experimental discoveries about the growing brain, language, and cognition in early child development, and new technology integration that, in turn, renders a powerful research platform for future scientific and technological exploration and expansion.
Figure 2. Composite image showing the multiparty interactions possible (thus, social conversational roles) among the Avatar, Robot, and infant.
Does RAVE Work?
Our experiments revealed the surprising (see Figures 2-4 of babies in Experimental set up): RAVE evidenced the potential to impart language learning in very young hearing and deaf babies ages 6-12 months. All babies preferred the avatar on the TV screen more than the physically embodied robot.
Experimental results demonstrated that this preference was due to the babies’ sensitivity to specific rhythmic temporal patterns underlying phonetic-syllabic structure in all human language (signed or spoken). Like a “lock and key,” we intentionally built these rhythmic temporal language patterns into the avatar’s ASL signing based on brain discoveries about the language patterns to which all infant brains are maximally sensitive (and at what maturational ages). Remarkably, all babies produced linguistic responses to the avatar’s linguistic productions, even hearing babies with no sign exposure.
Broad Impact
This language learning tool may break the communication barrier that widely impacts many young deaf and hearing children with minimal or no language input in early life.
NSF INSPIRE RAVE Team
Laura-Ann Petitto, PI, Cognitive Neuroscience, functional near infrared spectroscopy (fNIRS brain imaging), Gallaudet University
Arcangelo Merla, Applied Psychophysiology/Biomedical Engineer, Thermal IR Imaging, Universita D’Annunzio Chieti-Pescara, Italy
David Traum, virtual human science, University of Southern California
Brian Scassellati, robotics science, Yale University
Learn more about the NSF RAVE Revolutionary Learning Tool Prototype.
The NSF RAVE Revolutionary Learning Tool Prototype is funded by the National Science Foundation (NSF) and the Keck Foundation.