Measuring frequency of child-directed WH-question words for alternate preschool locations using speech recognition and location tracking technologies
Kothalkar, Datla, Dutta, Hansen, Seven, Irvin, Buzhardt
Companion Publication of the 2021 International Conference on Multimodal Interaction
Speech
and language development in children are crucial for ensuring effective
skills in their long-term learning ability. A child’s vocabulary size at the
time of entry into kindergarten is an early indicator of their learning
ability to read and potential long-term success in school. The preschool
classroom is thus a promising venue for assessing growth in young children by
measuring their interactions with teachers as well as classmates. However, to
date limited studies have explored such naturalistic audio communications.
Automatic Speech Recognition (ASR) technologies provide an opportunity for
’Early Childhood’ researchers to obtain knowledge through automatic analysis
of naturalistic classroom recordings in measuring such interactions. For this
purpose, 208 hours of audio recordings across 48 daylong sessions are
collected in a childcare learning center in the United States using Language
Environment Analysis (LENA) devices worn by the preschool children.
Approximately 29 hours of adult speech and 26 hours of child speech is
segmented using manual transcriptions provided by CRSS transcription team.
Traditional as well as End-to-End ASR models are trained on adult/child
speech data subset. Factorized Time Delay Neural Network provides a best Word-Error-Rate
(WER) of 35.05% on the adult subset of the test set. End-to-End transformer
models achieve 63.5% WER on the child subset of the test data. Next, bar
plots demonstrating the frequency of WH-question words in Science vs. Reading
activity areas of the preschool are presented for sessions in the test set.
It is suggested that learning spaces could be configured to encourage greater
adult-child conversational engagement given such speech/audio assessment
strategies.