The
LENA system has revolutionized research on language acquisition, providing
both a wearable device to collect day-long recordings of children’s
environments, and a set of automated outputs that process, identify, and
classify speech using proprietary algorithms. This output includes
information about input sources (e.g., adult male, electronics). While this
system has been tested across a variety of settings, here we delve deeper
into validating the accuracy and reliability of LENA’s automated diarization,
i.e., tags of who is talking. Specifically, we compare LENA’s output with a
gold standard set of manually generated talker tags from a dataset of 88
day-long recordings, taken from 44 infants at 6 and 7 months, which includes
57,983 utterances. We compare accuracy across a range of classifications from
the original Lena Technical Report, alongside a set of analyses examining
classification accuracy by utterance type (e.g., declarative, singing).
Consistent with previous validations, we find overall high agreement between
the human and LENA-generated speaker tags for adult speech in particular,
with poorer performance identifying child, overlap, noise, and electronic
speech (accuracy range across all measures: 0–92%). We discuss several clear
benefits of using this automated system alongside potential caveats based on
the error patterns we observe, concluding with implications for research
using LENA-generated speaker tags.
