Characterizing children’s naturalistic narratives via machine learning

Storytelling is a fundamental human behavior, and narrative skill is predictive of success in many domains, such that the development of narrative skill is an important process in childhood and adolescence. The goal of this research was to create and assess a novel method for identifying narratives produced in conversations between children and adults. To do so, we crafted a coding manual with a set of concrete rules and examples for identifying utterances containing narrative behavior. Then we asked trained coders to apply these rules to a set of transcribed conversations between adults and children, drawn from CHILDES. Our results indicated that coders could apply these rules with high inter-rater reliability. The utterances that coders identified had many of the characteristics of narrative defined by prior empirical literature. Further, we trained a version of a large language model (LLM) (GPT-4o-mini) to apply these rules and found that the model could successfully mimic human judgments. These results suggest that it is possible to automatize judgments of narrative behavior for rapid analysis of transcribed conversations at scale. This provides a novel avenue for investigation of the development of storytelling abilities, one which has the potential to generate new insights about the acquisition and use of narrative skill in social contexts.