Automatic assessment and feature-based characterisation of oral narratives for Afrikaans and isiXhosa children
type: CreativeWork
description: Developing narrative and comprehension skills in early childhood is critical for later literacy and academic success. Yet, less than 20% of South African 10-year-olds can read for meaning, highlighting an urgent need for early intervention. Accurate assessment is essential for identifying children who need support, but current methods rely on human judgement, which is resource-intensive and susceptible to inconsistencies. To address this, we design an automatic scoring system for low-income preschools, where large class sizes make it challenging for teachers to identify learners requiring assistance. Our research focuses on three core goals. First, we develop an automatic scoring model prioritising predictive accuracy. Our system employs automatic speech recognition (ASR), followed by linear and logistic regression models to predict language proficiency. To reduce the feature space, we apply an L1-regularised model to the vectorised text. Linear models are chosen for their simplicity and interpretability. We experiment with different linguistic units and scaling methods, finding that word-level tokens and raw feature values yield the best performance. We then compare our final linear model with a large language model (LLM) developed in parallel work. Although the LLM achieves higher accuracy in most cases, the linear model performs competitively given its simplicity. Second, beyond predictive accuracy, we examine which features the logistic model relies on most. Assuming perfect ASR transcriptions,we conduct quantitative analyses using permutation feature importance and exploratory data analysis, focusing on proficiency features, part-of speech counts and specific keywords. We also consult speech therapists to qualitatively interpret the relevance of these features relative to established measures of narrative development. Key indicators include lexical diversity and productivity, while speech production features such as articulation rate show unexpectedly limited p…
abstract: Developing narrative and comprehension skills in early childhood is critical for later literacy and academic success. Yet, less than 20% of South African 10-year-olds can read for meaning, highlighting an urgent need for early intervention. Accurate assessment is essential for identifying children who need support, but current methods rely on human judgement, which is resource-intensive and susceptible to inconsistencies. To address this, we design an automatic scoring system for low-income preschools, where large class sizes make it challenging for teachers to identify learners requiring assistance. Our research focuses on three core goals. First, we develop an automatic scoring model prioritising predictive accuracy. Our system employs automatic speech recognition (ASR), followed by linear and logistic regression models to predict language proficiency. To reduce the feature space, we apply an L1-regularised model to the vectorised text. Linear models are chosen for their simplicity and interpretability. We experiment with different linguistic units and scaling methods, finding that word-level tokens and raw feature values yield the best performance. We then compare our final linear model with a large language model (LLM) developed in parallel work. Although the LLM achieves higher accuracy in most cases, the linear model performs competitively given its simplicity. Second, beyond predictive accuracy, we examine which features the logistic model relies on most. Assuming perfect ASR transcriptions,we conduct quantitative analyses using permutation feature importance and exploratory data analysis, focusing on proficiency features, part-of speech counts and specific keywords. We also consult speech therapists to qualitatively interpret the relevance of these features relative to established measures of narrative development. Key indicators include lexical diversity and productivity, while speech production features such as articulation rate show unexpectedly limited p…
author: Sharratt, Emma Lori
language: en
date published: 2026-03
date modified: 2026-07-24T04:40:06Z
provider: Stellenbosch University
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: Stellenbosch : Stellenbosch University