A multi-dimensional transformer encoder framework for modeling fine-grained student programming behavior

Abstract Learning to program is not just about writing code. It is a complex process involving trial and error, hesitation, corrections, and sometimes silence. And each student experiences it in their own way. Understanding these behaviors, especially those that remain invisible, is one of the great challenges of teaching programming. In this study, we analyze the digital traces of 70 beginner computer science students engaged in programming exercises. Our goal? To propose a more refined framework for modeling their behavior, in order to better detect moments of blockage and enable more targeted educational assistance. We propose an approach based on Transformer encoders, which integrates three dimensions. The data comes from standardized practical work sessions more than 80 megabytes of timestamped traces, collected with consent and stored in a database. Each session is divided into coherent sequences, allowing for a detailed analysis of learning trajectories. Our method transforms keystrokes into meaningful action sequences, using a taxonomy enriched with 13 types of actions. The model is trained in a self-supervised manner on two datasets: an institutional dataset (70 students, 287,236 actions), and a public CodeWorkout dataset (487 students, 5.5 million actions). It learns to predict two things: the next action and a masked action. Results: 87.4% accuracy in predicting the next action, 95.2% accuracy in predicting masked actions. These performances show that beginner behavior follows a structure, a kind of implicit grammar, which the model manages to capture. This work contributes to the exploration of educational data by proposing an empirically validated framework, and highlights an essential trade-off: between data quantity and granularity, choosing the right balance is key to building useful models.