Generalizing Classification of Pilot Workload: Transfer Learning versus a JEPA-Inspired Transformer Architecture

<p>Within the context of learning, there poses difficulty when objectively measuring human performance. In this work, we investigate the evaluation of human performance via its relation to the individual's mental capacity by classification of cognitive load within the domain of aviation. By utilizing a mixed virtual and physical flight simulation environment in conjunction with biometric sensing, we create and evaluate the predictive capabilities of a Joint-Embedding Predictive Architecture (JEPA) and compare the architecture and results to traditional methods for transfer learning and domain adaptation. We find that our JEPA inspired architecture can achieve more than 70% accuracy of cognitive workload, compared to the 63% and 56% accuracies of traditional transfer learning methods. Through this foundation, we have made advancements in multi-modal and multi-task learning to classify various features across numerous pilots, operators, and novices within aviation. Our predictive model can automate the evaluation of cognitive load, enabling creation of generalizing features even when labeled examples are scarce.</p>