Explaining in context: Perceived informativeness of Explainable Artificial Intelligence (XAI) in Arabic hate speech detection

Hate speech is pervasive across most social media platforms, profoundly impacting users' lives and wellbeing. To address this critical issue, researchers are exploring explainable AI (XAI), which helps users understand AI decision-making and outputs. To ensure its effectiveness, XAI must align with users’ expectations, often assessed using user-centred approaches from human-computer interaction (HCI). However, the integration of XAI in hate speech detection, particularly in Arabic contexts, remains underexplored. This study proposed and evaluated a high-fidelity prototype of an Arabic hate speech detection system with explainable features. The focus was on assessing the informativeness of different XAI representation methods and how Arabic-speaking users perceived them. Both subjective and objective measures were used, including the Social Media Activity Questionnaire (SMAQ), Social Networking Usage Questionnaire (SNUE), a Perceived Informativeness (PI) questionnaire, and eye-tracking to assess visual attention. Results showed that users perceived both textual and visual explanations—specifically visual saliency—as more informative than other XAI methods. Social media usage duration had no significant effect on perceived informativeness. While the combination of textual and visual explanations (presented as a pie chart) resulted in longer fixation durations, this did not necessarily translate into higher perceived informativeness. The extended fixation was likely due to the additional visual element rather than increased clarity. These findings highlight the importance of designing visually clear and focused XAI representation methods. Elements like text highlighting through saliency can improve user perception of informativeness more effectively than simply adding visual complexity.