Development and content validation of a health assessment item bank for regional health big data: a sequential mixed-methods approach

Abstract Background Health assessment instruments are essential for individual health monitoring, yet most existing tools rely on periodic surveys and lack the capacity to integrate large-scale digital health data. With the increasing availability of regional health big data, there is a need to establish comprehensive indicator resources that can inform the development of validated instruments tailored to local health contexts. This study aimed to develop and validate a health assessment item bank based on regional health big data, providing a structured foundation for subsequent instrument development. Methods A sequential mixed-methods design was adopted. First, semi-structured interviews were conducted with experts from diverse health-related disciplines to identify key health categories and subcategories. A preliminary conceptual framework was produced through a directed qualitative content analysis process. Second, data elements were extracted from the national Regional Health Information Platform Interaction Standard (WS/T 790) of China and related standards. These data elements were vectorized using a Chinese pre-trained RoBERTa model, clustered with the DBSCAN algorithm, and matched to the conceptual framework through cosine similarity. Finally, experts reviewed and rated the semantic matching results by scoring the matching relevance from 1 to 5, agreement among experts were evaluated with the Fleiss’ Kappa analysis, and consensus discussions were conducted to refine the item pool and ensure content validity. Results The resulting item bank comprised 430 indicators distributed across five main categories and 17 subcategories. The main categories are Physiological health, Psychological health, Health behaviors, Social health, and Environment and healthcare services. Expert review yielded high agreement (mean score = 4.95/5.00; Fleiss’ Kappa = 0.626), supporting the adequacy of content validity. The Social support and Health service subcategories contained no mapped indicator, highlighting areas requiring integration of additional data sources. Conclusions This study established a comprehensive and validated item bank for health assessment based on regional health big data, offering a structured foundation for future development, calibration, and psychometric testing of health assessment instruments. This work contributes to advancing data-driven, context-specific approaches for monitoring health at the individual level.