Development and validation of the chilling effects in media scale

Chilling effects are inhibitions of legitimate behaviors and are an important ethical side effect of surveillance. With the centrality of media users’ data for digital communication it is important to gain insights into such ethical side effects, which requires a validated scale that can be used in future research. To develop and validate the Chilling Effects in Media (CEM) scale, we used a three-phased mixed-methods approach: (1) item generation using a literature review and qualitative interviews, (2) establishing content validity through experts interviews and a think-aloud study, and (3) establishing criterion and construct validity, and reliability through a pre-registered online panel survey (N = 415) in the context of corporate (e.g., personalized advertising) or government (e.g., security) surveillance. Factor analyses were conducted, resulting in three factors per context which individually and collectively scored adequately on reliability and validity. Between the two contexts, the factors were partially overlapping and partially distinct, showing that corporate and government surveillance result in unique chilling effects. In addition to the full-length scales per context (i.e., 16 items for corporate and 26 for government surveillance), shorter, more workable versions with respectively 8 and 9 were created too. These shorter versions perform just as well in terms of reliability and validity.