Infectious Disease · Medical Literature
Effectiveness of an online vaccine misinformation game for Arabic-speaking Australians: a randomized controlled trial.
Abstract
Vaccine misinformation can undermine vaccine confidence, institutional trust, and uptake. Arabic speakers in Australia are disproportionately susceptible to vaccine misinformation and have experienced adverse health outcomes. This randomized controlled trial tested the effectiveness of Cranky Uncle - Vaccine (Arabic), a gamified psychological inoculation intervention, in improving vaccine attitudes, misinformation discernment, and institutional trust among Arabic-speaking Australians. Participants (N = 198) were randomized 1:1 to either the intervention (game) or control (educational climate change video). Outcomes were assessed immediately and 3 weeks post-activity using a combination of validated and custom instruments: (1) the Vaccine Trust Indicator (VTI); (2) a misinformation discernment scale developed by the research team, and (3) items from the Institutional Trust subscale. Analyses were conducted using multiple imputation and adjusted linear regression models. The trial protocol was registered in the Australian and New Zealand Clinical Trials Registry (registration number: ACTRN12624000934549p). No statistically significant differences were found between the intervention and control groups at either timepoint across primary or secondary outcomes, and no subgroup effects were observed by age or education. Post-hoc item analyses found that items assessing trust in scientists and trust in pharmaceutical companies demonstrated the greatest difference between groups at the immediate post-activity timepoint. This trial is the first RCT of the Cranky Uncle - Vaccine game. Findings highlight methodological and measurement challenges in misinformation research. Developing and validating standardized measures that are appropriately sensitive to detect meaningful changes should be an urgent priority for the field. Future evaluation of the Cranky Uncle intervention for different populations could consider testing delivery in structured educational group settings.
Clinical relevance
Does playing the Cranky Uncle – Vaccine (Arabic) gamified psychological inoculation game improve vaccine attitudes, misinformation discernment, and institutional trust among Arabic-speaking Australians compared to watching an educational climate change video, immediately and at 3 weeks post-intervention? __SNAPSHOT__ {"studyDesign":"Parallel-group, randomized controlled trial (1:1 allocation), registered in the Australian and New Zealand Clinical Trials Registry (ACTRN12624000934549p)","setting":"Australia; community-based online trial targeting Arabic-speaking adults; affiliated with the University of Melbourne and Royal Children's Hospital (Melbourne, Victoria)","sampleSize":"N=198 (intervention: 99, control: 99); differential post-randomization dropout noted: 26 intervention-group participants dropped out before or during gameplay versus 3 in the control group","population":"Arabic-speaking adults residing in Australia; recruited via community networks through a project Advisory Group; a population identified as disproportionately susceptible to vaccine misinformation with documented adverse health outcomes","intervention":"Cranky Uncle – Vaccine (Arabic): a free, text-based browser/smartphone game using cartoons, humor, and gamified quizzes across multiple levels to teach recognition of vaccine misinformation rhetorical techniques (e.g., appeal to nature, fake experts) and provide factual vaccine information via a health worker character","comparator":"Educational climate change video (active control); participants viewed an educational video unrelated to vaccines or vaccine misinformation","primaryOutcome":"Vaccine attitudes as measured by the Vaccine Trust Indicator (VTI); secondary outcomes included misinformation discernment (custom scale developed by research team) and institutional trust (items from the Institutional Trust subscale)","followUp":"Two time points: immediately post-activity and 3 weeks post-activity","fundingSource":"Government (Victorian Medical Research Acceleration Fund [VMRAF-6, 2023], Department of Health, Government of Victoria); no industry funding reported"}
Key findings
No statistically significant differences were found between the intervention and control groups at either the immediate or 3-week follow-up time point across any primary or secondary outcome (Vaccine Trust Indicator, misinformation discernment, or institutional trust). Specific adjusted mean differences, 95% confidence intervals, and p-values for each outcome are not fully reported in the available abstract. Subgroup analyses stratified by age (≥36 vs. <36 years) and education level (less than Bachelor's degree vs. Bachelor's degree and above) also showed no evidence of differential effectiveness, with considerable 95% CI overlap and non-significant interaction p-values across all outcomes and timepoints. Post-hoc item-level analyses were performed using complete cases only (without multiple imputation), yielding n=73 (intervention) and n=93 (control) at immediate follow-up, reflecting the marked differential dropout between arms. No adverse events were reported (not applicable to this type of behavioral intervention). NNT/NNH: not calculable given null findings and absence of reported effect sizes.
Practice implications
This trial — the first RCT of the Cranky Uncle – Vaccine game — did not demonstrate statistically significant benefit of the gamified psychological inoculation intervention over an active control on any prespecified outcome at either post-intervention time point in Arabic-speaking Australians. The study did not establish that the game improves vaccine attitudes, misinformation discernment, or institutional trust in this population under the self-directed online delivery format tested. Importantly, prior non-randomized (pre-post) evaluations in African settings showed significant improvements, suggesting either that contextual factors, delivery modality, participant engagement, or measurement insensitivity may explain the discrepant findings. The authors themselves note that the findings highlight significant methodological and measurement challenges in misinformation research, and that community feedback suggested the game may be better received in structured educational group settings. The findings do not generalize beyond Arabic-speaking Australians tested in an online, self-directed format. Vaccine-hesitant individuals with the greatest potential to benefit from inoculation may have been underrepresented given the volunteer sample. Practice should not change based on this single null trial; the negative result may reflect implementation or measurement limitations rather than true inefficacy. NOTE: Only the abstract and indexed excerpts of this open-access article were available; the full quantitative results table (including all adjusted mean differences, CIs, and p-values by outcome) should be reviewed in the published manuscript before drawing definitive conclusions. __LIMITATIONS__ Marked differential dropout: 26 of 99 intervention-group participants dropped out before or during gameplay versus only 3 of 99 control-group participants, creating substantial post-randomization imbalance and raising concern about selection bias and reduced statistical power in the intervention arm despite multiple imputation,Relatively small total sample (N=198), which may have been underpowered to detect modest but clinically meaningful effect sizes, particularly after accounting for the high intervention-arm attrition,The misinformation discernment scale was custom-developed by the research team and not independently validated, raising questions about measurement sensitivity and construct validity; the authors acknowledge that development and validation of standardized, sensitive measures is an urgent priority for the field,Active control (educational climate change video) may itself have influenced vaccine-related outcomes in an unexpected direction or provided non-specific active engagement effects, complicating interpretation of the null result,Volunteer convenience sample recruited through community advisory networks may overrepresent vaccine-accepting individuals with higher health literacy, limiting generalizability and potentially introducing ceiling effects on outcome measures,No blinding of participants to treatment allocation was possible given the nature of the intervention; expectation effects and differential motivation between arms cannot be excluded,Short-term follow-up (3 weeks maximum); effects of psychological inoculation on real-world vaccine behavior and longer-term attitude change were not assessed,Self-directed online delivery format may not reflect the optimal implementation context; community and advisory group feedback suggested structured group educational settings might yield different results,Trial conducted in a post-pandemic period when baseline vaccine attitudes in the target population may have already shifted, potentially contributing to ceiling effects on outcome instruments,Government-funded study (Victorian state government); no industry conflicts, but investigator involvement in game development (some authors affiliated with Cranky Uncle game ecosystem) is a potential source of allegiance bias __BOTTOM_LINE_CATEGORY__ not_ready_for_clinical_application __BOTTOM_LINE__ In this first RCT of the Cranky Uncle – Vaccine game, a gamified psychological inoculation intervention delivered in Arabic did not significantly improve vaccine attitudes, misinformation discernment, or institutional trust compared to an active control among Arabic-speaking Australians at immediate or 3-week follow-up. The null result is substantially complicated by high differential dropout in the intervention arm, a small sample size, and the use of a non-validated custom outcome measure, meaning true inefficacy cannot be confidently distinguished from implementation failure or measurement insensitivity. Clinicians and public health practitioners should regard this intervention as not yet established for this population in self-directed online settings; exploration of structured group-based delivery and improved measurement tools is warranted before drawing broader conclusions.