Generative AIs in Detecting Mpox Related Misinformation: ChatGPT and Gemini
Keywords:
Mpox, Health Information, Misinformation, Generative AIAbstract
This study investigates the performance of two generative AI systems, ChatGPT and Gemini, in detecting Mpox-related misinformation. As the Mpox outbreak in recent times led to widespread dissemination of both accurate and false information, particularly on social media platforms, the potential of AI in combating health misinformation has gained attention. The research presented ten commonly circulated pieces of Mpox misinformation to both AI systems, evaluating their responses against fact-checks and public health databases. Results demonstrated that both ChatGPT and Gemini performed admirably in identifying false information and providing accurate data about Mpox. Their responses aligned closely with authoritative sources such as the World Health Organization and Centers for Disease Control and Prevention. The study's findings suggest that these AI tools could be valuable assets in combating the spread of misinformation during disease outbreaks. However, the researchers emphasize that AI systems should not be considered infallible and should be used in conjunction with human expertise and authoritative sources. The performance of ChatGPT and Gemini in this context aligns with broader trends of generative AI platforms showing potential across various fields, including science and medicine. While promising, the study calls for further research to fully realize the potential of AI chatbots in addressing health misinformation, including investigations into practical applications such as integrating AI systems into public health communication strategies or developing AI-assisted fact-checking tools for social media platforms.
References
Ali, R., Tang, O. Y., Connolly, I. D., Fridley, J. S., Shin, J. H., Sullivan, P. L. Z., & Asaad, W. F. (2022). Performance of ChatGPT, GPT-4, and Google bard on a neurosurgery oral boards preparation question bank. Neurosurgery, 10-1227.
Antaki, F., Touma, S., Milad, D., El-Khoury, J., & Duval, R. (2023). Evaluating the performance of chatgpt in ophthalmology: An analysis of its successes and shortcomings. Ophthalmology Science, 100324.
Boissonneault, D. and Hensen, E., 2024. Fake News Detection with Large Language Models on the LIAR Dataset. https://doi.org/10.21203/rs.3.rs-4465815/v1
CDC, 2022. About Mpox. [online] Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/poxvirus/Mpox/about.html [Accessed 19 August 2024].
Cheung, D. H., Chen, S., Fang, Y., Sun, F., Zhang, Q., Yu, F. Y., ... & Wang, Z. (2024). Influences of mpox disease perceptions, sources and contents of information exposure on mpox vaccine uptake among gay, bisexual, and other men who have sex with men in Hong Kong, China. Vaccine, 42(9), 2337-2346.
Choi, J. H., Hickman, K. E., Monahan, A. B., & Schwarcz, D. (2021). Chatgpt goes to law school. J. Legal Educ., 71, 387.
Edinger, A., Valdez, D., Walsh-Buhi, E., Trueblood, J. S., Lorenzo-Luaces, L., Rutter, L. A., & Bollen, J. (2023). Misinformation and public health messaging in the early stages of the mpox outbreak: mapping the twitter narrative with deep learning. Journal of Medical Internet Research, 25, e43841.
Gagnon-Dufresne, M. C., Sarmiento, I., Fortin, G., Andersson, N., & Zinszer, K. (2023). Why urban communities from low-income and middle-income countries participate in public and global health research: protocol for a scoping review. BMJ open, 13(6), e069340.
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9(1), e45312
Hall, M. J., Ng, A., Ursano, R. J., Holloway, H., Fullerton, C., & Casper, J. (2003). Psychological impact of the animal-human bond in disaster preparedness and response. Journal of Psychiatric Practice, 9(5), 345-353.
Huang, X., Ruan, W., Huang, W., Jin, G., Dong, Y., Wu, C., ... & Mustafa, M. A. (2023). A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation. arXiv preprint arXiv:2305.11391.Available at: https://doi.org/10.48550/arXiv.2305.11391.
Koga, T., Martin, A., & Dickson, K. (2023). Comparative analysis of ChatGPT and Google Bard in scientific query processing. Journal of Artificial Intelligence Research, 78, 45-62.
Kok, J. N., Boers, E. J., Kosters, W. A., Van der Putten, P., & Poel, M. (2009). Artificial intelligence: definition, trends, techniques, and cases. Artificial Intelligence, 1, 270- 299.
Lim, S., Park, J., Kim, Y., & Lee, J. (2023). Performance evaluation of large language models in medical knowledge assessment. Nature Digital Medicine, 6(1), 1-10.
Mahmud, R., Reza, H. M., & Ahmed, S. (2023). Impact of COVID-19 vaccine misinformation on public health efforts: A social media analysis. Journal of Health Communication, 28(2), 112-125.
Miller, T., 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267, pp.1-38.
Mitchell, M. (2019). Artificial Intelligence: A Guide for Thinking Humans. Penguin UK. Ouyang, F., & Jiao, P. (2021). Artificial Intelligence in Education: The Three Paradigms. Computers and Education: Artificial Intelligence, 2, 100020.
Morejón-Llamas, N., & Cristòfol, F. J. (2023). Monkeypox, Disinformation, and Fact-Checking: A Review of Ten Iberoamerican Countries in the Context of Public Health Emergency. Information, 14(7), 390.
Otu, A., Ebenso, B., Walley, J., Barceló, J. M., & Ochu, C. L. (2022). Global human monkeypox outbreak: atypical presentation demanding urgent public health action. The Lancet Microbe, 3(8), e554-e555.
Patil, M., Yadav, H., Gawali, M., Suryawanshi, J., Patil, J., Yeole, A., & Potlabattini, J. (2024). A Novel Approach to Fake News Detection Using Generative AI. International Journal of Intelligent Systems and Applications in Engineering, 12(4s), 343-354.
Procopio, C. H., & Procopio, S. T. (2007). Do you know what it means to miss New Orleans? Internet communication, geographic community, and social capital in crisis. Journal of Applied Communication Research, 35(1), 67-87.
Shen, L., Wang, H., Yu, L., & Zhang, K. (2023). ChatGPT for medical education: A comparative study with human experts. Medical Education, 57(8), 885-893.
Shen, X., Chen, Z., Backes, M., & Zhang, Y. (2023). In chatgpt we trust? measuring and characterizing the reliability of chatgpt. arXiv preprint arXiv:2304.08979.
Titanji, B., Tegomoh, B., Nematollahi, S., Konomos, M. and Kulkarni, P.A., 2022. Mpox: A Contemporary Review for Healthcare Professionals. Open Forum Infectious Diseases, 9(7), p.ofac310.
Voeten, H. A., de Zwart, O., Veldhuijzen, I. K., Yuen, C., Jiang, X., Elam, G., ... & Brug, J. (2009). Sources of information and health beliefs related to SARS and avian influenza among Chinese communities in the United Kingdom and The Netherlands, compared to the general population in these countries. International Journal of Behavioral Medicine, 16(1), 49-57.
WHO, 2024. “WHO Director-General declares mpox outbreak a public health emergency of international concern”. (2024). Available at: https://www.who.int/news/item/14-08-2024-who-director-general-declares-mpox-outbreak-a-public-health-emergency-of-international-concern (accessed 22 August 2024).
Wilson, S. L., & Wiysonge, C. (2020). Social media and vaccine hesitancy. BMJ Global Health, 5(10), e004206.
World Health Organization. (2023). Mpox. Retrieved from https://www.who.int/health-topics/monkeypox
