The 2024 Diagnose-a-thon was held in November 2024. View information on winners below, and watch this page for details about the upcoming challenges.
Generative AI tools like ChatGPT were transforming our daily lives by making tasks faster and easier, but at what cost? These tools could amplify inequalities, spread misinformation, and present serious risks. At CSRAI, we had been exploring both the benefits and dangers through a series of hackathons that highlighted generative AI’s shortcomings. The first challenge, Bias-a-thon, uncovered generative AI’s embedded stereotypes. The second challenge, Fake-a-thon, demonstrated how easy it was to create fake news with the help of generative AI and how difficult it was to detect AI-generated fake news. Then, we returned with our most exciting challenge yet: Diagnose-a-thon!
In the Diagnose-a-thon, participants explored how AI could help or harm when it was used to answer medical questions. The event focused on testing how well these GenAI tools could handle health-related information and provide advice to people seeking guidance about their personal health. At a high level, the Diagnose-a-thon required participants to prompt large language models (LLMs) with medical queries and evaluate whether the responses were accurate or contained inaccurate, misleading, and/or potentially harmful content. Through the competition entries, we hoped participants could see both the potential benefits and risks of using AI in healthcare-related queries. They joined us for a chance to learn, compete, and share ideas on making AI safe and helpful for health—all while competing for cash prizes!
2024 Diagnose-a-thon
The 2024 Bias-a-thon was held Monday, November 11, to Sunday, November 17, 2024, and was open to all members of the Penn State community with an active @psu.edu email address.
Read about the 2024 Diagnose-a-thon Winners
During the event, Diagnose-a-thon participants used an online form to submit entries across three tracks:
- Patient track: Acting as a patient, participants prompt an LLM to produce a diagnosis for their real or imaginary symptoms.
- Medical professional track: Acting as a medical professional, participants prompt an LLM to produce a diagnosis based on a hypothetical patient case.
- Out-of-the-box track: Participants prompt an LLM about a scenario not included in the patient or medical professional tracks that might lead to a potentially believable medical diagnosis.
Panel of Physicians
Each submissions was reviewed by a panel of Penn State physicians. We thank them for contributing their expertise!
- Jennifer Kraschnewski MD, MPH
Professor and Vice Chair for Research, Department of Medicine - Division of General Internal Medicine
Director, Penn State Clinical and Translational Science Institute (CTSI), Penn State Cancer Institute - Michael J. Beck MD
Associate Professor of Internal Medicine and Pediatrics
Division Chief, General Inpatient Pediatrics
Vice Chair, Clinical Affairs, Department of Pediatrics - Cynthia Chuang MD, MSc
Professor, Department of Medicine
Chief, Division of General Internal Medicine - Sarah K. Horvath MD, MSHP, FACOG
Physician and Associate Professor, Department of Obstetrics and Gynecology - Deepa L. Sekhar MD
Associate Professor of Pediatrics - Nicole M. Osevala MD
Physician, Geriatric Medicine, Internal Medicine - Matthew S. Nudy MD
Physician and Assistant Professor, Department of Medicine - Division of Cardiology - Paul N. Williams MD
Physician – Internal Medicine - Eliana V. Hempel MD
Physician – Internal Medicine