Yimu Pan, a doctoral candidate in the College of Information Sciences and Technology at Penn State, will deliver “Cross-Modal Learning and Foundational Models in Medical AI: From Techniques to Applications in Placenta Analysis” as part of CSRAI's Young Achievers Symposium. This lecture is free and open to the Penn State community.
About the Talk
Cross-modal learning and foundational models represent cutting-edge advancements in medical artificial intelligence (AI), enabling the integration of diverse data modalities—such as medical images and clinical text—to address challenges in data scarcity, heterogeneity, and clinical interpretability. Foundational models, pre-trained on large-scale multimodal datasets, provide a versatile framework for harmonizing information across clinical domains, uncovering latent patterns, and enhancing diagnostic precision. This work explores methodologies for cross-modal alignment, including contrastive learning and attention-based fusion, which empower models to bridge imaging and textual data for comprehensive insights. We demonstrate their application in placenta analysis, a critical yet under-researched domain in prenatal care, where multimodal integration improves the prediction of placental complications.
Generative models, such as diffusion models, are further leveraged to augment scarce medical image datasets, mitigate domain shifts, and enhance model robustness in real-world clinical settings. By synthesizing realistic medical images or generating synthetic clinical narratives, these approaches address data limitations while preserving biological relevance. Our study highlights the synergy of cross-modal learning and foundational models in advancing placenta-centric AI tools, emphasizing their potential to transform prenatal diagnostics, risk stratification, and personalized care. We conclude with practical considerations for deploying such systems in clinical workflows, underscoring the importance of interpretability, ethical data practices, and collaboration with domain experts to ensure translational impact.
About the Speaker
Yimu Pan is a doctoral student in Informatics in the College of Information Sciences and Technology at Penn State, under the supervision of James Z. Wang. His research focuses on computer vision and multimodal learning, with applications in health care, emotion understanding, and AI-assisted clinical decision-making. In 2024, he worked as an applied scientist intern at Amazon, where he collaborated on projects related to last mile delivery optimization. He earned bachelor's degrees from Penn State in Computer Science (Summa Cum Laude) and Mathematics and Statistics (Magna Cum Laude).
About the Young Achievers Symposium
The Young Achievers Symposium highlights early career researchers in diverse fields of AI for social impact. The symposium series seeks to focus on emerging research, stimulate discussions, and initiate collaborations that can advance research in artificial intelligence for societal benefit. All events in the series are free and open to the public unless otherwise noted. Penn State students, postdoctoral scholars, and faculty with an interest in socially responsible AI applications are encouraged to attend.