Multimodal Machine Learning Approach for Diagnosing Atopic Dermatitis
Author: Alida Widiawaty, Wresti Indriatmi, Wisnu Jatmiko, Endi Novianto, Aria Kekalih
Informations
JournalF1000Research
PublisherF1000 Research Limited, F1000 Research Ltd
Volume & IssueVol. 14
Pages952
Publication Year2025
ISSN20461402
Source TypeGoogle Scholar
Abstract
Background Atopic dermatitis (AD) is a prevalent, chronic inflammatory skin disease with diverse clinical presentations, often overlapping with other dermatoses. Its diagnosis remains largely dependent on clinical expertise, leading to variability and limited diagnostic accuracy, particularly among general practitioners. This study aimed to develop and evaluate a multimodal artificial intelligence (AI) model that integrates lesion image analysis and structured anamnesis to improve AD diagnosis. Methods This diagnostic study was conducted in two phases: Phase 1 used retrospective data from 2021–2024, and Phase 2 involved prospective external validation from multiple hospitals in 2025. Patients with AD or related skin conditions were included, with diagnoses based on AAD 2014 criteria. Multimodal fusion combined ResNet50-extracted image features and MPNet-based anamnesis text features using a late fusion model. This approach mimics clinical reasoning by integrating visual and contextual clinical information to classify cases as AD or non-AD. Results and Discussion The multimodal AI model integrating ResNet50 (image) and MPNet (anamnesis) achieved 98.28% accuracy in classifying AD vs non-AD, outperforming image-or text-only models. It offers clinical advantages by mimicking physician reasoning, improving diagnostic consistency, reducing subjectivity, and enabling mass triage. However, real-world generalizability remains a challenge due to limited training diversity, potential language constraints (Bahasa Indonesia), and …
Documents & Links
