Alzheimer's disease (AD) is the most common neurodegenerative disorder with one of the most complex pathogeneses, making effective and clinically actionable decision support difficult. The objective of this study was to develop a novel multimodal deep learning framework to aid medical professionals in AD diagnosis.We present a Multimodal Alzheimer's Disease Diagnosis framework (MADDi) to accurately detect the presence of AD and mild cognitive impairment (MCI) from imaging, genetic, and clinical data. MADDi is novel in that we use cross-modal attention, which captures interactions between modalities-a method not previously explored in this domain. We perform multi-class classification, a challenging task considering the strong similarities between MCI and AD. We compare with previous state-of-the-art models, evaluate the importance of attention, and examine the contribution of each modality to the model's performance.MADDi classifies MCI, AD, and controls with 96.88% accuracy on a held-out test set. When examining the contribution of different attention schemes, we found that the combination of cross-modal attention with self-attention performed the best, and no attention layers in the model performed the worst, with a 7.9% difference in F1-scores.Our experiments underlined the importance of structured clinical data to help machine learning models contextualize and interpret the remaining modalities. Extensive ablation studies showed that any multimodal mixture of input features without access to structured clinical information suffered marked performance losses.This study demonstrates the merit of combining multiple input modalities via cross-modal attention to deliver highly accurate AD diagnostic decision support.

Michal Golovanevsky

Carsten Eickhoff

Ritambhara Singh

Journal of the American Medical Informatics Association

JAMIA 是 AMIA 首屈一指的生物医学和健康信息学同行评审期刊。JAMIA 涵盖该领域的所有活动，包括临床护理、临床研究、转化科学、实施科学、成像、教育、消费者健康、公共卫生和政策。JAMIA 的文章描述了有助于推进生物医学科学和促进健康的创新信息学研究和系统。案例报告、观点和评论还可以帮助读者与实施、政策和教育中最重要的信息学发展保持联系。

The Journal of the American Medical Informatics Association: JAMIA is the premier peer-reviewed journal of the American Medical Informatics Association (AMIA) in the field of biomedical and health informatics. It covers all activities in this domain, including clinical care, clinical research, translational science, implementation science, imaging, education, consumer health, public health, and policy. Articles published in JAMIA describe innovative informatics research and systems that advance biomedical science and promote health. Case reports, viewpoints, and commentaries also help readers stay informed about the most significant developments in informatics related to implementation, policy, and education.

《Journal of the American Medical Informatics Association : JAMIA》是美国医学信息学协会（AMIA）最具权威的生物医学与健康信息学同行评审期刊。该期刊涵盖了该领域的所有活动，包括临床护理、临床研究、转化科学、实施科学、成像、教育、消费者健康、公共卫生和政策等。JAMIA发表的文章展示了推动生物医学科学和促进健康的创新信息学研究和系统。案例报告、观点和评论也帮助读者了解实施、政策和教育中最重要的信息学发展。

Journal of the American Medical Informatics Association : JAMIA

Multimodal attention-based deep learning for Alzheimer’s disease diagnosis

A novel multimodal deep learning framework to accurately detect the presence of AD and mild cognitive impairment from imaging, genetic, and clinical data and uses cross-modal attention, which captures interactions between modalities.

Category	Quartile
医学	2
医学, 卫生保健与服务	1
医学, 计算机信息系统	2
医学, 计算机跨学科应用	2
医学, 图书情报与档案管理	2
医学, 医学信息	2

Multimodal attention-based deep learning for Alzheimer’s disease diagnosis

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
MEDICAL INFORMATICS	1