Large Language Model vs. Deep Neural Network vs. Physicians: Accuracy and Consistency in STEMI Detection
Overview
This study compared three ways of reading 18 selected chest-pain ECGs with adjudicated outcomes (8 STEMI equivalents, 8 STEMI mimics, 1 STEMI and 1 normal ECG): a large language model (ChatGPT-5, queried 100 times per ECG), 95 physicians, and the PMcardio Queen of Hearts deep neural network (aOMI v1). The deep neural network was correct on 88.9% of ECGs, compared with 64.7% for physicians and 55.4% for the language model. On STEMI mimics, the language model's accuracy fell to 19.9%, versus 87.5% for the deep neural network and 72.5% for physicians. The deep neural network also gave identical answers on repeat runs (100% response consistency), compared with 84% for the language model and 77% for physicians.
Key findings
- The Queen of Hearts deep neural network reached 88.9% accuracy (95% CI 72.2–100%), compared with 64.7% (51.2–77.1%) for 95 physicians and 55.4% (38.9–71.6%) for ChatGPT-5.
- On STEMI mimics, the language model was accurate in only 19.9% of responses, compared with 87.5% for the deep neural network and 72.5% for physicians.
- Response consistency was 100% for the deep neural network, 84% for the language model and 77% for physicians.
- The language model gave unanimous answers across its 100 queries in only 6% of cases.
Published in: ACC.26 (American College of Cardiology Annual Scientific Session 2026); abstract published in the Journal of the American College of Cardiology (JACC)
Published on: 30 March 2026
Background
Large language models (LLMs) are increasingly explored for medical decision support, but their reliability for electrocardiogram (ECG) interpretation remains uncertain. Deep neural networks (DNNs) have demonstrated superior performance in detecting ST-elevation myocardial infarction (STEMI) and STEMI equivalents.
Methods
Eighteen chest-pain ECGs with adjudicated outcomes were selected: 8 STEMI equivalents, 8 STEMI mimics, 1 STEMI and 1 normal ECG. An LLM (ChatGPT-5) was queried 100 times per ECG. Its performance was benchmarked against 95 physicians and a deterministic DNN, the PMcardio Queen of Hearts model (aOMI v1), which yields identical outputs on repeat runs and was trained specifically for STEMI and STEMI-equivalent detection. Accuracy and the response consistency rate (agreement with the majority answer) were assessed.
Results
Among the 18 ECGs (50% STEMI by outcome), the LLM achieved 55.4% accuracy (95% CI 38.9–71.6%), compared with 88.9% (95% CI 72.2–100%) for the DNN and 64.7% (95% CI 51.2–77.1%) for physicians. LLM performance was significantly lower for STEMI mimics (19.9% accuracy; 95% CI 11.7–29.1%) than that of the DNN (87.5%; 95% CI 62.5–100%) and physicians (72.5%; 95% CI 61.3–83.0%). Response consistency was 84% for the LLM, with unanimous agreement in only 6% of cases, 100% for the DNN and 77% for physicians.
Conclusion
The deep neural network demonstrated superior accuracy and perfect reproducibility compared with physicians and a large language model, supporting its implementation for improved STEMI detection.
Share this article
About PMcardio
PMcardio is the market leader in AI-powered diagnostics, addressing the world’s leading cause of death – cardiovascular diseases. The innovative clinical assistant empowers healthcare professionals to detect up to 40 cardiovascular diseases. In the form of a smartphone application, the certified Class IIb medical device interprets any 12-lead ECG image in under 5 seconds to provide accurate diagnoses and individualized treatment recommendations tailored to each patient.
About Powerful Medical
Powerful Medical leads one of the most important shifts in modern medicine by augmenting human-made clinical decisions with artificial intelligence. Our primary focus is on cardiovascular diseases, the world’s leading cause of death.
Established in 2017, Powerful Medical has embarked on a mission to revolutionize the diagnosis and treatment of cardiovascular diseases. We are a medical company backed by 28 world-class cardiologists and led by our expert Scientific Board with decades of experience in daily patient care, clinical research, and medical devices. The results of our research are implemented, developed, certified, and brought to market by our 50+ strong interdisciplinary team of physicians, data scientists, AI experts, software engineers, regulatory specialists, and commercial teams.
