Claire Cassianni, MSc, Geoffrey D. Huntley, MD, Matteo Castrichini, MD, Ashley P. Akerman, PhD, Mihaela Porumb, PhD, Christopher G. Scott, MS, Halley N. Davison, MBA, William Hawkes, PhD, Gary Woodward, PhD, Barry Borlaug, MD, Ross Upton, PhD, Patricia A. Pellikka, MD
View the full publication here
Heart failure with preserved ejection fraction (HFpEF) accounts for approximately 50% of diagnoses of heart failure (HF) and frequently leads to hospitalization. Clinical algorithms developed for diagnosis have been applied to stratify risk for HF hospitalization or death.1,2,3 Deep learning has been applied to the automated interpretation of echocardiograms, but limited information exists regarding the potential of the learning models for predicting clinical outcomes.4 An artificial intelligence (AI) model was recently developed to identify patients with HFpEF using a single apical four-chamber video clip from a standard transthoracic echocardiographic examination.5 A convolutional neural network was applied to the video clip. The model comprised a series of three-dimensional convolutional layers designed to operate on two-dimensional videos over two in-plane spatial dimensions within the image frames and across the time dimension. The present study was conducted to assess the association between the model output and other HF biomarkers, risk for HF hospitalization and cardiac mortality, and to compare its performance with two clinical scores: H2FPEF (heavy, hypertensive, atrial fibrillation, pulmonary hypertension, elder, and filling pressure)6 and HFA-PEFF (Heart Failure Association pretest assessment, echocardiography and natriuretic peptide score, functional testing, and final etiology).7
This retrospective, multisite study was approved by our institutional review board.
The model was developed to classify patients with HFpEF vs individuals without HFpEF (control subjects). Patients with HFpEF were defined according to guidelines and included a diagnosis by the treating physician1 within 1 year of an echocardiographic examination demonstrating elevated left ventricular filling pressure. Control subjects were patients undergoing clinically indicated echocardiography who lacked these features (Supplemental Appendix). All patients had left ventricular ejection fractions ≥50%. The present analysis used the second version of the AI model (Supplemental Appendix). In the previously described independent test population consisting of 646 patients with HFpEF and 638 control subjects, the updated model produced 95 uncertain outputs (7.4%); in the remaining 607 patients and 582 control subjects, sensitivity was 89.8% (95% CI, 87.5%-92.5%), specificity was 86.3% (95% CI, 83.6%-89.7%), negative predictive value was 89.0% (95% CI, 87.0%-91.7%), and positive predictive value was 87.2% (95% CI, 84.7%-89.8%).
Incident HF hospitalization was obtained from electronic health record chart review using standardized definitions, using the first event after the echocardiographic examination. Mortality was obtained from the National Death Index, and causes of cardiac deaths were manually reviewed. End points were adjudicated by investigators blinded to AI analysis results. Cardiac mortality and HF hospitalization were plotted accounting for death as a competing risk. The method of Fine and Gray was used to estimate the hazard ratios (HRs) adjusted for differences in age and sex.
Among 1,284 patients followed for a median of 3.4 years (interquartile range, 1.7-6.5 years), there were 252 HF hospitalizations and 540 deaths. Figure 1 demonstrates the risk for HF hospitalization on the basis of HF categorical AI output (top) and quartiles of continuous probability output (bottom). After adjustment for age and sex, positive AI output was associated with a higher risk for HF hospitalization than negative output (HR, 3.76; 95% CI, 2.71-5.21; P < .001) and likewise for uncertain output (HR, 2.79; 95% CI, 1.60-4.62; P < .001). Cardiac deaths (n = 135) were attributable to HF in 63 patients (47%), to coronary artery disease in 55 (41%), to valve disease in five (4%), to arrhythmia in five (4%), and to other causes in seven (5%). Again adjusting for age and sex, cardiac mortality was higher in patients with positive output (HR, 5.55; 95% CI, 3.28-9.37; P < .001); patients with an uncertain output tended to have a higher mortality (HR, 2.22; 95% CI, 0.94-5.24; P = .07). Patients with higher continuous probability outputs demonstrated incrementally higher risk for cardiac mortality (fourth quartile vs first quartile: HR, 11.65; 95% CI, 4.65-29.20; P < .0001). Figure 2 demonstrates the risk for HF hospitalization on the basis of clinical H2FPEF score6 (top). The clinical score differentiated high and low risk for HF hospitalization, but 776 of 1,284 patients (60%) were indeterminate. Application of the AI model to the nondiagnostic H2FPEF outputs (bottom) allowed the classification of all but 68 of the 776 patients (8.8%). The AI model demonstrated a similar relationship between output and risk for HF hospitalization in patients with and those without diagnostic H2FPEF output. Findings were similar when patients were stratified according to HFA-PEFF score. HFA-PEFF score, brain natriuretic peptide, and N-terminal pro–brain natriuretic peptide also differed according to the AI model prediction (Table). Few patients underwent exercise testing; differences in exercise capacity were not significantly different.
Figure 1. Risk for HF hospitalization was higher in patients with positive and uncertain AI model output (top). Those with output scores that placed them in higher quartiles had incrementally greater risk for HF hospitalization (bottom), accounting for competing risk for death.
Figure 2. Risk for HF hospitalization was higher in patients with positive H2FPEF outputs (top), but many (n = 776) had nondiagnostic outputs. Application of the AI model was able to reclassify 708 (91%) of the nondiagnostic H2FPEF outputs (bottom).
Discussion
In this study we assessed the ability of a novel, HFpEF AI model using a single echocardiographic video clip to identify patients at increased risk for HF hospitalization and cardiac mortality. In summary, (1) positive model output was associated with higher risks for HF hospitalization and cardiac mortality, (2) patients with uncertain outputs demonstrated intermediate risks for these end points, (3) HF hospitalization and cardiac mortality risk were incrementally associated with higher model probability output scores, and (4) the AI model reclassified HF hospitalization risk in nondiagnostic clinical scores, including 91% for H2FPEF outputs and 92% for HFA-PEFF. This is the first AI echocardiographic model to produce outputs discriminating a specific disease (HFpEF) that are incrementally associated with risk for HF hospitalization and cardiac mortality. Prospective studies are required to confirm these retrospective results, to externally validate the AI model’s outputs in other echocardiographic laboratories, and to understand the implications for patient management. Studies using a broad representation of HFpEF phenotypes should be undertaken to understand the generalizability of this model in a naturally heterogeneous clinical syndrome.