Ross Upton PhD, Founder, CEO and Chief Scientific Officer, Ultromics, and AI Co-Chair, HeartShare, NIH (and team)
Ultromics, Oxford, UK
Read the full slides from the ESC 2024 HOT LINE Session
Artificial intelligence (AI) applied to cardiovascular imaging has been proposed to augment clinical decision-making. However, none have been evaluated in a multi-center, prospective randomized controlled trial that examines the impact on patient care pathway outcomes.
PROTEUS was designed to test, in real-world practice, whether AI interpretation of stress echocardiography augments clinician selection of patients for angiography. In coronary artery disease, clinicians rely on non-invasive imaging to identify disease and select a management approach. Stress echo is one of the most common tests but relies on visual qualitative assessment.
Previously, we reported the development of an AI device trained to identify patients with severe coronary artery disease from stress echo images. The AI device is a supervised machine learning classifier utilizing novel geometric features that capture LV motion between rest and stress. The AI device demonstrated excellent performance and improved less experienced clinicians to the level of expert clinicians in a reader study.
PROTEUS was a multi-center, randomized controlled trial in adults undergoing a clinically indicated stress echocardiogram to assess inducible ischemia from 20 NHS sites across the UK.
Inclusion Criteria:
Exclusion criteria: (i) Patients with more than moderate valvular heart disease.
(ii) Left ventricular outflow tract obstruction.
(iii) Significant co-morbidities with an expected life-expectancy of under 12 months.
(iv) Previous coronary artery bypass graft or other cardiac surgery.
(v) Congenital or inherited myocardial disease.
Patients were randomly assigned to either standard decision-making (Control), or AI-augmented decision-making (Intervention). Patients in Control underwent stress echocardiography per local healthcare protocols. Patients in Intervention also underwent stress echocardiography, but images were sent for additional assessment by the AI device.
All patients were followed up for six months after stress echocardiography, and their records adjudicated by a committee of cardiologists.
Secondary analyses examined non-inferiority in pre-specified subgroups where interpretation is known to be more challenging.
A total of 4,907 patients were screened at 20 NHS centres. 1,175 patients were randomized to Control and 1,166 to Intervention. Groups were well matched for demographics and cardiovascular disease history.
The trial did not meet its primary endpoint, most likely due to the low angiography referral rate. It was powered on data from the EVAREST observational study of stress echo practice within the NHS. EVAREST recruited ~10,000 patients between 2009 and 2020, and ~15% of the target population were referred for angiography. Within PROTEUS, the referral rate was only ~8%.
One potential driver for this drop in angiography referral may be recent large-scale randomized trials demonstrating non-inferiority of medical therapy. Despite the low number of angiography referrals, the results indicate AI-Augmentation may have utility in low volume stress echocardiography centers. Clinicians who perform fewer stress echocardiograms a year may have less opportunity to maintain their level of expertise.
These findings are supported by data in a previous reader study (Upton et al. 2022) demonstrating that AI-augmentation improved the lowest-performing operators, increasing accuracy to a level consistent with the highest-performing operators. These hypothesis-generating results should inform further trials, identifying and developing areas in the patient care pathway that might benefit from AI.
This is the first prospective, multi-center, randomized controlled trial for an AI device in cardiovascular imaging, examining the impact on patient care pathway outcomes. AI-augmentation did not meet the pre-specified non-inferiority endpoint for appropriate referral to angiography compared to standard decision-making, driven by a lower-than-expected angiography referral rate. However, AI-augmented decision-making may aid clinicians performing stress echocardiography in low volume centers.