Although Apple, based on its internal research, claims that the Apple Watch (AW) ECG has a 98% sensitivity and a 99% specificity for detection of atrial fibrillation, doubts have been raised about its accuracy in the real world.
I have recently reported on Apple Watch’s inability to diagnose atrial fibrillation (AF) when the heart rate is >120 beats per minute. This inherent limitation means AW has a built-in reduced sensitivity (which was not present in the testing group.)
In a Research Letter published online Feb. 24th in Circulation, Dr. Marc Gillinov, reports on the accuracy of Apple Watch in a population of patients who were post cardiac surgery and therefore on cardiac telemetry with a high risk of going in and out of AF.
Rhythm assessments using the Apple Watch ECG were performed 3 times per day over 2 days on 50 patients. Comparison was made between the watch reading (Sinus rhythm, AF, or inconclusive) and an expert human interpretation of the PDF from the watch and simultaneously obtained telemetry rhythm strip.
The results were disappointing for the AW.
The AW4 notification correctly identified AF in 34 of 90 instances, yielding a sensitivity of 41%. Of 25 patients with at least 1 episode of AF, AF was identified in 19. Among patients in SR, none was designated as AF (ie, no false positives); however, rhythm was deemed inconclusive in 31% of patients, and there was no additional attempt to assess rhythm. Overall agreement between AW4 notification and telemetry was 61% (κ statistic = 0.33 [95% CI, 0.24–0.41]).
This confirms my prediction that AW would identify less than half of AF cases.
I have to believe that the 29 cases diagnosed as “inconclusive” were due to the AW AF inherent blinding limitation related to rapid heart rate. If we presume these would all have been correctly identified as AF (if the AW had not been hamstrung) then the sensitivity increases to 70%.
The authors of this article don’t seem to understand the difference between unreadable (meaning too much artifact to make a diagnosis) versus inconclusive (which Apple only uses when the AF is > 120 BPM.) They conclude by saying:
The unreadable (ie, inconclusive) rate reported in that study was 6% compared with 31% in this pilot study.
They have muddled together unreadable and inconclusive.
I do strongly agree with their final conclusions
Variations in sensitivity between these 2 studies suggest the need for further validation before this technology is adopted by the public for AF detection. Physicians should exercise caution before undertaking action based on electrocardiographic diagnoses generated by this wrist-worn monitor.
Indeed, any diagnosis from the Apple Watch itself should be confirmed by a cardiologist who is an expert at interpreting these single-lead ECG recordings.
Conclusively Yours,
-ACP
17 thoughts on “New Study Confirms Poor Apple Watch ECG App Sensitivity For Atrial Fibrillation”
Do PVCs still confuse Kardia ECG devices, as you earlier wrote? Does that also apply to the Complete? Any point in someone with just PVCs obtaining one of these new devices?
The Complete is using the Kardia ECG algorithm with a very similar electrode configuration so expect the same performance with respect to PVCs.
PVCs are less likely to be interpreted as afib by Kardia than the frequent APCs. Kardia is still unable to diagnose PVCs and most likely will call an ECG with PVCs normal , sometimes unclassified. If you can learn how to recognize PVCS yourself the device is useful or if you have a cardiologist who doesn’t mind looking at the tracings.
To those with AW, do you get notice if you get any type of arrhythmia even if it doesn’t know it’s AFib? Eg if you go out of rhythm during the night will it catch it? Even if it doesn’t diagnose it properly?
Brian I wouldn’t be able to tell “in the night” unless I woke up to activate the ecg reading. It’s not continuous. I could infer arrhythmia by looking at my heart rate pattern when I woke up. If it went up and stayed up the I would likely be out of rhythm. Then I’d push the button to activate the ecg to check.
Thanks for that. So I’m inferring that the heart rate is continuous but not the EKG?
So if you had 30 mins of AFib during your sleep you would have information regarding that heart rate but not the EKG?
How do you know that you HR was e.g., very high? Does it specifically tell you that or do you have to go look in a history file?
Thanks again
Brian yes HR is monitored continuously with AW4. If my HR was higher than usual during my sleep I’d suspect afib but I can’t know just from looking at the data the next morning.
I know my typical HR range so when it goes up a lot I know it’s high. An increase of more than 15bpm sustained is suspicious to me. Especially while sleeping (though I do see variation thru the night depending on sleep cycles).
If you have more questions about AW4 and ECG or HR monitoring I encourage you to talk to your EP. I’m a patient who’s done a lot of reading and personal analysis on this subject.
I’m in a state of permanent flutter/afib now, with meds to control the rate. It’s been a frustrating process but at least I feel good enough to go about my life!
We have to clearly separate the two AW rhythm features when discussing results.
1. Monitoring of rhythm the PPG sensors which is continuous. If a significant irregular rhythm is found a notification should be registered. Theoretically this should cath atrial fibrillation of significant duration. It’s not clear what the sensitivity of AW is for afib. Specificity appears high. Also the AW can be set to notify you if HR<40 or > 100 or 120.
2.Recording of AW ECG which only happens if initiated by the wearer. this feature has the inherent flaw of being inconclusive if HR>120 BPM
Thanks Dr A. I had hoped that an irregular rhythm would trigger an ECG so that if it happened at night then I’d see what happened.
I have PAF with very minimal burden, hardly any in fact, but since next year I reach 65 and become CHADS-V of 1 I’d like to be certain of my AF burden. I’m not really depending on the AW to determine conclusively if it’s AF, but I want to see the ECG and normally I can take a good guess myself.
Nope. If you are awake and see the irregular rhythm notification and then make ECG recording that’s the only way you are going to capture an ECG and be able to make a diagnosis. You have to get your finger on that crown. Now I could see Apple creating a process by which two sensors are in the watch or one added to the band that would allow some recording of ECG activity but I don’t know what it would look like from these positions.
You are in a situation that many of my patients are in. We’ve identified brief episodes of atrial fibrillation (sometimes from pacemakers, sometimes from long term monitors applied for other reasons, sometimes serendipitously during a procedure). These episodes may or may not be felt. Most of the time these episodes result in an abrupt jump in the heart rate which AW will detect.
What is the risk of stroke in this situation? I hope to publish a post soon on my approach to these patients. Stay tuned.
Thank you!
I’ve almost never been able to generate anything other than an “inconclusive” result with my Apple watch (or Kardia). Last visit to my EP two weeks showed no afib in the past six months (yay), so the Dofetilide is doing what it is supposed to. But given my ASD repair, all the scarring from my surgery, ablation, and two different sets of pacemaker leads, rhythm issues are just par for the course. I’m just happy to be living as well as I am.
From a consumer’s perspective, I’d be happier with greater ‘consistency of recording (for later human interpretation) ‘ in place of inaccurate “Good News” – or even absent – diagnosis.
So o just signed up for the new study using this watch. I wonder if I should withdraw
The author’s conclude: “…Physicians should exercise caution before undertaking action based on electrocardiographic diagnoses generated by this wrist-worn monitor….”
————————
I think the author’s missed the big point — i.e that no action should be taken based on ANY machine generated diagnosis, from any ANY of the current crop of ecg monitors, be it AW, Kardia, etc. and perhaps even office machines!
That’s because all these algorithms are flawed and should be reviewed by a trained professional before undertaking a treatment decision.
That said, I find the quality of the AW tracings comparable to Kardia, and therefore, like the Kardia, very useful in people directed treatment decisions.
Jim
This study confirms my experience with my own AW4. I am in permanent flutter/afib and the ECG readings from it rarely indicate AF detected. Most readings are normal sinus rhythm, a few are inconclusive and none have had HR >120.
Interesting. If you have any of the tracings showing “inconclusive” at HR <120 BPM I'd be interested in seeing them. at drp@theskepticalcardiologist.com
I’m sorry I don’t find this article interesting. My Apple Watch reads my Afib every time. That is because I am in permanent a Afib.