Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Medicine AI Software Apple Hardware Science Technology

Apple Watch Can Detect An Abnormal Heart Rhythm With 97 Percent Accuracy, UCSF Study Says (techcrunch.com) 102

Posted by BeauHD from the abnormal-is-the-new-normal dept.
According to a study conducted through heartbeat measurement app Cardiogram and the University of California, San Francisco, the Apple Watch is 97 percent accurate in detecting the most common abnormal heart rhythm when paired with an AI-based algorithm. TechCrunch reports: The study involved 6,158 participants recruited through the Cardiogram app on Apple Watch. Most of the participants in the UCSF Health eHeart study had normal EKG readings. However, 200 of them had been diagnosed with paroxysmal atrial fibrillation (an abnormal heartbeat). Engineers then trained a deep neural network to identify these abnormal heart rhythms from Apple Watch heart rate data. Cardiogram began the study with UCSF in 2016 to discover whether the Apple Watch could detect an oncoming stroke. About a quarter of strokes are caused by an abnormal heart rhythm, according to Cardiogram co-founder and data scientist for UCSF's eHeart study Brandon Ballinger. Cardiogram tested the deep neural network it had built against 51 in-hospital cardioversions (a procedure that restores the heart's normal rhythm) and says it achieved a 97 percent accuracy in the neural network's ability to find irregular heart activity. Additional information available via a Cardiogram blog post.
This discussion has been archived. No new comments can be posted.

Apple Watch Can Detect An Abnormal Heart Rhythm With 97 Percent Accuracy, UCSF Study Says More

Apple Watch Can Detect An Abnormal Heart Rhythm With 97 Percent Accuracy, UCSF Study Says

Comments Filter:

  • Read any tutorial on Bayes theorem. Chances are most of the positive results will be false positives, but neither patients/consumers nor their doctors understand that, they hear "97 percent accuracy" and "You tested positive".

    • Chances are that the anonymous coward hasn't actually read anything, and that the results are actually useful, as in "97% of all people with problems detected". And just in case people don't know: You can have severe heart problems without feeling anything bad. _And_ you can have heart problems without feeling anything for some time, and your diagnosis will be inaccurate because you didn't actually feel when the problem started.

    • Re: (Score:2)

      by Shoten ( 260439 )

      Read any tutorial on Bayes theorem. Chances are most of the positive results will be false positives, but neither patients/consumers nor their doctors understand that, they hear "97 percent accuracy" and "You tested positive".

      This is a crucial point.

      When I see things like "97% accurate" with respect to a diagnostic function, I have to wonder about the definition of "success." Is that just a 3% false-negative rate? If so, what's the false-positive rate...because if it turns out that the watch is wrong half the time when it signals an abnormality, that's bad too. If a diagnostic function cries wolf too often, it gets ignored and becomes useless.

      If, on the other hand, the 97% accuracy rate covers both false positives and false n

      • What's the false negative rate for people who never go to the doctor and never monitor their heart? If you are only at the doctor's office for 10 minutes a year and the doctor is only checking your heart rate for 60 seconds under regular resting conditions, then what is the chance that they will catch an abnormal heart rate that may only come up when you're exercising or sleeping?

        This sounds like a very useful tool, even if it isn't 100 accurate for positives and negatives all the time. What's the big harm

        • Re: (Score:2)

          by Ihlosi ( 895663 )
          What's the big harm if it says you have a heart condition and you go to your doctor and they detect no problems?

          Err ... seriously?

          You have to spend time and money to go to the doctor.

          Your doctor has to spend time to examine an actually healthy patient, reassure him or her that nothing it wrong.

          Alarm fatigue. After the third false alarm, you'll stop listening to them.

          Just to name a few points.

        • What's the false negative rate for people who never go to the doctor and never monitor their heart? If you are only at the doctor's office for 10 minutes a year and the doctor is only checking your heart rate for 60 seconds under regular resting conditions, then what is the chance that they will catch an abnormal heart rate that may only come up when you're exercising or sleeping?

          They will put on an EKG, and usually find nothing, because many arrhythmias are in transient episodes . Then, if you are female they will decide the problem is you are too emotional, and emotional stress caused some minor heart racing. Protestations that you were calm and almost falling asleep when your heart racing strangely left you out of breath -- that is why you are scared and emotional now, will only prove to the doctor you are liar who probably needs to see a psychiatrist about anti-depressants.

          No,

    • With a heart beating at roughly 100 times per minute that is 3 false positives every minute.. Not quite good enough..

      Depending on how they measure the precisions.

      • Do you really think they're detecting this based on single heartbeats? Doubtful. It probably collects data over weeks or months and analyzes the aggregate.
  • Most patients can do it too. You don't really need an electronic device to tell you that you don't feel right. On the bright side most abnormal heart rhythms are harmless and quite common as you get older. The ones you have to watch for are the ones associated with effort/exercise, the ones that last more than a few minutes, the ones associated with pain, dizzyness or shortness of breath, or the ones that keep recurring.

    • I had AF 3 weeks ago. I felt fine, just little tired. When it got really bad it had been going on for quite a while and it was a problem already. The electronic device might have saved me 3 weeks in hospital.

  • A heartbeat is basically a one dimensional list of pairs of numbers (strength of beat, time since last beat). Creating an algorithm to figure out when something like that starts getting fucky doesn't sound like a problem that needs the full power of deeplearningAIneuralnetothermarketinggibberishAPPSMOTHERFUCKER brought to bear on it.

    • Atrial fibrillation is characterized by being 'irregularly irregular'. It is really pretty easy to identify, at least for humans and even EKG machines. The mathematical algorithms are well known and well characterized. The major difficulty that the iWatch has is that it is only using one EKG lead.

      But even three lead monitors have no problems with that.

      But yes, it has some useful medical implications. "Paroxysmal" atrial fibrillation is when the underlying rhythm is normal but occasionally jumps into afi

      • Re: (Score:2)

        by Ihlosi ( 895663 )
        The major difficulty that the iWatch has is that it is only using one EKG lead.

        As far as I understand the cardiogram app, it doesn't to an ECG. It measures the pulse.

        Measuring an ECG would involve at least sticking an electrode on two out of RA, LL and LA.

  • Accuracy is a stupid/useless metric (Score:1)

    by Anonymous Coward

    I can make an HIV test that is over 99% accurate by classifying all tests as negative. Accuracy is a stupid metric.

  • Meaningless (Score:1)

    by Anonymous Coward

    5958 / 6158 = 0.9675

    It probably classified everyone as the negative case. I couldn't find the paper or the confusion matrix, but this seems like a lot of noise. Accuracy is a useless metric for class imbalanced data.

  • Cardiogram on Android? (Score:4, Insightful)

    by Gussington ( 4512999 ) on Thursday May 11, 2017 @09:57PM (#54403609)
    Cardiogram is also available on Android devices. Is TFA paid for by Cardiogram or Apple?

    • Re: (Score:2)

      by Anonymous Coward

      And which Android wearable did you mean?

      Apple Watch only have two versions of hardware, making the analysis and result very clear cut. Trying to study and compare the results of myriads of Android wearable+device combination is most likely a waste of time for a study with limited time and budget. Not surprising to see only Apple Watch have a publishable result.

      • And which Android wearable did you mean?

        Apple Watch only have two versions of hardware, making the analysis and result very clear cut.

        The sensors all do the same stuff, it's the software making the difference.

    • Re: (Score:3, Informative)

      by Anonymous Coward

      The story here is the study showing it to be accurate when used with the Apple Watch sensors. There's no study showing it to be accurate with Android watches. You're letting your platform fanboyism make you do stupid things.

      • The story here is the study showing it to be accurate when used with the Apple Watch sensors. There's no study showing it to be accurate with Android watches.

        I know, that's why I made the comment. The sensors should make no difference since Apple probably use the same hardware component made in the same factory as every other piece of electronic hardware on the planet.

  • So let's say a false negative rate of 3% - no mention of false positives.

    So for every 100 people who *do* have a abnormal heart rhythym, 3 won't know it.

    So if you sell it to 100,000 people, 3,000 people won't get the proper answer.

    At a certain point, it feels like those false negatives are going to affect a lot of people, who might look at Apple as liable for giving them incorrect information...and we haven't even looked at consequences from false positives.

    • This is going to be the problem with this, and Personal Injury Sharks will take note. People will "rely" on these devices for monitoring critical health issues when strictly speaking they should not. It really doesn't matter how big and bold Apple, FitBit, or whoever makes the disclaimer that it's not certified by the FDA for this sort of thing, the layers will still sue.

      • This is going to be the problem with this, and Personal Injury Sharks will take note. People will "rely" on these devices for monitoring critical health issues when strictly speaking they should not. It really doesn't matter how big and bold Apple, FitBit, or whoever makes the disclaimer that it's not certified by the FDA for this sort of thing, the layers will still sue.

        In America.
        In the rest of the developed world we will use tools like these to continue improving the quality of life as normal.

        • In America.
          In the rest of the developed world we will use tools like these to continue improving the quality of life as normal.

          In America, hospitals cannot replace one broken X-ray machine unless the manufacturer still has a model identical to the other models they are using. Because if you have nine of last years model and one new X-ray machine, everyone who's X-ray is done on an older machine sues you.

  • False positives (Score:4, Insightful)

    by postglock ( 917809 ) on Thursday May 11, 2017 @10:24PM (#54403693)
    > About a quarter of strokes are caused by an abnormal heart rhythm

    But what about the opposite? How frequently does an abnormal heart rhythm result in a stroke? TFA doesn't mention it.

    If this is a low proportion, then there will be many false positives, making detection of abnormal heart rhythm useless in terms of stroke prediction. It will only serve in increase anxiety of users.

    • "If this is a low proportion, then there will be many false positives, making detection of abnormal heart rhythm useless in terms of stroke prediction. It will only serve in increase anxiety of users."

      Well then I guess that over time it will get fairly good at predicting heart attacks.

  • The way I see it, having a watch tell you that you have an abnormal heart rate might in some cases potentially help prevent problems but more often than not it's simply going to scare the shit out of people for no good reason. While it is true that a number of problems can be related to an abnormal heart rate, it's also a fact that the majority of people with an abnormal heart rate have no problems as a result of it. Besides, my regular low cost, no-frills blood pressure monitor detects my abnormal heartbea
  • I'd love to have such an app, but I don't have an iPhone nor am I going to be buying an iPhone. The data plans are prohibitively expensive and I don't need mobile calling or texting at that cost, when I can get it with pay-as-you go for $5/mo on a flip phone. I do have an iPad with Wifi though, so if the Apple Watch ever works with those, this app would be enough to make me want to buy one.
  • What do they mean by "accuracy"?

    Also, by how much is the "machine learning, neural network" approach better than simpler approaches? There's no point in shooting machine learning bullets at things that can be analyzed with much simpler means to a similar degree of sensitivity and specificity.

  • ... and process those into a yes/no answer.

    I bet that a much simpler algorithm could produce similar results. But nowadays it seems to be the latest fad to throw machine learning at fairly plain signal/data processing problems.

    • FFT even an audio signal of the beat, or light-level through the finger or whatever.

      Produce some simple stat on the regularity and speed of the heartbeat from that data.

      Use that number to establishment a limit, to use as a diagnostic against those who are medically diagnosed with such conditions.

      Apply that limit to Yes/No answer.

      If it got WORSE than 97% accuracy, I'd be surprised.
      It it took more than a handful of code coupled with an audio/camera and FFT library, I'd be amazed.
      The processing power required

      • FFT even an audio signal of the beat, or light-level through the finger or whatever.

        Produce some simple stat on the regularity and speed of the heartbeat from that data.

        You'll probably want to analyze a pulse signal in the time domain (e.g. autocorrelation, slope/maximum search or similar), as the beat to beat pulse rate of even a healthy person is too irregular to be easily analyzed in the frequency domain. Also, it's harder to extract beat-to-beat variations in the frequency domain, as they happen on

  • ...when paired with an AI-based algorithm

    Sounds to me like it's the AI-based algorithm that's doing the detecting here. Not the watch.

  • Accuracy of a test can be deceiving when the base rate is lower than the inaccuracy of the test. In other words, if the accuracy of this test is 97% and the base rate of arrythmia is 2.5% (wikipedia) then false positives will outnumber true positives meaning that if your phone says you have arrythmia, there's about a 55% chance it's right, not a 97% chance. Take 350,000,000 people. 2.5% or 8,750,000 have arrythmia. 8,487,500 (97%) will recieve a correct positive reading. 262,500 (3%) will have a false negat

Slashdot Top Deals

God made the integers; all else is the work of Man. -- Kronecker

Close