Forgot your password?
typodupeerror
Government Medicine Open Source Hardware Technology

SFLC Wants To Avoid Death by Code 247

Posted by timothy
from the me-too-me-too dept.
foregather writes "The Software Freedom Law Center has released some independent research on the safety of software close to our hearts: that inside of implantable medical devices like pacemakers and insulin pumps. It turns out that nobody is minding the store at the regulatory level and patients and doctors are blocked from examining the source code keeping them alive. From the article: 'The Food and Drug Administration (FDA) is responsible for evaluating the risks of new devices and monitoring the safety and efficacy of those currently on market. However, the agency is unlikely to scrutinize the software operating on devices during any phase of the regulatory process unless a model that has already been surgically implanted repeatedly malfunctions or is recalled. ... Despite the crucial importance of these devices and the absence of comprehensive federal oversight, medical device software is considered the exclusive property of its manufacturers, meaning neither patients nor their doctors are permitted to access their IMD's source code or test its security.'"
This discussion has been archived. No new comments can be posted.

SFLC Wants To Avoid Death by Code

Comments Filter:
  • by querky (1703040) on Thursday July 22, 2010 @07:00PM (#32998094)
    the software running your pacemaker is probably patented too!
  • So what (Score:5, Insightful)

    by clarkkent09 (1104833) on Thursday July 22, 2010 @07:05PM (#32998136)
    Does a government agency examine the source code which keeps airliners in the air, cars on the road, nuclear plants from blowing up etc etc? If the government is going to evaluate and approve every important piece of code line by line we will pretty soon run out of programmers. But then, chip designs will have to be evaluated too because they can fail as well. Next, mechanical designs, engines, turbines, reactors, better make sure that the government is stocked with experts in all those fields too.

    After all, nothing can possibly be safe until it is certified as such by the government. Just ask hundreds of thousands of people who died while the drugs that could have saved them were waiting for the FDA approval. They are pretty safe now.
    • Re:So what (Score:5, Insightful)

      by QuantumG (50515) * <qg@biodome.org> on Thursday July 22, 2010 @07:11PM (#32998202) Homepage Journal

      I think you miss the point. You should be able to examine the code in the pacemaker inside you - or hire an expert to do so.

      • That's not really what the summary says when it complains that "[FDA] is unlikely to scrutinize the software operating on devices during any phase of the regulatory process unless a model that has already been surgically implanted repeatedly malfunctions or is recalled". But even so, where do you draw the line. What is the principle involved here: that you should be able to examine the software (and presumably hardware?) design of any device that has impact on your survival? If so, that opensources a huge n
        • by QuantumG (50515) *

          However much a reasonable person agrees should be. Note that most reasonable people don't consider RMS a reasonable person, so it's somewhere between pacemakers and text editors.

        • by PopeRatzo (965947) *

          What is the principle involved here: that you should be able to examine the software (and presumably hardware?) design of any device that has impact on your survival?

          Of course. You, and your doctor, and any agency that is allowing that product to be sold for a particular therapeutic purpose should be able to examine the software (and hardware ) design of any device that is sold specifically for a medical purpose.

        • But even so, where do you draw the line. What is the principle involved here: that you should be able to examine the software (and presumably hardware?) design of any device that has impact on your survival?

          I would say that I should be able to examine the software (and of course hardware!) of any device that I own. People have certainly never needed anybody's permission to examine, and even modify, their own property since the beginning of time; why the heck should that magically change now just because the

          • by tehcyder (746570)
            But what happens the first time an adventurous but unskilled hacker fucks up his own pacemaker and kills himself? No doubt his relaives would sue his doctor and the manufacturer anyway (in the US).
            • what happens when that same person accidentally stabs himself in the eye with a screwdriver?

              or jumps off a bridge?
              no doubt his relaives would sue the company which built the bridge and his shoe manufacturer(since he walked to the bridge) anyway.

              "people might hurt themselves through stupidity" is an endless argument which could be applied to everything in the universe and every freedom or right.

              it is not a decent argument against this.

      • Oh please, so many problems in the US today are by enforced empowerment of litigious idiots. Not perfect? RARGH SUE. What's this? one of little jimmy's toys comes from a country where people sometimes fall over? RARGH SUE. What's this? my tap water has one atom per gallon of uranium, only detectable due to recent advances in hypersensitive ICP-AES? RARGH SUE. Liability is paralysing your nation, and all this will mean is less people making pacemakers.

    • by mirix (1649853)

      Mission critical things (life support, nuclear core monitors, etc) sure as fuck should have an independent code review.

    • by meerling (1487879)
      Whether or not the government looks at / approves of the code, it should be available to both the medical profession and those who's bodies it's being implanted in. As far as I'm concerned, the moment a piece of hardware is placed in my body, it totally freaking belongs to ME, just like the rest of my organs.
      • by XanC (644172)

        You don't have to have the device installed at all, you know. You're the one who needs a service from them.

    • Problem is that most of the stuff you listed, if it breaks and causes a death, the manufacturer and owner/operator can be held liable and sued. IIRC, most implantable medical devices are shielded from tort claims.
    • Re:So what (Score:5, Interesting)

      by wiredlogic (135348) on Thursday July 22, 2010 @07:44PM (#32998508)

      In the case of avionics, there are rigorous design and testing standards for electronics, software, and mechanical hardware that are mandated by the FAA. Passing them is part of the certification process. This task can be handled in house or by third parties that specialize in that task. The medical industry should largely be applying the same principles.

      • Re:So what (Score:5, Informative)

        by Achra (846023) on Thursday July 22, 2010 @09:09PM (#32999028) Journal

        In the case of avionics, there are rigorous design and testing standards for electronics, software, and mechanical hardware that are mandated by the FAA. Passing them is part of the certification process. This task can be handled in house or by third parties that specialize in that task. The medical industry should largely be applying the same principles.

        EXACTLY. First informed post I've read on this story. I've made a career out of working on medical devices of all levels of concern (yes, including a heart pump) and the V&V process is basically as the parent states. There is a fairly rigorous validation process which is performed on the device (over the course of months to years, depending on complexity of the product and level of concern). These things aren't exactly shuffled out the door like Microsoft shuffles out a new OS (yes, I've worked there too). There is a LOT of diligence involved in receiving 510k clearance on a new device.

        • Re:So what (Score:5, Informative)

          by gurudyne (126096) on Thursday July 22, 2010 @10:38PM (#32999522)

          I've tested medical device software and I had to sign my name on forms over 5K times for just one version. This was just for the behavior and appearance of the localized GUI, not the pure functionality. Each test was recorded via video. The 90GB of video, 4GB of datasets, and the 220 pounds (100kg) of signed test forms were shipped at the end of the 6 week series.

          At the medical device customer's end, all of the tens of thousands pages of signed and initialed test forms were scanned and burned to disks. The plan to hang on to these for about a century.

          Then, the forms are updated and reviewed, new languages and OS versions added and the cycle continues. Every step is reviewed and audited. We don't want the FDA asking 10 years from now if something was tested or considered for testing without giving defensible answers.

          The folks testing the functionality of the software had close to 100K of tests for each version of device software. (Different vendor, so I am going by what the device company told me.)

          We all reported to the same defect database, so we could be aware of progress and problems.

          Long hours, fun times.

      • by tsa (15680)

        Finally someone who talks sense. It's so easy to say that everybody and his dog should have access to the source code of my pacemaker, but then what? I sure wouldn't want my doctor to change the software because he thinks he knows better than the experts at the manufacturer's. I think it's completely useless to have anyone but the manufacturer have access to the source code.

    • by AHuxley (892839)
      Clark most of the world has tried to move beyond tombstone technology.
      Why wait for enough tombstones for the technology to get fixed?
      That seems to work well for airliners, trains and nuclear plants.
      Cars seem to need some help too.
      Drug approval is a given in the US, the idea because its a chip they can have a free pass seems great in the short term, but will catch up with many.
      Who will pay for new devices for a generational fault? Better to at least have some outside on average that just trusting the
      • by rcamans (252182)

        Wait a minute. Did he just say airliners are fixed for known safety issues?
        I call BS on this one.
        Most airplanes are full of foam plastic walls and flamable seating, an old problem airplane manufacturers refuse to fix.
        Many similar issues exist.

        • by AHuxley (892839)
          With every crash the FAA reports and new guidelines filter out, sort of :) Less doors falling off.
    • Not just government (Score:3, Interesting)

      by weston (16146)

      Does a government agency examine...

      How about the other entities mentioned in the summary (let alone TFA) -- patients and, more importantly, *doctors*? If not them -- who should review them?

      After all, nothing can possibly be safe until it is certified as such by the government. Just ask hundreds of thousands of people who died while the drugs that could have saved them were waiting for the FDA approval. They are pretty safe now.

      FDA approval works roughly about as well as "self-regulation" works, since the F

    • Just ask hundreds of thousands of people who died while the drugs that could have saved them were waiting for the FDA approval.

      What is your source for these numbers?

      I think you'll find that the experimental protocol at best simply extends the life of the terminally ill patient for some few weeks or months. It is not a miracle cure - it is an investment in the future.

      39% of lung cancer cases are diagnosed after the cancer has already metastasized (distant stage). The corresponding 5-year relative lung cance

      • the source is easy, just look at the number of drugs that are claimed to save thousands of lives per year and multiply by several years they spent waiting for approval
    • Did you completely miss the point of what the SFLC wants? Generally(somebody could probably dig up an exception somewhere) the free software types are not looking for a "Ministry of software" to enforce their aims. They are looking to secure the four freedoms for themselves and other software users and creators(and, on a cultural level, they tend to want most people to be at least a touch of both, rather than just "consumers").

      Medical devices are already blackbox tested for function, the SFLC presumably
      • These guys put together custom code and hardware that will run for years at a time on a single battery. It's hard to do at all and incredibly hard to do well. Not unreasonably, they are not excited about sharing that code with anyone outside the organization.
        • To be fair, black box testing is the foundation of device testing in the health field. And for simple devices it is exactly what you want: making sure that the outcomes are as specified.

          However, as any engineer knows, for complex devices there can be innumerable states, and no test can achieve good coverage of these states. So it is appropriate to vet the internals of these complex devices.

          Just because companies wish to keep these details as trade secrets does not mean that it is in the public interest

  • by Dunbal (464142) *

    The devices themselves are rigorously tested in clinical trials. If they pass those tests, what more do you want?

    • by Meshach (578918)
      Even more so how many doctors or patients are going to have the knowledge to "examine the source code" and tell whether it is working properly?
      • Re: (Score:3, Insightful)

        by julesh (229690)

        Even more so how many doctors or patients are going to have the knowledge to "examine the source code" and tell whether it is working properly?

        It only takes one or two to achieve useful results.

      • by evilviper (135110)

        how many doctors or patients are going to have the knowledge to "examine the source code" and tell whether it is working properly?

        I would be HIGHLY MOTIVATED to learn.

    • Well said.

      I've got one of these things - a result of conductive systems failure (CSF) - it means the top half can't talk to the bottom half to coordinate/synchronize pumps.
      In a way, I whole heartedly (pun intended) agree with your statement - but then I start to think - Windows 95 probably could have passed a clinical trial - and then came the hackers.
      So, I got this thing in my chest that keeps me alive, can be communicated with via an electromagnet, and has anyone ever really considered what would h
    • Re:Why? (Score:4, Interesting)

      by julesh (229690) on Thursday July 22, 2010 @07:15PM (#32998250)

      The devices themselves are rigorously tested in clinical trials. If they pass those tests, what more do you want?

      Software errors can (and in fact are most likely to) result in pathological behaviour in unusual circumstances. Example. [wikipedia.org] "The failure only occurred when a particular nonstandard sequence of keystrokes was entered on the VT-100 terminal which controlled the PDP-11 computer: an "X" to (erroneously) select 25MV photon mode followed by "cursor up", "E" to (correctly) select 25 MeV Electron mode, then "Enter", all within eight seconds. This sequence of keystrokes was improbable, and so the problem did not occur very often [i.e. not in any clinical trials] and went unnoticed for a long time." An independent source-code audit could have saved three lives in that case.

      • Re: (Score:3, Insightful)

        by Shinobi (19308)

        A source code audit would not necessarily have found it. Like with so many other obscure faults, most likely, you'd have to go through a full trial and error on an actually running system, since you do not always know beforehand if the error is introduced by the specific source code, the compiler or anything else.

      • Re:Why? (Score:5, Insightful)

        by vux984 (928602) on Thursday July 22, 2010 @08:41PM (#32998846)

        An independent source-code audit could have saved three lives in that case.

        =Could have= saved 3 lives.

        Would have cost 10s of thousands? millions?

        Pretty much every time someone on the planet dies of accidental causes there is some procedure or process that "could" have saved them.

        Life just isn't that safe. And I'd rather not spend every dime of the gdp trying to make it as safe as possible.

        When people die its tragic. If its something simple to fix, we fix it. But lets not lay guilt trip down every time anybody dies. Life is dangerous and it wouldn't be worth living if we made it safe, because the only way it will ever be safe is if we lock everyone up in straight jackets in padded rooms.

      • by drsmithy (35869)

        An independent source-code audit could have saved three lives in that case.

        What evidence do you have that an independent code audit would have had any more chance of catching the error than an internal code audit ?

    • Re:Why? (Score:4, Insightful)

      by mirix (1649853) on Thursday July 22, 2010 @07:15PM (#32998252)

      I'm sure Therac-25 [wikipedia.org] passed some sort of trials too. That didn't stop it from killing people, of course.

    • by msauve (701917)
      And if you have a bad ticker, are you going to refuse a pacemaker because they won't release the source code?

      Maybe the folks at the SFLC should consider building an Arduino based pacemaker, then they can write their own GPL licensed software. They can invest the money to get it FDA approved, too. But, I suspect what they really want is to force others who have already made that considerable investment to disclose their work for all others to see.
  • Too bad this story can't be combined with this story: http://www.nytimes.com/2010/07/20/health/20docs.html?_r=2 [nytimes.com]

    That would save us all a lot of trouble.
  • by Anonymous Coward on Thursday July 22, 2010 @07:09PM (#32998168)

    One of the July 2010 updates bluescreened my 81-year-old dad.

    The hospital backed out the update but they had to reboot him in safe mode and go up the back door.

     

  • This seems similar to other highly proprietary hardware platforms that vendors keep locked down, either for market dominance, or for *security*. Breathalyzers, police radar guns, ATMs, hearing aids, etc, etc.

    On the other side of things, imagine the scandal of somebody with a pacemaker installed for the purpose of athletic advantage, perhaps at the Olympics. Can you say heart hack? The winning line-up of the hacked-pacemaker 500, by embedded OS of choice:

    1. DSL (Damn Small Linux), lightweight, fast, an
    • Re: (Score:3, Insightful)

      by JustOK (667959)

      OSX: soon to be ad supported, will only beat during approved activities, phones home with details about your liver.

    • Re: (Score:3, Insightful)

      by JamesP (688957)

      No

      WIth the exception of ATMs (and some radar guns) I wouldn't even bother with an OS

      And that's GOOD. I DON'T want anything more complex than a couple (ok, 100) of lines of code in my pacemaker, thank you very much

      It doesn't NEED to be more complex than that, and it SHOULDN'T

      • Re: (Score:3, Informative)

        by demonlapin (527802)
        It needs to be a great deal more complex if you want to do something more than just be alive.

        Adaptable rates? You'll need a motion-detection routine in order to speed the heart up so that people can enjoy even the mildest exercise.

        Pacing only when needed, not when it's not? You'll need more code to identify when a beat has occurred within the correct time interval.

        How about automatic defibrillators? Those are the devices that will shock a heart back into a normal rhythm, which is far more than a regula
  • by chaim79 (898507) on Thursday July 22, 2010 @07:11PM (#32998208) Homepage

    I work for a company does full life-cycle development and verification of safety-critical software, the main areas we work in are aircraft instrumentation, smart munitions, and medical equipment (including pacemakers). The amount of testing and verification that goes into these software categories often exceed the development cost, and at every level it is documented and traced. What on earth do Doctors think they will see in the source code? We do verification, peer review, tracing, etc. what would an MD find that a room full of software, system, and QA engineers wouldn't? About the only thing that they would be able to look at and have a hope in understanding is criteria for taking action, and that is in the requirements and should be reviewed at that level, not at the code level.

    Next thing they know Pilots will demand the ability to review the code for their cockpit management system and soldiers the ability to review the code for their Anti-Tank rockets!

    • by mirix (1649853)

      So you do what people want the FDA to do, but are unable to. Not sure what you're getting at.

      They want a third party (the FDA) to review code on the manufacturers device to make sure there are no hidden bugs. No one said they want random MD's to do code review.

    • Re: (Score:3, Funny)

      by segin (883667)

      Oh, so because a few employees within a company (and maybe a closely related partner) have looked over the source, it's "peer reviewed"? Peer review means that EVERYONE can examine the source, including people you have never met nor have even heard their names. It means that people you absolutely hate can review your source, not just a few of your employees that have no qualms about lying and saying it's all good just to keep their jobs.

      In other words, your source code has had as much legitimate peer revi

      • Re: (Score:3, Insightful)

        by chaim79 (898507)

        yah, you have no clue.

        If you were able to sit down and listen in to any of our peer reviews or look through our test cases and procedures you might get an understanding. We work on Safety Critical software, there are no 'qualms about lying', and just 'saying it's all good' will in fact cause you to lose your job and fast. We regularly work on DO-178B Level A projects, that's the kind of project where if something fails people will die. As it stands I doubt there is an airline in the USA that doesn't have so

    • Re: (Score:3, Funny)

      by rcw-home (122017)

      The amount of testing and verification that goes into these software categories often exceed the development cost

      That puts the testing quality roughly somewhere between most video games and Windows.

    • Re: (Score:3, Insightful)

      What on earth do Doctors think they will see in the source code?

      That you did your job as you say you did. That something can go right and that laws were respected is no surprise to me. But I want to make sure that that is the case. You probably only see the cases that have a good testing. I want to make sure I am not depending my life on a device that was not tested adequately. I worked in both aviation and medical firms, and the security attitude of the medical world really scared the living daylight out of me.

      So no, I will not take adequate medical testing for granted

    • Re: (Score:3, Insightful)

      by StormReaver (59959)

      The amount of testing and verification that goes into these software categories often exceed the development cost...

      Then what's the harm in releasing the source code so those who are qualified to review it can do so?

      The most likely answer is: "to protect our proprietary secrets from competitors!"

      My response to that is, "what proprietary secrets?" If every company does the type of due-diligence you claim, then everyone in the field is already at the same level of competence and will not benefit from someone else's code. If not every company performs the same level of diligence, then that's all the more reason to have th

  • For safety-critical software, there indeed should be a required certification regime for reliability. In the security field there is, for example, the Common Criteria. Security is one aspect of reliability (not the other way around). For too long, we have lived without any way of knowing how much effort has been put into making a system reliable. For a phone app this might not matter, but for a pacemaker it does matter.
    • Re: (Score:2, Interesting)

      by htdrifter (1392761)

      The FDA requirements on software are strict. There are requirements for coding practices, testing, QA, etc. Inspectors show up, without notice, to check for compliance.
      The code reviews are very thorough and require a manager and at least two other programmers.
      All code has to be instrumented and scripts written to force execution of all code.
      The output traces from instrumentation have to be fully documented. Everything that happens is documented.

      They require the source code with all changes documented,

  • If they properly test the device, the everything should be covered.

    I think the FDA does need to realize there is a software component. For no other reason then to require a full recertification of the devise every time the firmware changes. The risk I see is that an item gets certified and then bugs get introduced later if future firmware updates.

    The FDA should also be notified of any bugs uncovered in existing firmware. Put the responsibility of deciding if an item needs recalled our of the hands of the

  • I have no doubt that the same issues that affect critical medical devices also affect automobile "drive-by-wire" systems like the Toyota runaway accelerator problem. Those systems need to be subject to inspection and validation by independent experts in the relevant hardware/software technology. And if there are problems, the hardware and software need to be even more thoroughly inspected.

  • Huh? (Score:2, Insightful)

    by jmactacular (1755734)

    "patients and doctors are blocked from examining the source code"

    huh? are either qualified to do so?

    • by AHuxley (892839)
      They can hire someone who can before the device is put in.
      Then make a selection from the devices on the market and at least know the software is "not faulty", all things been equal.
      Hardware may advance or fail, but software can be reviewed.
    • Dunno about the patients, but I'm a doctor - not a code security analyst. I am probably code-savvy enough to understand why a particular piece of code is a problem, if you explain it to me, but that's a long way from being able to identify the problem myself.
  • ....with the line "She hacked into my heart and crashed me."
  • by turing_m (1030530) on Thursday July 22, 2010 @08:08PM (#32998678)
    // max_int should be enough for anyone
    for(i = 0; i < max_int;i++){
      sleep(1);
      beat_heart();
    }

    // printf("hi!!!!!\n")
    • by Afty0r (263037)

      That *is* about 70 years... I think I'd be fairly happy if my Pacemaker lasted 70 years... :)

      What is the hardware life expectancy on those things anyway?

  • by Joe The Dragon (967727) on Thursday July 22, 2010 @09:11PM (#32999040)

    NEVADA GAMING COMMISSION has the code to slots games so why can't the FDA get the code to med systems?

    • by Ihlosi (895663)
      NEVADA GAMING COMMISSION has the code to slots games so why can't the FDA get the code to med systems?

      Why do you think the FDA can't do that? They can basically do anything they want, followed by the threat to kick you (the manufacturer) out of the US market and/or shut down your factories if they're in the US.

      Have a nice day.

  • "...neither patients nor their doctors are permitted to access their IMD's source code or test its security.'"

    "Aww Thufir, don't feel badly...everyone gets a heart-plug here..."

    Let's hope any vulnerabilities aren't wirelessly-exploitable!

    Strat

  • Medical device companies typically outsource hardware for a series of hardware tests. Similar arrangements can be made to test software similar to DO-178B test levels for avionics. This should be a documented process.

  • My girlfriend has an insulin pump made by Medco. It has to do certain things like, if she has a certain high blood sugar level, give the right amount of insulin dose for the next hour to bring her into a normal range. If she eats, she estimates the amount of carbs she's eaten, enters in a certain dose level, and the pump calculates how much insulin she needs, based on the type of insulin she's using.

    It uses a AAA battery. If the battery starts to run low, it beeps. If the battery is almost dead it beeps

    • IF the pump runs out of insulin---THE PUMP SAYS NOTHING. [...] Good thing she doesn't have to milk a hairless cat to live, huh?

      Its not so hard to milk a hairless cat. You just attach eight of those electronic insulin pumps to their nipples, and off you go.

      Now you have to watch closely, 'cuz the stupid things won't know to stop if the cat runs dry and if you're not looking, all you'll have left is what looks like somebody's nutsack laying on the floor.

  • Serious, WTF! Why are we still having to dick around with these issues of closed systems that you are prevented from reviewing, especially since they affect people's health directly! This should not require any kind of debate and if these medical devices that are certified by a government entity such as the Food & Drug Administration (FDA) then the manufacturers must be required to publicly disclose the design and software source code to the FDA for their review and additionally for public review sinc

The biggest mistake you can make is to believe that you are working for someone else.

Working...