Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
Microsoft Hardware Technology

Microsoft Dismisses Consumer Reports' Surface Complaints, But Doesn't Offer Much Evidence 66

Microsoft has publicly responded to Consumer Reports, saying that it disagrees with the publication's Surface reliability findings. But the company hasn't offered much in the way of evidence. In a blog post, Surface chief Panos Panay writes: In the Surface team we track quality constantly, using metrics that include failure and return rates -- both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%. Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1%. Surface also ranks highly in customer satisfaction. 98% of Surface Pro 4 users and Surface Book users say they are satisfied with their device, and our Surface Laptop and new Surface Pro continue to get rave reviews. Long-time watcher Paul Thurrott writes: Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime? Also, he introduces the notion of "return rates" here. By definition, the feedback that Consumer Reports receives is from product owners, not those who have returned products. If someone is almost two years into device ownership, they are not returning the product. They're just using it. And dealing with it. So consider the issue muddled, in just one carefully-constructed sentence. Which I believe was crafted to confuse the issue. But there is more. "Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1 percent," Panay offers. It's not possible to understand how an "incident" relates to a "failure." Mostly because he doesn't explain the term. Likely because doing so would betray that this is an apples to oranges comparison. [...] I will point the reader to Welcome to Surfacegate, my description of Microsoft's feeble attempts to ignore and then slowly fix endemic issues with those exact two Surface models. And anecdotally, I'll point to the fact that the three Surface Book models I've used have all had reliability problems. But the biggest issue I have with "customer satisfaction" is that it's kind of a bullshit measurement when it comes to premium products.
This discussion has been archived. No new comments can be posted.

Microsoft Dismisses Consumer Reports' Surface Complaints, But Doesn't Offer Much Evidence

Comments Filter:
  • by Mr D from 63 ( 3395377 ) on Friday August 11, 2017 @12:55PM (#54991967)

    Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime?

    Failure rates typically follow a bathtub curve, where most manufacturing/design issues are found early in the product life, then age related failures start increasing late in the life cycle. In between is where failures are typically the lowest.

    • by dgatwood ( 11270 )

      Yeah, and I'm sure they calibrated for that based on failure rate and historical failures of similar products.

      A failure rate of 25% over two years would indicate that even if they only sampled failure rate over the first month, they still got way more than a 1% failure rate, which IMO is an order of magnitude higher than is acceptable in a consumer product over the course of two years. Something is very wrong if CR is coming up with estimates that are that high. That's junk-level hardware by the time you

      • Yeah, and I'm sure they calibrated for that based on failure rate and historical failures of similar products.

        A failure rate of 25% over two years would indicate that even if they only sampled failure rate over the first month, they still got way more than a 1% failure rate, which IMO is an order of magnitude higher than is acceptable in a consumer product over the course of two years. Something is very wrong if CR is coming up with estimates that are that high. That's junk-level hardware by the time you get into a double-digit annual failure rate.

        Yeah, it seems ridiculously high. The two years is still in the initial failure phase, I was just pointing out the fallacy of the statement quoted in the summary and what it presumes.

        • Yeah, and I'm sure they calibrated for that based on failure rate and historical failures of similar products.

          A failure rate of 25% over two years would indicate that even if they only sampled failure rate over the first month, they still got way more than a 1% failure rate, which IMO is an order of magnitude higher than is acceptable in a consumer product over the course of two years. Something is very wrong if CR is coming up with estimates that are that high. That's junk-level hardware by the time you get into a double-digit annual failure rate.

          Yeah, it seems ridiculously high. The two years is still in the initial failure phase, I was just pointing out the fallacy of the statement quoted in the summary and what it presumes.

          Two years is NOT in the "Infant Mortality" phase, I was taught that those failure occur in under 90 days, and most often in the 0 to 30 day period.

          Two years is absolutely in the middle of the "bathtub". And if that 25% figure is even DOUBLE the actual failure rate, there is something seriously wrong with the product's design, components (including any software), and/or the manufacturing/QC methods.

          • Two years is NOT in the "Infant Mortality" phase, I was taught that those failure occur in under 90 days, and most often in the 0 to 30 day period.

            Two years covers the infant mortality phase, I agree its over before that two year period. The actual period varies with product type and use cases, but I'd agree that its closer to 30 for this type of product. The point was that the two years does not include the age related failure phase as the summary quote suggested.

            • Two years is NOT in the "Infant Mortality" phase, I was taught that those failure occur in under 90 days, and most often in the 0 to 30 day period.

              Two years covers the infant mortality phase, I agree its over before that two year period. The actual period varies with product type and use cases, but I'd agree that its closer to 30 for this type of product. The point was that the two years does not include the age related failure phase as the summary quote suggested.

              Two years may be part of the infant mortality phase for a bridge or a refrigerator; but it sure isn't for an electronic device with basically no moving parts, whose ENTIRE average lifespan is less than ten years.

      • It's hard to get an accurate picture without knowing the exact nature of the failures. If it's the result of components from a bad supplier that has been identified and dropped or an issue limited to a single production line since rectified, then the estimates aren't going to match reality. It's possible that Microsoft knows that something like this is the case, but isn't going to issue a recall because they're cheap.
      • by Anonymous Coward

        In general, the 'real grade' hardware in a tablet form factor is full of peril.

        You have high current over tiny breakable surface mount connectors. It's a problem in phones to an extent, but the bigger and more power hungry, the more abuse particularly the charging port is going to take. Also things like easily detachable keyboards will have connector problems much more likely than, say, a laptop.

        Also cramming the thermal and battery needed for desktop experience causes compromises.

        In other words, I haven'

        • Surface mount connectors in particular drive me insane... So easy to pop off... it doesn't seem like it is *that* much cheaper than through-mount.

          Learn to solder.

          SMT connectors can be plenty robust; although ones that get abused constantly should have at least one through-hole "staking" stub or holding screw.

  • by postmortem ( 906676 ) on Friday August 11, 2017 @12:56PM (#54991971) Journal

    Is why 3 generations of this device can't properly go to sleep mode. Every other wake is either: device dead or wifi dead.

    But of course, you can blame it to testing methodology...

    • by the_skywise ( 189793 ) on Friday August 11, 2017 @01:05PM (#54992049)
      Hear hear - Although I don't have as much problems with sleep mode as I do with dead wifi or dead camera coming out of sleep mode.
      I've been trained to expect that from windows PC manufacturers as MS and the hardware vendors go back and forth. But here, MS produces both the hardware AND the software and you'd think they could at least resolve the problems after 2 years (SP4 user here).
      • Nah, my windows phone was having issues too with shit they should have easily fixed. Windows 10 is just buggy as hell.
      • Hear hear - Although I don't have as much problems with sleep mode as I do with dead wifi or dead camera coming out of sleep mode.

        I've been trained to expect that from windows PC manufacturers as MS and the hardware vendors go back and forth. But here, MS produces both the hardware AND the software and you'd think they could at least resolve the problems after 2 years (SP4 user here).

        I have a Surface Pro 4 as well.

        I noticed early on that Bluetooth and WiFi were enabled during sleep by default and that they would drain the battery. I set these to be turned off during sleep. Now my battery life changes little during sleep mode. I haven't experienced a problem with WiFi not coming back up after sleep but I have seen occasional camera problems.

        I also tend to update the drivers as new releases come out. You can find the latest drivers at the link below. Looks like the latest version is

      • Hear hear - Although I don't have as much problems with sleep mode as I do with dead wifi or dead camera coming out of sleep mode.

        I've been trained to expect that from windows PC manufacturers as MS and the hardware vendors go back and forth. But here, MS produces both the hardware AND the software and you'd think they could at least resolve the problems after 2 years (SP4 user here).

        Exactly.

        That was always the Windows Fanboi's retort when people would point out that Macs didn't have these problems (at least not chronically across all designs and model-years). "Windows has to support SO MANY different configurations..."

        Well now they don't really have that excuse. If they can't get it right when they are controlling the "entire experience", like Apple, then when WILL they get it right?

    • Happens on my thinkpad too has it ever worked?
      • pfft - I've got a wide screen monitor and about every other NVIDIA driver update seems to make Windows scrunch all the icons and apps to the left hand side during sleep mode because the resolution "changes" during sleep.
        I mainly blame NVIDIA for that (they get it right half the time!) but I don't see why the desktop doesn't remember where the positions WERE when the resolution comes back.
        (I suppose it could be the Asus monitor or a DisplayPort quirk too but I've seen similar behavior on friends' non-wides
    • Laptops people installing linux on used to have problems with sleep mode too.. If microsoft just makes own linux distro and calls it "Windows" nobody gonna notice :P
    • I have a Surface Book and had a Surface Pro 3. I've not experienced that problem. But, I've gone through several models of both. So I have a lot of experience.

    • Is why 3 generations of this device can't properly go to sleep mode. Every other wake is either: device dead or wifi dead.

      But of course, you can blame it to testing methodology...

      Hmmm. My MacBook Pro never has that problem. And it wakes from sleep in about 1/2 second, too.

    • To be fair they did basically completely redesign sleep mode for the Surface line, and Microsoft isn't known for getting things right the first time, or the third time, and generally known quite well for fucking up the second and fourth completely. :)

  • I guess with the Surface, now that they have full control of the hardware, they can't blame anyone besides themselves for problems the user faces. All the drivers, hardware must have been fully tested by Microsoft's most capable QA team (what's left of it) before the Surface leaves the factory. So what could possibly go wrong?
    • by mysidia ( 191772 )

      they can't blame anyone besides themselves for problems the user faces.

      I guess that all that is left, then is to deny deny deny, and if someone finds errors, then their testing methods must be broken.

    • Well I don't think they put their most capable QA on it. The most capable QA team has been there a long time, and it would be foolish to pull them off their project that they're experts on and put them on a brand new hardware platform ("uh, boss, you know we're software QA and not hardware, right?").

      And QA doesn't matter if the management doesn't follow through and fix the bugs and put a premium on quality. Quality at Microsoft has always been an afterthought. Improving quality gets in the way of constant

  • We started with about eight Surface Pro 3's and being that I like to experiment with cost savings we got two of the Surface Non-Pro's with Atom processors inside to test for intern/assistant level stuff.

    We have two of those Pro's still functioning and I think we have one of the Atom's still in use but I'm not sure where it is, I may have to dig up inventory records.

    That being said we have MacBook Pro's that are four to five times the age of these oldest Surface in use still being used as well as Lenovo T400's, some Dell's that are just about as old and a couple of other clunkers hanging on in lesser roles. You have to see a Surface as a disposable system.

    I have a 64GB Surface Pro 1 at home that's still works like a champ and with the I5 processor it still functions as a champion coffee shop cruiser. That being said I care about my equipment. My daughter has a Surface Pro 4 that having received it at Christmas time has already outlasted a lot of the 3's in hours of use we've had around here many times over.

    I'm to say that a Surface is a good unit for people who give a shit about their equipment, but not in a company environment where replacement is someone else's problem. It's like buying a used rental car - which I have done. You just sort of expect to have to replace the tires and brakes even if it only has 20,000 miles on it. Buy a new Surface for yourself they're pretty awesome and the software problems that plagued them on Windows 8 seem to have gone away with WIndows 10, just don't waste your money in a corporate environment. *

    *I have not tested the Surface laptops (attached keyboard types) or the all-in-one desktops

  • In the Surface team we track quality constantly, using metrics that include failure and return rates -- both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%

    Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime?

    The Surface Pro 4 and Surface Book were introduced in October 2015. Surface Pro 3 in June 2014.

  • by xtal ( 49134 ) on Friday August 11, 2017 @01:21PM (#54992143)

    I have been looking at the Surface for awhile. It looked OK.

    Then I got one at work to replace my aging Dell Latitude. (Surface Pro 4)

    The keyboard is junk. It's flimsy and moves when you type on it; the trackpad, which is required if you're using it as a computer (which I have to) is very narrow in depth and is difficult to use when compounded by the fact everything flexes when you use it.

    Using this device on anything but a flat table or desk is right out.

    Also, it's covered in Microfiber. WTF. This stuff attracts lint, dust, and pet hair like nobody's business and I shudder to think how grimy it's going to be in a year or so.

    In short, Microsoft turned what looked like an OK tablet into a horrible computer.

    That I'm stuck with.

    Maybe for years.

    Sigh.

    • Not for power users that is for sure. I don't like seeing companies replace large well designed laptops with Surfaces. That said... I have one, and I will buy another if this one breaks. If you're not doing power user tasks and you can live with a device that requires ultimately a table or a very well reclined lounge chair to use then it's a great crossover device.

      I take mine to work for note taking, I use the pen a lot, but I'd be quite annoyed if they took my work laptop off me.

  • The NFL has a contract with Microsoft to use their Surface tablets on the sidelines. They tout it as a way for the team to get almost real time information on play calling, whether offense or defense. Unfortunately, and directly related to this article, there are those, particularly coaches and quarterbacks [techcrunch.com], who bemoan the unpredictability of the Surface and its many malfunctions [yahoo.com].

    Even worse, people were initially calling them iPads or iPad-like [businessinsider.com] which certainly didn't sit well with the marketing crew a
    • Microsoft can claim all they want their Surface is doing well, but from real world experiences,

      Microsoft's definition of doing well has nothing to do with people's specific use cases. The Surfaces are doing well. They are also not suited to every use case, and frankly probably a tad too bulky to be using as a tablet at a sideline.

      They do have problems now. Between my partner and I, with 2 Surface Pros in the house, we are on: 4th Surface Pro, 5th keyboard, and 4th pens, and 3rd Arc mice, a recalled PSU cable, and one which is frayed and likely a fire hazard.

      Other than the mice we've had a 100% failur

  • I've had multiple Surface products starting at the first Pro model. They are very versatile and "the ultimate Windows experience" but that experience doesn't speak to much in terms of quality.

    The operating system aside (Windows indexer breaking, several driver problems, basic functionality missing and no ability to use the Feedback Hub without turning on full telemetry, etc), the hardware is/was riffled with problems. The Broadcom multi-communications chip (WiFi/Bluetooth) is a mess. It's been revision afte

  • If I was Microsoft I would be less concerned about the Consumer Reports survey, and more concerned how many of those people surveyed felt compelled to give the Surface negative marks and how many of those people were spreading the word about their bad experiences.

  • both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%.

    Back when I was working developing industrial control products, we considered anything above a few percent (like less than 5%) to be "something's wrong here", prompting an evaluation of design, components (and software, if applicable), manufacturing processes, QC processes, etc.

    Something as alarming as 25% would have likely caused the entire product to be at least temporarily pulled-from-sale immediately, and more-than-likely, never sold again. In fact, even the most-failure-prone product I was aware of at

    • Can you imagine fast-food with a 25% failure rate? One out of four Big Macs got food poisoning. One out of four Wendy's Chilis got a finger in it [go.com].

      NOT acceptable (in Trump's America).

      • Can you imagine fast-food with a 25% failure rate? One out of four Big Macs got food poisoning. One out of four Wendy's Chilis got a finger in it [go.com].

        NOT acceptable (in Trump's America).

        Man, I just ate dinner!

        But you're exactly correct.

  • That's disastrous, assuming it's AFR.

    Excluding disk drives, 1.5% to 2% is a decent number.

  • Our experience hasn't reflected the Consumer Reports numbers. It has been far worse.

    My SP3:
    Replaced SP3 due to failure in the fan making a loud noise.
    Replaced keyboard due to complete failure to type.
    Replaced keyboard again (SP4 keyboard) due to mouse failing to click.
    Power brick cable subject to a recall.
    Replaced pen (SP4 pen) due to failing to pair.

    Currently living with infuriating wakeup bug that results from having an SP4 TypeCover on an SP3 which Microsoft know, acknowledge is a driver issue and have f

  • While annoying SW bugs generally get fixed. HW design weaknesses kind of stuck. The Surface has nifty features but appears fragile which should be understood by prospective buyers. I have a 5 yr old Vaio SVZ 1311AJ no issues but can't open to clean without risk of breaking due to very tight connections. Can clean moderately using pressurized air but over time dust collects inside so heats up more now. Would like to get a new device since overdue for a failure but SP4 pro seems risky.

When the weight of the paperwork equals the weight of the plane, the plane will fly. -- Donald Douglas

Working...