Microsoft Dismisses Consumer Reports' Surface Complaints, But Doesn't Offer Much Evidence 66
Microsoft has publicly responded to Consumer Reports, saying that it disagrees with the publication's Surface reliability findings. But the company hasn't offered much in the way of evidence. In a blog post, Surface chief Panos Panay writes: In the Surface team we track quality constantly, using metrics that include failure and return rates -- both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%. Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1%. Surface also ranks highly in customer satisfaction. 98% of Surface Pro 4 users and Surface Book users say they are satisfied with their device, and our Surface Laptop and new Surface Pro continue to get rave reviews. Long-time watcher Paul Thurrott writes: Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime? Also, he introduces the notion of "return rates" here. By definition, the feedback that Consumer Reports receives is from product owners, not those who have returned products. If someone is almost two years into device ownership, they are not returning the product. They're just using it. And dealing with it. So consider the issue muddled, in just one carefully-constructed sentence. Which I believe was crafted to confuse the issue. But there is more. "Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1 percent," Panay offers. It's not possible to understand how an "incident" relates to a "failure." Mostly because he doesn't explain the term. Likely because doing so would betray that this is an apples to oranges comparison. [...] I will point the reader to Welcome to Surfacegate, my description of Microsoft's feeble attempts to ignore and then slowly fix endemic issues with those exact two Surface models. And anecdotally, I'll point to the fact that the three Surface Book models I've used have all had reliability problems. But the biggest issue I have with "customer satisfaction" is that it's kind of a bullshit measurement when it comes to premium products.
Re: (Score:2)
OK. I'll bite. In what way is Consumer Reports evil?
Re: (Score:3)
You also have to be wary of the Hawthorne effect [wikipedia.org] - where worker productivity increases simply because they think you're doing something to help them, even if you secretly haven't really changed anything. So a better way to test is t
Re: (Score:3)
You also have to be wary of the Hawthorne effect
You, on the other hand, have to be wary about assuming anything stated in an anonymous posting actually happened. :-)
Re: (Score:2)
I gave Surface Pro to all my Mac and Linux developers. Profits plummeted!
Bathtub (Score:3)
Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime?
Failure rates typically follow a bathtub curve, where most manufacturing/design issues are found early in the product life, then age related failures start increasing late in the life cycle. In between is where failures are typically the lowest.
Re: (Score:2)
Yeah, and I'm sure they calibrated for that based on failure rate and historical failures of similar products.
A failure rate of 25% over two years would indicate that even if they only sampled failure rate over the first month, they still got way more than a 1% failure rate, which IMO is an order of magnitude higher than is acceptable in a consumer product over the course of two years. Something is very wrong if CR is coming up with estimates that are that high. That's junk-level hardware by the time you
Re: (Score:2)
Yeah, and I'm sure they calibrated for that based on failure rate and historical failures of similar products.
A failure rate of 25% over two years would indicate that even if they only sampled failure rate over the first month, they still got way more than a 1% failure rate, which IMO is an order of magnitude higher than is acceptable in a consumer product over the course of two years. Something is very wrong if CR is coming up with estimates that are that high. That's junk-level hardware by the time you get into a double-digit annual failure rate.
Yeah, it seems ridiculously high. The two years is still in the initial failure phase, I was just pointing out the fallacy of the statement quoted in the summary and what it presumes.
Re: (Score:2)
Yeah, and I'm sure they calibrated for that based on failure rate and historical failures of similar products.
A failure rate of 25% over two years would indicate that even if they only sampled failure rate over the first month, they still got way more than a 1% failure rate, which IMO is an order of magnitude higher than is acceptable in a consumer product over the course of two years. Something is very wrong if CR is coming up with estimates that are that high. That's junk-level hardware by the time you get into a double-digit annual failure rate.
Yeah, it seems ridiculously high. The two years is still in the initial failure phase, I was just pointing out the fallacy of the statement quoted in the summary and what it presumes.
Two years is NOT in the "Infant Mortality" phase, I was taught that those failure occur in under 90 days, and most often in the 0 to 30 day period.
Two years is absolutely in the middle of the "bathtub". And if that 25% figure is even DOUBLE the actual failure rate, there is something seriously wrong with the product's design, components (including any software), and/or the manufacturing/QC methods.
Re: (Score:2)
Two years is NOT in the "Infant Mortality" phase, I was taught that those failure occur in under 90 days, and most often in the 0 to 30 day period.
Two years covers the infant mortality phase, I agree its over before that two year period. The actual period varies with product type and use cases, but I'd agree that its closer to 30 for this type of product. The point was that the two years does not include the age related failure phase as the summary quote suggested.
Re: (Score:2)
Two years is NOT in the "Infant Mortality" phase, I was taught that those failure occur in under 90 days, and most often in the 0 to 30 day period.
Two years covers the infant mortality phase, I agree its over before that two year period. The actual period varies with product type and use cases, but I'd agree that its closer to 30 for this type of product. The point was that the two years does not include the age related failure phase as the summary quote suggested.
Two years may be part of the infant mortality phase for a bridge or a refrigerator; but it sure isn't for an electronic device with basically no moving parts, whose ENTIRE average lifespan is less than ten years.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
In general, the 'real grade' hardware in a tablet form factor is full of peril.
You have high current over tiny breakable surface mount connectors. It's a problem in phones to an extent, but the bigger and more power hungry, the more abuse particularly the charging port is going to take. Also things like easily detachable keyboards will have connector problems much more likely than, say, a laptop.
Also cramming the thermal and battery needed for desktop experience causes compromises.
In other words, I haven'
Re: (Score:2)
Surface mount connectors in particular drive me insane... So easy to pop off... it doesn't seem like it is *that* much cheaper than through-mount.
Learn to solder.
SMT connectors can be plenty robust; although ones that get abused constantly should have at least one through-hole "staking" stub or holding screw.
Stuff that they don't track (Score:3, Insightful)
Is why 3 generations of this device can't properly go to sleep mode. Every other wake is either: device dead or wifi dead.
But of course, you can blame it to testing methodology...
Re:Stuff that they don't track (Score:4, Insightful)
I've been trained to expect that from windows PC manufacturers as MS and the hardware vendors go back and forth. But here, MS produces both the hardware AND the software and you'd think they could at least resolve the problems after 2 years (SP4 user here).
Re: (Score:3)
Re: (Score:3)
Hear hear - Although I don't have as much problems with sleep mode as I do with dead wifi or dead camera coming out of sleep mode.
I've been trained to expect that from windows PC manufacturers as MS and the hardware vendors go back and forth. But here, MS produces both the hardware AND the software and you'd think they could at least resolve the problems after 2 years (SP4 user here).
I have a Surface Pro 4 as well.
I noticed early on that Bluetooth and WiFi were enabled during sleep by default and that they would drain the battery. I set these to be turned off during sleep. Now my battery life changes little during sleep mode. I haven't experienced a problem with WiFi not coming back up after sleep but I have seen occasional camera problems.
I also tend to update the drivers as new releases come out. You can find the latest drivers at the link below. Looks like the latest version is
Re: (Score:2)
Hear hear - Although I don't have as much problems with sleep mode as I do with dead wifi or dead camera coming out of sleep mode.
I've been trained to expect that from windows PC manufacturers as MS and the hardware vendors go back and forth. But here, MS produces both the hardware AND the software and you'd think they could at least resolve the problems after 2 years (SP4 user here).
Exactly.
That was always the Windows Fanboi's retort when people would point out that Macs didn't have these problems (at least not chronically across all designs and model-years). "Windows has to support SO MANY different configurations..."
Well now they don't really have that excuse. If they can't get it right when they are controlling the "entire experience", like Apple, then when WILL they get it right?
Re: (Score:2)
Re: (Score:3)
I mainly blame NVIDIA for that (they get it right half the time!) but I don't see why the desktop doesn't remember where the positions WERE when the resolution comes back.
(I suppose it could be the Asus monitor or a DisplayPort quirk too but I've seen similar behavior on friends' non-wides
Re: (Score:2)
Re: (Score:1)
I have a Surface Book and had a Surface Pro 3. I've not experienced that problem. But, I've gone through several models of both. So I have a lot of experience.
Re: (Score:2)
Is why 3 generations of this device can't properly go to sleep mode. Every other wake is either: device dead or wifi dead.
But of course, you can blame it to testing methodology...
Hmmm. My MacBook Pro never has that problem. And it wakes from sleep in about 1/2 second, too.
Re: (Score:2)
To be fair they did basically completely redesign sleep mode for the Surface line, and Microsoft isn't known for getting things right the first time, or the third time, and generally known quite well for fucking up the second and fourth completely. :)
Microsoft usually blames 3rd party h/w for issues (Score:2)
Re: (Score:2)
... I suspect the majority of the issues can be traced to the Skylake CPUs in the Surface Pro 4. The issues with that generation CPU in mobile are pretty well known...
Microsoft was the one who put those CPUs into the Surface. I didn't, you didn't and Intel didn't. So you seem to be saying that Microsoft knowingly put a CPU into the Surface that had known reliability issues, without doing any QA testing of those CPUs before using them. Wow, that sounds even worse than I had thought.
Re: (Score:2)
... I suspect the majority of the issues can be traced to the Skylake CPUs in the Surface Pro 4. The issues with that generation CPU in mobile are pretty well known...
Microsoft was the one who put those CPUs into the Surface. I didn't, you didn't and Intel didn't. So you seem to be saying that Microsoft knowingly put a CPU into the Surface that had known reliability issues, without doing any QA testing of those CPUs before using them. Wow, that sounds even worse than I had thought.
Hmmm. Then why don't the Macs that use those CPUs have the same issues?
Re: (Score:2)
they can't blame anyone besides themselves for problems the user faces.
I guess that all that is left, then is to deny deny deny, and if someone finds errors, then their testing methods must be broken.
Re: (Score:2)
Well I don't think they put their most capable QA on it. The most capable QA team has been there a long time, and it would be foolish to pull them off their project that they're experts on and put them on a brand new hardware platform ("uh, boss, you know we're software QA and not hardware, right?").
And QA doesn't matter if the management doesn't follow through and fix the bugs and put a premium on quality. Quality at Microsoft has always been an afterthought. Improving quality gets in the way of constant
We began to switch our laptop pool over to Surface (Score:3)
We started with about eight Surface Pro 3's and being that I like to experiment with cost savings we got two of the Surface Non-Pro's with Atom processors inside to test for intern/assistant level stuff.
We have two of those Pro's still functioning and I think we have one of the Atom's still in use but I'm not sure where it is, I may have to dig up inventory records.
That being said we have MacBook Pro's that are four to five times the age of these oldest Surface in use still being used as well as Lenovo T400's, some Dell's that are just about as old and a couple of other clunkers hanging on in lesser roles. You have to see a Surface as a disposable system.
I have a 64GB Surface Pro 1 at home that's still works like a champ and with the I5 processor it still functions as a champion coffee shop cruiser. That being said I care about my equipment. My daughter has a Surface Pro 4 that having received it at Christmas time has already outlasted a lot of the 3's in hours of use we've had around here many times over.
I'm to say that a Surface is a good unit for people who give a shit about their equipment, but not in a company environment where replacement is someone else's problem. It's like buying a used rental car - which I have done. You just sort of expect to have to replace the tires and brakes even if it only has 20,000 miles on it. Buy a new Surface for yourself they're pretty awesome and the software problems that plagued them on Windows 8 seem to have gone away with WIndows 10, just don't waste your money in a corporate environment. *
*I have not tested the Surface laptops (attached keyboard types) or the all-in-one desktops
The two claims aren't contradictory (Score:2)
The Surface Pro 4 and Surface Book were introduced in October 2015. Surface Pro 3 in June 2014.
Just given one of these at work.. (Score:5, Interesting)
I have been looking at the Surface for awhile. It looked OK.
Then I got one at work to replace my aging Dell Latitude. (Surface Pro 4)
The keyboard is junk. It's flimsy and moves when you type on it; the trackpad, which is required if you're using it as a computer (which I have to) is very narrow in depth and is difficult to use when compounded by the fact everything flexes when you use it.
Using this device on anything but a flat table or desk is right out.
Also, it's covered in Microfiber. WTF. This stuff attracts lint, dust, and pet hair like nobody's business and I shudder to think how grimy it's going to be in a year or so.
In short, Microsoft turned what looked like an OK tablet into a horrible computer.
That I'm stuck with.
Maybe for years.
Sigh.
Re: Just given one of these at work.. (Score:2)
I work for the government. They buy these things in the hundreds of thousands. Bathtub or not it's Surfaces all the way down.
I had to get a mouse and an external keyboard to write on it. That's pretty bad.
Re: (Score:2)
Not for power users that is for sure. I don't like seeing companies replace large well designed laptops with Surfaces. That said... I have one, and I will buy another if this one breaks. If you're not doing power user tasks and you can live with a device that requires ultimately a table or a very well reclined lounge chair to use then it's a great crossover device.
I take mine to work for note taking, I use the pen a lot, but I'd be quite annoyed if they took my work laptop off me.
This is why many in the NFL are frustrated (Score:2, Interesting)
Even worse, people were initially calling them iPads or iPad-like [businessinsider.com] which certainly didn't sit well with the marketing crew a
Re: (Score:2)
Microsoft can claim all they want their Surface is doing well, but from real world experiences,
Microsoft's definition of doing well has nothing to do with people's specific use cases. The Surfaces are doing well. They are also not suited to every use case, and frankly probably a tad too bulky to be using as a tablet at a sideline.
They do have problems now. Between my partner and I, with 2 Surface Pros in the house, we are on: 4th Surface Pro, 5th keyboard, and 4th pens, and 3rd Arc mice, a recalled PSU cable, and one which is frayed and likely a fire hazard.
Other than the mice we've had a 100% failur
Little to no quality control and horrible warrany (Score:1)
I've had multiple Surface products starting at the first Pro model. They are very versatile and "the ultimate Windows experience" but that experience doesn't speak to much in terms of quality.
The operating system aside (Windows indexer breaking, several driver problems, basic functionality missing and no ability to use the Feedback Hub without turning on full telemetry, etc), the hardware is/was riffled with problems. The Broadcom multi-communications chip (WiFi/Bluetooth) is a mess. It's been revision afte
How do you address the people who bought them? (Score:1)
If I was Microsoft I would be less concerned about the Consumer Reports survey, and more concerned how many of those people surveyed felt compelled to give the Surface negative marks and how many of those people were spreading the word about their bad experiences.
25% Failure is HIDEOUS!!! (Score:2)
both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%.
Back when I was working developing industrial control products, we considered anything above a few percent (like less than 5%) to be "something's wrong here", prompting an evaluation of design, components (and software, if applicable), manufacturing processes, QC processes, etc.
Something as alarming as 25% would have likely caused the entire product to be at least temporarily pulled-from-sale immediately, and more-than-likely, never sold again. In fact, even the most-failure-prone product I was aware of at
Re: (Score:2)
Can you imagine fast-food with a 25% failure rate? One out of four Big Macs got food poisoning. One out of four Wendy's Chilis got a finger in it [go.com].
NOT acceptable (in Trump's America).
Re: (Score:2)
Can you imagine fast-food with a 25% failure rate? One out of four Big Macs got food poisoning. One out of four Wendy's Chilis got a finger in it [go.com].
NOT acceptable (in Trump's America).
Man, I just ate dinner!
But you're exactly correct.
25%? (Score:2)
That's disastrous, assuming it's AFR.
Excluding disk drives, 1.5% to 2% is a decent number.
CR Failure Rate Too Low (Score:2)
Our experience hasn't reflected the Consumer Reports numbers. It has been far worse.
My SP3:
Replaced SP3 due to failure in the fan making a loud noise.
Replaced keyboard due to complete failure to type.
Replaced keyboard again (SP4 keyboard) due to mouse failing to click.
Power brick cable subject to a recall.
Replaced pen (SP4 pen) due to failing to pair.
Currently living with infuriating wakeup bug that results from having an SP4 TypeCover on an SP3 which Microsoft know, acknowledge is a driver issue and have f
HW vs SW (Score:1)