Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
The Almighty Buck The Media Hardware

Are Review Units Better Than Store Versions? 407

Posted by timothy from the dark-and-suspicious dept.
Anonymous Howard writes "Every now and then you hear about hardware manufacturers optimizing their hardware for certain tests or games to make their hardware look superior. I was surprised to hear of a new controversy brewing over reviewer units sent to hardware reviewers. This article claims that Samsung is sending LCD monitors with a contrast ratio of 700:1 when the consumer version of the same monitor has a contrast ratio of 450:1. Various sites list different specs for the same model, so it's somewhat confusing to know for sure which is correct. I don't doubt this happens, but I'm surprised that it would be this blatant. Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?"
This discussion has been archived. No new comments can be posted.

Are Review Units Better Than Store Versions? More

Are Review Units Better Than Store Versions?

Comments Filter:

  • Well (Score:5, Funny)

    by RedWolves2 ( 84305 ) on Wednesday October 29, 2003 @01:25PM (#7339640) Homepage Journal
    Review units are free and Store units are...well...not free. That would sway my opinion.
    • Maybe that's why you aren't a reviewer, thankfully.

    • Re:Well (Score:5, Insightful)

      by dnoyeb ( 547705 ) on Wednesday October 29, 2003 @01:36PM (#7339768) Homepage Journal
      A review is supposed to be done on a random sample anyway.

      Consumer reports had the right idea, that is why they have been so successful.

    • Re:Well (Score:3, Insightful)

      by diersing ( 679767 )
      And why aren't the reviewers performing their tests with retail purchased equipment for integrity sake anyway?

      • Re:Well (Score:2, Informative)

        by GreyPoopon ( 411036 )
        And why aren't the reviewers performing their tests with retail purchased equipment for integrity sake anyway?

        Most likely because doing so would be somewhat cost-prohibitive.

      • Small shops can't afford to purchase tons of review items. Also, a lot of shops (especially Computer HW shops) review items *before* they are available to the public. Just like movies, people want to buy/see something the day it is released, not a week later after everyone else has tried it.

      • Re:Well (Score:5, Interesting)

        by Ian Wolf ( 171633 ) on Wednesday October 29, 2003 @02:11PM (#7340081) Homepage
        Cost. Most review sites, especially in the PC industry, don't have much cash to purchase the products themselves. Instead they rely on vendors sending their hardware to them for free.

        It definitely isn't an objective model, but one that allows multiple people to review the same product. Ultimately, you have to make two really shaky assumptions in such a model.

        1. The vendor is sending the reviewer a consumer level product (or nearly so).
        2. The reviewer is objective and honest enough to verify the capabilities match that of the consumer available product and disclose where the product came from.

        For the most part the model works. Point 1 is completely out of the consumer's control and cannot even be known to the consumer without point 2. As a result, there is only one thing that consumers can do, and that is learn to spot the honest reviewers from the frauds, fanboys, and sponsored reviewers. For example, Anandtech [anandtech.com] and Tom's Hardware Guide [tomshardware.com] do a pretty good job. They clearly indicate where the hardware comes from, identify any differences between it and the shipping hardware, and do their homework as best they can to prevent getting duped. Contrast this with many reviewers who seem to be simply paying lip service to the vendors so they can get quoted in an advert and continue to get free hardware to play with. Researching the product also means researching the reviewer if you don't wanna get burned. It's like taking advice from the Gartner Group without seeing who paid for their latest study. :)

      • why aren't the reviewers performing their tests with retail purchased equipment for integrity sake anyway?

        Because integrity is secondary to ad revenues. Good review = more ad revenue.

  • Are Review Units Better Than Store Versions? (Score:4, Insightful)

    by Pingular ( 670773 ) on Wednesday October 29, 2003 @01:26PM (#7339643)
    Well, you know how some radeon graphics cards can be 'unlocked' and some can't? I'll give you one guess at which I bet radeon sent to all the reviewers.

  • Yeah, in my review version of the Ep IV DVD (Score:5, Funny)

    by Hairy_Potter ( 219096 ) on Wednesday October 29, 2003 @01:27PM (#7339657) Homepage
    Han shoots first. I think it's different in the retail version.

  • FARK : OBVIOUS (Score:3, Funny)

    by Anonymous Coward on Wednesday October 29, 2003 @01:27PM (#7339659)
    I heard Mandrake only sent copies of 9.2 to reviewers who didn't have LG CDROMS.

  • Ah, the Sad Effect of Technology (Score:3, Funny)

    by slagdogg ( 549983 ) on Wednesday October 29, 2003 @01:27PM (#7339664)
    What ever happened to the ancient art of bribing the reviewer?
    • I believe that's referred to as "advertising" in said publication.

      -r
    • They did. They sent him a free 700:1 LCD. It may not be cash, but it has definite value [ebay.com]. Bribery only depends on the currency accepted.

    • pressure on reviewers (Score:5, Insightful)

      by frovingslosh ( 582462 ) on Wednesday October 29, 2003 @01:38PM (#7339789)
      What ever happened to the ancient art of bribing the reviewer?

      Like in "here's a free expensive item for review that you get to keep. We'll be watching the review to see if you get anything else to review? Oh, it's still happening, but sending the reviewer a item that isn't the same as the crap they intend to sell you and me is just a little added insurance.

      You can pretty much see this in a lot of reviews that are written too. The only reviews that merit much trust are the independent ones where the reviewer actually went out and got an off-the-shelf item to review; but this is an almost dead pratice. No only does the reviewer not get neat fre stuff then, but his review may be months after the reviews by the company shills come out, and he ends up with the same crap you and I get rather than the free good working versions.

      • Re:pressure on reviewers (Score:4, Insightful)

        by kfg ( 145172 ) on Wednesday October 29, 2003 @02:09PM (#7340068)
        My mom is a freelance travel photographer. Nobody "bribes" her, but she sure gets to travel a lot for "free."

        Now what do you suppose happens to her free trips if she publishes unflattering copy about her "hosts?"

        You don't have to be bribed to be beholden.

        She could, of course, simply take her own trips on her own dime. . .if she were independantly wealthy and just playing at it instead of trying to make a living.

        And thus the media is corrupted without any application of coercion at all. No threat to remove advertising or anything. Just a loose understanding by everyone involved as to what's in their own personal interest.

        KFG

        KFG

  • Wait wait... (Score:2, Interesting)

    by fjordboy ( 169716 )
    Alright, I understand that this is false advertising, because the reviewed product is different from the actual product, but don't both products accurately describe the contrast? Like...the reviewed products are 700 to 1, and the consumer ones are 450 to 1...but aren't they both labelled as that? I think this would fall into one of those "check before you buy" categories...one of those common sense things maybe. As long as both products clearly indicate what their specs are, there is deception, but no ac

    • Re:Wait wait... (Score:5, Interesting)

      by slamb ( 119285 ) on Wednesday October 29, 2003 @01:32PM (#7339723) Homepage
      Alright, I understand that this is false advertising, because the reviewed product is different from the actual product, but don't both products accurately describe the contrast? Like...the reviewed products are 700 to 1, and the consumer ones are 450 to 1...but aren't they both labelled as that? I think this would fall into one of those "check before you buy" categories...one of those common sense things maybe. As long as both products clearly indicate what their specs are, there is deception, but no actual lies.

      Bah. They had the same model number on two different models. That's a lie.

      Is the difference between an outright lie and a deception really that important here anyway? They were expected to send the same product real consumers get. They didn't. That's enough to condemn them in my book, whether there's an outright lie there or not.

      • Isn't that like the Linksys WPC11 801.11b card? Same model, but different versions, with different specs, chipsets.

        The model may be the same, but some low level qualifier (version, revision, manufacturer ID, etc.) is very likely unique between the units. Otherwise the manufacturer can't tell them apart either.

        • Re:Wait wait... (Score:2, Interesting)

          by RoundSparrow ( 341175 )
          The first to do this on a large scale that I encountered...

          3com Ethernet cards, when they were the most popular (1997?). They came out with new chipsets that required different drivers under the same product name.

          Started doing the model "a", "b", crap.

          At least LinkSys puts "Version x.x" on their boxes!
    • nevermind, I changed my mind. I think deceit in itself is lying, so the technicality doesn't change the morality. Ignore my previous comment...I wasn't thinking clearly.
  • Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?

    Yes. Thank you for asking.

  • That's why Consumer Reports (Score:4, Informative)

    by eclectro ( 227083 ) on Wednesday October 29, 2003 @01:30PM (#7339689)

    buy their stuff off the shelf to use in reviews. Otherwise companies will send the cherries to reviewers.

    I worked for a couple of electronic manufacturers that had a standard operating policy to do this very thing.

    • Re:That's why Consumer Reports (Score:5, Interesting)

      by TimTheFoolMan ( 656432 ) on Wednesday October 29, 2003 @01:48PM (#7339898) Homepage Journal
      Just because they don't accept advertising, and they buy units off the shelf, doesn't mean the CR review is a de-facto "better review," or is definitely less biased than a trade magazine review. Based on my experience, they represent the worst possible example of "sound bite product reviews," and they rarely give me truly useful data. (For examples, read one of their loudspeaker, audio receiver, or sports car reviews. These are generally valuable only for the pictures.)

      CR tries to distill down all sorts of subtle performance parameters into a box score that ranks easily against competitive products, and in the meantime, miss the value of those parameters. Quite honestly, I'd be surprised if CR could accurately determine if they had a cherry LCD display or not, given the "rounding error" of their review/comparison methodology.

      Tim

      • CR's sports car reviews always make me laugh a little. It seems like in every one of them they always complain about the stiff ride, engine noise and fuel economy.

      • CR's two biggest problems (Score:3, Insightful)

        by swb ( 14022 )
        ...are its usage bias of the products they review -- they assume that every product they review is made to satisfy some everyman need, and products built for speciality audiences or specifically designed to do one thing REALLY well and 3 other things just OK get dinged badly in reviews, even though the one thing they do really well may make the difference (eg, 50" HDTV with shitty speakers but FANTASTIC PQ).

        This would be easily mitigated if the product reviews were serious articles with a lot more commenta
        • If you're talking about overall score, OK. But a TV that has poor sound, but good picture, will have that noted in the review. Furthermore, if CR took the time to note every specialty of every product, they would never be able to get the issues out in time. They don't do the most in depth reviews, and can miss some details, but generally their reviews are accurate and fair.
      • Consumer Reports is very good for appliances and basic household stuff, and very good for family cars and minivans. I wouldn't buy computer equipment based on their recommendations and, really, I wouldn't buy anything high-tech without hearing from another source in fact.

        That doesn't invalidate their methodology of buying review units off the shelf, though. Good magazines learn this lesson early. Car and Driver (which *ahem* does have good sports car reviews) has written about an early radar detector revi

      • (For examples, read one of their loudspeaker, audio receiver, or sports car reviews. These are generally valuable only for the pictures.)

        I heartily agree with this. Consumer Reports reviews ae pretty close to useless in many areas. Yes, they tell you which product they liked better. But they never seem to give you any of the numbers or any information on how they decided which product was better. They will tell you that one receiver performs better than the others, but they don't tell you how they te

      • My chief complaint is the lack of breadth in the product reviews. They don't review every model, or even every (or even most) competing products.

        So the review is incomplete, at least to me.

        For example, in the (aged) review on vaccuum cleaners, I beleive that I have purchased one of the better (if not the best) models on the market. Yet CR has never reviewed any machine from the company.

        Also, they do not update their product ratings. If they reviewed a series of toasters in 1997, and that's the latest
    • I worked for a couple of electronic manufacturers that had a standard operating policy to do this very thing.

      Cherries are one thing -- but did your employers really send out demo models with fundamentally different capacities than in the specs? I'm surprised this is the first time a reviewer noticed that, say, the 250 cc motorcycle he was reviewing looked suspiciously like a 600.

      • > I'm surprised this is the first time a reviewer noticed that,
        > say, the 250 cc motorcycle he was reviewing looked suspiciously like a 600.

        Oh, I'm sure that would get caught... the real question is, would he notice the port'n'polish job on the head, the blueprinted motor, carb tuning or EFI mapping (throwing it out of EPA spec), degreed cams (ditto), etc, etc?

        Of course, the manufacturer would have to be careful not to do anything to the motor which increased its octane requirement, or the reviewer

      • In any manufacturing line there are units that are built more carefully, look better, perfom to closer tolerances, more polished, that end up performing better/more reliably than a standard unit.

        Technically they are the same as every other unit. But due to their careful manufacturing would operate better than any unit pulled off the line. There are other specifications that are not mentioned in the literature that are equally important to operation. So it's not really a question of different operating spec

  • Scandalous! (Score:5, Funny)

    by Daniel Rutter ( 126873 ) <dan@dansdata.com> on Wednesday October 29, 2003 @01:31PM (#7339710) Homepage
    I am shocked, horrified, and revolted beyond human comprehension.

    But only by the fact that Samsung have never sent me [dansdata.com] any such thing.

    Dammit, I got into this business for the corruption. But do I get over-spec high-dollar hardware, automobiles or prostitutes? No, I do not. It's a bloody swindle, I tell you.

    Look, Samsung. 20 inch diagonal, 1600 by 1200, 700:1 contrast ratio, 16ms response time. Is that too much to ask?

    Delivery address provided on application. Favourable review [dansdata.com] guaranteed.

  • Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?

    Quack II, anyone?

  • Reviewers (Score:5, Informative)

    by pavon ( 30274 ) on Wednesday October 29, 2003 @01:32PM (#7339719)
    This is a crummy thing for the companies to do but it also makes you wonder about the reliability of reviewing companies. Like how he stated that designtechnica prided itself on reviewing retail products, but then never explained why they were using a review unit, and after noticing the discrepency did a lot of talking but still did not bother to pick up a shelf unit and test it, to see if it was true. Most of the hardware reviewers seem really flakey to me, more fan boys than reliable testing labs.

    Unfair tweeking is part of the reason why Consumer Reports never accepts review units from companies, but rather buys them from retail stores, just like anyone else would. The other reason is that receiving free stuff creates a potential conflict of interest which is why they also do not have any advertizing in their magazine or their website. This means that you won't have reviews out before products are released, and operating this way is more expensive, relying on subscribers to run, but it is worth it. I don't always agree with CR's subjective descriptions of products (cars especially), but the hard numbers they provide are the most usefull I have found, and have saved me plenty of money.

    I really wish that there was some site equally trustworthy in the computing world. For providing informative analysies there are usefull sites (I have always been impressed with anandtech). But for reviewing components, I have yet to find one I trust.

    • Re:Reviewers (Score:3, Insightful)

      by dasmegabyte ( 267018 )
      The reason why is that the expense and volume of computer hardware would require the site to be an instant success.

      Think about it...a good graphics card roundup should review cards from all of the companies that make a card based on a particular chipset. If there are 8 companies making that card, at $200+ retail each, that's $1600+ per review.

      Of course, I don't think it's a bad idea. Just one that will take a little bit of ingenuity. A good method MIGHT be to sell advertising space not to hardware manu
    • You buy me all the goodies, and I'll test and review them ;)

    • Re:Reviewers (Score:3, Informative)

      by _|()|\| ( 159991 )
      I really wish that there was some site equally trustworthy in the computing world.

      I'm not vouching for them, but Legit Reviews bought retail memory for a recent review [legitreviews.com]. I also liked Anand's recent test [anandtech.com] of OCZ memory, comparing pre-production and retail parts.

  • Wouldn't that be why some manufacturers send certain "test" samples directly to reviewers? The "test" units might have been factory optimized or best performance, whereas standard store shelf parts are factory defaults. Of course manufacturers will want everything optimized for testing purposes, especially if it's a product that hasn't been introduced to the public market yet. Isn't it sort of like how certain CPUs (engineering samples) are shipped to reviewers "factory unlocked" (e.g. P4), while the ret

  • It gets even worse - Best Buy for Example.... (Score:5, Informative)

    by Anonymous Coward on Wednesday October 29, 2003 @01:33PM (#7339731)
    has the same model number as other retailers, but a lower price. If you look at the Bestbuy HW vs the other retailers, the best Buy HW actually is missing some 'components/functionality'.

    Take a look real hard at that stereo reciever before you buy it....

    • News to me. Do you have any specifics?

      I thought the usual game was for these kinds of retailers to sell model numbers that don't exist anywhere else, but have the same specs as a commonly available one, so that they can "honestly" deny any compettitive pricing claims. ;-)

      B

      • Re:It gets even worse - Best Buy for Example.... (Score:5, Interesting)

        by zerocool^ ( 112121 ) on Wednesday October 29, 2003 @02:33PM (#7340268) Homepage Journal
        I thought the usual game was for these kinds of retailers to sell model numbers that don't exist anywhere else, but have the same specs as a commonly available one, so that they can "honestly" deny any compettitive pricing claims. ;-)

        Having worked at best buy, in the audio department, I can tell you that this is a fact. We sold products that other people did not sell; in fact, it was fairly rare to find products from well known manufacturers that had the same model number at Circut City and at Best Buy. It was done so that we could a.) say that we had exclusive products, and b.) say that we weren't doing competative pricing, but the much bigger one was 3.) when the models had different features, some people want the one from CC, some want the one from best buy. If they had Identical features, people would just go the place with the lower price, but because one may have an extra S-video input or what-have-you, they're willing to shell out the extra $20 for the extra stuff. The trick is for the corporation to get the model that looks more desireable to consumers.

        But, yeah, the main reason for the similar-but-different model numbers for similar-but-different products is to keep people from being able to compare identical products, and thus, simply wait for the sale. It's perfectly legal, and when you think about it, pretty smart.

        ~Will
        • I think another reason they may be doing this more is to stop bogus returns. What do I mean by that? Simple, buying on the internet, picking one up at BestBuy the same day, and returning the one you got via internet to Best Buy. I did that a few times with movies, but nowadays Best Buy is pretty competitive with online prices. That was probably even a worse problem with audio and video stuff. Heck, for video stuff I can't even find some of the same models between two different Best Buys!

          An example of

  • of course this happens (Score:3, Insightful)

    by steelerguy ( 172075 ) on Wednesday October 29, 2003 @01:33PM (#7339733) Homepage
    a lot of these units are sent to reviews before you can even buy the product in the store. with no actual consumer version to compare it against, everyone just pretty much accepts the results as long as they are within reason. by the time people are actually buying the products, reviewers have moved on to newer products.

    car companies used to do this all the time. they would send a 'ringer' to the review magazines. you would then get your car, put it on a dyno or take to a track and not be able to match the numbers.

    just one of those buyer beware things.

  • Ethics (Score:5, Insightful)

    by Shadow2097 ( 561710 ) <shadow2097@gm a i l . com> on Wednesday October 29, 2003 @01:33PM (#7339740)
    Why is it that more and more companies believe that turning a profit and being honest are mutually exclusive? Is there some secret, black ritual to remove ethics from the thought process during MBA classes?

    -Shadow

    • my last roommate was an mba condidate. her room was an absolute mess, she would often crank the heat and leave the windows open, and she never bothered to turn the stove off or shut the door when she left the house.

      maybe that has something to do with it.
    • I suspect it's less of an issue of ethics disapearing then it is that companies are becoming more transparent.

      The immediacy of todays news, people with personal weblogs, and web sites that can stir up interest in a little news item probably means that companies are just being exposed more.

      I'm not suggesting that corporations are filled with crouched over profit dwarves, physically drooling over the prospect of sucking some sap dry, it's just that the little decisions that 'sounded good at the time' are be
    • I'd have to say this is nothing new. Have you even HEARD of the evil conditions that led to unionizing and anti-trust acts at the beginning of the 20th century? How about inventions like the Corvair, or the pet rock?

      As long as there is success to be had from sketchy processes, there will be sketchy processes. Ethics are nothing more than a form of PR. If you believe otherwise, by all means start your own company. See how far you get.

    • Money has the power to let people forget the difference between good and evil. --paraphrased from the movie Sneakers

      At Business schools they try to teach business ethics. In fact with the recent stink of corporate scandals, many schools are finding it necessary and not an elective to have ethics traning. I heard an NPR interview with an ex-con who went to prison for fraud on a coporate level. He now works by speaking to MBA students about his experience with ethics and how not to fall into the same tr

  • Benchmarking developers have known this for years (Score:2, Informative)

    by Anonymous Coward
    I used to work for a company that produced benchmarks. We often found the reviewer machines had little extras (like more memory), more cache on the harddisk, or evil hacks (like the no-error-correction jumper) on harddisks. Sometimes they would even go as far as putting in a different processor and hope it was overlooked.

    More often than not you could catch this stuff and even the playing field when reviewing hardware.

    The video card hardware vendors were even more creative.

  • A story (Score:3, Informative)

    by cft ( 715198 ) on Wednesday October 29, 2003 @01:35PM (#7339758) Journal
    A while back, I bought a 17" samsung monitor which had 102kHz vertical refresh listed in the online "review" of the reseller, but upon closer examination, I discovered that it was, in fact, only 96kHz, so I informed them of this.

    What they told me was quite strange at the time, they said their review unit had a different refresh rate and that they checked with Samsung, but that there was no definite answer as to how this could have happened. All in all, they gave me a 19" for free for the trouble (which they apparently had no part of.)

    This happened in Toronto, Canada in 1998.

    It is good to know SlashDot picks up on such small things.
  • It certainly wouldn't be surprising if this were happening, although the Samsung case is certainly one of the biggest I've ever seen. This has been rumored to have been happening in the general hardware communities for years, with CPUs, memory, and video cards that are the cream of the crop, capible of overclocking far higher than normal chips. Unfortunately, it's rather hard to avoid, since even the obvious solution, buying products off the shelf, can resultin a product that could be better or worse than n
    • Unfortunately, it's rather hard to avoid, since even the obvious solution, buying products off the shelf, can resultin a product that could be better or worse than normal.

      Yeah, but it least it's random then, much like the actual consumer. When you get it directly from the manufacturer, there is little reason to believe they picked a random unit. Furthermore, we're talking a difference in specifications here (wrt Samsung). It's pretty hard to get, say, 2 HDs off the assembly line and one 'randomly' only ha
  • ...car manufacturers have been doing this with reviewers for decades. Is it really so surprising that the computer industry is catching on?

  • Consumer Reports (Score:3, Informative)

    by Rathian ( 187923 ) on Wednesday October 29, 2003 @01:36PM (#7339771)
    AFAIK, Consumer Reports does not take ANY units from manufacturers because there's always the chance they'll be sent a "ringer" unit that is better than the store bought models. It would seem that this is very much case in point.

    Review sites that take donated hardware and advertizing from those same hardware vendors should always be held somewhat suspect until you verify the quality through another source. Few sites are willing to give a bad item "both barrels" because they would be essentially slashing their own throat/revenue stream.

  • Reviews can make or break a company. Just look at the high-end graphics card market: their main customer base are the gamers who live and die by benchmark numbers.

    Hell, I work with commercial billing systems, and I can tell you nightmare stories of benchmarks being run on "special" data.
  • That's the reason I can't trust a majority of the review sites out there - just to many "what if's" (Not to mention the worthless benchmarks.... A FPS Game demo on a toughbook? What?)
    In a perfect world reviewers would be able to pay for the items they review without letting the manufacturer know. But unless your reviewing technology that's 5 years old the price is just to insane. Not to mention that most review sites online aren't anywhere near a real "money maker" and plaster ads everywhere trying to ma

  • It cuts both ways (Score:5, Insightful)

    by Dr. Spork ( 142693 ) on Wednesday October 29, 2003 @01:40PM (#7339809)
    Yes, companies will send their best stuff to reviewers, but there is a push from the opposite direction: they want reviewers to get their stuff early. In the computer world, this means that reviewers often get essentially prototypes. I've found that "first test" reviews of CPU's get processors that are worse than what the consumer will buy once production ramps up, because by then, many bugs get ironed out. AMD chips overclock much better later into the production process compared to the "for review only" samples. That's just one example.

  • Car and Driver (Score:5, Interesting)

    by batura ( 651273 ) on Wednesday October 29, 2003 @01:42PM (#7339829)
    I was reading a review of a Subrau WRX on caranddriver.com and it mentioned that their original review unit was able to post a 0-60 time of about 5.4 seconds. The requested another car from Subaru for a "long-term" test drive, which for them is about 60,000 miles over the span of about two years. This long-term car was equipped very similarly to the previous model, yet, it was never able do below a 5.9 0-60 time.

    Now, ever car enthusiast knows that 0-60 times and such the like are subject to various conditions, but that's a pretty large inequity in the difference between the two cars. They said they must have just gotten a lucky hot car, but I believe that perhaps they got a cherry that didn't have to last as long as the car on the long-term test. If they were only going to have the car for a few weeks, then it didn't matter if it was as reliable as a longterm car, so they upped a few things and gave it to them for review. Same thing with the monitors, I guess. Since its just for the flat panel review, they might as well spice it up. These companies base a ton of business on "independent" reviews, so I suppose its worth it to fix the results.
    • 5.4 seconds is unusually low for the WRX. The average 0-60 time I've seen is around 5.6-5.7. Seems C&D just got a car on each side of the spectrum.

    • Re:Car and Driver (Score:2, Informative)

      by shirai ( 42309 )
      While I agree that reviewers may sometimes receive the best of the crop, I don't think this example shows any such planned discrepancy (or certainly doesn't prove it).

      A 0-60 time difference of half a second could easily be attributed to a natural difference in each car. Cars differ more than you think from one to the other and I have read more than once about discrepancies in cars within the same model.

      In fact (I was trying to find it) but I believe Car and Driver (or some other car mag) said exactly this

  • It's called a "Golden Sample" (Score:3, Informative)

    by obsidianpreacher ( 316585 ) on Wednesday October 29, 2003 @01:43PM (#7339832)
    And there's even an advertisment campaign for Gainward's line of graphics cards that specifically pokes at this concept, and doing so for quite some time ... here's [hardocp.com] just an example, and a Google search [google.com] turns up many more results of this advertising campaign and the resulting products from it ...

    Perhaps I'm just overly cynical, but I tend to trust reviews where the reviewer went out and purchased an off-the-shelf retail copy of X rather than those where the company sent something. Of course, this is hard to do in print publications, because of the time-lag that magazines run through (ie, two months after it's released on the shelves, they have a review of it), but I see no reason (aside from money, which is a big reason) that online reviewers can't do things such as this. I also tend to look towards user-reviews and give those a pretty good weigh-in when I'm making a purchase decision. This is the first instance that I can recall where products are blatently better when given to reviewers than those that are store-bought, but I get the feeling that it's been done in the past.

    The above paragraph reflects what I do for my personal buying choices and should in no way construe that that's the optimal/correct/whatever way for large corporations/organizations/whatever to buy-in-bulk ...
  • Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?"

    Well, how about this [motherjones.com]?

    Search for the word 'Canada' to get to the falsification bit. Yes, this is a very old example, and no, it's not computer-related, but it still seems pretty relevant.
  • Have the companies send them a voucher or 'special id' that lets them get X product from any retail outlet. Then the hardware reviewers can pickup there stuff at Radio Shack, Future Shop, the local pc store or anywhere else.. and that way this negates any way of them 'upgrading' there products just for reviews.

  • I've heard a lot lately about car manufacturers flat out lying about their horsepower ratings, and every electrical engeneer and audiophile I know swears that Watt ratings on speakers mean absolutely jack shit. So what else is new?

  • Consumer Reports (Score:3, Informative)

    by devphaeton ( 695736 ) on Wednesday October 29, 2003 @01:48PM (#7339900)
    Every once in a while i read the Auto comparisons on Consumer Reports to get myself all worked up.

    Consumer Reports will not accept donations of vehicles or products from manufacturers or vendors just for this reason. They will discreetly send someone out 'under cover' to go acquire the products in an "off the lot" or "off the shelf" state.

    This is good, and commendable.

    However, i see a lot of times they will end up mis-matching the cars and trucks they compare. Usually it is simply a matter of trim levels on similar classed models. This *will* have an impact on the final outcome. Obviously it's difficult to do things *exactly*.

    Less often, but still wrongly, they will compare vehicles from incompatible classes- things like Buick Century vs. E-class Mercedes vs. Toyota Camry. Or the classic truck comparisons with the 3/4 ton, V8 powered Dodge and Chevy fullsize trucks, against a V6 F150, against V6 Toyota Tundra and Nissan.

    Consumer Reports might do this to other product reviews too, but i only pay attention to their auto ads for `entertainment'.

    I guess that no matter what, *any* test can be flawed.
  • Hardware manufactures have been doing this kind of thing for quite some time. They control who get's the hardware before a major release, and thus effectively control the reviews themselves. If your in the business long term, than you have to have the newest shiniest thing. Why do you think that the day a product is released that all of a sudden sites can have extensive reviews that took a week or more to do? They sign NDA's and on their release date they go public.

    The problem with reviews with hardware is

  • In some cases, the opposite is true (Score:5, Interesting)

    by The Llama King ( 187264 ) on Wednesday October 29, 2003 @01:53PM (#7339930) Homepage
    I review computer hardware/software for a daily newspaper, and I can tell you that, in many cases, the quality of the hardware I get is less than what you'd see on store shelves. This is because reviewers often get pre-production units, which are essentially lacking some of the bells/whistles and fit/finish of production-line products. Whenever possible, I try to insist on production models, but that's not always possible. It's even tougher for dead-tree magazines, which work on lead times of months, to get full production units.

    That said, there's no way for reviewers to know with pre-production units whether they are getting what will eventually be on the shelf - and it may not be a case of the manufacturer trying to get away with something. A processor in a pre-production unit may be faster, or an LCD screen have a greater contrast ratio, than what ends up at retail, but the reason often is that design changes are made at the last minute related to cost or part availability. In fact, sometimes the product may be less powerful in pre-production than what is finally delivered to buyers. This was particularly true in the days of falling RAM prices - I'd get review PCs with 128 MB of RAM, and when they shipped they'd have 256 MB.
  • The 172t [samsungusa.com] has a 700:1 contrast ratio. The 173t [samsungusa.com] has a 450:1. My guess is this was a typo. Never attribute to malice what can adequately be explained by stupidity.

    Caveat emptor also comes to mind.

  • Samsung 955DF CRT switcharoo (Score:3, Informative)

    by CFrankBernard ( 605994 ) <cfrankbNO@SPAMgmail.com> on Wednesday October 29, 2003 @01:57PM (#7339979)
    The early Samsung 955DF was a perfect 19" flat-black screen CRT with .20mm dotpitch. The control panel was a rectangle in the center that when pushed, slowly slid diagonally down to reveal the control buttons. Very slick. Early Samsung 955DF [ld-technology.com]

    Now the "Samsung 955DF" has controls on the front, the screens are much more reflective and oily-looking, and black appears grey even when the brightness is all the way down. More recent Samsung 955DF [balta.pl]
  • Comment removed based on user account deletion

  • my experience as a reviewer (Score:3, Informative)

    by drteknikal ( 67280 ) on Wednesday October 29, 2003 @02:03PM (#7340027) Homepage
    Two things I saw when I was writing product reviews. First, it was very common for the manufacturer to test the review unit before shipping it. It was uncommon to get hardware that hadn't been opened and resealed. Second, preproduction units often had different specs than the production models. Usually, known differences were noted, though.

    A lot of the manufacturer reps and pr reps I worked with would hand-select or pre-screen review units, but I never ran into any where I thought I was being given something better than what would ship just to get a better review.

  • Ian Bell: "We pride ourselves on reviewing boxed retail products, the very same you would purchase in your local store."

    That's obviously not true. If DesignTechnica has a monitor with a 700:1 contrast ratio, and the boxed retail product, the monitor you would purchase in your local store has a 450:1 contract ratio, then they are not reviewing boxed retail products.

    If they don't go to a retail store and buy the product in its retail package, they are not reviewing boxed retail products.

    We know these revi
  • When I was involved in the review of video games and related hardware the reviewables (especially software) were usually inferior to what was released. Sometimes we would even get hand built peripherals. The only cases where the hardware was superior was in the development consoles that were used for Beta testers. In many cases these had more physical RAM and such, and the games ran better than on the final hardware (smoother animation, etc...).

    The common situation involved getting sent a final beta while

  • Well Duh (Score:2, Insightful)

    by stratjakt ( 596332 )
    Do you think if Intel sends toms hardware a new CPU to review, they don't test the hell out of it to make sure they send the best one possible - the one that can be overclocked the furthest?

    Wouldn't you, if in Intels shoes?

    I was burned by this when I bought the first run of the Asus P4S8X motherboard. Review sites like toms were talking about this board being the cats ass, stable as a rock.

    However, the first runs of the retail version were garbage.

    They had a different clock gen, a different stepping on

  • Are you kidding? (Score:3, Informative)

    by jafac ( 1449 ) on Wednesday October 29, 2003 @03:05PM (#7340617) Homepage
    The software industry is RIFE with such abuses.

    In a previous job, my employer had a special team of people called "Product Managers" - but their job was to go visit magazine reviewers, ensure that they got top of the line grade A technical support during the review process, including onsite support, and coded patches directly from the developer's desktop to the reviewer's. Additionally, there was wining and dining, and talk of strippers and lapdances (though I never witnessed that). In that sense, what was reviewed in no way bore any resemblance to the shring-wrapped package some poor sucker paid $699 for.

    I'm no longer working in that sector, but for my 10 years, the practice was commonplace. Which is why I never read reviews.

  • Oh Please. Unit mix-up? Domestic vs Int'l? Typo? (Score:3, Informative)

    by ashitaka ( 27544 ) on Wednesday October 29, 2003 @03:45PM (#7341029) Homepage
    Everyone here is ranting on about "Yeah, I've seen it done before in cars, video cards, etc. Only one post brought up the point that it may be a simple mistake and a little more digging would have clarified the situation. Note this:

    The contrast specs on the Samsung USA site show the following:

    172T [samsungusa.com] - 700:1
    173T [samsungusa.com] - 450:1

    The specs on the Samsung Canada site say:

    172T [samsung.ca] - 500:1
    173T [samsung.ca] - 700:1

    Perhaps he got a Canadian unit although I don't know why they would be any different.

  • Here's a classic example (Score:3, Informative)

    by Potent ( 47920 ) on Wednesday October 29, 2003 @05:15PM (#7341905) Homepage
    Fuzzbuster gets busted by Escort and Car And Driver Mag for stuffing Escort guts inside of the Fuzzbuster supplied for radar detector review (1979):

    http://www.valentine1.com/lab/MikesLabRpt5.asp

    While you're there, please check out a Valentine One. Mike Valentine makes by far the best detector on the planet, and he's a heck of a nice guy! :)
    --

Slashdot Top Deals

In the long run, every program becomes rococco, and then rubble. -- Alan Perlis

Close