Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Games Entertainment Hardware

Tom's Hardware To Cardmakers : Game Over 49

Posted by Zonk from the hardware-reviewer-breaks-free-of-chains dept.
Merlynnus writes "Today, Tom's Hardware has announced that they're tired of being jerked around by the video card manufacturers and are going to publish reviews on their own schedule rather than one dictated by ATI or NVidia. From the site: "Reviews of cards with pre-launch buggy drivers and yet-to-come updates do no service to the user. We explain why we will no longer play that game."
This discussion has been archived. No new comments can be posted.

Tom's Hardware To Cardmakers : Game Over More

Tom's Hardware To Cardmakers : Game Over

Comments Filter:

  • Hopeful (Score:5, Insightful)

    by HappyKleenexDude ( 716112 ) on Monday October 11, 2004 @01:12PM (#10495374)
    A lot of that message, especially the end, sounds like it's directed more towards other review sites than their to their general readers. Maybe they're hoping others will follow their precedent.

    They're a large enough review site that they could hold out by themselves, but I doubt it will make any real difference unless some of the others adopt a similar position.

    • Re:Hopeful (Score:5, Insightful)

      by general_re ( 8883 ) on Monday October 11, 2004 @01:20PM (#10495464) Homepage
      ... I doubt it will make any real difference unless some of the others adopt a similar position.

      ...which they, for the most part, won't - there's too many sites out there wrapped up in being the first one to post "SNE4K p33K A+ THe H0t N3W C4Rd OMG 1+ phUck1N9 R0x0ReZ!!!!", and it basically ends up looking like PRNewswire with extra ads on top. Oh, well - if Tommy holds the line on this, the hell with the 0-day reviewers. I know who'll get my page views.

      • Re:Hopeful (Score:2, Insightful)

        by SuperRob ( 31516 )
        "Oh, well - if Tommy holds the line on this, the hell with the 0-day reviewers. I know who'll get my page views."

        So do I, and it'll be the one that gets the review up first. Information is free, and as soon as it's out there, the cat's out of the bag. What on earth could Tom's do better with more time? Another graph? Please. I want to know clock speeds and RAM size, general performance metrics, and if it'll fit in my case. Anything else is pretty pointless. If Tom's can't get that done in the same a

        • Re:Hopeful (Score:5, Insightful)

          by general_re ( 8883 ) on Monday October 11, 2004 @01:48PM (#10495776) Homepage
          I don't run out and buy $400 video cards on a whim, just so I can have bragging rights about riding the cutting edge. A fast card that crashes hourly is no good to me. A card that runs like a champ on game "X" when I don't play X is not valuable to me, especially if it turns out that it runs like a dog under my favorite game, Y. A card where half of the advanced hardware features are disabled because the drivers don't work yet is no good to me.

          Which car review would you put more stock in - the one written by a person who got to take the car home for a couple of weeks and subject it to regular daily usage, or the one written by the guy who got to drive it around the block twice while the salesguy was sitting right next to him? I mean, if all you want is info as fast as possible and you really don't care about the source, well, both ATI and nVidia put their press releases on their websites - just bookmark those pages and call it a day.

        • What on earth could Tom's do better with more time? Another graph? Please. I want to know clock speeds and RAM size, general performance metrics, and if it'll fit in my case. Anything else is pretty pointless. If Tom's can't get that done in the same amount of time as the HUNDREDS of other enthusiast sites out there, then that's their problem. If they can do it, but not with quality, guess what? Still their problem.

          Now let's see. Driver stability? Picture quality issues and artifacts? Compatibility with r
        • Information is free, and as soon as it's out there, the cat's out of the bag

          Ahh, and therein lies the rub. Sure the information is free, but if it's based on faulty testing/data, it isn't worth the price you paid for it.

          I mean, I have a magic rock on my desk that empirical testing says it keeps tigers away. I've never been attacked by a tiger, or seen one in my office, so it must work, right?

          Tom's is taking the stance that bullshit testing on buggy drivers, or on beta cards, or drivers optimized to ch
          • Yes, but I don't want numbers. Numbers are only useful when you're measuring your penis. All I care about is what games it'll run, and RELATIVE performance. Such as an X800XT will run a game at higher resolutions and relatively the same framerate as an 9800XT. I could care less if it's one or two frames better than this or that, because that's all within margin of error, statistically.

            So seriously, give me the raw information, because that's all that matters. At least, that's all that SHOULD matter.
    • Maybe everyone has lost sight of the fact that all the analysis that is done is supposed to serve the consumer, the interested reader and to help them make informed decisions.

      Well. Call me a cynic, but I doubt the marketing departments driving the rushed review circus has those motivations. Rather, the idea is to drive up the hype and snow the consumer from making throughly informed decisions. But then, that's just basic implications of the free market for you.

      Not that I have anything against a free m

  • who's fault is this? (Score:5, Interesting)

    by fireduck ( 197000 ) on Monday October 11, 2004 @01:13PM (#10495390)
    Tom's were the people putting the reviews out early using buggy drivers. They're the only ones to blame.

    Nvidia and ATI are all about getting as much free publicity as possible. If slipping a card to Tom's gets a pre-release review that breathlessly points out how it gets 3.5 fps more at 1600x with FSAA turned on, the manufactorer's are going to do it. You can't really fault them for trying to one up the competition.

    However, the article seems like TH are blaming the manufacturer's for this. Take this quote for example: "Maybe everyone has lost sight of the fact that all the analysis that is done is supposed to serve the consumer, the interested reader and to help them make informed decisions." The only people who seem to have forgotten this are the reviewers at TH and other sites that push to get their exclusive first reviews out before the product or any other website beats them to it.

    And then, there's this doozy in the opening paragraph: "It's always the same, with no time to do thorough testing, constant last minute changes, and a rush to print. If we were cynical, we'd have to think that the companies involved do not want us to have time to do thorough testing." Are the companies actually giving them the cards and saying "you have 40 minutes to post a review?", or is it more likely TH is given the card (maybe around the same time a competing site is given a card), and TH is just being quick and sloppy to get the first review out?

    No one is forcing them to publish reviews of products they've hardly tested except themselves. While it's good that they seem to recognize the symptom (shoddy reviews), it's be nice if they recognized the underlying problem (they want to be first to benchmark) rather than pawn it off on the manufacturers (who are just out to get free publicity).

    • Re:who's fault is this? (Score:4, Insightful)

      by Anonymous Coward on Monday October 11, 2004 @01:36PM (#10495623)
      You keep mentioning TH wanting to get the first review out. At best, they are guilty of wanting to be in on the NDA-lift-day first looks. Keep in mind that the "first" review goes to anyone and everyone who publishes at 12:01am on that day. No, TH isn't blameless, but their complaints are not without merit ... if NV and ATI change the drivers every coupld days leading up to the lift of the NDA, TH (and all the other sites) are stuck with a lot of running and rerunning to keep from being left out.

      • Re:who's fault is this? (Score:5, Interesting)

        by fireduck ( 197000 ) on Monday October 11, 2004 @02:01PM (#10495923)
        again, it boils down to sites wanting to be first, even if first means 1 minute after midnight when the NDA expires.

        The whole idea that we need a pre-release review is stupid. Every top of the line video card and CPU is going to be slower than the next top of the line product within a few months of release. That's practically a given. (so a pre-release review on a not-yet-ready for ready for primetime product is just going to qualitatively confirm this.) And its also a given that software is continually tweaked up until release, and even after release. So, if the review sites really wanted to do a service to their readers (rather than a service to their advertisers and their own pockets), they would wait until the product is finalized, and the shipping drivers are locked. Even if this means waiting until the product is shipping.

        And why are they even reviewing products in beta-test stage (as pointed out in the 2nd paragraph)? Particularly in a quantative sense. It's absolutely fine to review the beta for World of Warcraft and say "generally, its loads of fun". But it's no where near fine to review nVidia's beta BFGPU and say it's 4.36x faster than ATIs Wonderblunder. That's just shoddy reporting of test results that are going to change depending on which beta version they're using.

        The 4 points that they list in the article are fairly obvious. 1) We shouldn't rush a review because they tell us to. 2) we shouldn't review stuff that isn't finished. 3) we shouldn't review stuff that isn't going to be available for several months. 4) we (the collective review site community) shouldn't whore ourselves to all put out first day reviews simultaneously.

        While it's nice that they spell it out, why haven't they been practicing this since day 1? Where is the mea cupla? Nice plain English that says "We chose to be a PR machine for nVidia/ATI and have released shoddy reviews in the past. We're sorry and this is how we're making it up to you..." The whole article pawns responsibiilty for the problem off to the manufacturers, in essence saying they are forcing review sites to publish reviews quickly. No, they aren't. Its the review sites who have become too accustomed to nVidia/ATI's nipple too actually adhere to some sort of journalistic principle that are to blame.

        If the fear of not being first trumps being right, then perhaps these websites shouldn't be in the review business to begin with.
        • How about some more rules?
          • Vendor-supplied PR copy and a few overclocking benchmarks does not a review make.
          • You don't waste a whole page on the bleeding cardboard box the thing came in.
          • Selling the products you review makes people doubt your objectivity.
          • Same goes for advertising.
          • Reviews should include some sort of an opinion, otherwise it's just a product summary.

          I've stopped reading hardware "review" and "enthusiast" sites, as I can get the same pathetic excuses for insights straight from the manufacturer'

    • Re:who's fault is this? (Score:4, Insightful)

      by MMaestro ( 585010 ) on Monday October 11, 2004 @02:03PM (#10495949)
      The fault is the hardcore readers who bitch and gripe about a certain news site not having 'the most recent' information up A.S.A.P. even if that information is not yet reliable. If news sites stopped making reviews of early buggy drivers, why not stop making previews? Just about every EA game keeps getting marked as 'could use more polish' or 'it'd be interesting to see what they do between now and launch day'. Yet when games from EA get released, they turn out to be buggy pieces of shit but succeed thanks to the hype machine. You can say this about every game, from the heavily hyped Doom 3 (which most people ended up flaming) to the silent yet still crashed and burned Breed (which was hyped to be a 'Halo killer').
      • Don't forget, as an addendum, these are the same hardcore readers who spend upwards of thousands of dollars, keeping their machine "up to date" only to have it obsolete in 5 months or so anyway.. =) I don't think they are exactly the paragon of rational thinking...
    • I have read in a few hardware reviews from various sites that they couldnt test some things because they had to send the cards back to the companies by a certain date and just didnt have time to do it. Im not sure if that always happens at all the review sites, but I do remember seeing that at a couple places.

    • Strings attached (Score:3, Insightful)

      by DarrinWest ( 203204 )
      There are always strings attached. "We'll give you this pre-release card if you promise to get the review out on the 12th.".

      If they insult the manufacturers, or blow them off, they won't get an early card next time. There are only a few available anyway.

      What would happen if TH had to buy a card off the shelf? They'd have late reviews. Would they still get readers? Depends if the readers want a quick review of a pre-release card, or an indepth review that can be trusted. Both readers exist.
  • and they should pull this off, if they were not as good as they are they would probably lose readers.

  • Good for them (Score:3, Insightful)

    by sevensharpnine ( 231974 ) on Monday October 11, 2004 @01:17PM (#10495434)
    I started reading Pabst's site way back when his book was still relevant. I quit vising a few years ago due to the stupid advertisement-driven paginations and reviews that were evident of either stupidity or bribery. But if the review team is serious enough about their job to anger the two card manufacturers, I can give Tom's another chance. I, like many, will be watching closely for any signs of favoring certain manufacturers/advertisers (including the suspicious fact that most reviews just happen to take place with a cutting-edge Intel CPU--not like there's a contract in place...).

    So I'll put Tom's site back in my bookmark folder for now. If this is some stupid PR stunt, then I'll see it soon enough. But I seriously hope the reviewers over there are trying to bring back the credibility they once had. We really need more honest review sites to break away from the vendor's strong-arm tactics.
    • I, like many, will be watching closely for any signs of favoring certain manufacturers/advertisers (including the suspicious fact that most reviews just happen to take place with a cutting-edge Intel CPU--not like there's a contract in place...).

      Honestly, most folks simply have some preference.

      I don't see what the problem is with Tom's liking Intel and, oh, Anand liking AMD as long as it doesn't color their actual *CPU* reviews -- if they're reviewing some random game, it's fine to let them use whatever

  • There's a reason why I don't read Tom's anymore .. (Score:5, Insightful)

    by SuperRob ( 31516 ) on Monday October 11, 2004 @01:22PM (#10495484) Homepage
    I stopped reading Tom's a LONG time ago. He loves mudslinging, both with manufacturers and with other enthusiast sites. I'm close to giving HardOCP the heave-ho as well. I want INFORMATION, not grousing about how this company did so-and-so wrong, or what site is faking their benchmarks. I can figure that stuff out for myself.
    • The problem, or at least the thing that makes it different from everything else, is that in the world of gaming video cards, NOBODY has any customer loyalty. Gamer A doesn't care if ATI has always provided them a solid package if NVidia's new whiz-bang card is .04% faster at some benchmark that doesn't have anything to do with real gameplay.

      I buy ATI because I like the company and because I didn't like how NVidia basically killed of 3DFX. Does that mean I sometimes get video cards that aren't as fast as
      • You are not alone in being loyal, I've had heated arguments with people who swear by nVidia (even when their cards sounded like jet engines), ATI (when they had buggy drivers), and even Voodoo (even as their top-of-the-line card wouldn't run the latest version of DX).

        When the price and performance of two cards are almost equal (which is true most of the time between ATI and nVidia) then loyalty and marketing make a huge difference.

      • I buy ATI because I like the company and because I didn't like how NVidia basically killed of 3DFX. Does that mean I sometimes get video cards that aren't as fast as I could be getting? Most likely. Does that .04% of FPS loss bother me? Not at all.

        Running your graphics card on multiple operating systems may interest you.

        Going with a card company that supports AMD, OpenGL, Linux etc may be more beneficial to us all.

        I always though nvidia bought 3DFXs technology and some of it shows today.

        3DFX was great

      • ...because I didn't like how NVidia basically killed of 3DFX.

        Yeah. How dare NVidia offer better cards at competitive prices. Shame on them.
    • Personally, I find [H]'s (seemingly increasing) "Look at me" attitude with all their hidden-as-controversy PR 'lawsuits' of late becomming unbearable. I would rather see a site stand up and not have their reviews be such a function of what the manufactures want them to be.
      • Which is why HardOCP is always getting those threatning letters. They *do* stand up and not have their reviews be a function of what manufacturers want them to be. They stick to their reviews, rather than be dictated to.

        Now whether or not you like the review is besides the point. Having a site stand up under pressure of a lawsuit, which costs the reader nothing, gets my readership.

    • Expectations for reviewers (Score:5, Interesting)

      by 0x0d0a ( 568518 ) on Monday October 11, 2004 @03:49PM (#10496985) Journal
      I moved from Tom's Hardware to AnandTech, but I think that you're doing Tom a disservice here. While I agree that Tom's isn't perfect, that doesn't make his point here any less valid. The "preview culture" and hype does customers no good and gives them a good deal of hype.

      As a matter of fact, I would expect a good, reputable product review site to follow the following policies:

      1) No non-shipping products. This is a *review* not an advertisement (i.e. "preview"). What good is a review two days earlier if it's incorrect and misleading? If it isn't shipping, it's a "preview" and should be called that (and those of us looking for hard product data can skip it).

      2) Absolutely no reviewing based on promises. If a driver is buggy and the product crashes, then the product crashes. Calling the vendor and saying "Hi, this is Mike's Review Center and we'd like to know whether you plan to fix this issue soon so that we know what to tell our readers" is fruitless. Of *course* they will tell you that they're working on the problem and will have it fixed in the moment. You are reviewing the customer experience. If a customer can go out to a store and buy the product and it crashes, then that product has problems that need to be mentioned. If there is an existing patch, then it is certainly reasonable to apply the patch (and mention to readers that you had problems that had to be corrected with a patch). However, reference to what "NVidia says" or "Adobe says" has no place in a review.

      3) Use a sane resolution for ratings. Frequently reviewers use an absolutely insane resolution for their product ratings, such as a 1-20 scale (often 1-10 with half-point increments) or even a 1-100 scale). There is absolutely no way that they have the ability to make that kind of call. Opinions are already subjective, and it's a good bet that the reviewer is unable to (a) consistently give the same rating within a 10% range and (b) consistently give the same rating as other reviewers within even a 30% range. Eliminate this kind of false data. I could see using a "Very impressed", "Somewhat impressed", "unimpressed", "Somewhat disappointed", "Very disappointed" scale.

      4) Eliminate the concept of a "combined rating". Frequently, products (especially games) are reviewed based on a scale where they are given ratings in several areas. "Graphics", "Audio", "Sound", and so forth. There is no reasonable mathematical way to make an effective "combined" rating of these scores. If you want to have a category for "Overall opinion", that's great -- but don't try to make it a mechanical synthesis of the other scores. I remember when themes.org tried doing an essentially mechanical score, with very little leeway given to the reviewer -- it was disappointing. If a user cares most about reviewers having good "overall opinion" of a game, great -- let them search based on that criteria. If a reviewer really likes or dislikes a game, even though it has fantastic/really bad sound, they shouldn't be restricted from panning/extolling the game.

      5) Expect reviewers to note what percentage of the game they played/how many hours they played. If a reviewer played half the game and gave a review (not unreasonable for a large game, in my opinion -- in the case of a strategy game with long campaigns, 50% of the levels is probably far overkill). If they played the whole game, great. If they played five levels at the beginning and skipped to the last level to play that, they should mention it. There's no secrets involved, but I think that it is important to let people have honest data on what portions of the games were included in the review.

  • perhaps I'm on the wrong page (Score:4, Funny)

    by kendoka ( 473386 ) on Monday October 11, 2004 @01:26PM (#10495522)
    but didn't that guy sound a little self-important? Sure the readers deserve a good review, but it sounds like this guy went and publicly demonized NVIDIA and ATI because he's under a lot of pressure. To me that seems a little immature given that many people in those 'large, faceless companies' probably think they're locked in the 'battle of their lives...'

    To me the guy seems to think his job is really that important, when it's not... it's just a review, and if he wanted to clarify by saying that they sent it to him last-minute, then he should say so. He's getting a free card (at least for a little while) out of it, I wouldn't complain too much...

    • Re:perhaps I'm on the wrong page (Score:4, Insightful)

      by Anonymous Coward on Monday October 11, 2004 @01:42PM (#10495689)
      "To me the guy seems to think his job is really that important, when it's not..."

      250,000 fanboys deciding which $600 card to buy on the basis of your website and, say, four others.... you don't think that's important? We're talking about $150,000,000 in revenue that's directly tied to what these sites write.

      People base card buying decisions on reviews, not advertisements. Being ethical when in a position like that is not self-important.

      And fuck it all anyway.... since when is it self-important to do your job right in ANY situation? The only reason he made this announcement is that if he didn't, people would think his site hadn't been able to *get* the new card, and he'd lose his business. If you're going to take a stand, you have to tell the people who are affected why things have changed.
    • Toms is the site majorly responsible for the AMD heat issue scandal, the ATI and NVidia optimization and quality scandals and other's.

      They may not be the first to market or have the most popular products but they often have the best reviews and it's investigative journalism.

      Basically they are saying that many of the products that pass through their labs have glaring faults that they simply cannot assess with 12 hours of benchmarking.

      They also need to look around at some of the customer complaints and
    • Seriously compared to other review sites like Anandtech, Toms hardware no longer does the best reviews. The video card sections are milking advertising money. There is no way in hell you need to span 25 pages of graph charts.

      And the conclusion always say the same thing.... "All cards are designed from a reference board. Therefore it's no different if you buy from any vendor." Then why the hell do you need to review 20 boards? Just do 1 ATI, and 1 Nvidia card and finish.

      I don't know who to defend. ATI

  • Good for Tom's (Score:4, Interesting)

    by Derkec ( 463377 ) on Monday October 11, 2004 @01:51PM (#10495818)
    I tend to go visit Tom's when I'm looking to purchase new hardware. Do I really care if a review comes out a week later than the other sites? Hell no. I won't be reading it for a month anyway.

    I hope they are able to pull this off without losing readership. I think they can cede the first review of a product and go for providing the best review. That's strong marketting.

  • Objective Reviews (Score:5, Insightful)

    by drivers ( 45076 ) on Monday October 11, 2004 @02:18PM (#10496089)
    If they really wanted to be objective the review sites would buy their own products retail with no indication that it is for a review site. This is what Consumer Reports does. The goes for software reviews as well. Yes, this requires a different business model than review sites are using now, but right now they are not objective; they are a part of the business they are covering.

    • Re:Objective Reviews (Score:3, Insightful)

      by fireduck ( 197000 )
      exactly. and perhaps they shouldn't be accepting ad revenue directly or indirectly from companies they review. (the current ad i get when loading the homepage is an MSI motherboard based on nVidia's nforce3, which interestingly quotes an Anandtech review in it.)

      but this is internet journalism, where ethical rules don't seem to apply...
      • The poster you respond to implicitly answers your point. It requires a different revenue model to address that. Consumer reports ain't free. Tom's hardware is. Online advertising revenue is not easy to come by.

        On the brightside, while benchmark numbers are potentially perilous, they are way better then nothing. Most product reviews have no standards to go by and you just get extremely subjective and vague impressions the reviewer had in regards to it. "It felt kinda funny" "Seemed sturdy enough" "B
  • If it is standard for BOTH companies to provide review sites with samples and pre-release cards, I don't really see a problem. If THG can't get thier stuff together in time to to a review, why would I want to visit thier site.

    They must not be very familiar with the PC enthusiast community. We love reading news about beta, pre-release, rumor, alpha, next-gen, etc. products. This is how we plan ahead. Sure, we will also read about the product once it's fully release and tested. Until then, we want to hear a
  • I wonder if Tom thinks the vendors will start to give him $$$ if he does a 0-day reviews?
  • That was the point I liked the most. I hate seeing reviews for cool things I can't get. Also the reviews often have no bearing on the real product since they have different versions. What's the point of reviewing a motherboard with bad drivers? This article should have been on the front page of Slashdot.

Slashdot Top Deals

The opossum is a very sophisticated animal. It doesn't even get up until 5 or 6 PM.

Close