Slashdot stories can be listened to in audio form via an RSS feed, as read by our own robotic overlord.

 



Forgot your password?
typodupeerror
Intel Hardware

PCMark Memory Benchmark Favors GenuineIntel 298

Posted by kdawson
from the not-what-you'd-call-neutral dept.
javy_tahu writes "A review by Ars Technica disclosed that PCMark 2005 Memory benchmark favors GenuineIntel CPUID. A VIA Nano CPU has had its CPUID changed from the original VIA to fake GenuineAMD and GenuineIntel. An improvement of, respectively, 10% and 47% of the score was seen. The reasons of this behavior of FutureMark product are not yet known."
This discussion has been archived. No new comments can be posted.

PCMark Memory Benchmark Favors GenuineIntel

Comments Filter:
  • by RandoX (828285) on Friday August 01, 2008 @08:38AM (#24432317)

    Is this like changing the user agent in a browser?

  • by k_187 (61692) on Friday August 01, 2008 @08:38AM (#24432325) Journal
    Exactly, what happens when you run an AMD chip under both IDs? or an Intel? If the Intel optimizations are faster on all three, then there might be a problem, as it stands the via chip might just benefit more from the Intel optimizations than the AMD ones.
  • Re:Money (Score:5, Interesting)

    by SimonGhent (57578) on Friday August 01, 2008 @08:40AM (#24432353)

    Easy. Intel paid them to make it that way.

    If anyone can come up with a better explanation I'd be interested to hear it.

  • by SimonGhent (57578) on Friday August 01, 2008 @08:45AM (#24432453)

    That's an insightful explanation, but IMO the benchmark is then only valid is the operating systems people use make the same allowances for the different chips.

    One thing that doesn't seen to have been investigated is the permutations of the test - marking an Intel chip as a VIA, etc. If the differences are the same drop in performance as the improvement in marking a VIA as an Intel then your explanation has effectively been disproved.

  • CPUIDs (Score:5, Interesting)

    by bestinshow (985111) on Friday August 01, 2008 @09:09AM (#24432925)

    VIA's is "CentaurHauls"
    AMD's is "AuthenticAMD"
    Intel's is "GenuineIntel"

    There's no "VIA" nor is there "GenuineAMD".

    Clearly PCMark2005 is buggy (at the best) and cannot be used to compare different CPU families in this test. At the worst it is intentionally flawed, and shouldn't be used at all.

    It's a shame that not one VIA Nano review benchmarked the built-in Padlock functionality. Not one OpenSSL benchmark.

  • by WindBourne (631190) on Friday August 01, 2008 @09:12AM (#24432977) Journal
    The benchmarks is looking at the ID and making assumptions. These benchmarks run on Windows. So another possibility is that MS does an optimization that few know about. Any of these are plausible. Simplest answer should rule until proof shown otherwise; Bad assumptions made in OS or program.
  • Re:Money (Score:5, Interesting)

    by JamesP (688957) on Friday August 01, 2008 @09:22AM (#24433219)

    Yes, I remember that...

    But why would icc make AMD better than "no name" beats me.

  • by Albert Sandberg (315235) on Friday August 01, 2008 @09:24AM (#24433239) Homepage

    you got a point there which is important to the discussion, if the source is closed, how can we know if the test is fair?

  • Re:Money (Score:5, Interesting)

    by Fumus (1258966) on Friday August 01, 2008 @09:46AM (#24433693)
    What I don't get is why game developers don't release freeware benchmark versions of their engines.
    Saying that a config has 9000 points is pretty much useless. Saying that it gets an average of 40FPS in the UT3 benchmark at high details, and 1680x1050 is much more informative.

    Unfortunately, this also is a little bit more complicated, and as we know everything simpler is more popular with the dumb masses.
  • Re:Money (Score:3, Interesting)

    by CastrTroy (595695) on Friday August 01, 2008 @09:49AM (#24433731) Homepage
    That's what I would think they are doing. If you write a processor benchmark in C, you don't really know what it gets assembled into, you don't know if it's actually even using the SSE/SSE2/SSE3 optimizations. It's hard to know what path the program is taking unless you only give it one path. You'll still get some discrepancies between processors that don't support the same features, because the one that doesn't support SSE will take longer than the one that does, but at least you won't show one processor being faster, when it should in all senses be the same.
  • Re:Money (Score:3, Interesting)

    by Cheesey (70139) on Friday August 01, 2008 @10:07AM (#24434037)

    Fine, testing is one reason for two code paths; you want to make sure the generated code works on all x86 CPUs, but you don't want to thoroughly test on all of them. But that means that icc isn't suitable for compiling benchmarks. because different code is being run depending on the CPUID. The comparison is not a fair test; you have two variables (the code and the CPU) instead of one (the CPU).

    Two questions:

    - Why didn't Intel tell benchmark writers not to use icc? Obviously the results will be unfair if an Intel CPU is being compared against a non-Intel CPU, and Intel will be accused of cheating.

    - Why not call the memcpy in the C library instead of doing a one-byte-at-a-time memcpy? Amateur coding? Or professional evilness masquerading as incompetence?

  • Re:Money (Score:2, Interesting)

    by Fumus (1258966) on Friday August 01, 2008 @10:10AM (#24434099)
    Non-trivial amount of work? If they already have a demo of the game ready (and they bloody should. no demos IMO result in more people pirating the game to 'try it out') I doubt a big effort is needed to make a floating-camera benchmark and some scripted effects.
  • Re:Money (Score:5, Interesting)

    by Bert64 (520050) <bert@s[ ]hdot.fi ... m ['las' in gap]> on Friday August 01, 2008 @11:04AM (#24435027) Homepage

    If your benchmark tool is going to use multiple code paths, then they should be configurable, so that you can benchmark different systems both using the same code and more optimal code. That way you'd get an idea of how much speedup various features provide.

    As an example, john the ripper's SSE2 support for cracking DES - on a core2 the SSE2 version is considerably faster and compiling the C version never comes anywhere close regardless of compiler and flags, but on an AMD compiling the generic C code with appropriate flags and a modern version of gcc produces slightly faster code.

    Running the SSE2 ver on a 2.3ghz core2 quad achieves about 2mil c/s per core, while a 2.3ghz amd phenom yields about 1.6, but compiling the C source with various flags and gcc versions makes amd slightly faster, while the core2 is nowhere close.

  • Re:Money (Score:2, Interesting)

    by Anonymous Coward on Friday August 01, 2008 @11:42AM (#24435805)

    Intel crossed the line when they made their compiler do optimizations based on the CPU vendor-ID "GenuineIntel" instead of the CPU feature-flags.

    The feature-flags are what Intel documented as the correct way to check if advanced features are supported by the CPU.

    And that is why our shop won't use Intel's ICC compiler.

  • Re:Money (Score:1, Interesting)

    by Anonymous Coward on Friday August 01, 2008 @11:53AM (#24436011)

    "I'll give 10:1 odds that Futuremark simply compiled their benchmark with Intel's C++ compiler.

    I wrote a detailed explanation back in 2005 about how the Intel C++ compiler generates separate code paths for memory operations to make AMD processors appear significantly slower, and how you can trick the compiled code into believing your AMD processor is an Intel one to see incredibly increased performance. See this article for additional details." - by Eponymous Cowboy (706996) * on Friday August 01, @10:09AM (#24432917)

    Good job man, & that's completely 'no sarcasm' on my part - congrats/kudos/salutations.

    That sounds COMPLETELY logical, & reasonable + "right on the mark", about futuremark & other benchmarks... especially, as others state here, IF you "follow the money trail" & look @ the motivations of billions of potential profit.

    Hey - I'll even "go out on a limb", & say YOU are probably right here - even though "the great arstechnica" (NOT) apparently couldn't figure that out, & you basically DID, long ago (3++ yrs. back)...

    ----

    HEY - between the processor errata that INTEL has, that has folks in security circles 'terrified' to a big extent, since hacker-cracker types are out to exploit them, per this news:

    Researcher to demonstrate attack code for Intel chips:

    http://www.infoworld.com/article/08/07/14/Researcher_to_demonstrate_attack_code_for_Intel_chips_1.html [infoworld.com]

    SALIENT/PERTINENT EXCERPT:
    ----
    "Kaspersky says CPU bugs are a growing threat, with malware being written that targets these vulnerabilities... Security researcher and author Kris Kaspersky plans to demonstrate how an attacker can target flaws in Intel's microprocessors to remotely attack a computer using JavaScript or TCP/IP packets, regardless of what operating system the computer is running."
    ----

    I am GLAD I have an AMD here, by all means, because still, to date? AMD has not been shown as vulnerable to the above, either... not yet @ least. So far, so good.

    (I would have told anyone that INTEL CPU's were better/faster, until I saw this article of yours AND the one I point out above - regarding CPU errata exploitation being possible on INTEL cpu's, where so far, no such thing has surfaced on AMD stuff!)

    ----

    Ordinarily?

    I'd say that is an accomplishment on YOUR part, in figuring out YEARS AGO, what all of arstechnica apparently cannot!

    Then again - Comparing yourself vs. arstechnica's general populace in computing know-how isn't saying much man (it's like comparing a world-class athlete vs. a newborn child, in a footrace - not exactly a fair comparison (& most anyone could win)).

    No HUGE trick, getting the better of them & their "experts" (like the "no degree, no certification, no years-to-decades of professional hands-on experience in the trenches in the art & sciecne of computing" Jeremy Reimer, for example. No contest).

    APK

    P.S.=> Good job man, good job... &, I liked your article as well - I must have missed it years ago, thanks for bringing it up again too! apk

  • by KWTm (808824) on Friday August 01, 2008 @12:49PM (#24437059) Journal

    The problem with this argument is that with open source software, you don't just have to trust a single random guy for your information. When the source is open, it is often the case that MANY people in the online community will examine the code, and through discussion there emerges a consensus which is far more reliable than the opinion of just one random guy. That isn't to say that the community as a whole is never wrong, but it's vastly more trustworthy and reliable than just some $randomInternetDude.

    I agree with you.

    I was wondering if there is some way we can get code audited by the community on a more formal basis, perhaps with a bounty system and a reputation system, so that one might donate to get the KDE4 code audited by me ($10), or some KDE contributor ($300), or Linus Torvalds ($10000). Then these people could develop a formal reputation system, like + or - votes on SourceforgeAuditVoting.org. They'd use their PGP signature to sign the audits.

    Or something. I would view this as the next phase of the open source economy. Eventually companies might hire people with good reputations, to audit their own intra-company code.

"Consider a spherical bear, in simple harmonic motion..." -- Professor in the UCB physics department

Working...