PCMark Memory Benchmark Favors GenuineIntel

javy_tahu writes "A review by Ars Technica disclosed that PCMark 2005 Memory benchmark favors GenuineIntel CPUID. A VIA Nano CPU has had its CPUID changed from the original VIA to fake GenuineAMD and GenuineIntel. An improvement of, respectively, 10% and 47% of the score was seen. The reasons of this behavior of FutureMark product are not yet known."

Re:Do I understand this correctly?

[chalkyj]

Is this like changing the user agent in a browser?

Pretty much, yes.

That should be AuthenticAMD...

[LiquidFire_HK]

That should be AuthenticAMD, not GenuineAMD.

But that would be expecting editors to actually, you know, edit.

Re:Money

[ArcherB]

Easy. Intel paid them to make it that way.

If anyone can come up with a better explanation I'd be interested to hear it.

TFA offers the following:

At the very least, this suggests some incredibly sloppy coding on Futuremark's part, as the company may be enabling or disabling CPU optimizations based on a processor's vendor name in CPUID instead of actually checking CPUID for SIMD support. In this case, PCMark 2005's memory subsystem test doesn't appear to be aware that Nano supports SSE2 and SSE3, and is instead running a decidedly less-optimized code path. There are two factors, however, that make this explanation a bit difficult to swallow.

First, there's the issue of timing. PCMark 2005 was released (obviously) in 2005, and was obviously coded with an eye towards supporting current and future processors. This is standard operating procedure for Futuremark, which always builds benchmarks designed to last for at least a year, and often two. VIA's C5N-T (Nehemiah) core may have only supported MMX and 3DNow!, but the C7 launched in 2005, and that processor supported SSE2 and SSE3 from day one. Even if proper extension support wasn't built into the first version of PCM2K5, we tested version 1.2.0, and that patch was released on or around 11-29-2006.

Second, there's the issue of performance when Nano is identified as AuthenticAMD. If performance between the AMD and Intel CPUIDs was identical, there wouldn't really be a story here, but it isn't, and that's curious. Futuremark could plausibly argue that VIA's C3/C7 processors weren't exactly on the radar back in 2004-2005, but AMD and K8 certainly were, and K8 launched with full SSE and SSE2 support, with SSE3 added in 2005.

There's more, but I don't want to quote the entire article.

Re:Additional instructions

[CTho9305]

The CPUID instruction provides feature bits that software should use to determine which instructions are available. Using the vendor string is not a reasonable way of detecting the presence/absence of instruction set extensions like SSE.

Re:Money

[Eponymous Cowboy]

I'll give 10:1 odds that Futuremark simply compiled their benchmark with Intel's C++ compiler.

I wrote a detailed explanation [slashdot.org] back in 2005 about how the Intel C++ compiler generates separate code paths for memory operations to make AMD processors appear significantly slower, and how you can trick the compiled code into believing your AMD processor is an Intel one to see incredibly increased performance. See this article [slashdot.org] for additional details.

Re:Compiler Optimization?

[afidel]

CPUID can be intercepted so it shouldn't be a big deal to grab the call and return whatever you want.

Re:Compiler Optimization?

[lastchance_000]

Yes, it does: http://en.wikipedia.org/wiki/CPUID [wikipedia.org]

Re:MMX/SMD Extensions

[AcidPenguin9873]

The thing is, the x86 CPUID instruction [sandpile.org] gives you many different things in EAX/EBX/ECX/EDX depending on what you put into EAX before you execute it. The "GenuineIntel"/"AuthenticAMD" strings are what you get when you put 0 in EAX. When you put 1 in EAX, you get various feature bits in ECX and EDX, such as support for SSE/SSE2/SSE3. Any code worth its salt will base any decisions on those feature bits from CPUID function 1, not on the string you get from CPUID function 0.

Re:Money

[CastrTroy]

You have to still be pretty careful when looking at game benchmark scores. ATI has cheated [techreport.com] before with popular games. I wouldn't put it past a lot of companies.

Re:Money?

[Anonymous Coward]

Seems obvious, but follow the money trail, does PCMark get backing from Intel?

Yes, and from VIA. And a whole lot of other companies who want to pay the (annual, if I remember correctly) entrance fee of the PCMark BDP (Benchmark Development Program) - a support contract, in essence.

http://www.futuremark.com/bdp/pcmark/

They have pestered and been pestered by other companies regarding "cheating" and so far always prevailed. Why they would all of a sudden (well, three years ago) have started making shady deals with Intel (and poor deals, if you look at the size of the company) is beyond me.

Pick me, pick me!

[pxc]

The Phoronix Test Suite [phoronix-test-suite.com].

It's Linux only, but a CPU that performs better on Linux will perform better on Windows.

Re:troll? really? mod up again!

[NitroWolf]

By whom? Someone trustworthy? Mathematics? You're clutching at straws there, dude.

Uh, prior experience. Isn't that obvious? I don't have to trust someone to use their stuff. For example, I don't trust the people who put cracks up on gamecopyworld.com, necessarily... I just don't care. I use their stuff even though I don't trust them, and they could theoretically build up trust over time.

The point is not that open-source is inherently more trustworthy than closed source, it's that an open-source vendor who claimed that their code could do something it couldn't do would lose credibility.
Closed-source products give the vendor "credibility through obscurity", i.e. something for nothing.

Uh... any vendor who claims their product does something it can't loses credibility. And when they claim they can do something that they can, in fact, do, they gain it. This is all very elementary, and has nothing to do with closed or open source.

I have to think you are trolling, because I can't possibly believe that someone that's capable of typing on a keyboard can be so inept and unable to discern the difference between the level of trustworthiness on a given topic when an unrelated sample of the population gives it a thumbs up and a single vendor gives it a thumbs up.

There's a reason science and statistic gathering uses random swaths of the population for information gathering. It provides a fairly accurate sample of the whole. Using a single data point, rarely, if ever, provides an accurate depiction of a larger system.

When you accept the consensus of 1000 unrelated programmers who have no coinciding agenda nor any stake in the product succeeding or failing, you have a much more unbiased and therefore a much more likely to be accurate assessment of the code.

When you accept what a single company tells you about their code, which they have an agenda (to sell more software) and a stake in the outcome/product success (to make more money), you automatically have a less accurate assessment and a HUGE bias in the assessment. You are likely to get a very inaccurate assessment of the code.

The fact that you can't discern the difference between the two is evidence of mental retardation or trolling. Since you don't seem to write like someone with a metal handicap, I'm going to go with trolling. So stop.