New Analysis Casts Doubt On Intel's Smartphone Performance vs. ARM Devices 94
MojoKid writes "A few weeks ago, the analyst company ABI Research published a report claiming that Intel's new CloverTrail+ platform (dual-core Medfield) for smartphones was significantly faster and more power efficient than anything ARM's various partners were shipping. If you follow the smartphone market, that was a very surprising claim. Medfield was a decent midrange platform when it launched in 2012, but Intel made it clear that its goal for Medfield was to compete with other platforms in its division — not seize the performance crown outright. Further investigation by other analysts has blown serious holes in the ABI Research report. Not only does it focus on a single, highly questionable benchmark (AnTuTu), the x86 version of that benchmark is running different code than the ARM flavors. Furthermore, the recently released Version 3.3 of the test is much faster on Intel hardware than on any of the other platforms. But even with those caveats in place, the ABI Research report is bad science. Single-source performance comparisons almost inevitably are."
Re:Queue (Score:5, Funny)
Re: (Score:1)
So Intel comes second once again.
Re:Queue (Score:5, Insightful)
The arm fanboys
We are not "ARM fanboys". We are "Intel-haters". There is a difference.
Re: (Score:1)
Android apps (if they're made correctly,) run on Dalvik, which is platform indapendent. The virtual machine is compiled for ARM & x86 both, and it in turn runs the android apps.
Re:It just kind of seems DOA (Score:5, Insightful)
Not if you use the NDK, which most games and video applications will use for performance reasons.
You can of course compile for both.
Re: (Score:2)
If that is the only way to get the performance you need, it is the right way.
Using the NDK is not wrong, it is just an option. The play store should really demand x86 and ARM executable for all apps.
Re: (Score:2)
And why not MIPS? It's a pretty popular CPU in China, and has certainly been used in Android based phones and tablets.
PowerPC is a pretty good-performing mobile CPU, too. And I'm sure SuperH CPUs still have their adherents somewhere.
Re: (Score:2)
Personally I think the best case would be the code is uploaded not any binaries. That way the store can make binaries as needed for new architectures.
Re: (Score:2)
I always thought MIPS would be a better choice than ARM for low power servers... MIPS already has a 64bit variant which is well tried and tested.
Re: (Score:1)
So how do you propose writing code which is portable between Android and iPhone?
I'm REALLY curious about this one...
Re: (Score:2)
Re:It just kind of seems DOA (Score:5, Insightful)
Which would you rather do, use the NDK and recompile, or write once for each platform? "The right way" isn't always a single choice, it's usually a compromise.....
If Intel processors become popular in Android phones, Google will probably introduce a multiple-architecture executable format, much like iPhone does with FAT and MACH (currently around 70% of apps for the iPhone have two architectures, one for ARM7 and one for ARM7s).
Re: (Score:2)
The last Atom/Android phone I read about had an ARM emulator for running NDK apps. Performance suffers, but it's better than not running at all (particularly since Atom is more powerful than the common Cortex-A9 cores, so it has some performance to burn).
ndk has x86 support (Score:2)
Not if you use the NDK, which most games and video applications will use for performance reasons.
You can of course compile for both.
extend the if done correctly to enabling x86 compiling for the native parts.
I really doubt that there's that many apps there that have hand crafted assembly in them...
though this myth that it doesn't have x86 support might have been pushed forward by the action from intel of writing an arm translator for those apps that haven't been ndk compiled for x86.
Re: (Score:2)
Android supports compiling for x86.
So not all of those are ARM exclusive.
Re: (Score:2)
Every iOS app was compiled for x86 also (iOS simulator), but it doesn't mean any games ship with that binary.
Just because you can easily cross compile does not mean a platform will not have a huge hurdle out of the gate just getting someone to re-compile a build.
Re: (Score:3)
App stores make this less of an issue. If Apple wants to switch the iDevices to x86, they'll tell developers that they must have an x86 build posted by such-and-such date, or their app will be dropped from the store.
Which they do not do (Score:5, Informative)
App stores make this less of an issue. If Apple wants to switch the iDevices to x86, they'll tell developers that they must have an x86 build posted by such-and-such date.
Apple DOES NOT do that for older apps. It will often place restrictions on app updates or new app submissions (like must include iPhone5 screen shots) but they don't ever force older apps to do any kind of update - nor should they.
both binaries can be included (Score:2)
Every iOS app was compiled for x86 also (iOS simulator), but it doesn't mean any games ship with that binary.
Just because you can easily cross compile does not mean a platform will not have a huge hurdle out of the gate just getting someone to re-compile a build.
if you enable the x86 portion then the app ships with both x86 and arm ndk compiled binary parts... and mips too.
all major apps would have that shit working instantly if they got even 10% of the android share.. or sooner, if it makes a good showcase.
Re: (Score:2)
if you enable the x86 portion then the app ships with both x86 and arm ndk compiled binary parts... and mips too.
But it requires every app to re-compile, so it's not like you can launch with a full selection.
all major apps would have that shit working instantly if they got even 10% of the android share
No way, it would take a lot of testing and even for a 10% share a lot of companies would approach it cautiously. It would take years to get 50% of the app market onto a new platform even IF (Huge, huge IF) yo
Re: (Score:2)
Apple too might benefit from a shared architecture between the desktop and phone
It'd have to be a hell of a lot more than a "might benefit" for them to do it. It'd have to be the inability of one of the platforms to support future requirements in their existing niche. Which is a very far distant prospect in each case.
Re: (Score:1)
2 billion?
For small values of 2 billion.
Re: (Score:2)
Most android apps are compiled for dalvik and don't care what processor is running dalvik.
Re: (Score:2)
Optimization is another matter, but that's mostly fro GPU stuff.
Re: (Score:2)
Java Android apks run just fine on Medfield.
Clearly this can't be true (Score:3)
Re: (Score:2)
Where, in any of the linked articles, was any data provided by Intel?
Re: (Score:3)
Whenever an analyst produces a dishonest research report, look to the company that benefited from the dishonesty for the source of funding of the report.
Re:Clearly this can't be true (Score:5, Insightful)
Re: (Score:3)
Intel has a VERY long history of questionable ;) benchmarks, all the way to tweaking processor designs to run benchmark code faster. Microsoft's "Get The Facts" propaganda is just a pale imitation of Intel's history.
Supposedly, benchmarks are written to simulate real workloads. It seems to me that tweaking processor designs to run benchmarks faster is a good idea. If you have a better idea for what applications to design a processor for perhaps you should join a processor company in the workload analysis or planning group.
I agree that Intel, or any company with enough "muscle" (see Nvidia, ATI/AMD, MSFT, IBM, HP, Oracle, etc.), will try to influence the press and show their products in the best possible way, even dish
Re: (Score:2)
Supposedly, benchmarks are written to simulate real workloads. It seems to me that tweaking processor designs to run benchmarks faster is a good idea.
That may be true, *if* you don't break the benchmark in the process. This link, shared by another posted further down the /. thread, accuses AnTuTu of exactly that:
http://forums.anandtech.com/showthread.php?t=2330027 [anandtech.com]
In summary, while the compiled ARM code performed a series of bitwise operations as written in order to excercise the CPU instructions, the Intel compiler seems to have applied some compile-time smarts to effectively bypass a lot of the work but achieve the same end result. In a real-world progr
Re:Clearly this can't be true (Score:4, Insightful)
Wow. What's more, apparently that optimisation was added by Intel after the benchmark was developed:
What's more, this optimization wasn't present in ICC until a recent release. Somehow I don't think that they just now discovered it has general purpose value. More likely case is that they discovered is they could manipulate AnTuTu's scores. Seems to coincide well with this third-party report appearing showing how amazing Atom's perf/W is - using nothing but AnTuTu. Or the leaked scores seen for CloverTrail+ and now BayTrail that are AnTuTu. Is this really a coincidence?
So basically they modified their compiler to optimise away the actual benchmark, then got someone to release a third-party report based solely on the benchmark they'd just manipulated the results of.
Re: Clearly this can't be true (Score:2)
The interpretation of whether the benchmark is broken or not depends on what the benchmark was trying to ascertain. If it was trying to ascertain which CPU could process instructions fastest and most efficiently then it could be argued that it was broken because the result is heavily influenced by the logic of the compiler rather than the performance of the CPU. If it was trying to ascertain which platform had a better result allowing for toolchain-based optimizations, then you could argue the benchmark was
Re: (Score:1)
So in this particular case Intel is not reporting wild benchmarking results
Re: (Score:3)
In addition, how much did Intel pay ABI for the report I wonder?
With Intel wanting to get a bigger share in the mobile market as bad as they do (they restructured around it), paying for a favorable research report doesn't sound all that out of scope.
Re: (Score:1)
The intel code was just optimized with a smarter icc compiler that unrolled a loop doing trivial bitops, which is a fairly standard thing for compilers to do. If this breaks the benchmark because optimization wasn't in the spirit of what it was trying to measure, fix the benchmark.
Re: (Score:2)
It was so standard that the ICC compiler apparently didn't bother to do it until after the benchmark was released, probably because it's unusual for anyone to write code that benefits from that optimisation outside of benchmarks.
Re: (Score:2)
The article is on face value self admitted FUD. If, maybe, possibly. Due to the lack of hard facts, the article is raising concerns of what might be happening.
Has anyone tried to use one of the phones hands on? I had the oppertunity to try one of the earlier Intel phones, but not the new one, so I don't have a comparison between the older Intel phone and the newer one. A benchmark between the new and older model would be interesting. I know how the older Intel phone works. It worked very well. The on
Clover Trail vs Medfield (Score:5, Interesting)
I design chips and work for intel.
Clover Trail is significantly more power efficient than Medfield because it has a lot more power control stuff in it, so more of it is turned off most of the time. This is not a big secret as far as I know.
The rest of the phone consumes power too, particularly the screen. So YMMV.
Surprised? (Score:1)
Re: (Score:2)
Problem is, Intel's kingdom is shrinking [arstechnica.com] and while it's the bread and butter, it won't be large enough to sust
Re: (Score:1)
Well, Intel tried to go away from x86 once. They probably don't want a second Itanic.
Re: (Score:2)
x86 is a horrible kludge and always has been, it survives not because it's any good but because there is a large amount of closed source code out there compiled specifically for it.
Trying to shoehorn x86 into another market tho, one that isn't previously locked in to it just seems ridiculous... Even if Intel manage to become competitive with ARM by using more efficient fabbing processes, this is only detrimental to end users as an ARM chip fabbed on the same process would be better still.
Re: (Score:2)
Well, the best thing Intel may do to society is just dying... But I see how they'll think differently.
Anyway, one of the basic tenets of evolution is that it can only happen when you change things. Intel is trying to evolve without changing anything... They'll stay dependent of Microsoft, they'll keep the centralized product development, they'll stay compatible with the power wasting x86 applications, and looks like they are keeping the dishonest marketing department.
By the current trends, even if Intel cre
Re: (Score:2)
It depends upon how their architecture is licensed. ARM sells a licenses and then licenses on steroids. The latter allow you much latitude in how you design your SoC. Intel so far has an "I know best" attitude. The other problem for Intel is their chips are too expensive. To fight on price means their profit margins shrink and that makes chasing ARM less of profitable proposition. On the other hand, Intel might feel their future is threatened by ARM so much that they must go after ARM.
Intel, like Microsoft,
Re: (Score:2)
Yep, the power relation is similar, but ARM was successfull competing with them up to now. In fact, that's the main reason they have a monopoly at mobiles, because Intel killed everybody else... And time is playing at the ARM team, not Intel. With the end of Moore's Law (that's not a certainty yet, but a possilbe outcome for the next generation of fabs), Intel's lead in fab technology becomes less relevant.
Comment removed (Score:5, Insightful)
Re: (Score:3)
I anxiously await Intel's new "Oregon Trail" CPU.
Re:Biased benchmarks endemic in chip industry (Score:4, Funny)
I anxiously await Intel's new "Oregon Trail" CPU.
Might be a long wait, I heard a bunch of their engineers mysteriously died of dysentery while working on the design.
Re: (Score:2)
Single-source performance comparisons ....? (Score:1)
Single-source performance comparisons almost inevitably are, yet this point is being made by none other than a single person. I love the irony.
Streisand effect in action! (Score:3, Interesting)
What's really hilarious is that this supposedly "rigged" benchmark got almost zero press (never posted on Slashdot or mentioned on major tech websites) when it first came out. Now the ARM religion has to wage a jihad on anybody who claims that it is physically possible to use silicon that doesn't include royalty payments to the Church of ARM to run your smartphone.
Using my own ARM-based smartphone and the Dexplorer app, I looked at Antutu and other common Android benchmarks. One interesting thing that stood out was that Antutu uses the NDK (i.e. there are C-compiled libraries for both ARM + x86 ABIs). The other benchmarks like quadrant and Linpack were pure java based. Basically what I'm seeing is that the Clovertrail+ x86 hardware absolutely is in the same ballpark as the snapdragon 600 for performance, but that the ARM vendors and Android developers have been optimizing Dalvik for the ARM architecture since 2009 while little has been done for x86. Once a real compiler like GCC gets to generate C-code for the x86 Atoms though (and also for the ARM parts BTW, it's a level playing field), then we see that Clovertrail+ puts up decent competition for modern ARM chips in the same power envelope.
As for the usual: "Intel cheats using compilers!" whine, please take a big dose of vitamin STFU. I've been bored to death for over a decade by the drone of the ARM jihadists who claim that their architecture is so magical that you can literally knock back a fifth of Tequila and vomit up a perfectly optimized web browser before breakfast. If ARM is truly so beautiful and if the engineers at ARM are truly such geniuses, then it should be trivial for them to implement compilers that blow away x86.
Re: (Score:1)
Re: (Score:3)
When it comes to compiled code the situation is reversed, gcc has been heavily optimized for x86 whereas other architectures although supported have had far less work done on them. There's also Intel's compiler which generally produces faster code than gcc.
What happened to "don't be evil"? (Score:1)
Re: (Score:2)
Or right forgot, it was just a marketing slogan.
...for Google, not Intel.
Where to buy? (Score:1)
AnTuTu assembly code analysed (Score:5, Informative)
Some guy on the Anandtech forums analysed the AnTuTu code and found that it indeed had been tweeked to favor x86 processors:
http://forums.anandtech.com/showthread.php?t=2330027 [anandtech.com]
Re: (Score:2)
1) Massive increase in die area for units that translate the rotten x86 ISA to the internal RISC one.
2) Massive power usage by that translation block
3) Massive IP costs for everything that makes x86 'special' (special in the short-bus sense)
4) Massive coding inefficiency overhead for having to control the true internal RISC ISA with code written to the x86 ISA (usually emitted by a compiler, of course)
5) Massive cost of supporting multiple overlapping compute models such as x87, mmx, sse, sse2, blah, blah, blah...
6) Massive cost to support stupid number of obsolete and legacy instructions that nobody care about but still need to perform well to avoid performance regressions on old binaries
Of course they're running different code. (Score:1)