Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Cellphones Hardware

Samsung Caught Boosting Galaxy S4 Benchmarks 234

Posted by Soulskill from the not-that-you-should-buy-a-phone-for-its-benchmarks dept.
A recent forum post at Beyond3D made an interesting claim: that the Samsung Galaxy S4's GPU ran at 532 MHz for certain whitelisted benchmark applications, and at 480 MHz for everything else. The folks at AnandTech decided to investigate and found out that the phone does indeed let its GPU run at a higher frequency when particular benchmark software is running. They found a similar oddity with the CPU — it wasn't restricted for other apps, but it was forced to run at max speed during benchmarks. Then they decided to look for direct evidence that this was intentional. "Poking around I came across the application changing the DVFS behavior to allow these frequency changes – TwDVFSApp.apk. Opening the file in a hex editor and looking at strings inside (or just running strings on the .odex file) pointed at what appeared to be hard coded profiles/exceptions for certain applications. The string 'BenchmarkBooster' is a particularly telling one. ... Quadrant standard, advanced, and professional, linpack (free, not paid), Benchmark Pi, and AnTuTu are all called out specifically. Nothing for GLBenchmark 2.5.1 though, despite its similar behavior."
This discussion has been archived. No new comments can be posted.

Samsung Caught Boosting Galaxy S4 Benchmarks More

Samsung Caught Boosting Galaxy S4 Benchmarks

Comments Filter:

  • Government Regulation (Score:5, Insightful)

    by ElementOfDestruction ( 2024308 ) on Wednesday July 31, 2013 @07:15AM (#44433531)
    When every sixth topic on Slashdot is about the evils and perils of Government Regulation, why are we constantly seeing examples of companies misleading, blatantly lying, to their customers? We need more teeth on consumer regulation. I bought my Samsung Galaxy S4 on certain assumptions of power. Remember Hyundai blatantly lying about their fuel numbers for half a decade? They were doled out a punishment, but the boost in sales due to in part by their chain-wide efficiency offset any net losses.

    Slashdot readers will remember this, and probably choose an S4 when faced with so few choices. Samsung sees no benefit to not skewing numbers in the future.

    • Re: (Score:2, Insightful)

      by Bornhuetter ( 2804691 )
      Arguments about whether "more regulation" is good or bad are in my experience almost always misguided. We need better regulation, but that is much harder to define, and doesn't make for as good a soundbite.

    • You bought the phone on certain assumptions of power. OK, that's fine. Would a difference of at most 10.8% difference in the benchmark results have changed your choice? Or would you still have bought the Galaxy S4 for its features? Do your applications run acceptably fast? Has the slow speed made you wish you had bought a different phone?

      This is not at all the same thing that Hyundai did. They actually lied about their test results from standard tests and were caught when the EPA did the same test and

      • Re:Government Regulation (Score:5, Funny)

        by ElementOfDestruction ( 2024308 ) on Wednesday July 31, 2013 @07:59AM (#44433867)
        It's about getting lied to. It fucking sucks. If the operates 10% slower than it performs on tests, I would expect - at the least - an explanation or a discount off the outrageous selling price. Yes, I really don't care if it operates at peak performance, the way I use my phone I suspect I wouldn't see any performance differential.

        But I wouldn't want to buy a Delorean advertised to be capable of going 95 mph, only to find out that it can go 95 mph when it's being timed on a closed course; when normally used, it can only physically run at 86mph. I need 88 mph in a mall parking lot, otherwise the mother fucking Libyans will get me.

        • Re: (Score:2)

          by Xest ( 935314 )

          "But I wouldn't want to buy a Delorean advertised to be capable of going 95 mph, only to find out that it can go 95 mph when it's being timed on a closed course; when normally used, it can only physically run at 86mph."

          You jest but to be fair that's how it usually works. Mine has something like 150mph on the clock but I'm pretty sure if I even approached 130mph in it it'd probably start breaking up.

          • I'm sure it is how it works, but your example is not valid at all. A speedometer is not a claim of maximum speed, it never has been and it never will. I've driven in cars that have >50mph larger difference between the maximum number on the speed limit and the maximum top speed of the unmodified vehicle. It's a gauge, not a specification, and anybody who reads that number and thinks that is their car's top speed is a dolt.

            I've also driven in cars which have odometers that stop at 999,999 miles.

    • Your post is ironic given that this article is about the public policing itself. I wouldn't be surprised if civil litigation came out of this. We'll see.

    • Re: (Score:2)

      by dywolf ( 2673597 )

      because lying is not itself illegal.

      only certain kinds of lies are, and there's damned few of those.

      this is a small article from just yesterday: http://www.cracked.com/article_19485_5-outrageous-lies-companies-are-legally-allowed-to-tell-you.html [cracked.com]

    • Re:Government Regulation (Score:5, Insightful)

      by MachineShedFred ( 621896 ) on Wednesday July 31, 2013 @08:45AM (#44434399) Journal

      And Samsung still wouldn't care, evidenced by past behavior (otherwise known as the best predictor of future behavior):

      Samsung could face 15B Euro fine [bgr.com]
      Samsung, LG fined for LCD price fixing [cnet.com]
      Tax evasion, bribery, and price fixing: how Samsung became the giant that ate Korea [independent.co.uk]
      Samsung agrees to plead guilty to DRAM price fixing, pay $300M fine [justice.gov]
      6 Samsung executives headed to jail for price fixing [edn.com]
      Samsung, LG fined for mobile price fixing scheme [techcrunch.com]

      Everyone is holding these guys up to be some kind of saints in their battle against the evil Apple Empire, when they are thrice-convicted price fixers that screw their customers over at every opportunity, legal or otherwise; and try to screw the competition by suing over standards-essential patents that they don't license for FRAND terms (allegedly).

      Samsung is not a friendly company, but I'll likely be modded down for saying so. Whatever, I've got the karma to burn.

    • Aside from some of the more extreme Libertarians in the crowd, most people here are fine with sensible regulations (though the definition of "sensible" would still be debated). For instance, it makes sense to designate different frequencies in the radio spectrum for different purposes and then enforce that through regulation, rather than allowing it to be a Wild West scenario with products stomping over each other's signals. The stuff where we rant about the "evils and perils" of regulation tend to be the r

    • When every sixth topic on Slashdot is about the evils and perils of Government Regulation,

      The argument over 'more regulation' vs 'less regulation' is about the stupidest argument out there. It's so unnuanced, and the arguments are based primarily on campaign soundbytes, that I just hang my head and cry everytime I hear someone get in that argument.

      The argument is so simple that both sides are right:
      1) YES, we need more regulation, good regulation that improves the world. Also,
      2) YES, we need less regulation, less bad regulation that hurts the economy for no real benefit.

      Every regulation ha

  • I remember old articles where ATI and Nvidia were both caught out gaming benchmarks, in one case by embedding particular benchmark game strings in their driver, and short cutting a few algorithms to boost their score.
     

  • Official answer from Samsung (Score:4, Informative)

    by pieleric ( 917714 ) on Wednesday July 31, 2013 @07:37AM (#44433665) Homepage

    There seems to be an official answer from Samsung here: http://samsungtomorrow.com/4676 [samsungtomorrow.com]

    It's in Korean, but here is the translation, provided by sammobile.com:
    "Under ordinary conditions, the Galaxy S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

    The maximum GPU frequencies for the Galaxy S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

    We remain committed to providing our customers with the best possible user experience."

    • Re:Official answer from Samsung (Score:5, Insightful)

      by Sockatume ( 732728 ) on Wednesday July 31, 2013 @07:46AM (#44433717)

      That doesn't tally with the information extracted from the S4 code: it lists several benchmark apps, which when detected activate a "boost" feature that changes the CPU clock.

      • Re: (Score:3, Insightful)

        by Xest ( 935314 )

        That's because the app has to tell the phone to stop saving battery and start performing at it's most optimal speed and the danger is that if the benchmark apps aren't built to do this then the benchmark apps will only give a benchmark for the phones power saving mode rather than at it's optimal performance.

        There's no overclocking going on, the GPU is rated for 533mhz so running at 532mhz in that configuration isn't any kind of fudge but a genuine representation of how the phone can perform at peak.

        • If "saving battery" is the phone's state whenever it is not running a benchmarking application, it is the phone's normal state.

        • How much does Samsung pay you to spread this misinformation?

          This thread is littered with multiple posts from you spreading the same misinformation that is clearly wrong to anyone who's read the article - the 533MHZ speed is ONLY available to benchmark apps (the code it triggers is even called BenchmarkBooster). Other apps and games cannot access that speed and are limited to 480MHz.

          http://www.anandtech.com/show/7187/looking-at-cpugpu-benchmark-optimizations-galaxy-s-4 [anandtech.com]

          ...all other apps/games were limited to 480MHz.

          Given how many times you've posted mis

          • Re:Official answer from Samsung (Score:4, Informative)

            by Xest ( 935314 ) on Wednesday July 31, 2013 @09:04AM (#44434683)

            "How much does Samsung pay you to spread this misinformation?"

            Most likely the same as Apple pays you, nothing, because I'm not a shill and presumably neither are you. Unless you are of course, in which case then they still pay me nothing.

            "This thread is littered with multiple posts from you spreading the same misinformation that is clearly wrong to anyone who's read the article - the 533MHZ speed is ONLY available to benchmark apps"

            What is it about Samsung's official response that confuses you so much? I think it's quite easy to understand from their simple response that they do this for more than just benchmark apps and also do it for a number of other every day apps:

            "Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance."

            "What is the going rate for spreading misinformation in a Slashdot discussion?"

            Now I've explained that I'm not a shill and that I've explained why you're wrong I have a question for you instead - what is the going rate for attacking Samsung despite being wrong? Nothing? I thought so, so why exactly do you insist on doing it?

            If you really want to see someone spamming this thread go look at your fan Sockatume, who has posted many more posts backing up your incorrect position than I have correcting the both of you. I don't think he's a shill though, I just think he refuses to back down when he's clearly and demonstrably wrong and instead just likes to witter on even more as if if he posts enough he'll somehow become right, even though that wont happen because you can't change reality.

            If you can prove to me that it's all a lie, and that Samsung's other apps listed above don't run at 533mhz, and that no other apps can utilise this feature other than benchmarking apps, and that all the quotes of the GPU in the S4 running at 533mhz are lies and that this was never the case and it's simply being overclocked for benchmarking then I'll gladly concede defeat and agree that Samsung are in the wrong.

            Good luck with that though, but kudos if you can in fact somehow uncover such a massive deception campaign dating back to articles all the way from March.

            Protip: Just because you disagree with someone doesn't make that other person a shill.

        • So the benchmarks would show scores indicative of real world performance, then?

          Isn't that the point?

        • You can't handle full power! (Score:5, Funny)

          by mveloso ( 325617 ) on Wednesday July 31, 2013 @09:16AM (#44434835)

          It's not overclocking, it's just that Samsung underclocks their phones to save battery and to stay within the specified thermal envelope.

          Only the benchmarking apps run at full speed, because they're the only apps that need the full power of the phone at all times.

          Other apps can't handle the full power of the Samsung ecosystem, thus Samsung protects them from the overwhelmingly high power coolness that is the Samsung platform.

          So really, everything we do is in the best interest of our customer. We protect our customers from experiencing the full power of our phones to preserve their mental cohesiveness. Anything less would open a wormhole in the fabric of reality, and we wouldn't want that.

      • Re: (Score:2)

        by jovius ( 974690 )

        I'm trying to understand what's wrong in making a device to run unrestrained when making the benchmarks. The very idea is to test what the device is capable of.

        This calls for more extensive and hands-on comparison and feature testing. It's funny how much people are tuned to numbers.

        The benchmark app producers could also be provided with a flag to turn the limiting features on and off. The more transparency the better anyway.

        • "What the device is capable of" is a function of the device's current state - clock speed, cooling, voltages, power supply, etc.. You want to test the device in the same state that it will actually be used. A 533MHz benchmark is a good indication of what this particular chipset would be capable of when it is running at 533MHz. It is not a good indicator of what this chipset would be capable of if you clocked it to 400MHz, or at 1600MHz, or 3GHz.

        • I'm trying to understand what's wrong in making a device to run unrestrained when making the benchmarks. The very idea is to test what the device is capable of.

          The idea is not to test how fast a device can run benchmarks, but to use benchmarks to be able to draw conclusions about how fast other apps would run. And this kind of manipulation means the conclusion will be wrong.

          Example: I want to know how fast my far can go - but I want to know how fast it can go while still lasting a reasonable time. The manufacturer has a switch that creates 50 more horse powers but makes the engine break down after 20,000 miles. The top speed with the special switch turned on is

      • Re: (Score:2)

        by AmiMoJo ( 196126 ) *

        Both AMD and nVidia have "profiles" for specific games that try to get the best performance out of them with per-executable tweaks. It's not cheating, just optimization. The problem is people get upset when they do it for benchmarks, but really all it does it show how pointless benchmarks are. Either they don't tweak in which case the benchmark performs worse than it would if it were a game/app, or they do tweak and get accused of cheating.

    • Except that all testing thus far shows this to not be true, including the discovery of the benchmark booster....

    • The section of code that activates the changes is actually called "BenchmarkBooster". Someone will be fired for that I'm sure.

    • And in fact, Anandtech specifically points out the opposite:

      It's interesting that this is sort of the reverse of what we saw GPU vendors do in FurMark. [...] In order to avoid creating a situation where thermals were higher than they'd be while playing a normal game (and to avoid damaging graphics cards without thermal protection), we saw GPU vendors limit the clock frequency of their GPUs when they detected these power-virus style of apps.

  • Still way behind even after stacking the deck (Score:4, Informative)

    by the computer guy nex ( 916959 ) on Wednesday July 31, 2013 @07:42AM (#44433689)
    Shows how far behind Samsung is in terms of hardware engineering. They stack the deck and still can't touch a 9 month old phone. Both browser performance and gaming performance, the 2 most stressful use cases on a smartphone, are way behind Apple.

    http://images.anandtech.com/graphs/graph6914/54294.png [anandtech.com]
    http://images.anandtech.com/graphs/graph6914/54296.png [anandtech.com]
    http://images.anandtech.com/graphs/graph6914/54300.png [anandtech.com]
    http://images.anandtech.com/graphs/graph6914/54298.png [anandtech.com]
    http://images.anandtech.com/graphs/graph6914/54305.png [anandtech.com]

    • Shows how far behind Samsung is in terms of hardware engineering. They stack the deck and still can't touch a 9 month old phone. Both browser performance and gaming performance, the 2 most stressful use cases on a smartphone, are way behind Apple.
      http://images.anandtech.com/graphs/graph6914/54305.png [anandtech.com]

      Look at your link. It shows the S4 beating the iP5. Also Sunspider is kind of weird. I think that current Windows Phones with underpowered SoCs post the best scores in more recent comparisons, and that doesn't make a lot of sense.

      Regarding your other links, yes, the iP5 has oddly good GPU performance.

      • Re:Still way behind even after stacking the deck (Score:4, Informative)

        by the computer guy nex ( 916959 ) on Wednesday July 31, 2013 @11:10AM (#44436281)

        Shows how far behind Samsung is in terms of hardware engineering. They stack the deck and still can't touch a 9 month old phone. Both browser performance and gaming performance, the 2 most stressful use cases on a smartphone, are way behind Apple. http://images.anandtech.com/graphs/graph6914/54305.png [anandtech.com]

        Look at your link. It shows the S4 beating the iP5. Also Sunspider is kind of weird. I think that current Windows Phones with underpowered SoCs post the best scores in more recent comparisons, and that doesn't make a lot of sense.

        Regarding your other links, yes, the iP5 has oddly good GPU performance.

        The S4 beats the iPhone 5 while in a freezer. It has heat dissipation issues due to poor built quality.

    • Shows how far behind Samsung is in terms of hardware engineering.

      If by engineering, you meen choosing an off the shelf triple core PowerVR graphics processor over the single core Adreno 320 that Samsung uses.

  • I think we should end this (Score:5, Insightful)

    by TheSkepticalOptimist ( 898384 ) on Wednesday July 31, 2013 @07:51AM (#44433775)

    I don't think we need to celebrate benchmarking phones, period. This was one of those flamebait trolling things that happened in the PC era where people boasted how superficially fast their beloved shoebox was by putting $10k worth of equipment into and liquid cooling it just to get some high number result in 3D Mark or some other meaningless program.

    We don't need this for phones.

    Yes phones play games, yes phones are getting faster, but realize that phones and tablets are a HUGE step back from the PC era in terms of performance so benchmarking them means you may as well drag out your dusty Pentium era PC and start boasting about good its benchmark numbers are.

    Also when 80% of the apps on the Android platform are unstable POS then I don't care about how fast they crash. Even Chrome quits unexpectedly repeatedly and this is by the company that makes the Android platform on their own Nexus brand devices.

  • This was probably done by the Engineers who designed the phone so they could get an accurate benchmark of the final product without it kicking into power save mode. Then again was it left in for negligence or marketing reasons? Whatever the reasons it doesn't actually appear to be over clocking the GPU so what's the issue? And why does everyone have to look so surprised when corporations lie to us they do it all the time anyway!

    /Rant

  • (This is an HTC device rather than Samsung) When I origionally stumbled on this thread I assumed yea yea cyanogen must be doing something wrong using a shit driver or not doing something quite right using a conspiracy theory as an excuse to blurt out a lazy response.

    http://forum.cyanogenmod.com/topic/75172-why-is-cyanogenmod-htc-one-so-slow-6250-on-quadrant-standard-instead-12500/ [cyanogenmod.com]

    I suspect that A benchmark manipulation is not limited to Samsung. B there is still something screwy going on in cyanogen and C

Slashdot Top Deals

Every nonzero finite dimensional inner product space has an orthonormal basis. It makes sense, when you don't think about it.

Close