Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
Data Storage Software Bug Desktops (Apple) Music Windows Linux Technology

Spotify Is Writing Massive Amounts of Junk Data To Storage Drives (arstechnica.com) 196

An anonymous reader quotes a report from Ars Technica: For almost five months -- possibly longer -- the Spotify music streaming app has been assaulting users' storage devices with enough data to potentially take years off their expected lifespans. Reports of tens or in some cases hundreds of gigabytes being written in an hour aren't uncommon, and occasionally the recorded amounts are measured in terabytes. The overload happens even when Spotify is idle and isn't storing any songs locally. The behavior poses an unnecessary burden on users' storage devices, particularly solid state drives, which come with a finite amount of write capacity. Continuously writing hundreds of gigabytes of needless data to a drive every day for months or years on end has the potential to cause an SSD to die years earlier than it otherwise would. And yet, Spotify apps for Windows, Mac, and Linux have engaged in this data assault since at least the middle of June, when multiple users reported the problem in the company's official support forum. Three Ars reporters who ran Spotify on Macs and PCs had no trouble reproducing the problem reported not only in the above-mentioned Spotify forum but also on Reddit, Hacker News, and elsewhere. Typically, the app wrote from 5 to 10 GB of data in less than an hour on Ars reporters' machines, even when the app was idle. Leaving Spotify running for periods longer than a day resulted in amounts as high as 700 GB. According to comments left in the Spotify forum in the past 24 hours, the bug has been fixed in version 1.0.42, which is in the process of being rolled out.
This discussion has been archived. No new comments can be posted.

Spotify Is Writing Massive Amounts of Junk Data To Storage Drives

Comments Filter:
  • by rfengr ( 910026 ) on Friday November 11, 2016 @08:04AM (#53264275)
    Bandwidth, memory, clock cycles....don't matter. Use more shitty layers of abstraction over layers built into high level languages, then kick it out the door.
    • by MitchDev ( 2526834 ) on Friday November 11, 2016 @08:15AM (#53264329)

      Wonder what this does to people's data plans and consumption of their monthly limits...

      So glad I just use local music files and don't stream. Write once, maybe again to add some more music, then just read many,,,

      • Who cares? Thanks to weak net neutrality we can just get our traffic exempt! Fuck the competition who actually have to write performant stuff!

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      Today's programmers? It's been rampant since at least the 1990's...

      • Re: (Score:3, Interesting)

        by Anonymous Coward

        It's called Gates Law, because it's the opposite of Moore's Law.

        Every 18 months hardware became[1] twice as fast, and every 18 months software becomes[1] half as fast.

        [1] This trend has mostly stopped for hardware, but software is still becoming slower with each new version, something I can see at the office where everybody is complaining about how slow the PCs are running with Windows 10, where as mine is running Windows 7 just fine[2].

        [2] Well, fine for Windows anyway. Of course things don't happen instan

        • Remember when the OS fit on a floppy and only did the most basic tasks rather than spying on the user and trying to be everything including the kitchen sink?

          • by Yvan256 ( 722131 )

            I remember when the "OS" was stored in ROM ICs, computers didn't even have floppy drives and could boot under one second.

            • Good point...forgot to go back that far

            • I also remember when the OS ran on top of a Basic interpreter. I never experienced it, but I understand there were OSs written in Basic.

              Let's not even start with entering start points with toggle switches to get the system to boot.

              Now a real old timer will show up to regale us with stories of stringing his own core memory, getting a stitch wrong...

              • I also remember when the OS ran on top of a Basic interpreter. I never experienced it, but I understand there were OSs written in Basic.

                To clarify what others have said, the OS wasn't written in- nor ran under- BASIC. (#) Both the OS and the BASIC interpreter were themselves written in machine code.

                What *was* the case (AFAIK) is that on many 8-bit machines there wasn't such a clear-cut distinction between the functionality of the BASIC interpreter and that of the OS itself; or, at least, much of the OS functionality was accessed through the BASIC command line by default.

                For example, on the Sinclair ZX Spectrum or Commodore 64 (or most o

            • by haruchai ( 17472 )

              "I remember when the "OS" was stored in ROM ICs, computers didn't even have floppy drives and could boot under one second"

              And accomplish what else? Compared to modern PCs, those were hopelessly archaic & difficult to use.
              But what we have no is hopelessly bloated, no argument there.

              • On my Atari ST,with it's OS in 192 kB of ROM and 2.5 MB of RAM (upgraded from the stock 1 MB by soldering in two 1 MB SIMMs), I could run Calamus Desktop Publishing, Finale music scoring, PureC C-compiler (very compact and fast code), Bugaboo debugger, Signum typesetting, and many, many more. All in a windowed desktop environment. It may not have been as refined as contemporary counterparts, but is was not at all as difficult to use as you indicate.

                • Yeah, but not all at the same time, since TOS didn't support multitasking except via some limited hacks analogous to the PC's "Terminate and Stay Resident" utilities.

                  Unlike, say, the Amiga with its full pre-emptive multitasking OS. Did I just reopen the ST vs. Amiga holy war? I think I did! ;-)

                  Joking aside, while I know that the ST did later get "proper" multitasking via Mint/MultiTOS, it's interesting to note that while the Amiga hardware was more advanced in many respects, the OS itself- including the
            • If you are genuinely nostalgic for that era, you can still get a device with that level of complexity and computing power from companies like HP and Texas Instruments.

              • by Yvan256 ( 722131 )

                I routinely use the ATmega328P, it does a pretty similar job when used with the Arduino IDE.

          • Yep. (A)bort, (R)etry, (F)ail.

            Good times.

            Setting IRQs with tiny little DIP switches.
            Swapping out the floppy drive with the spell checker.
            80 x 25 screens.
            Hercules Graphics cards! Whoooeeee!
            33K "high speed' modems.

            Indeed. Those days were the pinnacle of Western civilization.

    • by Anonymous Coward on Friday November 11, 2016 @08:33AM (#53264445)

      If you can't differentiate between bad programming and high-level programming with abstractions, you're part of the problem.

      PS lots of great software is written in higher level languages than you're probably capable of ever reading.

      • Re: (Score:2, Funny)

        by Anonymous Coward

        lots of great software is written in higher level languages than you're probably capable of ever reading

        English isn't apparently among them.

    • by jareth-0205 ( 525594 ) on Friday November 11, 2016 @08:54AM (#53264549) Homepage

      Bandwidth, memory, clock cycles....don't matter. Use more shitty layers of abstraction over layers built into high level languages, then kick it out the door.

      Well, what do you expect? Everyone expects client programmers to support more devices, more user for less money, cheaper / free apps. The last 3 places I've worked at had no QA department whatsoever.

      I know it's fashionable to shake the fist at 'lazy' programmers, but the fact is we expect more functionality from less dev time, requiring abstractions, libraries that aren't completely controlled or understood, testing skipped, etc. Programmers aren't the problem, relentless competition is.

      • Re: (Score:2, Insightful)

        by Anonymous Coward

        Well, what do you expect? Everyone expects client programmers to support more devices, more user for less money, cheaper / free apps. The last 3 places I've worked at had no QA department whatsoever.

        I know it's fashionable to shake the fist at 'lazy' programmers, but the fact is we expect more functionality from less dev time, requiring abstractions, libraries that aren't completely controlled or understood, testing skipped, etc. Programmers aren't the problem, relentless competition is.

        I certainly expect better. Open source delivers quality - again and again. Any organization with an actual budget ought to do better. And please note that the competition is on quality, not on prettyness, and not on delivery date either.

        Also, this can't be a bug resulting from sloppy programming. Sloppy/quick programming results in apps that crash "occationally" and a lot of corner cases that aren't quite right. This MASSIVE writing is something else entirely. Fortunately, spotify is not necessary. In my ca

        • by Nemyst ( 1383049 )
          Open source can deliver quality, but take open source as a whole rather than by just cherry picking the successful projects and you'll find just as much crap as in closed source projects.
        • Open source is not a magic panacea that fixes all ills. It requires dedicated programmers with alot of time, just like anything else. The many-eyes-make-all-bugs-shallow mantra has failed many times, have you followed the OpenSSL Heartbleed?

          If you don't think that this can happen easily then I guess you've not been in programming very long, or at all. Computers will quite happily do something repetitive and destructive in a loop forever, and in a way that is almost invisible to the programmer unless they're

        • I certainly expect better. Open source delivers quality - again and again. Any organization with an actual budget ought to do better.

          This statement demonstrates deep misunderstanding of the open source process.

          Yes, successful open source projects do deliver high quality, and the result is free to user and other developers... but it is by no means effortless. In fact, open source projects require a lot of overhead that projects internal to a reasonably-efficient organization do not. Communication is slower and more difficult, individual developers tend to have less of the context and less focus, etc. Open source succeeds not because it

        • And please note that the competition is on quality, not on prettyness, and not on delivery date either.

          Cheap, Fast, Good: Pick any two.

    • What do layers of abstraction have to do with writing massive amounts of data to the storage device? It's not the layers that write the data, it's concrete code that does, irrelevant if it is behind, 0, 1 or n layers of abstraction.
      • When it's about the behaviour of an underlying library (as it was in this case) that's not properly understood by the programmers using it.

        • If you don't understand an API and what the functions do, it doesn't matter if the code you write has any abstraction. You're still probably going to screw up.

        • An abstraction layer has nothing to do with an underlying library. The abstraction layer, or 2 or 3 above that underlying library still calls the same library function.
    • It has very little to do with abstraction layers.

      It's poor implementation, lack of appropriate testing and, in a lot of cases the aforementioned is a result of unrealistic deadlines.
    • high level languages

      I agree. We need to fire all C programmers and go back to Assembly only. None of this high level abstraction stuff.

    • by gweihir ( 88907 )

      Very much so. And when you tell them that they are doing it wrong, they first do not believe you and then they start to cry. We have far too many coders and most of them really bad.

      • Very much so. And when you tell them that they are doing it wrong, they first do not believe you and then they start to cry. We have far too many coders and most of them really bad.

        We don't have too many coders, we have an industry that is immature because it's far too hard to avoid making stupid mistakes. Other industries can handle below-average participants without collapsing (/causing catastrophic outages, security leaks, whatever). You or I might be awesome, but there can only ever be so many great programmers, half of all programmers are below average. And we need them too. All industries attempt to make the skill easier & safer, and that's a *good* thing. You can always jus

    • To be fair you can program using Java (for example) using as few layers of abstraction as you want. The problem is that the schools insist on teaching the exact opposite and in the companies there is a leveling down between those who can optimize and who can only use frameworks.
    • Skype does this too, sometimes it goes into a mode where, during a call, the hard drive light stays on solid throughout the entire call. Same problem with SSD wear. I deal with it by switching to Skype on my Android tablet. Not sure if the same problem exists there, since it doesn't have a HDD activity light.
  • ... for highlighting the potential for damage as news, don't ya think?

  • by Qbertino ( 265505 ) <moiraNO@SPAMmodparlor.com> on Friday November 11, 2016 @08:39AM (#53264475)

    This sounds like some smart software architect to the abstraction of the persistance/storage layer of the Spotify stack too far whilst at the same time storing to much of miniscule datapoints in Spotifys objects. Because once abstracted properly, adding attributes to your objects and the entire stack is trivial.

    Think of it:
    If your stacks ORM neatly abstracts everything concerning persistance and on the backside syncs on neatly whenever it has the opportunity, all you need is app-side developers and software designers storing every little piece of data they can find and that changes evers millisecond and then you have your bandwidth/load disaster as described.

    If something like this is the case with Spotify, which I do strongly suspect, it is a good example that goes to show that you can take clean-room design too far. And that a haphazard duct-tape and chickenwire approach to product development can have significant advantages, as you build around unforseen roadblocks on a daily basis and only add the features really needed.

    I see an example of this every day, as I am currently doing WordPress development and building a WordPress pipeline for an agency. Large parts of the WP legacy architecture are an abysmally convoluted mess built by people who shouldn't have been let near a keyboard 15 years ago. But having a non-developer build a production capable demo of a website in WP is significantly faster than starting with an actual UX prototype, which quickly leads our team into real-world problems that we often haven't suspected. And suddenly a proper ORM and cleanroom design would cause hassle at one end or the other.

    My to eurocents.

  • Spotify, why (Score:5, Informative)

    by lucm ( 889690 ) on Friday November 11, 2016 @08:41AM (#53264489)

    I use Google Play Music. Not only can it cache songs, you can also upload your own collection. And now that Google has acquired and integrated Songza, their playlists are awesome.

    • by pnutjam ( 523990 )
      Google won't allow me to use the family plan with my custom domain, thanks for rubbing it in.
      • Google won't allow me to use the family plan with my custom domain, thanks for rubbing it in.

        I finally just had to have everyone in the family get a gmail.com account and use the family plan on that. It's a minor annoyance on laptop/desktop, because you have to have a separate browser profile logged into the gmail account you don't use for anything else. It's not a problem at all on Android, which supports multiple Google accounts very cleanly, and I imagine it's fine on iOS because iOS has no notion of Google accounts device-wide, so you'd just log the Google Music app into the gmail account.

  • by aisaac ( 247911 ) on Friday November 11, 2016 @09:36AM (#53264797)

    Here is a possibly related complaint [spotify.com] from almost three years ago.

  • If you leave browsers up all all the time, they have the same problem. Firefox and Chrome. https://www.grc.com/sn/sn-580.... [grc.com]

  • Seems developers don't consider to optimize disk I/O. Recently I saw a live event streamed using firefox (from a not so great website, i guess it uses flash) and it kept my disk 100% all the time. Why should a streaming service write all those video data into disk, can't it just cache in RAM n display n forget the bits?

    Such unnecessary disk i/o wears my disk down, increases power use (if I'm on say battery on my laptop) and of course creates a kind of internal DoS as it hogs the disk i/o and rest of proc
  • Spotify Is Writing Massive Amounts of Junk Data To Storage Drives

    Or are they talking about the music files?

  • That will teach 'em. It is absurd what programmers can do and get away with it, simply because you click on "I agree" on EULA starting with "No warranty"

    Yes, it is going to be expensive, but software is getting worse.

Quark! Quark! Beware the quantum duck!

Working...