A User Archived Nearly 2 Million Gigabytes of Porn to Test Amazon's 'Unlimited' Cloud Storage (vice.com) 233
An anonymous reader shares a report: Reddit user beaston02 was determined to find the true ceiling of Amazon's cloud storage plan, which was killed off in June. He decided to push its limits with a petabyte of porn. For reference, a petabyte is one million gigabytes. "It is nearly entirely porn," he told me in a Reddit message. "Ever since I got into computers, I found myself learning more, and faster when it was something more interesting. Call me crazy, but women interest me more than most other things on the internet and there is a huge amount of data being created daily, so it was a good fit for the project." He said it took five or six months to collect one petabyte of porn, and he stopped collecting just shy of 1.8 petabytes. How long would it take one to consume 1.8 petabytes of porn? 1.8 petabytes is about 23.4 years of HD-TV video, but webcam streams are nowhere near that quality. A few good folks crunched the numbers: 720p is about two gigabytes per hour, and at 900,000 hours, that's 102 years of straight calendar time. If the videos are even lower quality, say, 480p, that's around 0.7 gigabytes per hour, or 293 years and six months.
2 million gigabytes of porn (Score:5, Funny)
Re: 2 million gigabytes of porn (Score:5, Funny)
Rephrasing one guy - if 90% of internet is porn, nobody forces you to watch it all
Re: (Score:2)
Rephrasing a guy rephrasing one guy - if 90% of internet is porn and kittens [youtube.com], nobody forces you to watch it all
Re: (Score:2)
Forget the kittens, pity the tax payers, they have to pay taxes to protect all that porn, why, just why? When by constitutional law, that protection is against the letter and the principles of that constitution, porn based upon the US Constitution which so many Americans prattle on about, should not be protected by copyright. If content does not pass a value to society test, which under user pays system, the person applying for copyright protection should pay, then that would should under law being denied c
Re: (Score:3)
> porn based upon the US Constitution
I'm guessing some sort of parchment-based cosplay figures highly in your fantasies.
Re: (Score:2)
> porn based upon the US Constitution
I'm guessing some sort of parchment-based cosplay figures highly in your fantasies.
Powdered wigs is an obscure fetish, to be sure.
If the videos are even lower quality, say, 480p, that's around 0.7 gigabytes per hour, or 293 years and six months.
There can't be more than 10 minutes of good stuff in that. Sad!
Re: 2 million gigabytes of porn (Score:5, Funny)
Re: 2 million gigabytes of porn (Score:5, Funny)
Rephrasing one guy - if 90% of internet is porn, nobody forces you to watch it all
More specifically, quoting Dr. Cox (SCRUBS [wikipedia.org]):, "I’m fairly sure if they took porn off the internet, there’d only be one website left, and it’d be called Bring back the porn!”
Re: 2 million gigabytes of porn (Score:5, Funny)
Re: (Score:2)
Only if her first name is Courtney...
Re: (Score:2)
I see that you, too, love Cox?
Only if her first name is Courtney...
I can agree with that, though... I have a bigger crush on Christa Miller - and her characters Jordan Sullivan (Scrubs) and Ellie Torres (Cougar Town).
Re: 2 million gigabytes of porn (Score:4, Interesting)
Preaching against watching porn does not reduce the amount of porn being watched but only makes people lie and feel bad about watching porn. There was a time in the distant past when human survival was dependent on humans having multiple partners. Those whose brain rewarded them for this behavior reproduced more and passed this on to their descendants. Tiger Woods is a perfect example of this. Even with an almost perfect mate, his brain rewarded him when he had sex with other women. Which is worse?
1. Watching a movie where it shows how to give pleasure to another human.
2. Watching a movie where it shows how to murder another human.
I just watched a Netflix movie where a large number of humans were killed. The worst was when a woman who was holding on to a ledge of a building was shot point blank and another where a young girl was placed in a box and burned to death.
I am sure there would be calls to shut down Netflix if it shows two women giving oral sex to each other but I am sure this movie will not generate much opposition.
Re: (Score:2)
Re: 2 million gigabytes of porn (Score:2)
Re: (Score:2)
Yes. you shouldn't be forced to watch all of it... Problem is finding the good 15%....
Re: (Score:3)
We could help if he gives us a link.
Re: (Score:2)
I suppose we should be glad they didn't measure that in football fields.
How many is that in Library Of Congress? /rhetorical
Re: (Score:2, Insightful)
Re: (Score:2)
293 years and six months.
If you fast forward through the foreplay/scene setting/grinding parts it's probably only a couple of days worth
Re: (Score:2)
At work I found things Just Ran Better when I applied lz4 compression on large ZFS pools. Space is cheap, if more storage is needed, just buy it. At home I've experimented on a relatively small 2 TB mirror with lz4 and ZFS' dedupe enabled. It took some tweaking to eliminate most of the 'bursty-ness' associated with it. The other ~24 TB is simply lz4 with no dedupe and it has never needed any real
Re: (Score:2)
dedupe is voodoo that can offer wonderful things, but can also bite you right in the anus.
Is that part of that collection TFS mentions? If so, I'd like to subscribe to your newsletter.
Re: (Score:2)
"Git" seems to have solved the deduping problem rather well.
Re: (Score:2)
Re: (Score:2)
Git works by hashing the data. Hashing data works great for both text and data.
Yes, hashing a large file takes longer, but you do it as it is uploaded, and the impact is minimal.
Re: (Score:2)
Deduping my local copies of all my personal data has netted decent results, but that's mainly document-type files. Like anything: YMMV
Re: (Score:2)
--Would you mind going into minor detail regarding the tweaking?
/ zfs fan/user (on Linux)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
The question to my mind was: Did that guy dedupe his own collection before uploading?
Re: (Score:3, Insightful)
For research (Score:2)
violating the T&Cs of the sites in question.
The fact is, he never re-distributed the records.
And probably, given the huge archive (over 250 years worth of video according to TFA) there's simply no physical way he would even re-whatch himself.
So, at the end of the day, the data was exclusively used only to test the limits of the storage.
Meaning it's probably one of the rare occurrence where the guy could genuinely argue it was done solely for research.
Almost certainly without permission of the girls involved,
again, given the huge amount of data and how systematic and indiscriminate his grabbing scripts are,
Uh (Score:2)
Glasses (Score:2)
Eternal Flame (Score:5, Funny)
Imagine a grave stone that instead of an eternal flame there is an eternal stream of porn on a small display. The location may be problematic for the viewer to fully enjoy though.
Re: (Score:2)
With tissue boxes to the left.
Re: (Score:2)
And to the right... https://hardware.slashdot.org/... [slashdot.org]
That.... (Score:5, Funny)
My other zinger (Score:5, Funny)
Re: (Score:2)
Probably offtopic but do men without foreskins wank in a different way?
Probably with furious resentment.
Pied Piper will compress that down big time (Score:4, Informative)
Pied Piper will compress that down big time so that real hit is way less.
Re:Pied Piper will compress that down big time (Score:5, Funny)
Pied Piper compression works pretty well on gay porn video.
I've heard it's specially suited for dual jerk off material.
Re: (Score:2)
I'm sure you meant Piper Perry...
x's and o's (Score:2)
My collection of ASCII porn can fit on 5 double-sided floppy disks and it's lasted me 30 years.
Re: (Score:2)
Re: (Score:2)
Just don't punch the floppy... it will hurt.
Re: (Score:2)
I don't get it (Score:5, Interesting)
Re:I don't get it (Score:4, Interesting)
why not just randomly generate various files.
Because then this wouldn't be on Slashdot. The only reason this story is interesting is the porn angle. Look at the posts so far: 90% are about porn and maybe 5% are about cloud storage.
Re: (Score:2)
why not just randomly generate various files.
Because then this wouldn't be on Slashdot. The only reason this story is interesting is the porn angle. Look at the posts so far: 90% are about porn and maybe 5% are about cloud storage.
What about procedurally generated porn? :)
Re: (Score:2)
... Look at the posts so far: 90% are about porn and maybe 5% are about cloud storage.
Just like the internet...
Re: (Score:2)
So, what, "ALL YOU PR0N IS MINE"?
Re: (Score:2)
*ALL YOUR PR0N IS BELONG TO US!*
This probably is showing my age.... And the inability of the /. spam filter complaining about yelling.
Re:I don't get it (Score:5, Insightful)
That's so much porn you can't even possibly look at most of it.
And your excuse for saving years of old emails you'll never read again is what exactly?
If you wanted to test Amazon's "unlimited' storage, why not just randomly generate various files. You could probably have a computer make shitty modern art paintings much faster than you could curate a 1 petabyte porn collection, and you still get to test out how much you can store on Amazon's cloud storage service before they pull the plug. And when they invariably do, you won't lose your porn collection.
Hrm...random worthless bullshit vs. porn. No surprise which was deemed more entertaining. There's a reason porn has always been a predominant force online.
Re: (Score:2)
Last week I needed to lookup an e-mail from 2006. It happens.
Re: (Score:2)
And what happened when you wanted to look up some porn from 2006, ay?!
Yeah, thought so.
Check and mate.
Re: (Score:2)
Using porn means that the test was flawed. He didn't really store petabytes of data. He had a relatively modest collection of porn and uploaded it multiple times, and the server de-duplicated 99.99% of it into a few bytes of a reference to the original instance.
Re:I don't get it (Score:5, Funny)
"That's so much porn you can't even possibly look at most of it. "
You don't have multiple monitors and split screen's for dozens of concurrent streams when looking at porn?
Fucken amateur.
Re: (Score:2)
Re: (Score:2)
he already had it so it was just a matter of uploading
Assuming there are no repeats (Score:5, Funny)
All this does is make me realize there's a lot of porn I will never see. And that makes me sad.
Re: (Score:2)
Fortunately, there are rarely surprise endings.
Re: (Score:2)
Re: (Score:2)
Dang! I live up on the rez in Tulalip and only get 25/3 Mb/s.
Still remember downloading porn pix at 9600 baud, waiting for the nipples.
Re: (Score:2)
It was heartbreaking when the modem crapped out right before they appeared
There goes one more nice thing (Score:5, Interesting)
Whether it's unlimited data, unlimited backups, unlimited soup, salad and breadsticks- this is why we can't have nice things. Regardless of motive, people like this ruin for the rest of us.
Re: (Score:2)
There's a big difference: bandwidth comes and goes but storage stays.
With storage people put 50-500TBs on amazon (now on google) in a matter of days (they're using fast VPSes, google compute stuff, etc). Then they're using 1000 times the regular user forever. This doesn't happen with bandwidth. You can't just use your pipe for 3 days then cost your ISP forever as much as 1000 "average" customers (even if you turn your computer off). Then when you are in the mood do some uploads again and start to cost the
Re: (Score:2)
You mean people who use what they bought? Those bastards.
No they aren't ruining anything other than the ability for companies to lie in marketing materials.
Re: (Score:2)
I wonder if he really did use petabytes, or if he uploaded multiple copies of the same dataset and the server deduplicated it all down to a few gigs.
I find it hard to believe that if it wasn't deduplicated Amazon wouldn't have cut him off by now. Their service is "unlimited" in the modern sense of the word, where there is actually a secret "unlimit" that they won't tell you but if you exceed it they will ask you to pay for a metered business account.
Re: (Score:2)
I think dedupe is probably a reasonable explanation. I usually see deduplication systems cut data consumption by 30-40% but in some cases it can go even higher than that.
The limit I would have expected him to run into was creating a data set that exceeded the storage system, hitting some size that required spanning storage buckets in a way it couldn't do.
Re: (Score:2)
people like this ruin for the rest of us.
Not if he makes his collection public. In fact, that would remove 90% of the need for unlimited storage in the first place. Win-win!
(Backs away slowly) (Score:5, Funny)
"But not nearly as interesting as the assorted women parts I have archived in my basement freezer," he continued.
Re: (Score:3)
What's the dedup rate on this (Score:2)
Since many hosting services use Amazon as a backend, I'm wondering how much Amazon actually stored. 2PB in and of itself isn't that impressive anymore, you can fit it in half a rack or less with current densities.
Re: (Score:2)
Intel's new 'ruler' SSD form factor lets you put 1PB in a single 1U rack space: https://www.theverge.com/circu... [theverge.com]
Well, (Score:2)
I guess we can figure out why they shut it down.
The inverse of Internet Rule 34 (Score:2)
If it's porn, this guy has it.
Re: (Score:2)
...Of course if it takes the cops 100+ years to go through all of it...
I expect that they would "put out a call for volunteers" and knock it out in a year.
idiot (Score:2, Redundant)
This is why we can't have nice things like unlimited storage on cloud servers. Some peon comes along and decides to childishly see if they can prove that the word, unlimited really isn't true by doing something inane and stupid. He satisfies his urge to say, "nay-nya-na-nay-nya," and everyone else suffers. Moreover, by loading up on content of questionable social value, he reduces what might have been a valuable resource (say, backing up the Internet Archive), to an unabashedly immature, indefensible act of
Argh (Score:5, Interesting)
Someone has already mentioned the tragedy of this ruining it for other people and in a way I agree, but since he did it using the remainder of the unlimited storage period, I'm not going to feel bad about it either. This dude is a fucking hero even just for making the world's most obvious joke about Amazon's Cloud storage into a reality.
But it's a damned shame that the unlimited storage plans are going away because I really did have about 6TB of actual personal data (mostly photos, which are still unlimited for Prime subscribers anyway) on my account and the amazing thing about Amazon as opposed to every other big cloud service is that they never, ever seemed to run out of upstream bandwidth. I took a 4TB drive to my server colo and sent data to Cloud Drive. Amazon took it at the physical speed of the hard disk. Compared to Google or Carbonite or Backblaze, where data transfer throttles seem to be the name of the game, the idea that Amazon will take THAT MUCH data in so short a period of time is more impressive to me than just having the disk space to store it.
Provide LINKS for all of us or it didn't happen. (Score:2)
And HEY I'm being altruistic here -- I could have said "Just for me." You're welcome.
(Now if it's all slightly modified versions of goatse.cx and tubgirl I'm going to be REALLY upset. 1.7TB of THAT??? And you thought looking at the eclipse w/o glasses was blinding.)
Other units? (Score:3)
How much is that in litres of sperm?
Re: (Score:2)
Ha! Or rods per fortnight.
But what's the actual disk usage? (Score:2)
He said it took five or six months to collect one petabyte of porn, and he stopped collecting just shy of 1.8 petabytes.
Yet the actual disk usage was probably much smaller thanks to deduplication. The fact that he downloads porn rather than producing it means someone else already had it, and if it's already somewhere in Amazon's system, they just add him to the list of "owners" of each file and use next to no disk space. This is probably why they didn't care.
How is that possible? (Score:2)
Hm ... (Score:2)
I hope he cataloged and tagged it properly or he never finds his most favourite again.
Epiphany (Score:2)
How much is redundant? (Score:2)
2 petabytes of porn? (Score:2)
How does this guy get anything done?
vision problems ... (Score:2)
... OMG they were right, I've start to go blind! Oh wait, that's just presbyopia [wikipedia.org].
Re: (Score:2)
So, do you deny them your essence [npr.org]?
Re:Porn is unwatchable. (Score:5, Insightful)
Only disgusting whores do it. They are not sexy in any way, in the vast majority of cases. Only disgusting. The females you would *want* to see naked never would do such things (in public).
Dude, we get it, only gay male porn for you!
Re: (Score:3)
Only disgusting whores do it. They are not sexy in any way, in the vast majority of cases. Only disgusting. The females you would *want* to see naked never would do such things (in public).
You must watch a lot of porn to be able to speak with authority about what the "vast majority" looks like.
Re: (Score:2)
Re: (Score:2)
I guess at one point the "collecting" part becomes more satisfying than that.
Wow.. that's REALLY sick!
Re: upload time (Score:3)
Blame the bad businesses, not compliant users. (Score:2)
If a data storage service advertises "unlimited" storage they're lying to you. Nobody has the capacity to store unlimited data and businesses aren't keen to purchase storage because users take the business at their word.
No, if a data storage business shuts down or sharply curtails a so-called "unlimited" storage plan, it's because they got caught being taken at their word and learned they were unwilling to make good on a promise of service they never should have made in the first place. It's not possible to
Re: (Score:2)
Have you considered the possibility that women also watch porn? Or are they supposed to be pure little angels with 0 knowledge of human reproduction because that suits your world view?
Re: (Score:2)
You're the one using the wrong units.
https://en.wikipedia.org/wiki/... [wikipedia.org]
Re: (Score:2)
Ah, I see the mistake. But that's 1000 Terabytes, not 1024 which was my point.
Re: (Score:2)
Turns out, even if there's only a tiny percentage of the population who are exhibitionists, with the total population of the planet equipped to record and upload their sexual acts, the result is a lot of porn.