'Hobbit' Creates Big Data Challenge

CowboyRobot writes "In the past five years there has been an 8x increase in the amount of content being generated per every two-hour cinematic piece. Although 3D is not new, modern 3D technologies add from 100% to 200% more data per frame. In 2009, Avatar was one of the first movies to generate about a petabyte of information. The Hobbit: An Unexpected Journey was shot in a new digital format called High Frame Rate 3-D, which displays the movie at 48 frames per second, twice the standard 24-fps rate that's been in place for more than 80 years." But with digital storage transcending some other limitations of conventional projection techniques, it's not just framerate that directors are now able to play with more easily; it's the length of movies themselves, which stats suggest just keep getting longer.

Re:Smart play by the studios

[MightyYar]

Huh? Most of the stuff you find on the various pirate channels is compressed down - at the most, you'll get a raw Blu-Ray rip. You can still get an xvid (avi usually under 2GB) version of just about anything that is available in any other format.

Re:Smart play by the studios

[jandrese]

The Petabyte figure is almost certainly for all of the working copies of the movies while it was being produced. Nobody is sending a Petabyte to every theater in the country, and much less to every home. Once the movie is finished a final copy is compressed and sent to theaters and the disc authoring house. The disc authors have to further compress the image to make it fit on the Blu-Ray or DVD. Your average pirate is going to compress the movie even further because full Blu-Ray rips are still rather unwieldy for most broadband connections and personal storage solutions.

Re:it's the length of movies themselves

[Sir_Eptishous]

This.

How many times do we need to see Goblins getting knocked off wooden plank bridges by dwarves with a pole?
Not enough it seems.

Re:How big was the hobbit?

[bughunter]

Well, I can't find an official number, but we can estimate using data from here [wikipedia.org] and here [imdb.com]:

From the first link, which says the max data rate is 250Mbps, and doubling that to account for HFR, we have a 500Mbps data rate. Multiply that by the 169 minute running time and you get

500e6 bit/sec x 1/8,589,934,592 GB/bit [google.com] x 169 min x 3600 sec/min = 35,400 GB

(assuming the limit on precision is the running time at three significant digits).

Divide that by 1024 GB/TB and you have about 34.6 TB. Not impossible to set up, and probably far less expensive than the projector... but that's for the non-IMAX version, which probably explains why I could only find three theaters with the HFR IMAX version near my house in Pasadena CA.

I also expect that some theaters will not operate at the maximum data rate but use some other, more lossy compression. It's probably safe to assume a lot of theaters are showing distributed versions that are about 10 or 20 TB large.

Re:How big was the hobbit?

[mill3d]

Actually, it's all the data required as input to make the *final* frames. We're talking many layers of video at 32 bit per channel (128bit images), VFX cache data which can be GBs per second of footage, thousands of textures that are also GBs in size, point clouds... All of that is meant to retain a maximum amount of flexibility before finalizing the footage. Read up on the REYES pipeline for detailed info.

Disclaimer: Film and animation professional and professor.

Re:Smart play by the studios

[SuricouRaven]

The standard pirate sizes ate 700MB for a recompressed DVD-rip and 4.4GB for a recompressed blu-ray rip. These sizes are used because they are just small enough to fit onto a CD-R and DVD-R respectively. It's not universal though. Especially long or difficult films might go up to 8GB in HD, and there has been a recent trend towards smaller files where quality wouldn't be compromised rather than just assuming media-size for everything.