Catch up on stories from the past week (and beyond) at the Slashdot story archive


Forgot your password?
Data Storage Movies Entertainment Technology

'Hobbit' Creates Big Data Challenge 245

CowboyRobot writes "In the past five years there has been an 8x increase in the amount of content being generated per every two-hour cinematic piece. Although 3D is not new, modern 3D technologies add from 100% to 200% more data per frame. In 2009, Avatar was one of the first movies to generate about a petabyte of information. The Hobbit: An Unexpected Journey was shot in a new digital format called High Frame Rate 3-D, which displays the movie at 48 frames per second, twice the standard 24-fps rate that's been in place for more than 80 years." But with digital storage transcending some other limitations of conventional projection techniques, it's not just framerate that directors are now able to play with more easily; it's the length of movies themselves, which stats suggest just keep getting longer.
This discussion has been archived. No new comments can be posted.

'Hobbit' Creates Big Data Challenge

Comments Filter:
  • by iONiUM ( 530420 ) on Thursday January 03, 2013 @01:19PM (#42463763) Journal

    I read TFA, and nowhere does it say how big The Hobbit was.. only that Avatar was about a Petabyte. Why isn't this stated anywhere? It's very frustrating, and also makes the article less useful, since its entire premise is that "The Hobbit creates big data challenge" with no specificity.

    • by Anonymous Coward on Thursday January 03, 2013 @01:30PM (#42463973)

      According to the torrent sites; 2.32 GB though they do use the lossy video camera conversion...

      • Re: (Score:3, Funny)

        For some reason I find this observation hilariously funny...

    • by Anonymous Coward on Thursday January 03, 2013 @01:31PM (#42463991)
      Tolkien writes that Hobbits are between two and four feet (0.61–1.22 m) tall, the average height being three feet six inches (1.07 m).
    • That petabyte must be without any compression. The Hobbit (HFR, 3D) as used by a digital cinema projector still fits on a 500GiB hard drive.

      • by mill3d ( 1647417 ) on Thursday January 03, 2013 @02:56PM (#42465107)

        Actually, it's all the data required as input to make the *final* frames. We're talking many layers of video at 32 bit per channel (128bit images), VFX cache data which can be GBs per second of footage, thousands of textures that are also GBs in size, point clouds... All of that is meant to retain a maximum amount of flexibility before finalizing the footage. Read up on the REYES pipeline for detailed info.

        Disclaimer: Film and animation professional and professor.

    • The data space isn't about how large the end result film is. It's the data storage requirements for the entire production. This would include raw footage, cgi generation, presumably all archiving, and probably many other things. As in, I have a new movie I'm creating and need a petabyte of disk space to accomplish this. Not, I'm creating a 1 petabyte movie file.
  • by Dartz-IRL ( 1640117 ) on Thursday January 03, 2013 @01:19PM (#42463769)

    Am I the only one who longs for the return of an intermission? If only for a little relief rather than ducking out for 3-4 minutes and missing that one important little line of dialogue on which the whole thing pins?

    • by interkin3tic ( 1469267 ) on Thursday January 03, 2013 @01:27PM (#42463905)
      Until they do bring it back, there is an app for that []
    • by cervesaebraciator ( 2352888 ) on Thursday January 03, 2013 @01:28PM (#42463935)
      This would be helpful. Statistics may show that movies are getting longer, but my experience shows also that minute-for-minute they feel longer. At least they do when they're something like Michael Bay movies with their interminably long CGI-gasms (I mention Michael Bay, but most directors seem to be doing action sequences in his style; as much as I like the Hobbit, the best comment I saw about it was [paraphrased] 'I kept waiting for Peter Jackson to put down his X-Box controller and get on with the movie). An intermission would give me just enough time to think seriously about the horrible decision I've made and how hours of my life would be better spent by going home for a beer and a book.
      • I kept waiting for Peter Jackson to put down his X-Box controller and get on with the movie

        Thats brilliant! A more concise assessment of the film could not be had. We can only hope the next two films have less of a premature ejaculation vibe to them.

      • by RDW ( 41497 )

        An intermission would give me just enough time to think seriously about the horrible decision I've made and how hours of my life would be better spent by going home for a beer and a book.

        May I suggest The Hobbit? At 1.4Mb (including the illustrations) the .epub would fit on a floppy and, I suspect, still end up saying more than Jackson's multi-petabyte trilogygasm.

        "Why, I feel all thin, sort of stretched, if you know what I mean: like butter that has been scraped over too much bread." - Bilbo Baggins.

    • When I lived in Germany the movies had about a 10 - 15 intermission. They'd come around with snack trays like venders at a ball game here in the states. Even for 90 minute comedies there were intermissions. It was wonderful during longer movies. And probably made the theaters more money as people would use the restroom and usually by another beer, soda, snacks (the concept of free refills doesn't really exist in Europe).

      • We have a few dine in theaters here in Florida that do that too. They're rare (and more expensive per person) but it's a nice change from the standard theater style

      • by vurian ( 645456 ) on Thursday January 03, 2013 @02:21PM (#42464665) Homepage
        I had a free refill, once, in Europe. In Linkoping airport, where I was allowed to fill my coffee cup again from the can. I was so surprised... Never happened anywhere else. And given the quality of the coffee, I didn't bother in Linkoping airport either.
        • by arth1 ( 260657 )

          Coffee in Scandinavia is often with free refills and sometimes free altogether, especially with meals. And much, much stronger[*] than what Americans are used to, so I'm not surprised you didn't want a refill.

          Soda, on the other hand, is typically served in much smaller glasses, often without ice unless you ask for ice, and no refills. Scandinavians just drink a lot more coffee than soda, unlike over here.

          Kastrup airport outside Copenhagen was, incidentally, the first place I went where I got charged for a

    • I don't long for it. It's called the space bar. mplayer pauses exactly as long as I need, whenever I need it.
    • Yes please.

      One of the reasons I don't buy concessions is because if I do, I'll want a drink. If I drink something, I'll have to use the restroom during the movie.

      I there were an intermission, I would buy those overpriced snacks.

    • I would rather see movies become shorter. I liked The Hobbit, but I think it would have benefitted by leaving another 20 minutes or so on the cutting room floor, including that entire first segment between Frodo and Bilbo.
    • My copy of "Doctor Zhivago" on DVD is on two discs, because it didn't fit on one. When it gets to the end of the first disc it plays the original theatrical intermission card on a loop. I just wish the DVD allowed for it, like buffering the scene so it could keep playing it while you changed disc.

  • by Naatach ( 574111 ) on Thursday January 03, 2013 @01:20PM (#42463785)
    It was so classy. I'm sure it would help with the theater owners concession sales as well.
    • by vurian ( 645456 )
      Er... You mean movie theatres in the US don't have intermissions? That would mean that people get up all the time to go for a pee. Should be pretty disturbing.
      • by seinman ( 463076 )
        Most people pee before the film starts, and do not need to again until it is over. Two to three hours is not long to wait. The few idiots who don't think of that or drink so much pop that they can't hold it anymore are the only ones who would need an intermission.
    • The Cooper had not one, but TWO concession stands IN the theater at the sides, totally unobtrusive, it was glorious. 800+ seats, Cinerama (146-degree arc) screen.
      I saw one of the last movies shown there - re-mastered Lawrence of Arabia.
      I sat at the absolute focal point of the screens, having skipped my morning classes to catch that. I was one of 5 people in the theater. []

    • by axl917 ( 1542205 )

      Yes it was. I love the intermission during 2001, there's just something about a pause in the story accompanied by Ligeti that's just surreal.

  • by Spy Handler ( 822350 ) on Thursday January 03, 2013 @01:31PM (#42463983) Homepage Journal

    which stats suggest just keep getting longer"

    And in the Hobbit's case, longer, and longer, and...... just waay too long. LOTR movies had 1000 pages of book to fill them with interesting content. Hobbit, not so much. In many of the scenes you can almost feel the director guy just out of camera view making that "stretch" motion with his hands.

    • by Sir_Eptishous ( 873977 ) on Thursday January 03, 2013 @01:57PM (#42464339) Homepage

      How many times do we need to see Goblins getting knocked off wooden plank bridges by dwarves with a pole?
      Not enough it seems.
    • by h4rr4r ( 612664 )

      If it was shorter you would be bitching about all the things he skipped.

      I loved the movie and I wish LOTR would have been done in a similar style. This is how you convert a book into a movie.

    • by JackDW ( 904211 )

      "The Hobbit" needs a fan edit to bring it below the two hour mark. This should be easy for part 1, though the real editing challenge would be to do it for the entire trilogy. Tricky, but possible, because it's not a long book.

      I much preferred the LotR approach of releasing shorter versions to theaters and then releasing long versions on DVD for dedicated fans.

      • by h4rr4r ( 612664 )

        A fan edit to shorten it?
        Sounds like the opposite of fan to me, if you want to shorten it.

        If your attention span is so short how did you ever make it through the book?

        • by JackDW ( 904211 )

          Well, I thought I was a Peter Jackson fan, but I guess I'm not, since real fans don't criticise.

          I don't recall getting bored during the book at all. But I was bored during the film. It really dragged on. It's not so much the plot development and the story - those are fine. It's the action sequences. They are repetitive and interminable. Some of them could be cut out completely, while others could be significantly shortened, and the film would be better for it. There is a tradition of "fan edits" that make

    • The Hobbit: An Unexpectedly Long Journey
    • I mean they did the 1000 page Battlefield Earth in under 2 hours in film and that turned out great...

      Look at even LOTR, special editions probably make it a 12 hour film for the same page count.

      • Re: (Score:3, Insightful)

        by 7bit ( 1031746 )

        I mean they did the 1000 page Battlefield Earth in under 2 hours in film and that turned out great...

        "Battlefield Earth" + "turned out great" in the same sentence?... Travolta, is that you?

      • by Pizza ( 87623 )

        Except the Battlefield Earth movie covered less than half of the book.

    • I watched it yesterday. and all in all i thought it was a pretty good movie an decent story. I am also biased because I spent 3 years working for howard shore, the man who writes the score.
      • by afgam28 ( 48611 ) on Thursday January 03, 2013 @04:30PM (#42466311)

        Granted, the special effects were well done, but "a decent story"? All I remember was this:


        Some dwarves, a hobbit and a wizard go on an "adventure". They get into trouble, and then the wizard saves them. Then they get into trouble again, and the wizard saves them again. Then they get into trouble again, and the wizard summons some really big birds to save them again.

        I still don't understand why they didn't just take the birds from the start, and all the way to the end. It would've saved a lot of trouble, not to mention hard disk space.

  • by catmistake ( 814204 ) on Thursday January 03, 2013 @01:32PM (#42463997) Journal
    I'm ok with advances in technology and the new challenges it creates. What I'm not OK with is a director deciding to make the source material "better" by changing the narrative. Jackson completely gutted Tolkien's Hobbit, rearranged the important events, and has replaced a light-hearted adventure story with the dark themes from LotR. Mr. Peter Jackson, why do you hate the work of JRR Tolkien?
    • ... and has replaced a light-hearted adventure story with the dark themes from LotR.

      You mean like the changes Tolkien himself made and wanted to make? []

      • The more I see this debte, the more I realize much of LOTR was Tolkien's personal "fan fiction". It was stuff he wrote down for fun.. Mostly to practice applying "archeology" skills for languages and myths. Languages and history is what he taught at university, tracing back origins of languages and myths to the events and people that it happened to. LOTR was some fun working "backwards" with the same principals to build his personal LARP.

        I begging to agree with George R R Martin in that he's going to wrap

    • completely gutted

      What, changing a few lines of dialogue to make the transition from book to screen easier? That's gutting?

      Adding some scenes that tell the backstory of the Oakenshields? That's gutting?

      Bringing in some canon characters to make the story a better prequel to LOTR, entirely consistent with the canon of the milieu? That's gutting?

      The only element that was out of place was having Azog gallavant all over the place chasing Thorin, but still, the basic conflict is entirely consistent [] with canon.

      • Well *obviously* authors are allowed to gut their own stuff, because it's *theirs*. Unless it's George Lucas.

        I don't really find any of Jackson's changes all that galling. Some of the additions are a bit "why?" but most are "fair enough".

    • I didn't mind him trying to make the Hobbit part of the LOTR cycle. it wasn't originally written that way, but as the other reply stated, Tolkien wanted to do that too. in fact, some of the dialogue changes he made when the book was re-released DID change certain things to make the Hobbit fit better with the later books. And I guess he pulled in some of the pieces of The Quest of Erebor and the Similarrion (sp?) to try to tie things together. But really, the main difference is that The Hobbit was written a

  • by Sir_Eptishous ( 873977 ) on Thursday January 03, 2013 @01:50PM (#42464239) Homepage
    there is a reason for them to be long(>2 hours).

    I'm a HUGE Tolkien fan, and went to the LOTR Extended Version Trilogy Marathon recently before seeing The Hobbit.
    I was surprised at how well the longer versions of the films held up, after not watching them for around five years.

    However, The Hobbit film was a let down on several levels, most of which I won't go into here. My main complaint? You do not need three films to tell the story. PJ has thrown in everything but the kitchen sink into The Hobbit, and it drags. Even the uber-videogame-esque "escape from the Goblins" scene drags... Too much of a good thing can ruin a film.

    I would also say the same thing about the last Batman film. Too long and drawn out. Scenes that should be edited or removed alltogether. Thats why they call it the Directors Cut!

    It makes me wonder if there aren't people involved in the film such as producers or editors who tell guys like PJ or Nolan, "hey bro, you might want to trim things down, just a smidge... You know, just to kind of keep the flow of the film going"
    • Even the uber-videogame-esque "escape from the Goblins" scene drags

      Ironic, isn't it? By adding more action, the movie became less riveting. You don't need a threat of violence to keep a movie entertaining (not that there was a threat, since one guy got smashed full-force in the face with a mace and remained conscious).

      For me, one of the most attention-holding scenes was at the beginning, when the dwarves ate, sang, and cleaned dishes. That was a party I'd love to hang out with. The farther they got from Tolkien in telling the story, the weaker the story became. The reven

      • Ironic, isn't it? By adding more action, the movie became less riveting.

        Good point (The Matrix: Reloaded immediately comes to mind as the posterchild movie in this regard).

        Also, it is a little odd to see some people complain about too long of a runtime when most of the complaints about the original LOTR movies was that too much was cut out of the story (Tom Bombadil, Scouring of the Shire, etc.).

        • Also, it is a little odd to see some people complain about too long of a runtime when most of the complaints about the original LOTR movies was that too much was cut out of the story (Tom Bombadil, Scouring of the Shire, etc.).

          Yeah, I don't think the problem with the Hobbit is the long runtime, the problem is the runtime was extended by adding things that were made up out of nowhere.

          If they'd stayed true to the story (or even true to the Tolkien universe) no one would mind a long runtime.

        • by tycoex ( 1832784 )

          I think most people would be fine with the length of the Hobbit if it only included stuff that actually needed to be in the movie.

          It all depends on how much material they actually have to work with. I don't want to watch a 2 hour movie stretched in to 3, and I don't want to watch a 3 hour movie cut down to 2. The Hobbit was the former.

    • by ADRA ( 37398 )

      I appreciated Batman Rises much more after the second watching from home.

      As for scene cuts, a movie this large where scenes cost millions of dollars to shoot / produce, you can damn well bet that the movie was story boarded to death before a single roll of film was shot. Its not like the old days were you can just randomly have great actors run on a scene longer, or just make up an extra dialogue inter-cut and then drop it depending on how well it fit in with the rest of the movie. Hell it happens, but ulti

      • Well, then the storyboarding is too long...

        Yea, it's too bad, because Nolan is a brilliant director. I mean, "Inception" is the best sci-fi I've seen in years. It's hard for me to square it really, because I love most of his films.

        But "The Dark Knight Rises", with Bane and his annoying and non-understandable heavily accented blabber via too much audio effects to the police being sent notes via the sewer locked underground, the whole thing was a mess, a much too long mess.
  • The Hobbit was also shot (and maybe shown?) at 4K resolution.

    That's another bump in the data size.

  • At the London preview screening Peter Jackson said that because 48fps + 3d is 4x the frames it's taken longer to render and the last scene with the coins was only finished a couple days before the premiere. He did mention the complexity in moving coins though

  • I wonder why this is news... I guess it's a good time to invest in ENTERPRISE level storage!!

    The Hobbit was shot in 5k resolution, and in "true 3D" with two cameras on every mount. And shot at 48fps (5k x 48 fps x 2 cameras) is MASSIVE film stock. Add multiple shoots for setup, testing lights, and then the actual acting. Not to mention all the digital elements that have to be stocked at high resolution as well.

    Yup that's a LOT of data. But then so was Star Wars. I suppose with those really long copyrights

  • by ranton ( 36917 ) on Thursday January 03, 2013 @02:41PM (#42464917)

    How does this have anything to do with Big Data? Storing large amount of data isn't the important part, it is being able to analyze that data. You do not analyze a movie's data file. You just load and display the movie, which can easy be stored in one large continous file. A Big Data problem would be Netflix trying to determine what kinds of movies to recommend, not storing and then displaying a long movie to users.

  • Tolkien's work is love-it-or-hate-it and unfortunately I fall squarely on the "hate-it" side. I guess it's good to know that we can enjoy hours of tedium at a higher-than-normal frame rate, though.

  • I remember watching Ghandi in the theaters in 1982, and it had an intermission because it was too large for a single reel, so there was 10 minutes while they changed the reel. Spartacus, and Ben Hur were also over 3 hours. They are just listing the longest upcoming movies expected, with no actual analysis of percent movies over 2 hours and 3 hours per year. It's multiple anecdotes pieced together, ignoring all data contradicting the hypothesis.

    Call me when the average time of a Disnet movie is 150 minut
  • short, reader's digest versions of stories.

    Growing up, my dad ran a video tape repair business. He had many popular video chains (including the now defunct Blockbuster) as customers.
    We watched movies ALL THE TIME. As I entered adulthood, the movies themselves got so watered down and predictable, that it became a struggle
    to find something worth watching. As I hit my teens, I understood that what was missing was the actual story.

    In effect, Hollywood was giving us 60 minute T&A action flicks, Family

  • Everybody wants to go to digital distribution, but I refuse to accept compromise for the experience. I want uncompressed sound and stunning visual clarity in my movies, not some overly compressed barely HD content with stereo sound split to 5.1 false channels.

    Everybody wants to move to the cloud but I live in a G8 country where my bandwidth is throttled and still stuck at 20th century download speeds and upload speeds that are barely better then dial-up.

    So yes, the next big challenge for big data is to del

    • You'll have to learn Korean or Japanese to have that promise delivered. If the G8 country you're referring to is the US, don't hold your breath.

The unfacts, did we have them, are too imprecisely few to warrant our certitude.