Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

ILM's Datacenter

Posted by CmdrTaco on Thu Mar 30, 2006 11:14 AM
from the something-to-read dept.
Data Storage
kylegordon writes "CGW has inside scoop on Industrial Light and Magic's facilities after they moved from San Rafeal to San Franciscos Presidio. With 3000 disks, it can shift 170Tb to 5000 rendernodes over 10GbE and 1GbE network links. It's an impressive system, for impressive films."
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

ILM's Datacenter 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • With 3000 disks, it can shift 170Tb to 5000 rendernodes over 10GbE and 1GbE network links. It's an impressive system, for impressive films.
    Unfortunately, all that storage can't provide decent acting, quality humor or plot lines without holes for their movies.

    I bet I could make a graph that represents how the quality of movies is characteristically inversely proportionate to the amount of CGI effects in them. Oftentimes, eye candy is used to shroud the plot and mask the bad acting/directing. American audiences especially just go looking for explosion sequences and CGI in the annual summer action flick hunt. We often fear a movie that might prove to be too cerebral and that pretty much disgusts me. Way to reinforce bad movies that are only good for one viewing with volume set to 'loud' and TV set to 'huge.'

    ILM is responsible for making movies like The Mask [imdb.com] (of which there are seven films) and characters like Jar-Jar Binks [wikipedia.org] possible. Be sure to thank them for that.
    • There is absolutely nothing wrong with "The Mask". It's a great film. One of the few Jim Carey films I can bare to watch.
    • ILM is responsible for making movies like The Mask [imdb.com] (of which there are seven films)

      There were only two "The Mask" films. And only the first one counts if you don't like stuff that sucks.
    • American audiences especially just go looking for explosion sequences and CGI in the annual summer action flick hunt.

      You say that like it's a bad thing.

      Okay, CGI of explosions is bad, though. Stuff should get really blown up. Big stuff, not just models. It's the American way!
    • I bet I could make a graph that represents how the quality of movies is characteristically inversely proportionate to the amount of CGI effects in them. Oftentimes, eye candy is used to shroud the plot and mask the bad acting/directing. American audiences especially just go looking for explosion sequences and CGI in the annual summer action flick hunt. We often fear a movie that might prove to be too cerebral and that pretty much disgusts me.

      OK, here's the thing. Movies that are "cerebral" and thought-prov
      • Look at "Air Force One." Decent movie, spoiled by absolutely terrible special effects at the end.

        You mean the final splashdown? It was fine. What stretched things just a bit was a middle-aged politician who could beat the shit out of a whole squad of elite Spetsnaz; or trying to work out the motivation of the Secret Service traitor; or.... the plot was a much bigger obstacle to enjoyment than the SFX, for me. Nevertheless, Gary Oldman, William Macy, and even Ford made it watchable.

        • The theatrical edit showed painted matte plates (obvious - huh? kind of error) between the CG composites when the plane started going down. It was *Very* noticable and distracting. I believe the made for DVD and TV edits fixed it.
    • Seven Mask films? I suppose that the other 5 haven't been released yet? (or even made yet). Also, ILM wasn't responsible for Jar-Jar....you can only thank Lucas for that.

      Also, ILM didn't "make" those movies...and hasn't actually "made" any movies. They're an FX house. They provide a service to someone that pays them. Like any other service industry really.

      But as the other posters said above, don't blame ILM for movies sucking. They did some effects for "Schindler's List" and "Saving Private Ryan"...so shoul
    • Unfortunately, all that storage can't provide decent acting, quality humor or plot lines without holes for their movies.

      This statement is more true than you think. One of my high-school friends who went to work for ILM lamented that, as the most expensive special effects house in the business, they attract particulary the films that have nothing going for them but a high budget. No engaging plot, no spectacular acting, just a dumptruck full of money.

      What they end up with, and why he was so upset, is t

  • Way to go, guys! Who would have known that a small startup from Pittsburgh with some killer engineers, could make it into ILM's datacenter. Hi, Gus!
    • Pittsburgh has CMU, which has a top-tier CS program, a top-tier ECE program, and the ETC (Entertainment Technology Center); what boggles the mind is why there aren't _more_ success stories like spinnaker comming out of Pittsburgh.
      • I think that those people fucking leave almost instantly. There are a couple of small companies and startups, but I think it is more difficult to make it as a tech business here in Pittsburgh, than in places like San Francisco, or Boston -- even though there's a whole hell of a lot more competition. There are a few standouts, and it is easier to stand out here, but people are generally less receptive to new ideas.

        I met lots of CMU people in Boston, though.

  • By my calculation ... (Score:3, Interesting)

    by LaughingCoder (914424) on Thursday March 30 2006, @11:25AM (#15026668)
    According to Mapquest a trip from San Rafeal to San Francisco would take about 35 minutes (Est. Distance: 21.06 miles). Therefore, if I loaded up all 170TB on a truck my effective bandwidth would be about 3.06e28 bps (or roughly 3e16 Tbps). Once again for huge data repositories there is no substitute for shipping physical media.
    • How long would it take you to copy the data for transit (you're not suggesting you transport your master copy, surely), and then restore it?
    • Wow, you must drive really fast dude! 21 miles in 45 femtosecond! What's your secret?
    • What I love about the 'station wagon full of backup tapes hurtling down the highway' analogy is that it never seems to account for the integrity, security, or transfer time of the data to and from the medium itself.
    • You obviously don't know what traffic is like in the SF Bay Area. During rush hour, you'd be working at 801.11b speeds.
    • Try .65 Tbps...

      170TB*8=1360Tb

      1360Tb/(35*60)s = .65 (rounded)

      As another poster said, your travel time would have to be VERY VERY fast to get 3e16 Tbps...

    • Therefore, if I loaded up all 170TB on a truck my effective bandwidth would be about 3.06e28 bps (or roughly 3e16 Tbps). Once again for huge data repositories there is no substitute for shipping physical media.

      Bandwidth is not the issue here - latency is...

      A 70-minute Ping can really ruin your killing spree in Quake ;-)

      • Re:By my calculation ... (Score:4, Funny)

        by LaughingCoder (914424) on Thursday March 30 2006, @12:03PM (#15027042)
        I wasn't suggesting moving the media was necessarily a better way to go -- I was just curious what the bitrate would turn out to be.

        As an aside, the other day in the lab a coworker asked me for an application I had on my thumbdrive. I tossed it across the room to him and then observed that I had just moves 1GB of data in 1 second, wirelessly!

        BTW, I tried to read the article but the site was slashdotted at the time.
  • Okay... not everything they do is shit. That, and CG doesn't make movies worse... only if it sucks. You can go watch claymation if you would like.

    http://www.ilm.com/ilm_services.html [ilm.com]

    Look at all they have done. While some of the stuff on there may have sucked... there is some really fucking good stuff on there.

    Also, if I remember correctly, they were some of the first to experiment with particle renders for CG (they used it in the Mask to create some of the storm/tornado transformations). Anyways...
    • It isn't that fucking good yet, so get off your high horse.

      • Liar (Score:5, Insightful)

        by Kombat (93720) <kombat@kombat.org> on Thursday March 30 2006, @11:50AM (#15026895) Homepage
        I ALWAYS notice CGI.

        No you don't. You think you do, but you don't. When you do notice it, you point it out and say to yourself, "that was so obvious, CGI sucks." But when you don't notice it, you don't realize that what you're looking at is CGI. You think it's real. You think the man really has had his legs amputated ("Forrest Gump") or Arnie really did jump his motorcycle off a 15 foot ramp ("Terminator 2"). CGI is used all over the place in movies now, not just for the big explosions that still may not look 100% convincing (however, it's much better than stop-motion animation).
        • you don't realize that what you're looking at is CGI. You think it's real.

          Stop, man, you're tripping me out... This sounds like Total Recall or something. It's heavy.
        • however, it's much better than stop-motion animation What are you talking about. Gumby kicks ass! In fact, I didn't realize it was fake until last year. Alas, I will never get Gumby's autograph.
        • Umm...the motorcycle jump in Terminator 2 wasn't CGI.

          It wasn't Arnie on the bike, but it wasn't CGI.
          • Someone jumped that bike off a high drop in the T2 scene. That stunt was performed by a trained professional on a real bike.

            With a crane and steel wires that were digitally removed. That was my point.
          • IIRC the jump was real and so was the stuntman, but the bike was being guided/supported by wires that were painted out later on.

    • Re:What?!?! (Score:4, Insightful)

      by east coast (590680) on Thursday March 30 2006, @11:56AM (#15026972)
      CG doesn't make movies worse...

      IMHO, you are wrong. CG can make a movie suck. Once Hollywood understands this maybe good films will not be as uncommon as they are today. Good CG and/or a good story can both do something for the fantasy aspect of things but when you put too much CG in to make up for a lame storyline than CG does suck (*cough* matrix 2 *cough*).

      It really doesn't bother me to watch an episode of (the old) Dr. Who, ST:TOS or Twilight Zone and notice that rocks are made of foam rubber or that a costume is little more than a pie plate and a grocery bag painted green on someone's head when the story is good and these series put out consistantly good stories. No amount of CG can make up for poor acting/writing.
        • Bad CG makes movies suck when you notice it.

          Not to me, but that's just me. I can accept having a budget that can't make everything spiffy or that effects just aren't that important if the writers and actors dedicate themselves to the overall concept.
  • The new ILM DataCenter [jedinet.com] looks alot like the one at Trinity College in Dublin, Ireland.
  • Nice setup you have there ILM. Its a shame if something should happen to it ;)
    • "...and ten petabytes of fiber channel disk."

      "Fiber Channel, Dino."

      "Be a shame if someone was to set fire to them."

      "Set fire to them?!?"

      "Things burn, Colonel."
  • It's an impressive system, for impressive films.

    If it's for impressive films, why are they using it for soulless dreck? Some sort of beta testing period maybe?
  • I've got to echo a lot of the comments here, that sadly ILM and hearing details about ILM, has lost its luster a bit since the old Star Wars days. Not that the acting or the plot was that great in the early Star Wars films either, but there was just a rebelliousness to it, a certain type of moxy, plus a lot of us Gen Xers were very young back then, so maybe it has to do with that.

    Still, there was Jurassic Park, which had that wow effect, but only in a suburban, sterilized kind of way. Maybe it was just t

  • am i the only person who thinks its funny that there is quite a few machinima (game engine rendered movies) that are of better film quality than anything ILM makes on their billion dollar systems?

  • Virtualisation (Score:3, Funny)

    by ettlz (639203) on Thursday March 30 2006, @12:24PM (#15027234) Homepage Journal
    Why have 5000 render nodes when you could virtualise with 250 physical processors running 20 apiece?
  • now Lucas can rape my childhood memories faster than ever.
  • That's San Rafael, not San Rafeal. A sidenote on place names: Skywalker Ranch was located in Lucas Valley, but the valley had its name long before George located there.
  • Imagine a Beowulf cluster of these things.
  • I wonder if they are using NICs that have offloading capabilities in the compute nodes. InfiniBand, Myrinet, and iWarp NICs are all designed to get rid of TCP/IP processing. One would think with relativly large data sets, TCP could be a big CPU consumer. Also, standard NICs using TCP have horrible latency compared to InfinBand, Myrinet, etc. That latency really eats up cluster performance when the nodes all wait for something (like new data, results, etc.) In lots of high performance computing applicat
    • The real question is:

      If 3000 disks are spun up in the forest, and no-one is there to listen, will there still be a nationwide blackout?

    • Re:Nice network (Score:5, Interesting)

      by flaming-opus (8186) on Thursday March 30 2006, @11:55AM (#15026961)
      That's a funny question because I used to work at ILM's (San rafael, much less shiny) lab, benchmarking raids, including the first version of the IBM shark. At that time we came to the conclussion that the IBM raid was reliable, and reasonably fast, but the price was so far out of line, that it wasn't a real contender.

      The shark, and many of the high-end raids, are really designed around transaction oriented applications (databases). ILM's application are classic video codes, which work better on a classic raid5, than they do on the data-sprinkler style raids like the shark, eva, clariion, etc. Netapp makes pretty decent storage boxes, and they're highly configurable, so I'm sure they have them fine tuned to the apps' preffered i/o size.

      Furthermore, the nas/san has more to do with the spinaker software than the raid of choice. Back when I worked there, ILM was testing cluster sollutions, but the renderfarm was a bunch of sgi origins. The storage was hung off of a couple of 8-way irix boxes, and pushed around with NFS. Since then they've upped their compute capacity by a factor of 30, there's no way they'd be able to do all that I/O with NFS to a couple of big servers. The san setup lets them distribute the NFS load to a large number of servers, all sharing access to the storage on a san. A lot of other cluster filesystems allow this too.

      From the benchmarking I've done of these types of storage clusters, you don't get the same single stream performance as you do from a big-iron server setup, but the aggregate across a large number of nodes is pretty good. Managing the mess, and reliability can be problematic. I've never used spinaker, but I've used almost all the other products in this space, and they're all in the "pretty good" category. My current favorite is apple's xsan, because it is really inexpensive, and so is the hardware.
      • Have you head of BlueArk? What do you think about their SAN's ?
        • I've tried bluearc. It works alright, though not as well as the whitepapers say. This is true of most everything, though. What really pissed me off about bluearc is the pre-sales engineers who seemed to have drunk a whole hell of a lot of the company coolaid. The whole story is that the filesystem is "implemented all in hardware", so it's really fast, and that should solve all your problems.

          Well, I've been around the block enough times to know that no filesystem is actually implemented in hardware. They may

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.