Forgot your password?
typodupeerror
Facebook

+ - Facebook's Corona: When Hadoop MapReduce Wasn't Enough->

Submitted by
Nerval's Lobster
Nerval's Lobster writes "Facebook’s engineers face a considerable challenge when it comes to managing the tidal wave of data flowing through the company’s infrastructure. Its data warehouse, which handles over half a petabyte of information each day, has expanded some 2500x in the past four years—and that growth isn’t going to end anytime soon.

Until early 2011, those engineers relied on a MapReduce implementation from Apache Hadoop as the foundation of Facebook’s data infrastructure. Still, despite Hadoop MapReduce’s ability to handle large datasets, Facebook’s scheduling framework (in which a large number of task trackers that handle duties assigned by a job tracker) began to reach its limits. So Facebook’s engineers went to the whiteboard and designed a new scheduling framework named “Corona.”"

Link to Original Source
This discussion was created for logged-in users only, but now has been archived. No new comments can be posted.

Facebook's Corona: When Hadoop MapReduce Wasn't Enough

Comments Filter:

"The greatest warriors are the ones who fight for peace." -- Holly Near

Working...