Data Center Designers In High Demand 140
Hugh Pickens writes "For years, data center designers have toiled in obscurity in the engine rooms of the digital economy, amid the racks of servers and storage devices that power everything from online videos to corporate e-mail systems but now people with the skills to design, build and run a data center that does not endanger the power grid are suddenly in demand. 'The data center energy problem is growing fast, and it has an economic importance that far outweighs the electricity use,' said Jonathan G. Koomey of Stanford University. 'So that explains why these data center people, who haven't gotten a lot of glory in their careers, are in the spotlight now.' The pace of the data center build-up is the result of the surging use of servers, which in the United States rose to 11.8 million in 2007, from 2.6 million a decade earlier. 'For years and years, the attitude was just buy it, install it and don't worry about it,' says Vernon Turner, an analyst for IDC. 'That led to all sorts of inefficiencies. Now, we're paying for that behavior.'" On a related note, an anonymous reader contributes this link to an interesting look at how a data center gets built.
News at 11 (Score:5, Funny)
Re: (Score:2, Funny)
I have proposed a solution! (Score:2, Insightful)
We can send people to college and have them study things like thermodynamics, the flow of air and water in a system, physics, electricity, scale, and perhaps even a little economics. (Things that would be useful for data center design)
And here's the real kicker: They can APPLY what they have learned in classrooms and labs to actual mechanical and electrical systems in a datacenter!
Wow, that was rough sailing for a while there.
Note, ho
Re: (Score:2)
And what would be a "real salary" and how do you arrive at that number?
I don't see what's so hard about it.. (Score:5, Funny)
Re: (Score:3, Funny)
Re: (Score:2)
OK - what do they do? (Score:2, Interesting)
Do they constrain the end user to particular hardware, or is it just basic civil engineering?
I can see that a well planned installation can reduce cooling costs, but if Customer A insists on having his Superdome rather than a more energy efficient alternative, what does the designer do then?
Re:OK - what do they do? (Score:4, Insightful)
I think that is the point here: Data centers have become large enough that you don't want to just stuff them into a random office building and hope everything will work out fine. Specialization is valuable in this case.
Re: (Score:3, Informative)
Re: (Score:2, Interesting)
I had the same several month long arguements in planning our new office. It's expensive. We cannot raise the ceiling (the building HVAC systems are in the plenum.) Do we really need 5ton air handlers. Do we have to have 2 of them. etc. etc. Well, my 12" floor became a 10" floor -- a compromise to make the ramp 2ft shorter, and 2 Lieberts became one because no one listened to
Re: (Score:2)
As for that carpet, a bunch of us showed up on a Saturday and busted our collective asses scraping that shit up. Was glued to slab!
Re:OK - what do they do? (Score:5, Informative)
Those of you who have been in data centers have seen forced air cooling that is not used correctly; cabinets not over vent tiles, vent tiles in the middle of the floor, cabinets over air vent tiles but with a bottom in the cabinet so no air flows.
When equipment is nearing end of life and hardly being used, it sits there and turns electricity into heat while doing nothing. There are often a grand mix of cabinet types that do not all make best use of the cooling system, undersized cooling systems, very dense blade style cabinets replacing cabinets that were not so dense unbalances the heat/cooling process in the whole data center. Not to mention what doing so does to the backup power system when needed.
There are hundreds of 'mistakes' made in data centers all over the country. Correcting them and pushing the efficiency of the data center is a big job that not many people were interested in paying for in years gone by.
If you are interested in what you can do for your small data center, try looking at what APC does, or any cabinet manufacturer. They have lots of glossy marketing materials and websites and stuff. There is plenty of information available. Here's a first link for you http://www.datacenterknowledge.com/archives/apc-index.html [datacenterknowledge.com]
Re: (Score:2)
Servers vent, in their standard setup, front to back. While a vertical system with a hot air plenum is fantastic, its going to require specialized hardware, and since most datacenters are a odd mix of new and old, is never going to happen. Google could afford to do it, but Random Company X or Colo Y can't.
If you're talking about a full cold-aisle containment system, in which a standard cold-air subfloor is much more closely controlled, the
Re:OK - what do they do? (Score:5, Informative)
- Power - redundancy, and resisiliency as much as 'just having enough'
- Cooling - air conditioning is a BIG deal for a data centre - you need good air flow, and it probably doubles your electric bill.
- Specialist equipment - datacentres are _mostly_ modular, based around 19" racks. But there's exceptions, such as stuff that is 'multi-rack' like tape silos.
- Equipment accessibility - you'll need to add and remove servers, and possibly some really quite big and scary bits of big iron - IIRC A Symmetrix is 1.8 tonnes. You'll need a way to get that into a datacentre, which doesn't involve '10 big blokes' - spacing of your racks might not help
- Putting new stuff in - a rack is 42U high. Right at the top of that rack, is going to require overhead lifting.
- Cabling. Servers use a lot of cables. Some are for power, some are for networking, some are for serial terminals. You've got a mix of power cable, copper cables, fiber cables. They need to fit, they need to be possible to manipulate on the fly, and they need to not break the fibers when you lay them. You also need to be aware that a massive bundle of copper cables is not perfectly shielded, so you'll get crosstalk and intereference. And every machine in your datacentre will have 4 or more cables running into it, probably from different sources, so you need to 'deal' with that.
- Operator access - if that server over there blows up, how to I get on the console to fix it. If I am on the console to fix it, how do you ensure I'm not twiddling that red button over there that I shouldn't be.
- Remote/DR facilities - most datacenters have some concept of disaster planning - things as simple as 'farmer joe dug up the cable to the ISP' all the way to 'plane flew into primary data centre'. These things are relatively cheap and easy to deal with on day one, and utter nightmares to retroengineer onto a live data centre.
- Expansion - power needs change, space needs change, technology changes and
... well, demand for servers increases steadily. It's something to be considered that you will, sooner or later, run out of space, or have to swap out assets.
That's what springs to mind off the top of my head. There's probably a few more things. So yes, civil engineering, but with a splattering of IT contraints and difficulties.Re: (Score:2)
Putting new stuff in - a rack is 42U high. Right at the top of that rack, is going to require overhead lifting.
With blades and other new high density things (48 disk trays, anyone), a lift will be required regardless.
But yah, good analysis.
The thing about being a data center designer is that it is multidisciplinary. You have to know a lot about facilities in particular to be anywhere near qualified, and all the computer stuff is there as well on so many levels.
if the engineers in the datacenter (Score:5, Funny)
Doing it for the glory? (Score:5, Funny)
Amen (Score:3, Insightful)
Only now they want people like myself? Screw that. I gave up on that long ago when it was a dead end.
Re: (Score:2)
Re: (Score:2)
'nuff said.
And he built a crooked data center... (Score:2)
Because the power grid has become very fragile (Score:4, Interesting)
Electricity generation hasn't grown ahead of demand due to government meddling, atom-ophobia, and environmentalist obstruction in the courts and on planning boards.
The rolling blackouts will be coming soon. It'll start with small ones. Then everyone will buy battery backups that draw a lot of power to recharge once power is restored. This will cause the duration of the periodic blackouts to go from a few minutes to a few hours in about 2 years.
Not long after that, we'll start building power generation capacity in the US again.
Re:Because the power grid has become very fragile (Score:5, Insightful)
Re: (Score:2, Insightful)
Re: (Score:3, Informative)
Re: (Score:2)
How many letters have you written to your congressman advocating that the government build a coal plant on your block? That's the fuel that America has the most of.
Because it would make less sense to import uranium. And I'd have no problem living next to a nuclear reactor plant, except of course living in Queens (NYC Borough) and they would never out one that close to Manhattan. Assuming I lived in Sachem though, I'd be all for them firing up the plant there. Its a small risk for cheap energy.
Re:Because the power grid has become very fragile (Score:5, Interesting)
I would dare say that the future looks good for ARM and Via on that last account, at least.
Re: (Score:2)
It's like asking a hungry person to cut back on his food intake just a little more to share with the other hungry people in his family or village.
Or we could just generate some more power and have good lives instead of slowly starving ourselves.
Re:Because the power grid has become very fragile (Score:4, Insightful)
Of course, those numbers are all just pulled out of my ass for an example, but still -- you get the point. Cost saving measures at home are also going to lead to energy savings at large. With power prices going up ~30% next month, I think more people will start looking at the alternatives and where they can cut costs.
I agree that things need to be done on the supply side as well, but they should be done in a responsible manner. Building more nuclear plants, for instance. But by reducing consumption, we can then close down fossil plants instead of doing a 1:1 replacement.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Insightful)
No, energy efficiency is like getting a well fed person, who currently throws away 30% of their food because it goes bad before they eat it, to do a bit more planning and a bit less impulse buying so they reduce their wastage to (say) 10% or less and save themselves significant money for a very small effort, and do their bit for the environment as a side effect.
Re: (Score:2)
Why not just make more? It's easy and we know how and it's cheap and people are happy to buy it because they want their lives to be lives of plenty instead of lives of desperate want.
Re: (Score:2)
Why not just make more? It's easy and we know how and it's cheap and people are happy to buy it because they want their lives to be lives of plenty instead of lives of desperate want."
I disagree. I think many people are becoming more energy conscious, but there is still a ton of room to improve. Many people are still very wasteful in their energy consumption (leaving the tv on all hours of the day, not turni
Re: (Score:2)
How do we know when we've sacrificed enough of our wealth and wellbeing to meet your standards?
The blackouts are coming.
Re: (Score:2)
How do we know when we've sacrificed enough of our wealth and wellbeing to meet your standards?"
Turning the lights out when I leave my office or my home does not decrease my supposed wealth. If anything it helps increase my monetary resources as they are not being spent on wasted electricity. Certainly it
Re: (Score:2)
As energy is made more and more artificially scarce, production will move to areas of t
Re: (Score:2)
I'll supply the gas^h^h^hhydr^h^h^h^hethan^h^h^h^h^hbeer.
Green IT (Score:4, Informative)
BOFH style designers? (Score:3, Interesting)
The BOFH cares about important things:Like service:
So, besides electricity usage, what else should you care about? How about heat? Your room can't be too hot (you can send all the heat to the swimming pool in the fitness centre...).
What about wires? Both a OHS issue, and a potential to kill off half your servers if you trip over an exposed power cord or network line. So you lay them under the floor?
Complicated stuff this...
Re: (Score:2)
Your room can't be too hot (you can send all the heat to the swimming pool in the fitness centre...).
I've been wondering about heat storage when we had the article about the new data center in the desert near Las Vegas. If you have a large enough mass (e.g. a couple 100 tons of water) you could store some amount of heat during the day until you can vent it during the night. That would require water cooled heat exchangers everywhere but you would need less power for active cooling.
Virtualization in equally high demand. (Score:4, Insightful)
It's amazing to see the differences VMware has made in my career in just a few short years... going from deploying hardware servers in weeks to a virtual in seconds.
Re: (Score:2)
In our operation, virtual servers would NOT cut the mustard. They are nice for development and QA -- maybe -- but when you are dealing with thousands of simultaneous users, you need real iron, not virtual.
Re: (Score:3, Insightful)
Re:Virtualization in equally high demand. (Score:4, Insightful)
I've seen companies that turned to virtualization to solve their power and cooling problems. Yes, you can serve more OS instances with less hardware. That is good.
However, these places are generally not managing their infrastructure well in the first place. Now you start running into problems with server sprawl and storage management. The management costs are going up because you have more servers running more applications. That takes more management, not less.
I think it is great that we are seeing more specialization in this space. I think that SysAdmins need to look at how they want to specialize moving forward. Are you going to manage hardware & resources? Are you going to be more OS and application tuning? We can't expect one person to have enough breadth to go from HVAC/electric/network/storage/OS/application. I'd hate to see a tape ape get into Data Center design because "hey they're down in the Data Center anyway"
Don't get me wrong, I think virtualization (server & application & desktop) is the wave of the future. But I don't think a lot of firms see this yet. I think they are still trading one problem for another.
Re: (Score:2)
However, these places are generally not managing their infrastructure well in the first place. Now you start running into problems with server sprawl and storage management. The management costs are going up because you have more servers running more applications. That takes more management, not less.
Solving their infrastructure problems would be a good thing for everyone.
But if they don't, It'd also be better for everyone if their increased costs go towards management and not towards increased electricity usage.
I think they are still trading one problem for another.
Labor is a "problem" with a solution that can be ramped up to full speed a lot faster than a new power plant. Not to mention the external costs from labor are preferable to the external costs of more coal fired power.
Re: (Score:2)
Companies with full data centers and in need of more servers are turning to virtualization technologies to increase their server density, reduce their physical server deployment, and improve efficiency in cooling, hardware maintenance, and administration.
It buys you a few years, but that's all. There's massive growth going on in the amount of server capacity needed, and all virtualization can do is to get your utilization up closer to the 95% level (bad idea to go much over that; you need a little space for admin overhead). But once you've done that, you're going to need more real capacity anyway (or the business isn't growing...) At best, use virtualization to buy yourself the time to get your physical server systems in order so you can host more physical
Re: (Score:2)
endanger the power grid? (Score:4, Interesting)
I don't understand the peculiar emphasis the New York Times places on "endangering" the power grid. Even though a data center uses a lot of electricity, it's a high value operation that needs a stable power supply. What's wrong with the idea of paying more to insure that your power supply is sufficiently stable for your needs? The power company accepting those checks can then work on delivering that power. It's like saying that I'm somehow responsible for the stability of the oil production and distribution infrastructure because I drive a car. Perhaps, if I tweak my engine just so, I can engineer a democratic transformation of Saudi Arabia. I'll see if changing the oil does the trick.
At some point, you have to realize that the consumer, no matter how big, isn't responsible for the supply of resources by another party. If there's a problem with how those resources are supplied, be it fixed price (regardless of demand) power transmission lines, pollution, or deforestation, then that problem should appear as an increase in cost to the consumer. If it isn't, then it's a problem with how the resource is distributed, not a problem with the consumer.
Re: (Score:3, Informative)
We are building high power devices on a power grid that was designed 20 years ago before such concepts weren't even thought of.
Environmentalists won't let new plants be built. Solar, Wind are nt yet generating enough to even begin to offset the demand.
it isn't distribution it is availability. by 2015 I expect rolling blackouts during hot summers, simply to keep the air conditioners
Re: (Score:2)
For smaller applications like home, a Radioisotope Thermoelectric Generator might do, they've been used in military applications since the middle
Re: (Score:2)
Re: (Score:2, Interesting)
But yes, it's a priority application too - datacentres score as 'business critical' in most companies, so no matter how much it costs to run, it's cheaper than it 'not running'.
Part of the point of DC design is resiliency, and therefore you _do_ have to consider available services and supp
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:2)
Suggestion for Data Center Design(ers) (Score:2)
That way, the server dude who is wrangling my server issue CAN HEAR ME when he calls me, elbow deep in whirring fans, spinning disks and humming thingees. Even if he listens, gives me a BRB, puts me on hold and dives into the machines.
Or if not feasible, maybe the server dudes could wear priestly robes made of this stuff [physorg.com]
Re:Suggestion for Data Center Design(ers) (Score:4, Funny)
Re: (Score:2)
Blackbox computing - scaling design skills (Score:2, Funny)
Re: (Score:2, Interesting)
Re: (Score:2)
I know of one case where the whole trailer disappeared overnight.
There has always been a shortage (Score:4, Interesting)
It us a shame because it really has a lot of great career opportunities.
Data center work is just a subset of that-- it is hard to find people with the experience, but not impossible to train.
UK Government (Score:2, Funny)
kinda offtopic (Score:4, Funny)
Their UPS was pretty impressive. It was about a 2 thousand square foot room full of what looked like car batteries. I didn't like to go in there, I don't like being around large, uninsulated, potential. (I was electrocuted pretty badly as a kid once)
incremental datacentre design ... (Score:5, Insightful)
While I admit that these datacentres are huge and get a lot of publicity, thus a lot of pressure to design right and "green" I don't think that level of advanced knowledge is typical for SMBs and even most non-IT centric businesses regardless of size.
In practice a company has a few servers and one or two system admins, then they grow, staff leaves, they start thinking about different technologies, required software changes etc. What they end up with is a few vendors servers, a few vendors disk arrays probably a few flavors of networking etc.
In short the "real world" problem for the majority of companies/sys-admins isn't the very academic concept of building a single purpose datacentre, but handling growth and change. I'm yet to see a good reference for how to handle this. At best I see vendors showing how great there new server/rack combination is in isolation, Another popular thing is the ever popular look how low our power needs per FLOP are for a data centre based on our products. Yeah like we are likely to use identical systems for databases as we do for LDAP, and the same one for a fileserver as we use for a MPI cluster.
Anyways, does anyone know a good reference to deal with these "real world" problems?
Re: (Score:3, Interesting)
The biggest secret is in providing enough space to allow for growth, changing needs, and eventually equipment replacement.
As for efficiency, I have to tell an aspiring co-lo that they will
weatherproofing (Score:2)
a) Stoopid computer parts can't just sit outside and work right.
b) Cheap heartland desert acres are beloved by accountants.
No roof == no heat problem. If there's wind. And you're not in the sahara.
If only my data center was water-resistant...
Re: (Score:2)
How many jobs per data center? (Score:5, Insightful)
Data center employment often comes up in discussions of economic development. Many communities are eager to attract data center projects, but struggle to define the economic benefits of these facilities. Jobs have always been the primary benchmark by which economic development projects are measured. Incentive packages offered by state and local governments are often based on the number of full-time jobs created by a new business. And do data centers really hire locally, or do trained data center engineers migrate from other existing data center hubs? In some cases, local officials try to stipulate local hires, which is a sticky wicket.
Re: (Score:2)
Well that might be an issue with skilled labor. Some skillsets cannot simply be trained, for example mechanical and electrical engineers. They require degrees and, in some cases, professional certification. Computer admins might be trained but more likely require experience and some certification. Electricians, HVAC technicians can be found locally but the question is then a matter of skill and availability. Unskille
Re: (Score:2)
Re: (Score:2)
And how the hell do people get degrees and professional certification? Sure, those things tend to have a much longer training period, but it can't be that difficult to find people capable of Doing The Math, which is really to what a huge amount of engineering boils down.
Re: (Score:2)
A new data center is coming to town in 6 months. They need a mechanical engineer. Can a local convenience clerk be trained for that position in that amount of time? No. They need a degree (4+ years). If they are doing any work that involves construction, then they need to be a licensed Professional Engineer to sign off on documents. Professional Engineer License requires up to 4 years of work experience (depends on state) and passing of two technical exams similar to the bar for lawyer or medical boar
Re: (Score:2)
If a new data center is coming to town in 6 months, they better have had the planning of construction and staffing done already.
As far as the design of the construction goes, there's nothing saying that the engineers have to be located in the town. I've designed systems constructed in about 35 different states without moving from my state.
For mechanical operation and maintenance, they might want, but probably don't need, to employ a l
Re: (Score:2)
Re: (Score:3, Insightful)
When Google plants a data center in The Dalles, OR, and MSFT plants one in Moses Lake, WA, I guarantee you that most of the hires aren't local. Initially. They just plain don't exist there.
As far as the local economy goes, though, even if every hire comes in from out of the area, it's likely to be goo
Data center managers in demand, too (Score:2)
Video tour of data center design & constructio (Score:4, Interesting)
Re: (Score:2)
Here's another inside peek, and a spam for my employer:
http://scobleizer.com/2008/03/13/a-real-business-leader/ [scobleizer.com]
It's about Rackspace's new datacenter in San Antonio.
~W
Re: (Score:2)
http://www.fastcompany.tv/video/rackspace-tears-new-headquarters [fastcompany.tv]
~W
SkyNet is assembling itself ... (Score:2)
IT = Volatile (Score:4, Insightful)
The problem is that if you have a family, such volatility can be problematic. Possible solutions are to save during the good times (nearly impossible if you are married), or be a generalist, such as the only IT person at a small company or department. Generalists tend not to be paid well, but they do seem to weather downturns or paradigm changes better. It's a trade-off.
Re: (Score:3, Interesting)
Re: (Score:2)
How much is used for advertising delivery? (Score:2)
What is all this computing infrastructure doing that's useful? Other than advertising delivery?
Use geography to solve the cooling problems... (Score:3, Insightful)
1)We can now pump oil out of the national reserves in Alaska
2)We don't have to work very hard to cool the data centers.
Win Win if you ask me
Re:Use geography to solve the cooling problems... (Score:5, Interesting)
One answer may be heat wheels, but they are fairly new and unproven in the data center space. Take a look at http://www.kyotocooling.com/ [kyotocooling.com]
Re: (Score:2)
Second, just bubble your dry warm air through some cold water. You'll get cold wet air.
Re: (Score:2)
As the low-humidity cool air you've blown in heats up, its relative humidity drops even lower. You may find the humidifying the volume of air present in a 100,000-200,000 square foot data center with a 4 foot raised floor, a 12-15 foot ceiling, and a 4 foot ceiling plenum a little challenging. Espec
Green Computing (Score:2)
Re:I thought it would be higher (Score:4, Insightful)
That's probably part of the demand for the "data center in a box" concept. With a shortage of engineers to design centers it's an obvious move to try and start mass manufacturing them.
Re:I thought it would be higher (Score:5, Insightful)
Now data centers can have lots of change-- as the servers themselves change along with equipment that's located in a data center. The -48vdc telephony equipment is now housed there, along with blade server chassis that breathe fire and suck power like an SUV-- let alone the heat generation problems.
Add in mandates of 5-9's availability (a new concept in the computer industry), earthquake, hurricane/tornado/flooding, power grid availability, the liablities of co-los, legal mandates and constraints, and data center design has become a discipline unto itself. It's not necessarily constrained by good personnel, rather it's constrained by the huge number of changes in the industry overall, and the numerous disciplines needed to bring asset life out of a data center investment.
Re: (Score:2)
Am I the only one who has seen the IBM commercial where they talk about blades fixing their heat problem?
Never understood that one. Blades may be something in the future, but I've found no use for them yet. They sound great in theory, but they do have limitations.
Re:I thought it would be higher (Score:5, Informative)
Re: (Score:2)
Re: (Score:2)
I am the raised floor nazi: no more raised floor for you!
C//
Re: (Score:2)
C//
Re: (Score:2)
Yes, I know cables do not cause heat, but they do block the hot air from escaping. (hint: never use rack "arms")
However, this is a "hot spot" issue, causing higher server temps. You're still sucking the same power, or close.