Ideas For a Great Control Room? 421
lewko writes "Our company is about to build a central monitoring facility and I'm looking for ideas/suggestions about the best hardware and the best way to make it comfortable for those manning a screen. It will be manned 24x7 and operators will be monitoring a variety of systems including security, network, fire, video and more. These will be observed via local multi-monitor workstations and a common videowall. This is going to be a massively expensive exercise and we only get one chance to get it right. The facility is in a secure windowless bunker and staff will generally be in there for many hours at a time. So we have to implement design elements which make it a 'happy' place. At the same time, it has to be ergonomically sound. Lastly, we will be showing it to our clients, so without undoing the above objectives, it would be nice if it was 'cool' (yet functional). Whilst Television doesn't transfer to real life always, think 'CTU' from 24."
Tapes... (Score:3, Funny)
Tape reels, and an ergonomics person (Score:2, Informative)
Re: (Score:3, Funny)
It's endearing how earnestly you try to help the OP. But let me assure you, when an Ask Slashdot appears starting with "My company is undertaking a multi-million dollar project, wat do I do lol," no comment ever is seriously given the remotest consideration.
Fake windows (Score:5, Insightful)
Re: (Score:3, Funny)
Also, make it so that people have to get up from chairs once in a while.
AWFUL! What you want are high-back chairs that support the neck so you don't have to support your body the entire time. Also full recline capabilities help, too. Ideally, they'll never have to get up from their chairs. Think fridge under the table with microwave on top, etc. Also, if they do need to move around for who know why, make it so that they can roll over to their destination. I modified my desk so that I can put my legs forward on a padded leather cushion instead of having them dangle, this
Re: (Score:2)
Clearly you have no friends.
Re: (Score:3, Funny)
Re: (Score:2)
At least my office has a window.
Re: (Score:2)
Re: (Score:2)
Look at people who do it well (Score:4, Informative)
I had a short stint with a fire department in a fairly large metro area. They handle emergency response for a few million people and have a pretty efficient set up for their war room.
It is fairly roomy so that you can get up and walk around.
desks are set up in pods with groups a people sitting facing towards the center of the hexagons they are on the perimeter of.
They have phone communications with headsets that allow them to move around
there are multiple monitors on each desk and large monitors on every wall that can be switched to show any desktop
The lighting and noise were somewhat subdued, but in no way dimly lit or overly comfortable (no high back chairs)
I do not remember the snack food situation, but there is a lot of security and it would be a pain to have to walk in and out all the time)
fwiw, I would save your company a ton of money and just use IP kvms and a software kvm management solution to tie together a bunch of desktops in a relatively open area together where the operators have some room to walk around, but are not overly distracted or lulled to sleep
ya know, massive operations centers are just soooo glass house IS anyway, totally 80's thinkin
Re: (Score:2)
Re: (Score:2)
Sitting for too long a period if time, over time, will kill you - or didn't you read the story here on slashdot.
You want to make it a happy place? Don't have cameras on people all the time. It not only shows a lack of trust, but it's dehumanizing. get off YOUR butt and pay them a visit if you want to be kept in the loop.
What you have described is not conducive to continued alertness. People need variety. What you have described is a "show room", where people will be LESS alert, not more.
Re: (Score:2)
make sure you put in some decent aircon and heaters.
In winter feeling your fingers slowly turn to ice can be horrible but not nearly as bad as having to wipe sweat off your keyboard in summer.
and put the dial where the people in the room can actually change it without going through 3 layers of bitter old building maintenance staff who like to watch the office staff sweat.
I cannot stress that one enough.
Also
Put a bathroom less than 10 minutes walk away.
make sure it's clean and remains stocked with plenty of
Re: (Score:2)
Another small one.
A few fans can make things far more pleasant even if the temperature is right.
When you're a bit tired a breeze in the face can wake you up a bit and make you feel more alert.
Cold water or ice dispensers are good as well as the coffee.
Re: (Score:2)
Put a bathroom less than 10 minutes walk away.
I would hope it would be closer than that.
Re: (Score:3, Funny)
Bathroom is within the bunker and very close.
Toilet paper is a great idea. Thank you.
Management want the Three Seashells...
Good lighting (Score:5, Insightful)
Whatever you do, don't make it dark with blue-ish lighting, like on TV. That strains your eyes. Provide good lighting, and make sure the persons can sit or stand comfortably while watching the screens.
Re: (Score:2)
Re: (Score:2)
Good point. Make the lighted area well lit, but the big screens on the wall, make them 12 - 15 feet away and the area they are in is not lighted (to reduce glare, improve contrast, reduce eye strain)
Problem with lighting too well is it supposedly causes eye strain. A darkened room is supposedly better for long term LCD/CRT viewing, hence you will find that most large, professional sit rooms like this have lower lighting.
The better alternative is to use zone/pool lighting to keep eye strain down, while providing areas of brightness more appropriate to other activities.
Re: (Score:2)
I work with an employer who has done the right thing when it comes to atmosphere. The buildings allow enormous amounts of natural lights in through a combination of a wall-of-glass as windows and a sun light in the middle, where there are even more windows (imagine a building with a building inside it that has glass as a ceiling), not to mention good quality lighting that is not overbearing. Plus the ability to shade the windows on the exterior in cases of too-bright sunlight. I'll grant that we need bet
Re: (Score:2)
Oh yeah and we have an on-site, full service gym. So got that covered too :)
Re: (Score:2, Insightful)
I do complain about the flicker from PAL/NTSC TV. It can't watch PAL TV at all. It's like staring into a strobe for me. This could be because I only have peripheral vision to work with. LCDs are the best thing that ever happened to me.
Secure windowless bunker (Score:5, Funny)
Plenty of ideas here:
http://www.villainsource.com/lairs.html [villainsource.com]
Re: (Score:2)
You could just post what we are all really looking for and thinking....
Hollowed Out Magma Lair with retractable Mojo Relaxation Center (mood lighting included).
The Red Button (Score:5, Funny)
Re:The Red Button (Score:5, Funny)
And when someone presses it, make another sign ligth up saying "Please do not press this button again."
Re: (Score:2)
No, no, no. When you push the button, the sensei has to come out and kick you in the nuts.
Re: (Score:3, Interesting)
Natural light (Score:5, Interesting)
Well, it sort of goes against 'being in a bunker', but if I was going to work somewhere for many hours, I'd like some natural light.
Of course it's still possible to achieve that using reflective tubing or such like, though it might still undo whatever it is you seek to achieve by being underground.
If it's not possible, I'd suggest paying lots of attention to lighting. And add some real plants too - they'll generate oxygen as well as making the environment seem less bunker like.
Re: (Score:2)
Re: (Score:2)
What kind of nature do you come from anyway?
Re: (Score:3, Interesting)
Fiber Optics (Score:2)
Use fiber optics to transmit the light. There are some sky scrapers that do this (and use it to grow plants too) to help decrease power consumption. The only ones I know of are in Japan but it may be worth looking into.
Re: (Score:2)
Well, I was working on the assumption that any such place would have corridors, somewhere to eat etc.
It's also quite possible to focus grow lights on a plant without spreading light everywhere else.
Someone else commented on the issue of natural light hitting computer screens. That too sounds like a design problem that allows unwanted light to hit those screens.
These are issues that are hard to tackle in a retrofit. They should be an integral design aspect of a new build property.
Some things you'll want to do (Score:3, Insightful)
Borrow some ideas from the utility control rooms I've been in. Everyone has and uses their own headsets, I might extend that to keyboards. Keeps people from passing the contagious thing of the week around a confined space when sharing monitoring stations.
Re: (Score:2)
I would recommend having a 100% outside air economizer, distance viewing elements to reduce eye strain, and time-of-day lighting to keep body clock normal-- brightest at noon, can even create asubtle artificial east-west fade with wall washers.
Re: (Score:2)
Borrow some ideas from the utility control rooms I've been in.
Well now *that's* helpful.... ;)
Control rooms at CERN (Score:5, Informative)
Re: (Score:2)
USG Contractor? (Score:4, Insightful)
Re: (Score:2)
So you're saying, make it look like WarGames?
Re: (Score:2)
Make it Functional, But install a Cool-Mode Button (Score:5, Funny)
Numerous commentors here point out the error of using cool colored dim lighting for a facility where actual work gets done. But you can have your cake (or Cake, if you are using cool indie rock background music) and eat it too - just have a button that switches the lighting to "cool mode" whenever a visitor comes in. Meaningless but cool looking graphic "screen savers" could also pop up on the screens.
Re: (Score:2)
No matter what you do, the people working there are going to be spending at least a little time looking at things that aren't work related, especially on breaks. Make sure there's a "boss key" that flips things back to something that at least looks work related, so they can look busy when visitors come in. Yes, that includes their own managers, but so what? Presumably there's a way to monitor what they're doing, and i
Environment factors (Score:4, Insightful)
Do ensure that it has nearby and adequate washroom facilities. Nothing worse than having to travel up a couple flights of stairs every time personnel has to take care of nature, know what I mean?
Adequate lighting, and ventilation / heating / air conditioning is also something to consider. Nothing worse then working in winter with cold fingers, let me tell you.
I've been NOC for just shy of 3 years now, and I can tell you the environment you work in plays a huge role in how comfortably you handle the workflow. Its nice to focus on the more technical bits such as equipment and infrastructure, monitors, etc, but do not forget that people have to comfortably be there for hours at a time. We do 12 hour shifts here, and the most important consideration would be the temperature and air quality, imho.
Re:Environment factors (Score:5, Informative)
I worked for a couple of months in doing NOC work, you pick up some basic must-haves really quickly in regards to comfort, I'd like to add them. Temperature and air quality is among the very top priorities for sure, here are a few other points.
Get the best and most comfortable chairs you can find and then get the ones that are even better. People will be sitting in these 24/7, you need the best you can get or they'll get worn down in mere months, both chairs and people. Full range of adjustability is essential, as is full back and lumbar support. Look at the seats in taxis, cop cars, any sort of car where people do long shifts, look at long-haul trucks if you can. Taxis where I live are almost uniformly Mercedes-Benz or Volvo, partly because of their excellent seat ergonomics.
Get people out of the chairs. I know this sounds silly considering you've just bought amazing chairs, but give people options. Exercise balls, kneeling chairs etc., let people mix it up so they aren't stuck in an ordinary sitting position for hours on end. Some basic exercise equipment is good too, doesn't have to be anything more fancy than some wall-mounted bars etc., just to let people stretch a bit.
Quick access to toilets and kitchen(ette). This type of work demands as little downtime as possible. Make sure the kitchen has at least two microwave ovens, depending on the number of people who'll be working there, a good meal is essential to getting through a night shift and even the slight inconvenience of having to wait a bit longer can be amplified by being sleepy and in a job where you deal with very stressful situations on a daily basis. Night shifts have been shown to have a correlation with heart disease and possibly cancer. Don't make people wait too for their food as well :-) The place where I worked had lots of people doing 9-5 jobs as well and the cafeteria always kept well-stocked vending machines with sandwiches and the hot dish of the day for the night shifters. It was highly appreciated.
Speaking of the kitchen, get really good coffee. No scratch that, get great coffee and reliable high-capacity coffee makers. These people will suck down black coffee like you wouldn't believe. Keep the fridge(s) well-stocked with ice-cold Coke (or whatever caffeinated soft drink they like) for the non-coffee drinkers.
I cannot stress this enough: Have fresh fruit available at all times. This was an absolutely life saver for me. You haven't had a tasty apple until you've bitten into a fresh, ripe Golden Delicious at 4:30 in the morning after a long nights stressful work. Keeps people from gorging on chips and other unhealthy snack foods as well.
Lastly, let people have their distractions. We used fancy multi-monitor setups where we could put just about any system we monitored on whichever monitor we wanted, very slick. I can't remember what they called it, but it seemed to work pretty well. My point is that the universal setup seemed to be one monitor for desktop stuff, one for TV (most people had 24/7 news on) and the rest for monitoring etc.. The TV feed was brilliant, during the downtime you could catch up on the news and so on, it helped you get through the boring parts of a night shift.
I'm sure I've forgotten a lot of stuff, but these things stood out in my mind as the most important.
Re: (Score:3, Interesting)
To add to your kitchen comments make sure that all the outlets are at least waist high. Also (because the electrician won't care) I would put at the maximum two outlets per circuit and make sure your microwaves are on separate circuits. If possible have your kitchen circuit box separate from anything else. You won't believe the things people will want to buy for the kitchen:
Ice Tea Maker
Popcorn Popper
Crock pots
Their "special" espresso maker
Food processor
Juicer
Deep fryer
Hot plates
Toaster ovens
I
Get help from pros (Score:5, Funny)
Re: (Score:2)
Indeed.
Most oil companies for instance spend quite a lot of money to do it right. And they rarely if ever do it themselves. They come up with requirements and work with a company that does control room design. Having spent some time in a brand new control room for an offshore installation in Norway I can say that all the little details matter. Do not try to do it yourself. Get someone with experience. It is worth the money.
Re: (Score:2)
One was in a fertilizer works made from assorted imported bits of plants of various ages that leaked dangerous amounts of ammonia at times. It had three impressive walls of lights and annoying buzzers, and it would light up like a christmas tree when things were a bit high or low but well within operating parameters, but then wouldn't do much different when things were going seriously wrong.
Another was in a power station with sc
Re: (Score:2)
Those sound more like seriously bad alarm handling than control room design.
Almost everyone group it together, but they are distinct if you think about it.
Re: (Score:2)
Are you kidding?!
Most of us have graduated from forts of blankets and chairs and couch cushions to full on underground lairs (inourparentsbasements) where we've resided in for months at a time.
C'mon, who knows better than us?
Re:Get help from pros (Score:5, Funny)
I am the OP.
We asked a specialist company. They charged $5k and said "ask Slashdot".
Careful What You Wish For (Score:5, Interesting)
My company invested millions of dollars into a central monitoring facility, with a large video wall driven by Crestron equipment. The idea was the video wall could display news/weather alongside alarms/outages in real time, with geographic mapping capabilities. Workstations were quad displays on adjustable motorized desks which sat atop a raised platform for simple network runs. A large executive "war room" style conference room was built with a glass wall overlooking the platform and video wall believed to be useful in the event of some catastrophic failure. All other staff sat in cubicles surrounding the platform with glass cube walls anywhere that would otherwise obstruct the view of the platform/video wall. A secure mantrap was put in place to restrict access to the facility. Dedicated bathrooms were installed with showers in the monitoring area in case critical staff were quarantined for extended periods of time.
It was impressive when it was built, but within a couple years, the video wall has been dismantled and parts sold off due to its impracticality. The right software was never found to perform the type of "geographic" monitoring conceived, partly due to bureaucracy. Network redundancy was overlooked, which made the monitoring facility itself non-functional during an outage. The facility lacked appropriate backup generators and UPS to keep the facility running during a thunderstorm. The platform desks required too much real estate and allowed no room for growth, so they have been replaced by cubicles. The secure mantrap was an inconvenience for upper management, so the inner door was disabled, defeating the mantrap. The quad displays ironically obstructed the view of the video wall when it was still in place, and did not fit in the cubicles when they were installed, so these were reduced to 2. All critical staff were sent home to telecommute because they took up too much real-estate required for day-to-day operations, and it made more sense to not have critical staff in a single central location anyway.
The point is, don't get too caught up in building 'CTU' from 24. The right monitoring software platform makes all the difference, as does intelligent network redundancy, telephony and backup power.
Re: (Score:3, Insightful)
Some experience (Score:2, Informative)
See http://www.tbcconsoles.com for a few ideas.
Chairs - Haven't found anything to beat the Herman Miller Aeron yet.
If you're placing PC's in the control room, beware of heating and cooling requirements. Most common mistake I've seen.
Second most common mistake, way over-spec'ing the AV system for the videowall. Keep it modest, in keeping with your actual requirements and conop, with room to grow.
I like the single tier monitor approach. Get monitors above other monitors makes for an ergonomically uncomfort
Its going to all be for show. (Score:4, Informative)
Lastly, we will be showing it to our clients
In other words its going to all be for show.
Big projected screens showing something management finds important to brag about but the employees will never glance at.
All individuality quashed, no pics of the family etc.
My advice, in all honesty, is to build two. One that actually works, and one that is a star trek mock up. Whenever they did marketing picture shows they hired college age models to "staff" our network management center anyway, so non-operational equipment is not exactly a problem for the models to pose with.
Re: (Score:2)
"My advice, in all honesty, is to build two. One that actually works, and one that is a star trek mock up."
That's not even your advice but one I remember seriously pushed within the Rainbow Series (you know, the Orange Book et al). It was not for a NOC but for a datacenter, but it makes no difference: no matter your security policies, your boss is your boss and he will want to show his power to some important visitors. So use your decomissioned equipment to build a mock up; incidentally since computers ar
difficult choice (Score:2)
windowless bunker or happy place. choose one.
Plan for fast depreciation (Score:5, Interesting)
I used to know someone who had worked for an alarm monitoring company. She said the chairs and other furniture were used 24 hours a day, yet the accountants were depreciating them like ordinary office furniture. As a result, the furniture was not replaced often enough and was falling apart and uncomfortable. Make sure to plan on fast depreciation for your furniture.
steveha
Much harder than it looks in the movies (Score:5, Insightful)
So, a couple things to think about:
1. How much money do you really have? Lots of people think they have money, but run out when it comes to all the details.
2. Do you want flash, or functionality? The two are sometimes complementary, but often one trumps the other.
3. How many people will staff? What's the schedule? This helps you figure out workstation configurations.
4. Are you putting multiple tiers in the same room? This is "best practice" if you do it right.
5. Are you handling customer calls directly? Do you deal with customers?
Basically, you need to figure out a lot of goals. A true "global" NOC can cost $50M easily for a telecom or comparable organization.
I've been a big fan of Barco for large projectors, and their IP-based solution is quite powerful. Recently, I rolled out a "public safety" SOC (security operations center) with 8 SXGA+ rear-projection displays. The largest I've worked on was 40+ of that style of display. Your garden-variety projector isn't cut out to handle this kind of duty-cycle. They're not cheap, but they're designed to operate 24x7x365, and many models have multiple lamps, etc. so that you can service them while they're online. So here's a few more things to think about:
* What goes on the "big screen" has to be useful. It must be grokable in a very short period of time. If you can't look at it for 2 seconds and get a good idea of what's going on, it's too complicated.
* Multiple displays per operations person
* Operational "graphs" that show overall statistics that matter to the people working, not to management.
* Good task lighting. Good lighting period is everything. Pay a real designer to do this.
* Good seating. We have let operations people pick chairs that fit their needs. Expect to spend $800-1k/person on seating.
* Sound deadening/management. NOCs get loud, and managing the acoustics is important to make sure that people can "think" and they can interact with one another.
* Ticketing is everything. Look at systems that are available commercially and for free. Consider writing your own if needed. If the system is streamlined to your own business, it will always be an impediment to getting the job done, which means people won't use it. If they don't use it, lots of knowledge is lost and post-mortems are more difficult.
Also, a few things that seem superfluous, but ended up being critical in some places I've worked (not all these were at the same place):
* Virtualized desktops (think RDP, X11, etc.) so that people can move and maintain their setup :)
* Color-shifting lighting to compensate for normal rhythms of people on weird shifts. Turns out green is effective after lunch at helping people maintain focus. This isn't cheap, but it sure does have a big impact.
* Keep your customers OUT OF THE NOC. A glass wall into the NOC is fine, but actually letting them in is distracting, and depending, can come with legal issues around privacy, HIPAA, etc. Best to keep them at a distance.
* Before you let customers see the NOC, you warn people. We had a blinking lighting strip under the displays that was linked into the Crestron system so that you couldn't flip the LCD-glass for 10 seconds to give NOC operators a warning. You don't want customers seeing people picking their nose.
Finally, as nice as good facilities are, if you don't have the process and people, it's useless. People people people people. Good people create good processes. Promote from within, and develop a strategy to give people a career path. Otherwise, you'll burn people out, and get huge turnover. That sucks for everyone.
Re:Much harder than it looks in the movies (Score:4, Interesting)
Oh a few more thoughts:
1. Buy high end task furniture (Haworth, Herman-Miller, etc.) but buy it "used". It's 1/2 or less the price, and often you can get the used high-end stuff for less than commodity new.
2. Get a telephone system that doesn't suck. This is harder than you might think. Today, I'd build something with Asterix/VOIP integrated with a customer database to do some real-time CTI. In the past, I've used Aspect successfully as well. Cisco's VOIP gear is nice, but overpriced.
3. Everyone gets their own . Whether it be a headset, keyboard, etc. Trust me, it makes sense.
4. Lockers outside the NOC for staff. Make them nice, tall and big, and nobody shares.
5. Plan for actual breaks from operations. Nobody can stare at a computer screen that many hours and stay alert.
There's a million more details.
Re: (Score:3, Insightful)
4. Lockers outside the NOC for staff. Make them nice, tall and big, and nobody shares.
A locker-room with showers, connected to a workout room with treadmills and stuff, is almost a requirement for shift work. Nothing eliminates the "mid shift snoozies" like 5 minutes on the treadie. Or a semi-serious workout in place of lunch.
My advice is no webcams, maybe even no security cams. Guys don't care, but the women found it extremely creepy that some dirt bag was watching them stretch and bounce around, so they either complained or refused to use the facility. Now a glass wall, where they can
Re: (Score:3, Informative)
4. Are you putting multiple tiers in the same room? This is "best practice" if you do it right.
Horrible idea. I encourage all my competitors to implement that idea as fast as possible.
You ever actually work in a NOC? I have, for decades.
You ever watched "Office Space" with the scene where the guy is utterly unable to concentrate because his cube is next to the receptionists cube "Welcome to innitech, please hold ... Welcome to innitech, please hold .. Welcome to innitech, please hold". Horrible awful idea. Good if you are my competitor, or if you intentionally are trying to destroy your own produ
Not my area (Score:3, Insightful)
But my first somewhat obvious thought would be to build a very detailed list of what has to happen in the room, then use that to drive your design.
Positioning of people I would also imagine is quite important.. which groups of users need to communicate with each other.. who will be using the video wall.. who is going to be making the most noise (is someone going to be on the phone every 10 minutes.. if so a separate sound proof cage might be in order)
Things like white boards might also be a good idea. For all the high tech collaboration solutions out there, I've found nothing beats a whiteboard for figuring something out or just tracking status of a short term issue.
I'd also watch the cool factor stuff. A lot of the stuff that looks really neat on TV actually sucks in real life. Moody blue lighting for instance is depressing and hard on the eyes. Maybe you could have some kind of "holywood mode" switch or something for when people are being toured through.. though that is a little extreme.
Finally I'd say good quality monitors and the most comfortable chairs that the budget allows.
Great Hazards (Score:3, Insightful)
In such a situation the greatest hazard will probably be from your own company executives. Establish a great monitoring process and it is carved in granite that some executive will add tasks that take attention away from the primary task. Reports leap to mind.
The second hazard will be from employees that man the monitoring station. They can become disgruntled or even be paid to do foul deeds. Good encrypted backups kept off site may help as will a monitor that watches the people that man the station.
It may help if you disallow electronic gizmos of all types from being brought to work in that monitoring station. Also tools that could open a computer case and install a USB card make it clear that you need to have absolute control of all items brought into the room.
Beyond those factors have you considered Faraday shielding?
Treadmill Desks (Score:3, Insightful)
I'm serious. If people are going to be there for long periods of time, they should get a little exercise.
I'm not talking about running, just a leisurely ~1mph walk.
Standing and walking are probably the absolute best options for maintaining good ergonomics.
The office furniture company Steelcase makes one. [steelcase.com]
I'm not saying to ban chairs, keep them as backup and for people who physically can't stand for extended periods of time.
Stuff you need (Score:2)
Sod! (Score:2)
Multiple Computers and Synnergy for Videowall (Score:4, Insightful)
You can purchase some really high end equipment to manage multiple monitors on a videowall, but you shouldn't. Use standard PC level hardware (or lower end rackmount depending on space requirements) with no more than two display cards each. Drive all your monitors separately then tie them together with Synnergy. You can still administer them all from a single workstation, fairly seamlessly, but you don't have a single point of failure, and you've probably saved hundreds of dollars. The videowall systems can also run some light duty servers especially system monitoring. (I like Xymon over Nagios, but it depends on what you want to do with it.)
So far as the monitors themselves, purchase flat-panel HDTV's. They are likely to be cheaper than similarly sized monitors, and you won't want greater resolution than an HDTV can handle for a video wall anyway. This gives you the added benefit of being able to tie in training videos, or third shift entertainment on to one or more screens if needed. Also, if one of your videowall servers goes down right before clients come to view the installation, you can quickly switch those monitors over to CNN, CNBC or another relevant channel.
The workstation tables should be glass or some other surface that can support either dry erase or grease-pen writing. Being able do simple notes on your desk will reduce scratch paper usage and make maximum use of available areas. Glass cubicle walls will cut down on noise like a cubicle would, but does not give as much of the feeling of being in a box as standard cubicles. They allow unobstructed view of the video-wall and you can write on them with grease pens.
Have more workstations than you need, and do not tie people their workstations. If someone wants to claim one that is fine, but some people will really like being able to log off, walk across the room, and log back on. This will also allow you to bring in off-shift workers when shit hits the fan.
As a security measure, get a dot-matrix printer on your firewall. Feed tail -f /var/log/authlog directly to it. If anyone gets in that shouldn't they will NOT be able to erase their tracks.
Put in a breakroom or break area that still has a view of the common videowall. When your people are taking a break during downtime, they should still be able to see if it is suddenly no longer downtime.
For the love of God (and your staff) put in a drink fridge or soda fountain and a coffee pot.
Don't fuck this up. (Score:5, Funny)
ESPN (Score:2)
Don't forget the full ESPN package. In HD, games look awesome on a 20 x 35 - foot screen.
Just to be on the safe side, make a big screen shot of an all-nodes-green Net Manager layout or whatever you use, and keep an image viewer running with that image handy. (I had a previous employer that actually did put up a screen shot of all-OK Net View or whatever, for VIP visits.)
get a Directv box for that as you can use a RF rem (Score:2)
get a Directv box for that as you can use a RF remote to control the box with out line of site.
Bathrooms, Janitorial Services & Housekeeping (Score:5, Insightful)
24x7 ops in a confined space that you want to actually work inside means caring about the two things overlooked by both IT and Management.
1) Human waste
Unless able/willing to get time outside the tank, people _are_ going to: snack,drink coffee & water, etc at their stations.
The bathroom (mens and womens) is going where? Outside the double tier biometric locked doors past the guard?
In a crisis or crunch the time use for bathrooms actually goes up as more coffee and crap food (fatty, sugary, glutten, etc) are ingested & a trip to the sewage system is required.
Also bathroom facilities need to be built with those things necessary for those with disabilities; with obestity / diabetes and those with really bad digestion (IE get a really good odour neutralizaing vent system) Also some sound damping is necessary. After a shift change would you want to hear 14 flushes in a row at your station?
If you want a real good idea of how human smells take over a confined space take Greyhound to anywhere on a trip longer than 8 hours.
After a few hours you'll be begging for fresh air & a decent restroom as well. You can figure out the max time people can hold it as would need to if they pass outside the security areas of your ops center.
2) Housekeeping & Janitorial
How are and just who will clean the bathrooms as well as the control room area?
The guys & gals making so little as opposed to the IT guys, but do the really important grunt work during the night to porter the bathroom with their cleaning carts and supplies.
Really, although the janitors have a key to everywhere they typically will block doors open. The cleaning team is so "trusted" it isn't questioned about bypass of security doors.
If Solid Snake could hide in a janitors cart not a cardboard box - he'd go anywhere.
Additionally these are also the people that are going to empty the waste bins, recycle bins, spritz down empty cubes/stations with disinfectant / de oderizers.
My former company had a policy: if techs are on a station that station will not be cleaned.
You have to have some sort of desk rotation to move out your personnel (that monitor now covered by a different station) so the area can be cleaned adequately.
If this is not done you deserve the thick stank that will descend upon your control room.
If your running really critical Ops: A HR policy on proper hygiene and showers should also a clause in the personnel contract. Everyone laughs until you have to term a tech for stinking to much.
As a bonus I'll add that my experience is that the AC design will never be adequate nor will the heating. It will be visited often by AC techs in the first 5 years until everyone gives up hope.
The failure is that AC is typically the retail mall design of a large scale dumping of cold dry air into a large volume of space and somebody's desk (hopefully not yours) is just under where this happens. Hot and cold spots are intractable in a large open floor plan arrangement.
Re: (Score:3, Interesting)
2) Housekeeping & Janitorial
Don't only think what, but also think how. There's nothing more fun than trying to solve a critical time sensitive problem to the background music of a noisy vacuum cleaner. The control room at our refinery has a novel remote compressor with some air ducting that runs under the floor to vacuum ports at the wall. Rather than drag a vacuum around the cleaner will come in with essentially the tube and handle, and plug that tube straight into the wall.
The end result is the only part of the vacuum cleaner y
Re: (Score:3, Interesting)
If Solid Snake could hide in a janitors cart not a cardboard box - he'd go anywhere.
In my unfortunately extensive experience with this over the decades, the problem is not so much James Bond sneaking in, its "trash bags" full of laptops and HDTVs sneaking out. Even in the "best cities". Everythings gotta be locked down, even/especially in a security theater environment. And all employees need at least one lockable drawer for their purse, cellphone, spare change, insulin syringes, whatever.
You really need to understand what security theater is before you plan your "cool security system".
Re: (Score:3, Interesting)
In my experience employers that directly or indirectly turn a blind eye, ie who hire against national/local labor policies, already have deep problems anyhow. I wouldn't work nor recommend anyone work for such businesses.
For legit businesses, most housekeeping staff I've known are decent hard working folks often looked down upon by the Management and IT guys if even recognized at all. Most won't steal nor want to be implicated because it costs them their job or the contract instantly. No investigation perio
Be practical (Score:2)
Look, you need to be practical here. There's a good reason control rooms like the one in 24 don't exist in the real world: they don't work. Forget the video wall and just make sure that your employees have access to what they need on their own workstations. You want this thing to be functional in a crisis situation, right? Then that means that the facility may need to run on backup power, and a bunch of huge monitors are going to use a lot of that power. Not to mention that that expensive video wall i
Abnormal Situation Management (Score:3, Insightful)
The chemical processing industry have been working together for at least 15 years on related topics.
http://www.asmconsortium.net/deployment/guidelines/Pages/default.aspx [asmconsortium.net]
Might be something in there that help.
Specifically display standards and alarm rationalisation ("every alarm should have a unique action", if there is nothing the operator on shift can do about an alarm it should be journaled).
ZombieEngineer
Talk to the operators (Score:2)
My control room experience at fermilab (Score:3, Informative)
These are just my experience. I've worked in SSN maneuvering (reactor plant control) rooms, and at Fermilab, the Experimental Areas, D0, CDF control rooms and now days, the Main Control Room.
Keep the racks sealed, front and back as much as possible, to keep dust in the room from getting into the electronics. You might (or not) be amazed at how much dust a control room can collect over a year.
Include a fair amount of sound deadening foam. again, it's remarkable how loud pcs can get, way moreso if you have to add fan cooled crates and such.
Stay away from trackballs unless you can get *really* *awesome* trackballs, they tend to collect crap inside and are usu. kinda a PITA to clean.
Beware of the temptation to put in q00l tracklights, as they have a tendency to cause a lot of glare. Keep bookshelvs away from the consoles as much as possible.
Have your operators face AWAY from the hallway. Consider keycard access to keep distractions down. Spend the money on comfy high-backed chairs. Kitchen immediately adjacent to the control room. Bathroom nearby. Consoles should go UP, not out; it's easier to look up than turn your head and it _will_ make a difference.
I would not curve your workstations; skooching from one point to the next in a curved layout requires a unique trajectory for each endpoint; straight layouts are easier to run your chair along.
Here's mine http://www-visualmedia.fnal.gov/VMS_Site/gallery/stillphotos/2006/0000/06-0022-30D.hr.jpg [fnal.gov] kinda big, and i'm in the picture! Looks old and clunky but we manage.
HTH
DOCUMENTATION. (Score:5, Informative)
What hardware you have doesn't matter much. Any run of the mill dual headed desktop will do what I need - a few browser windows, a dozen xterms, and some email up in the corner. Anything else is just for show.
What's really valuable when I'm on point is documentation. I need to know:
What does this server do?
What's the procedure to rebuild it?
Who built it last time, in case I hit trouble?
Who's the business owner?
How do I reach them? A cell number would help.
What's the escalation path when they're AWOL?
Where are the contact numbers for our transit providers?
What are their SLAs?
Where's a map of our network, in case I have to creatively reroute traffic in ways that OSPF won't?
Is it up to date?
What services are exposed in the DMZ?
Where's the ticket requesting this port be opened?
Are there supposed to be 100,000 different IPs connecting here, or just three?
Where is the password vault?
What's the procedure to update the password vault if I have to change one?
Being able to find these things quickly will make me a much happier sysadmin than any creature comforts, excluding caffeine.
If you want to get into creature comforts in a windowless bunker, make it lighting. I don't want it bright, but it should be well designed to cover the space well. Good warm triphosphor fluorescents with high frequency electronic ballasts are much much nicer than the old cold ones with 120Hz flicker from magnetic ballasts. Color rendering index matters. That makes a good base lighting for the workspace. Then get a few of those industrial HID grow lamps, and have them light a big picture of a forest scene covering an entire wall. Or actually grow plants under them, and don't pay too close attention to what else people plant when you're not looking. Careful not to make it too bright, but the sun-like spectrum will break up the monotony of the fluorescents. Add some bright halogen task lights for when you need to see something well, without having to flood the whole room.
Raise the ceiling as high as you can. Rip out the ceiling tiles. Suspend the lights on cables. Let the ductwork show. Paint it all black. I hate living in a box. Exposing all the HVAC and cabling breaks it up, and I actually like the minimalist industrial aesthetic. If you want a softer look, hang some tapestries up amongst the machinery.
Re: (Score:2)
Good coffee machine. (Score:2)
And a sofa.
Needs a better name (Score:2)
Call it "the NOC". People go 'Whoah'.
This (Score:2)
Why would I tell you for free... (Score:2)
I've worked in a control room before (Score:2)
Noise (Score:3, Interesting)
Having worked in several such control rooms as a railway dispatcher, what made me mad was that the morons installed the computers under the desks (dozens, also every person working there had 6 monitors) with lots of ventilators and cold air coming from the AC in the room below. They could just as easily have put the computers in the room below, since they were in enclosed, locked metal boxes for security reasons, that way all the noise and cold air under the desk would have been avoided.
OTOH the noise-cancellation walls, floors and ceilings were OK.
If many different people sitting in the same seats, with varying weights and habits, the (expensive) seats were usually broken after 3-4 months.
One other thing, if they run a blocked, single app all the time, lefthanders need still a way to change the R/L mouse buttons, we didn't have that, it sucked for lefties.
Also, since the machines were totally blocked from accessing them, even changing a keyboard or mouse needed a certified tech, who had to be called from home during the night and holidays, that really sucked, how hard would it be to place connectors on the desk.
Great control room setup (Score:3, Interesting)
I used to work for a large insurance company in Chicago. The director charged with building our NOC in 2000 basically traveled throughout the country visiting other large IT organization's NOC's and took the best ideas and made them work for us - and it did resemble 24.
Take a large crescent shaped room with a 30' or more ceiling. The video wall was three different sections (this is important for separation of displays and multiple tools at the same time). The display units were high end rear projection systems that were each hooked up to computers that drove the display and were roughly 3'x5' each. Of course there's no seam or separation between the screens. Any group of screens can be used to display anything you want (1 screen, 2, 4, 6, all, etc). Pretty basic stuff nowadays, but it was great ten years ago. The left and right banks had three screens stacked on top of eachother, by either 4 or 5 wide. The center bank was 3 high by either 8 or 10 wide.
Three rows of crescent tables with low walls in front separating them, and minimal separation between workspaces - you want people in a NOC to work very closely with eachother, especially in case of an outage. Each station had two or three LCD screens mounted on articulating arms, but not to be stacked on top of eachother like those trading desks you see with 6 or 8 LCD screens at them. That would be too tall, and you couldn't look over the top to see the main video wall without standing. The room sat close to 50 people. Around the edges of the room are various cabinets, printers, personal storage for the three shifts of employees that work in the NOC, etc. Of course high end chairs are important as others have noted. Lighting is also equally important. You have to be very careful with making sure it is as close to natural lighting as possible. The lighting we used was recessed and inset so that no lightbulb shone directly out or down on the people - it made it less harsh, but still very bright in the room based on a good design. Wireless headsets are important, and also minimizing speakerphones and any other distracting noise.
Behind the rows of tables at the back of the crescent in the donut hole section if you will is an enclosed room large enough to sit 30 people comfortably with power, phones, and network connections to cover it. The walls facing the NOC are floor to ceiling glass, and it has connectivity to the videowall of the NOC so that displays from their can be sent to the meeting room as well. It has every high end normal conference room tool you could need - multiple video conferences, smartboard, integrated microphones and speakers, etc. Everything was hidden inside builtin cabinets made of high end wood. This main room is the situation room. During a large outage, 2nd and 3rd level staff will work out of the room in conjunction with the NOC teams. Directly upstairs from the situation room is another identical room, also with floor to ceiling glass walls looking out to the video wall of the NOC. This upper room was reserved for senior and executive management use during a large outage. Engineers and Executive management have different needs during an outage and require separate spaces and separate functions, although constant information does need to feed between the two. The upper room was more of the showpiece room. It had a motorized curtain that you could press a button on the wireless control panel to open and close. The entrance from the building going up to the second floor board room does not give anything away for what the NOC itself looked like, so once everybody was assembled in the room and the button was hit, it never failed to impress first time visitors. They would always leave their chairs at the conf table and walk right up to the glass wall to look down at the people working in the NOC and see what was displayed on the board.
It was an extremely impressive setup. I am now in sales and visit customer sites on a daily basis and I have yet to see something that even approaches what this
Re: (Score:2)
Completely unbelievable troll. I don't own a dog.
Re: (Score:2, Informative)
Re: (Score:3, Informative)
Re: (Score:2)
Wow, I'm a trek fan (and a Voyager fan, yes, one of the few) and even I think this dude is a mega-ultra dweeb.
Re: (Score:3, Funny)
Re: (Score:2)
> The white buzzing fluorescent lights overhead suck...
You can get quiet ballasts. There are also many choices of color other than "cool white" such as "daylight", "deluxe warm white" etc. but make sure maintainence is instructed to use correct replacement lamps. As someone said earlier, hire a lighting designer.
Re:if you need to ask (Score:5, Insightful)
I am the OP. Thanks for your advice. You've saved us millions. No, billions.
Now that you've established I am clueless, go away and read some of the genuinely insightful comments on this thread and you'll see precisely why I asked Slashdot. It's called canvassing a broad range of opinions. Even the world's best consultants/designers would have only designed a handful of these personally. I don't want to make any expensive mistakes and I am quite happy to consider the opinion of a low-ranking staff member earning a few bucks an hour sitting in these places before I subject our low-ranking staff to the same mistakes.
And FYI, my consultants are reading this thread and loving it.
Re: (Score:3, Interesting)
Xybix. They make desks.
Everyone seems to talk about all the benefits of standing, vs. sitting, or more importantly the ability to do either one, but nobody ever bothers to propose something that does both at the discretion of the operator.
Xybix does both. Motorized desks that move up and down (desk surface and monitor platform independently), foot warmers (keeps people from fighting over the thermostat), air circulation, etc. I don't work for them, and I don't sell them, but I have worked at a few instal
Re: (Score:3, Interesting)
If I could afford it, I'd be sitting at a Xybix desk right now.
Not standing?
We've looked at motorized desks. The seem cute but having spoken to people who invested MASSIVE cash in them, their users seldom if ever adjust them. I asked their management for an opinion and they were adamant that they were a waste of money. Moreover, they've had a hard time maintaining a consistent look with various proprietary furniture that needs to be replaced over time and they suggested avoiding bespoke furniture wherever possible and keeping it simple.
I am however considering having a