Do Data Center Audits Mean Anything? 84
1sockchuck writes "Data center service providers often tout certifications such as SAS 70, SSAE 16 and SOC 2 as evidence that they meet lofty operational standards. But some of these certifications are based on self-defined standards, and the entire situation is confusing and frustrating to customers, according to one critic, who says data center shoppers are poorly served by the jumble of acronyms and standards. Do these certifications matter when users are seeking data center space? Should they?"
Not really (Score:5, Informative)
Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.
Re: (Score:2)
Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.
The first thing you'll probably discover is that 'Certificate' isn't spelled with an 'S', like all these acronyms are starting with. :-\
Re: (Score:3)
Them:What? I zipped it with a password, doesnt that protect the patient information?
Me: Sure, for about 5 minutes. *doublefacepalm*
Re: (Score:2)
But it's disclosed to you, and based on that you can make a decision on whether to give a certain datacenter your business. (not sure if there's as much choice in the medical field, but that's a topic for another time)
Re: (Score:2)
The certifications amount to "This shell company we created to report whatever we want it to says we're tight and secure. Yay for us!!!!!"
Re: (Score:2)
The certifications amount to "This shell company we created to report whatever we want it to says we're tight and secure. Yay for us!!!!!"
And if you pay enough for our services, we'll frame the certificate, as soon as it comes off the laser printer.
Re: (Score:3)
Re:Not really (Score:4, Insightful)
Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.
It depends on the purpose of the audit. If the purpose is to appease middle managers and the like, then the auditors (good or bad) will be able to read the request "We need to ensure we are certified for [insert current buzzword]." and see that this is nothing short of an easy way to make a costly fee. If on the other hand, the request is to find ways to break into the systems and comes from sysadmins or the like, then it is much more likely that the company wants to patch vulnerabilities.
Business is business. If a sales person sees easy money walking into the office, they will probably sell them overpriced and needless goods/services. If they see someone who knows exactly what it is they want, they will more likely give them exactly what they ask for and for a reasonable price.
Re:Not really (Score:4, Insightful)
These aren't intrusion tests they're talking about but certification audits.
My experience with those (ISO, SAS, etc.) is that a company hires someone to write up a bunch of documents to match what the auditors want to see, and tell the employees where to find it. Then the auditors come and get told/shown what they want to hear/see so they'll go away and let us get back to real work. The documentation isn't looked at again by regular employees until the next audit.
Those certs are just like professional certs like MCSE, CCNA, etc. They don't really have any bearing on whether or not you're good at what you do, but they sound good to customers/employers.
Re: (Score:2)
And yet the minimal standards are much higher than most people actually do.
My take: Do audits and auditors do anything to secure your systems? Rarely. Do having real hoops to jump through and jobs/salaries/bonuses on the line for failure prompt people to try to secure their systems? Frequently.
These "audits" are bogus. (Score:1)
It's like when any of the US government agencies "audit" themselves for instance, The Federal Reserve. Remember that "audit" recently?
Re: (Score:3)
Bad example, the Federal Reserve is a private entity.
It's better than no cert at all (Score:4, Interesting)
Re: (Score:1)
Re: (Score:1)
No, he obviously meant Master of Science in Clinical Epidemiology (MSCE) [upenn.edu] Whoo boy, that cert isn't worth the paper it's printed on!
Re:It's better than no cert at all (Score:4, Funny)
Re: (Score:2)
Re: (Score:2)
Comment removed (Score:5, Funny)
Re: (Score:1)
Really? Wow. I've never had a company claim they have a SAS70 type II (or whatever it's called now) but refuse to release the actual report ... that just seems so non-sensical. Not having a SAS70 isn't the end of the world, but lying about it certainly would be.
Re: (Score:1)
Re: (Score:2)
Uses for Audits/Certifications (Score:5, Insightful)
Re: (Score:2)
Rent-seeking, it's not just about buying rack space.
should they? no. do they? not to me. (Score:2)
The fact of the matter is a lot of stupid certification acronyms were specifically designed to allow spenders to make decisions without being actually informed in any way about what they're spending their money on. That's actually the *point* here. The problem isn't the certifications, the problem is that to make an informed decision about which ISP should host your servers you shouldn't be the type of CTO who insists on using outlook express and ie6 still and can't even configure their own email client.
Re: (Score:1)
The fact of the matter is a lot of stupid certification acronyms were specifically designed to allow spenders to make decisions without being actually informed in any way about what they're spending their money on. That's actually the *point* here. The problem isn't the certifications, the problem is that to make an informed decision about which ISP should host your servers you shouldn't be the type of CTO who insists on using outlook express and ie6 still and can't even configure their own email client. You need to know bandwidth from ass-width.
I couldn't help but notice this post wasn't ESU 77A Certified - sign up for our Seminar[www.joesbarandtrainingcenter.com] only $1,500, availibility is limited
Like ISO 9000 (Score:2)
Re: (Score:1)
Why not come up with a similar certification, one that's available under Creative Commons? That way, anyone could actually read the specifications, and you can go so far as to have a third party certify the results, either by self-certifying and having that notarized, or having a trusted third party perform the audit.
Re: (Score:2)
Will that certification be meaningless to not create any kind of trouble, and yet obscure enough for potential clients not discovering it is meaningless?
Re: (Score:2)
The actual ISO 9001 standard only costs about $130 on the ISO's website, the verification labor is by far a bigger part of the cost. Nobody foregoes ISO 9001 certification on account of the non-libre status of the standard.
short answer: no (Score:4, Informative)
I'm a work at a somewhat large financial services company that provides customer information to various other large financial institutions (chase, wells fargo, capital one, amex, discover, just to name a few). We receive this customer information from pretty much everywhere - those self same banks, government agencies, credit card companies, universities. Basically, if you've ever had a loan or grant, credit card, bank account, paid a utility bill, child support or been in prison then we have that data. Your address, phone number, social security number, bank account information, etc.
The majority of this information is stored unencrypted on systems that are accessible to any employee, often with 777 permissions. While the majority of the systems are patched pretty regularly, many aren't. I recently had to convert over an old apache 1.3 server that hadn't been patched since 2006 - there's another similar server that is regularly used by outside contributors to drop off customer information.
We have customer facing IPlanet servers that haven't been patched since 2004 - the software isn't even under support anymore.
We have session recording software on our unix servers that is so ridiculously trivial to bypass that the company that sells it (centrify) should be ashamed to sell it.
Yet we've had PCI certification for 3 years, we've passed the SAS70 certification every time - they are rubber-stamps, nothing more.
Re: (Score:2)
Basically, if you've ever had a loan or grant, credit card, bank account, paid a utility bill, child support or been in prison then we have that data. Your address, phone number, social security number, bank account information, etc.
The majority of this information is stored unencrypted on systems that are accessible to any employee, often with 777 permissions. While the majority of the systems are patched pretty regularly, many aren't. I recently had to convert over an old apache 1.3 server that hadn't been patched since 2006 - there's another similar server that is regularly used by outside contributors to drop off customer information.
Well, looks like I won't be able to sleep ever again.
What country is this in? My only hope is that it is not where I reside and things are much more secure here. Please don't shatter my illusion! Looks like ignorance really is bliss.
Re: (Score:3)
Nothing about SAS-70 requires any measure of quality or completeness. When the business claims, "yes, we have a disaster recovery plan. yes, we have a business continuity plan. yes, we have a backup, fault tolerance, order handling, fulfillment, budgeting, a
Re: (Score:1)
Re: (Score:2)
And Your QSA is ?
Re: (Score:1)
Yes your QSA should be fired and your company should be fined for running antiquated software that has security holes the size of the grand canyon.
Of course it matters (Score:5, Informative)
Well, it certainly matter for regulation purpose. If you handle data that need to be covered under a specific standard (say, PCI), you'll seek out a certified data center. In this context, the certification isn't about security, it's about risk transfer. It's the provider who become liable if there's a breach if it can't show to have respected the standard properly.
Now as security references, they certainly have their problems. We can take solace in the thought that they help enforce the bare minimum at the very least. As a security professional, I would say their best benefit is how well they can be used as a big stick, "encouraging" management to perform necessary changes. It's a hard sell to convince an average manager to invest in security for the sake of security. But if there's a legal penalty associated with whatever standard must be put in place, as well as a big dollar sign attached to it, they'll suddenly start to listen. That's a language they understand.
Need to see the criteria (Score:3)
Better Measurement of Success (Score:3)
The problem is that they are trying to get certifications when what they really need are Achievements! Just ask Microsoft, it worked for the XBOX, they are throwing it into Visual Studio, it will work for data centers.
Include a Facebook Like button and a Twitter link and your done!
Would you put your money in a non-FDIC bank? (Score:3)
Just like when hiring a new employee, you look for certifications and credentials. When trying to separate the legitimate companies from the fly-by-nights, seeing audits every year going back 2, 3, 4 years can help verify that they've been around for a while. Datacenter space isn't cheap, and if you find a good deal you want to make sure that your server with all the company data on it is still there on monday morning, and not on a plane to China.
You put your money in an FDIC-insured bank account because it's registered with someone who's taken the task of keeping an eye on their registrants and staked their reputation on it. If you don't agree with that, I have an offshore bank account you can transfer some money in to for the International Bank Of Hadlock, we just opened yesterday, but we offer 3000% interest daily and don't keep money laundering records.
Re: (Score:2)
I have some of my money go through a non-FDIC credit union. They have a non-federal deposit share insurance provided by http://www.americanshare.com./ [www.americanshare.com]
Only as good as the auditor (Score:3)
I'd say that data center audits, just like financial audits, are only as good as the auditor. If you're a big enough client, the auditor will say pretty much any thing you want:
http://en.wikipedia.org/wiki/Arthur_Andersen#Demise [wikipedia.org]
But still, I look for the certifications to cover my butt. Of course, that's what all of these standards are about - just saying that you've implemented procedures to cover your butt. It doesn't matter whether or not the procedure actually does anything worthwhile or even if there are big gaping holes elsewhere. As long as you can say you've implemented it fully, then you're covered.
These are NOT Certifications! Nobody Gets This! (Score:1)
There is a number of problems with how data centers make these statements and what people interpret.
The main problem is that people say things like "SAS70 Certified". That is terribly bad wording. There is no such thing. The SAS70 (now SSAE16 or SOC1 report) is not a certification. There is no preset/predetermined criteria that is universal to all companies that receive such a report. Each report is specific to that particular company/data center. It's almost like saying I have a diploma as an indepen
These are NOT Certifications! (Score:5, Informative)
Re: (Score:1)
How much is PHB speak that what people who don't (Score:2)
How much is PHB speak that what people who don't know that they are running come up with BS to make it look like they do.
Re: (Score:1)
I have no idea what that sentence means.
If you want security and reliability... (Score:5, Insightful)
Security and reliability are processes, they are not something you can do once and then forget about. So, yes, I would say that having regular audits are a useful thing. As far as whether these specific standards are useful, the facility we have most of our servers in we have been in since before their SAS 70 audit, and their procedures were good before, but there's a noticeable improvement after. Things like a man-trap with a live security person comparing you with your on-file photo before you enter the raised floor, 2-factor auth on all doors rather than just on the key doors, maintenance lock-outs displayed more prominently, EPOs installed (not a benefit to me, but they did put alarmed doors around the EPOs to prevent the common problems).
As far as it being "based on self-defined standards", I'm ok with that. I'm ok with the requirement being that they *HAVE* standards for certain things rather than dictating what exactly those standards are. One size does not fit all, but having standards for what you do, I have found in my own business, improves quality.
sas70 quality (Score:1)
SAS70 and the new SSAE 16 require that the assessor and the reader both recognize the limitations and the scope of the work. Like many have said, its dependent on the quality of the auditor. This is why there are a "Big 4" in the audit world, their name carries weight to the quality of review. We will exclude the perceptions about the quality that may actually be provided. When I did SAS70 and similar audits, one of the first things i look at is the company that performed the review, the quality of the
like all audits (Score:1)
rarely
Conversation with an Auditor (Score:1)
Re: (Score:2, Insightful)
I don't understand why people have such a hard time with this Audit concept. In these cases, an Auditor audits your processes as defined by your management. It doesn't matter what your "process" does in real life. That is NOT what an Auditor is checking. You are not being graded on what you do or how you do it. An IT/Financial/Process Audit is NOT an employee performance review. Something that Engineers and Programmers can't seem to get through their thick heads.
An Auditor is providing a report to the
The answer you need to show your boss (Score:5, Informative)
Right here, pure gold: http://www.gartner.com/it/page.jsp?id=1400813 [gartner.com]
Read that 5 times, carefully, and then get your bosses to do the same. Seriously.
SAS70 is a *questionnaire* that the vendor completes, and then the auditors just go in and confirm that their answers are correct.
So I could say "we don't do backups" in my answer to the questionnaire, the auditors would verify that I didn't do backups, and I'd "complete" the SAS70 process (not a certification!) successfully.
It is the client that is resoponsible for reviewing the questionnaire and ensuring that the audited answers are sufficient for the needs of their business. That's called "vendor management" and is a core practice area in ITIL.
Re: (Score:2)
Pretty much. If your procedure is that you never do backups, or don't have redundancy, and the auditors verify that as fact: ta-da SAS70 seal of approval.
Re: (Score:2)
We are way ahead of you! We did a backup once, and threw it away!
Yes, they do mean something ... (Score:2)
The presence of a certification unaccompanied by an invitation to come and look over the place yourself should be a pretty good warning.
only if it involves testing (Score:1)
Re: (Score:2)
The bigger picture (Score:2)
Certainly at the worker bee level we all can agree that most certs are not worth the paper that they're printed on, however when you start looking at what the company needs and requires then, yes, the certs do provide some value.
Imagine signing a multi-million dollar contract with a data center. Part of the contract are clauses for things like QOS, DR, and a whole host of other very tiny details which are so, so important in the contract. Now, lets say that the data center goes down (fire for example), al
NOT A CERTIFICATION!!!! (Score:1)
There are two types of SSAE16 audits (Score:1)
In a Type 1 audit, all the auditors look for is whether the company has policies/procedures/controls in effect to obtain the objectives of the company (whatever those may be)
In a Type 2 audit, the auditors will attempt to determine whether the policies and procedures in place are being followed. Whether the controls are effective in achieving the objectives that have been stated.
I work for a software company that recently went through a Type 2 audit. In our case most of what was looked at was our SDLC (sof