Bush's Electronic Archives Threaten To Swamp National Archives 185
ColdWetDog writes "The New York Times reports that the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data. This includes the Barney Cam and even 'formats not previously dealt with.'
By way of comparison, the Clinton White House dumped less than a single terabyte into the archives. Of course, Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.'
Glad to see that somebody over there is trying to clean up the cruft for posterity."
Formats not dealt with? (Score:5, Funny)
Re:Formats not dealt with? (Score:5, Insightful)
When trying to hide something in plain sight, drown 'em in irrelevant crap.
Re: (Score:2)
It wouldn't take the general internet public of curious people very long to pick through 100TB.
We've been saying this for years (Score:5, Insightful)
The contingency plan, quietly approved by the National Archives on Nov. 7, emphasizes the difficulties posed by large numbers of White House records created with proprietary commercial software.
Re:We've been saying this for years (Score:4, Insightful)
Actually, there are advanced national and international programmes for digital preservation around the world. Quite a few people in those programmes have participated in these discussions.
But the message hasn't entirely got through to all the other government departments, who are still stuck in a paper mentality. Most are willing but don't know quite what to do (and certainly aren't sure how much it will all cost), and a few are actively difficult to work with, for whatever reasons.
Re:We've been saying this for years (Score:4, Insightful)
Exactly, back in 2001 do you really have a way to restore, read or repair Office 97 and exchange 5 files? Especially if the data has been archived "according to law".. which means they wipe everybody's current mailbox every 6 months to a backup tape and start over.. now you have 8 years history of backup tapes of varying formats and varying versions of the software, none of it overlapping in time frames.
This is the same game Microsoft pull when you sue them they can produce lots of "documents" but your ability to actually read thru them and get something meaningful is greatly diminished. With paper, every body expects boxes of 8 1/2 by 11... with a computer you can make every page take the secret decoder ring from your box of Cracker Jacks!
Re: (Score:2)
To be fair, stuff like that is usually sent to "electronic discovery" firms, who have the means to convert/process data like that.
Re: (Score:3, Insightful)
An ounce of prevention...
Re: (Score:3, Insightful)
Oh come on, now the Bush Administration is evil because they are giving too much data to the archives? That's complete garbage.
Sure, Clinton only had 1 TB of data, but for the day that would have easily equaled 100TB now. I mean really, in 2008 I had like 10gb that more than held all of my entire world.
in `2008 the family desktop machine in my living room has 2 TB of storage with all of my family movies, photos, music etc, and that's just one machine in my house. That doesn't count my linux box, my lapto
Re: (Score:3, Interesting)
I agree, they are evil for witholding information, they are evil for dumping too much information - either pick a side OR admit your hatred of the Bush Administration isn't based on the free-flow (or lack ther-of) of information from this white house.
In a recent article about Obama, they mentioned that George Bush decided to stop sending emails to his daughters while in office because he didn't want those private note part of a public record at some point. Link to the article [politico.com]. The collection of records know
In the Information Age sensorship is the haystack (Score:4, Informative)
No need to draw attention by stopping something; flood the infonet with too much bad information and create a bigger haystack. Not to mention all the FUD that can be done much easier now. You could even put out almost true information with slight crazy distortions to make the truth look bad.
The IMPORTANT INFORMATION WAS LARGELY DESTROYED ("mistakenly lost") and whatever might be useful will take forever to dig out and make any sense of.
Criminal neglect never applies to politicians and don't think that they do not know this.
Re: (Score:2)
Hey I'm tired. I know I have typos in there. Hell, why does English need this old junk like the letter C anyhow? K and S fill the void just fine.
Re: (Score:2)
National Archivist: "What's a .gpg file and how do I open it?"
Bush/Cheney: "heh heh heh..."
Re: (Score:2)
Re: (Score:2)
If they can't open on a typical machine...
Like these [wikimedia.org] files? (all OGG)
Re: (Score:2)
Re: (Score:3, Informative)
I don't think the National Archives NEEDS a license. I'm not sure if you US folks have a law like it, but here in NZ the government can infringe copyright on a whim for certain purposes - and archiving is one of them.
Not much of a threat (Score:2, Interesting)
While the National Archives obviously must catalog and make available all the data in some form or another (honestly I do not know their rules & regs for that sort of thing, and it seems a good bit is missing), the mere act of storing 100 terabytes hardly seems all that daunting. NewEgg has 1TB Samsung Spinpoint harddrives available for $100 with free shipping. You can't tell me the folks over at the National Archives couldn't afford 100 of those plus some additional hardware to oversee the transfer o
Re:Not much of a threat (Score:5, Interesting)
Wouldn't work here (Score:5, Informative)
What isn't classified or private needs to be available to the public, what is classified or private needs to be available to people with the proper credentials. Some of it will be automatically declassified in 5 years, some of it in 7 years.
In short, somebody has to look at it before it is added to an index. Probably security will dictate that the classified information is stored on different machines, probably different networks, than the publicly available stuff. You might be able to write an algorithm that automates this process, but Google certainly isn't it.
Re:Wouldn't work here (Score:4, Funny)
>somebody has to look at it before it is added to an index.
Sure, they can just outsource it to India...
Re: (Score:3, Insightful)
A question that should be asked. Is why was there not a network admin/security offical managing this data durring the administration? Did all of this data just get dumped into a hard drive with no organization?
Sounds like another bad excuse for not planning for the future... This should have been proactivly handled and organized. Now the "Oh Shit" factor is very large and daunting.
Time to hire a security/system admin for the president administration...
Re: (Score:2, Interesting)
If you RTFA (I know, I know) you'd see that the information is principally in email systems and in a records management system storing all the textual material (e.g. documents, but also spreadsheets, pictures, audio, videa and everything else people like to plonk in those systems).
By focussing on volume of data, the article is a bit misleading. It's 100Tb of stuff in several different email and records management systems, with no easy or obvious way to suck all the data out of them and preserve it and the
Re: (Score:2)
Re: (Score:2)
The problem is that the Executive branch hires the admin who has been managing this data and we happen to be getting to the end of an administration that does everything they can to prevent the office of the president from being held accountable to the good citizens of our nation.
From the very beginning with Executive Order 13233 to the present day with Cheney claiming whatever he wants are personal papers. They just don't think we have any right to know what they are doing.
Re: (Score:2)
Security and Admin won't do any more good. If the mandated retention is 6 months on Inboxes, then the admin dutifully archives every 6 months to tape, wipes the mailboxes, and files it away for security. As long as that tape is "available" (remember, retention legally means GONE, daily and weekly backup tapes for operations are not "archive") for the archivist the admin has done his legal duty... anything more than that, like choosing formats or organizing is the President's job to tell somebody to do....
More to it than that (Score:5, Informative)
What's more, the interest is not just historical, they should be able to access this information immediately. For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why. Additionally, they need to know what overtures the North has made and what they could mean. It will save the new administration lots of time to read the old administration's analyses instead of having to generate their own. Theoretically, the transition team should be assuring that this kind of institutional knowledge is passed, but in reality something always gets missed.
With this amount of data, you are looking at something a lot more complicated than a mysql database and a web based front-end. To be quite honest, I would be surprised if there is any off-the-shelf software capable of this task.
Re: (Score:2)
For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why.
But shouldn't all that data then (at the least) be archived in whatever database they already use for that stuff (probably kept at some intelligence agency) and/or in the Secretary of State's computer system or whatever databases he or she, and whoever else needs the info, already uses. Also at the archives for posterity, sure, but why keep the working copy there if it adds to the cost unnecessarily?
I doubt there is any pressing need for the Barney cam (and probably some of the other date in the 100TB total
Re: (Score:2)
It should be in those databases, but sometimes it's not, simply because someone mentioned something in passing in an unrelated e-mail, a document was saved in the wrong directory, or a file got the wrong name and it wasn't opened to check the contents. A semi-manual review is about the only way to go through these documents to check them.
Re: (Score:2)
For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why.
But shouldn't all that data then (at the least) be archived in whatever database they already use for that stuff (probably kept at some intelligence agency) and/or in the Secretary of State's computer system or whatever databases he or she, and whoever else needs the info, already uses. Also at the archives for posterity, sure, but why keep the working copy there if it adds to the cost unnecessarily?
I doubt there is any pressing need for the Barney cam (and probably some of the other date in the 100TB total) to be protected or accessible at anywhere near the levels of some of the other stuff, and most of the data will mostly be accessed by different people.
The impression I have of the way Governments operate is that only specific bits of information are preserved in the long run, and these specific bits were defined by legislation and convention hundreds of years ago. I think the secretary of state's computer system will get taken from the building and erased the minute she leaves office.
I don't think there exists specific Secretary Of State groupware, just waiting to bring Hillary up to speed and pass on her next hundred tasks.
Re:More to it than that (Score:4, Funny)
they need to know exactly what the old administration was doing and why
The 'what' is probably relatively easy to answer. The 'why', I doubt even they know...
Re: (Score:3, Interesting)
They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.
The cost for 100 TB of data storage from Newegg is currently $9322 [newegg.com]. So the cost for 100 backup systems, distributed geographically, would probably be less than a million dollars, after government bulk purchase rates are figured in. When we are giving 7,000 times that amount to the banks, etc., and hoping that they will be able to survive and pay the loans back, I don't think the cost is too much to worry about, when the figure calculated is an obscene amount of redundancy.
Re: (Score:2)
They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.
We are talking about the U.S. Government. Even if we can't afford it, we will spend it anyway.
You are also ignoring the possibility of an agency already having what's required.
I'm sure the NSA could set up a colo for the Whitehouse if needed, for instance.
Heck, I would feel a lot better with that data living under the NSA's roof than some random Whitehouse hodgepodge.
Re:Not much of a threat (Score:4, Funny)
Re: (Score:2)
No, no, they have all of your records for the past 8 years in impeccable detail. When asked if they could help organize the White House files they threw their hands up in the air and walked away.
It's just unreal (Score:5, Interesting)
Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.'
Thereby making what he was doing immune to FOIA requests. Nice.
It's just unreal how unabashedly criminal Cheney is. Nobody ever calls him on it. Anyone in a position to do anything about him (other than Dennis Kucinich anyways) strangely...doesn't.
Of course, he's also the same guy who shot a hunting buddy in the face. And had the victim apologize. [cnn.com]
Far more dangerous than W. Will not be sorry to see him go. Good riddance. Go retire on your inflated Halliburton stock and please leave my country alone.
Re:It's just unreal (Score:4, Interesting)
Re:It's just unreal (Score:5, Insightful)
Sorry, Right-wingers, that wasn't a troll. Just because this guy said something you don't like doesn't make it any less true, or any less valid of an opinion to express.
Re: (Score:2, Insightful)
Does Cheney really follow the rules of typical partisan politics? (If so, someone should let him know - he refuses to follow any other rules) He's involved in too many places to go away just because we elected a democrat.
Re: (Score:2)
Re: (Score:3, Funny)
Expressing an opinion is something entirely different from a troll, and Dick Cheney is an objective scumbag. Anyone who thinks he's an honest or caring person is verifiably wrong. If the comment said something like "Every Republican should die" then that would probably be a troll, but saying that a senior party member will continue to have a large interest in the party after his tenure is both true (for any major party, not just the Republican party) and defensible.
You really should drop your persecution co
Re: (Score:2)
If you change that to "if they go back in time before he dies off", you have the plot for a Summer Hollywood blockbuster movie.
Re:It's just unreal (Score:5, Insightful)
McCain would probably have stood up to him
Two points:
1. McCain lost any credibility he had when he endorsed US torture of foreigners.
2. If McCain was someone who would stand up to Cheney, he'd never have been chosen for the Republican ticket.
Re: (Score:2)
Check the record (Score:5, Informative)
He may have been loud, but he wasn't very firm. When Bush effectively nullified the ban on the military using torture with a signing statement, McCain said and did nothing. When congress tried to extend the ban to prevent the CIA from using torture as well, McCain voted against it. [talkingpointsmemo.com]
He may have been against torture at some level, but not as much as he was in favor of getting the nomination. When the two goals came into conflict, he caved.
--MarkusQ
Ballony (Score:4, Informative)
No, he loudly and publicly proclaimed his absolute unconditional refusal to endorse torture. Then, when his bill to prohibit it was quietly circumvented, he said and did nothing. Given the opportunity to vote for a revised bill that would have had teeth (by specifically prohibiting the CIA from torturing people, thus closing the loophole) he voted against it.
And then, on the campaign trail, he continued to play up his POW history and his objection to torture.
That isn't wisely refraining from shouting at your colleges, that's showing your true colors and folding like a hypocrite when it counts, and hoping the saps you pander to are too dumb to notice.
--MarkusQ
Re: (Score:2)
No, I'm recounting recent history. Check the links. McCain spoke out against torture right and left, but when the time came to do something about it, he caved. Twice. And then continued to make the same damned speeches about how much he opposed it. Check the record.
And when bananas are nothing but earwax, mom, it's time to wash your bike.
N
Re: (Score:2)
But he lost.
Then Hillary was supposed to win, but she lost the nomination to Obama.
So Obama isn't part of the Clinton masterplan. He's more like the Mule in Asimov's "Foundation" series, the statistically-unlikely possibility that appears from nowhere and has the planners going "Whu? Where the hell did HE come from?".
Re: (Score:3, Funny)
And Obama isn't a third Clinton administration? :-)
No, he's the second Carter administration!
Re:It's just unreal (Score:4, Interesting)
Re: (Score:2)
senseless reply to undo accidental modding
I'd rather go hunting with Cheney (Score:2, Insightful)
And where in the Constitution does it say that the Executive cedes power to Congress by a mere passage of a law (FOIA)? I thought the actual Executive was in charge of that branch.
For all of you constitutional purists, I'd ask where in Article I Congress has the right to limit executive authority, whether it's FOIA, FISA, the War Powers Act, "congressional oversight," etc. I am looking but can't find in the Constitution where Congress can do this without ratifying a
Re: (Score:2)
No, just that the Executive decides what is done within its own, co-equal branch of government.
Article 1, Section 8
The Congress shall have Power To . . . provide for the common Defence and general Welfare of the United States;
Yes, and how the hell does that clause grant Congress a check over another branch? That's a new one. Provide means it has a duty itself to do something, not tell another branc
Gah. Unclosed tag. (Score:2)
Okay. (Score:2, Insightful)
(2) [As a result of (1)]: Treason
I don't pay attention to any hate sites. But his actions have been pretty blatant.
You know what, folks? (Score:2)
Yes, I have been victim, but I have seen many, many other victims as well.
Let's try to be ADULTS here, people, and accept the idea that others can hold opposing views without being "trolls"!!!
Disgusting (Score:5, Insightful)
"I'm told researchers like to come and dig through my files, to see if anything interesting turns up," Mr. Cheney said. "I want to wish them luck, but the files are pretty thin. I learned early on that if you don't want your memos to get you in trouble some day, just don't write any."
This really says it all, doesn't it? I mean, wasn't this essentially Nixon's view on things? That if the president (or his puppet master, vice-president Cheney) deems it not for the public's purview, it's none of your damn business? I mean, what part of PUBLIC office does this numbskull not understand? (Excuse me, the mastermind understands, just doesn't care.)
Sickening. What's even worse is that no one's gonna make this administration accountable for anything they've done. In fact, I'm sure no one's gonna really take a hard look at what exactly this administration has done until a looong time later; everyone's too preoccupied with moving on.
Re: (Score:2)
Politician is the only well paid position I know that requires neither any kind of credentials that you're able to do your job nor comes with any kind of responsibility if you do a bad job. What's the worst thing that could happen? You don't get reelected? Duh, if you don't manage to line your pockets sensibly in four years you're doing something wrong anyway.
Re: (Score:2)
I think 5 is "getting a Nobel prize".
Re: (Score:2)
rather "write books"
Re:Disgusting (Score:4, Informative)
Legally there are distinctions between public and private data release, even ( especially ) if you are an elected official.
As along as the legal boundaries are followed, then the politician is completely in the right and your 'feelings' are null and void.
Don't like this idea? Lobby and get the laws changed.
Re:Disgusting (Score:5, Interesting)
Looking from overseas I find it sadly funny that people who call themselves "Republicans" really want to act like absolute monarchs instead. Hopefully future administrations will act a little bit more like the government George Washington and others wanted instead of like some 18th century German principality.
Re: (Score:2)
Actually Nixon's stance was that the president was above wrongdoing as long as he felt whatever he was doing was in the best interest of the nation. That may be the same thing Cheney thinks, but Nixon had no problem writing memo's and documenting everything because he felt that he was above the law.
Re: (Score:2)
I mean, what part of PUBLIC office does this numbskull not understand? (Excuse me, the mastermind understands, just doesn't care.) Sickening. What's even worse is that no one's gonna make this administration accountable for anything they've done. In fact, I'm sure no one's gonna really take a hard look at what exactly this administration has done until a looong time later; everyone's too preoccupied with moving on.
Exactly, it's time to move on already. I mean, Clinton has been out of office for almost 8 years now. What's the point of stirring up the past again?
Or, were you talking about Bush? It's hard to tell. Every time a president gets ready to leave office, the zealots on the other side start to complain about what criminals they were and how they should be prosecuted for their crimes and all of the dead bodies that were left behind and can't understand how it's not obvious to everyone what a snowjob they're pull
For the uninformed.. (Score:2)
If all you haters really want to look at something slimy how about the case was made that the NSA was "not an
Re:For the uninformed.. (Score:4, Funny)
Re: (Score:2)
"Leader of the Legislative Branch"... that has a Limbaugh Horse Hockey smell to it. If the Files of the President of the Senate contain anything more than discussions about tie votes and parliamentary procedure, then we've got some fraud on our hands.
Speak and Spell (Score:2)
I'd really have thought with Bush's user level and Cheney's underhanded nature you'd have been able to archive their entire administration on a few floppies.
Make that 101 TB (Score:2, Funny)
Wow! So many HDs (Score:5, Funny)
100 TB! Wow! That's about 60 hard disks or so! And I don't even want to imagine how much money you'd have to pay for those drives - it'd be a *four digit sum*!
Well, just imagine the next one (Score:2)
Obama's probably already stored a few TB just with the fundraising email campaigns.
the next white house team... (Score:4, Interesting)
there is always talk of obsolete formats here on
Re: (Score:2)
It's not really any administration's fault they use proprietary formats either. I have one of the first DVD video cameras. The format it records into is awful, and it's almost impossible to convert to anything else other than playing the movie off the camera, then recording that stream into another format. It's very aggravating.
These aren't the bytes you're looking for... (Score:2)
Does anyone else find it ironic this is the same administration that couldn't keep track of a few years worth of official emails? I seem to remember a lack of storage space being one of the excuses, too.
hide it (Score:2)
Before this turns into a political flame fest: (Score:4, Insightful)
If you think this is bad today, this is only the tip of the ice burg. The national archives better ramp up for a drastic increasing curve of data to store as each new president is elected.
Not that i have the answer, but i can see it happening. Just look at the exponential increases in personal information for the average citizen.
Re: (Score:2)
Of course, scrubbed like Nixon's tapes (Score:2)
Maybe we can find out how (Score:4, Interesting)
Here's the real question (Score:2)
Is there an 18 minute gap in the Barney Cam tapes?
Barney Cam... WTF? (Score:2)
I don't know if it was worth the 10 minutes of my life I just spent watching it, but it was cute.
Re: (Score:2)
In such a smart community... (Score:2)
I really do believe most people who post here are very smart and intelligent people involved in some of the most important fields we currently have: technology of all kinds.
So why are we paying attention to this? Does this matter? Does it matter that some content is going to stored away (as IF the public will be able to access it easily and as IF the information will even matter)?
To me, this is a sign of Slashdot turning into Fox News and any other mainstream media. Slashdot is supposed to be the other side
Cheney's personal files (Score:4, Interesting)
Government employees aren't supposed to be using government computers for non-government purposes. I can't complain if Cheney has a few shopping lists and personal emails lying around, but truly personal files should be few and small. If he claims that any substantial portion of his files are personal, he's either lying or he has been misusing government property.
the 'lost' emails .. (Score:2)
Does that include the lost emails [cnn.com], the ones that were on the non-existent backup tapes ?
Moore's Law: well, yeah! (Score:2)
This includes the Barney Cam and even 'formats not previously dealt with.' By way of comparison, the Clinton White House dumped less than a single terabyte into the archives.
2^(8/1.5)=40, so the fact that Bush's term ended 8 years after Clinton's would automatically account for 40% of the increase. Reckoning that digital records are more widely used now than even a decade ago easily takes care of the remainder.
Yeah, 100TB is a lot of data, but no more than expected. The National Archive better gear up for at least 10PB from President Obama, given those two factors alone.
Re:100TB (Score:5, Funny)
I hear you can't look at porn on the White House computers. NOW does it sound impressive?
Re: (Score:2, Troll)
It's not very impressive that the centre of Freedom In The World censors the Internet. The word I'd use is "ironic".
Re: (Score:2)
Re:100TB (Score:5, Insightful)
Needle.
In a haystack.
"If there's anything we haven't classified or destroyed, let's make it impossible to locate."
Re: (Score:2, Insightful)
"If there's anything we haven't classified or destroyed, let's make it impossible to locate."
No problem.. send the media to Google. I'm 100% sure they can figure it out (unless the archive contents are encrypted).
Re: (Score:2)
As I say: ironic, huh?
Re: (Score:2)
Re: (Score:2)
That's the sort of thing they got Oliver North on - although the shopping list was for things like a convertable and air conditioning for his house embezzled out of the money from selling guns to Iran. The funny thing is the emails were all deleted individually which would have taken ages - and that attracted enough attention that the backup tapes were taken out of their cycle and stored seperately. Questions by either Poindexter or North about
Re: (Score:2)
Re:64 TB (Score:2)
That should be enough for anybody.
Re: (Score:2)
It's true. It all started back when President Eisenhower submitted exactly 1 byte to the National Archives when he left office.
Re: (Score:2)
Re: (Score:3, Insightful)
What you do have are laws preventing the use of federal systems for purely political uses. This is what was the problem with the mailing system a year or so ago. People were given blackberries and other systems for political and party related messages and because of government requirements not given those same types of tools for government business. People being people and since they had those tools availab
Re: (Score:2)