Bush's Electronic Archives Threaten To Swamp National Archives

ColdWetDog writes "The New York Times reports that the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data. This includes the Barney Cam and even 'formats not previously dealt with.' By way of comparison, the Clinton White House dumped less than a single terabyte into the archives. Of course, Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.' Glad to see that somebody over there is trying to clean up the cruft for posterity."

Data volumes grow exponentially

[Anonymous Coward]

Data capacities and volumes appear to be following the approximate path of Mooreâ(TM)s law â" doubling every 12â"18 months. If Clinton submitted 1 TB eight years ago, I would expect Bush to submit (very) roughly 1/8 * (2 + 4 + 8 + 16 + 32 + 64 + 128 + 256) = 64 TB.

A president ten years before Clinton would probably only have submitted ~1 GB of digital data. This is to be expected.

Stop complaining. Google handles this volume of information every minute.

More to it than that

[an.echte.trilingue]

They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.

What's more, the interest is not just historical, they should be able to access this information immediately. For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why. Additionally, they need to know what overtures the North has made and what they could mean. It will save the new administration lots of time to read the old administration's analyses instead of having to generate their own. Theoretically, the transition team should be assuring that this kind of institutional knowledge is passed, but in reality something always gets missed.

With this amount of data, you are looking at something a lot more complicated than a mysql database and a web based front-end. To be quite honest, I would be surprised if there is any off-the-shelf software capable of this task.

Re:Disgusting

[Zippy_wonderslug]

Secret Service protection is now limited to 10 years after leaving office. Clinton has done pretty well on the speaking circuit, over $30 million in the first 4 years out of office http://projects.washingtonpost.com/2007/clinton-speeches/ [washingtonpost.com] Everyone involved seems to make out pretty well if they so choose.

Wouldn't work here

[an.echte.trilingue]

Problem is, some of this information is classified, and most of it isn't. Some of it legitimately contains private information that should not be made public yet (things like job applications). It contains personal information that should probably be permanently removed from what the public sees, such as employee social security numbers.

What isn't classified or private needs to be available to the public, what is classified or private needs to be available to people with the proper credentials. Some of it will be automatically declassified in 5 years, some of it in 7 years.

In short, somebody has to look at it before it is added to an index. Probably security will dictate that the classified information is stored on different machines, probably different networks, than the publicly available stuff. You might be able to write an algorithm that automates this process, but Google certainly isn't it.

Re:Disgusting

[nurb432]

Legally there are distinctions between public and private data release, even ( especially ) if you are an elected official.

As along as the legal boundaries are followed, then the politician is completely in the right and your 'feelings' are null and void.

Don't like this idea? Lobby and get the laws changed.

Check the record

[MarkusQ]

Umm, McCain was the loudest critic of torture in the Senate. That's one of the reasons a lot of people DIDN'T like him as a candidate for president. He would have let thousands die rather than twist someones arm. Remember he was tortured, that changed his psyche.

He may have been loud, but he wasn't very firm. When Bush effectively nullified the ban on the military using torture with a signing statement, McCain said and did nothing. When congress tried to extend the ban to prevent the CIA from using torture as well, McCain voted against it. [talkingpointsmemo.com]

He may have been against torture at some level, but not as much as he was in favor of getting the nomination. When the two goals came into conflict, he caved.

--MarkusQ

Ballony

[MarkusQ]

He recognized that his dissent would do nothing to mollify his rabid colleagues, and so wisely chose not to provoke their ire by continuing to shout at them, knowing that his shouting would have no effect.He recognized that his dissent would do nothing to mollify his rabid colleagues, and so wisely chose not to provoke their ire by continuing to shout at them, knowing that his shouting would have no effect.

No, he loudly and publicly proclaimed his absolute unconditional refusal to endorse torture. Then, when his bill to prohibit it was quietly circumvented, he said and did nothing. Given the opportunity to vote for a revised bill that would have had teeth (by specifically prohibiting the CIA from torturing people, thus closing the loophole) he voted against it.

And then, on the campaign trail, he continued to play up his POW history and his objection to torture.

That isn't wisely refraining from shouting at your colleges, that's showing your true colors and folding like a hypocrite when it counts, and hoping the saps you pander to are too dumb to notice.

--MarkusQ

In the Information Age sensorship is the haystack

[bussdriver]

No need to draw attention by stopping something; flood the infonet with too much bad information and create a bigger haystack. Not to mention all the FUD that can be done much easier now. You could even put out almost true information with slight crazy distortions to make the truth look bad.

The IMPORTANT INFORMATION WAS LARGELY DESTROYED ("mistakenly lost") and whatever might be useful will take forever to dig out and make any sense of.

Criminal neglect never applies to politicians and don't think that they do not know this.

Re:Formats not dealt with?

[Kalriath]

I don't think the National Archives NEEDS a license. I'm not sure if you US folks have a law like it, but here in NZ the government can infringe copyright on a whim for certain purposes - and archiving is one of them.