Forgot your password?
typodupeerror
Data Storage Republicans United States

Bush's Electronic Archives Threaten To Swamp National Archives 185

Posted by timothy
from the if-you're-smart-this-can-all-end-well dept.
ColdWetDog writes "The New York Times reports that the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data. This includes the Barney Cam and even 'formats not previously dealt with.' By way of comparison, the Clinton White House dumped less than a single terabyte into the archives. Of course, Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.' Glad to see that somebody over there is trying to clean up the cruft for posterity."
This discussion has been archived. No new comments can be posted.

Bush's Electronic Archives Threaten To Swamp National Archives

Comments Filter:
  • by eggman9713 (714915) <{eggman97132007} {at} {mac.com}> on Sunday December 28, 2008 @04:57AM (#26247801)
    'formats not previously dealt with.' If they can't open on a typical machine, they are probably just corrupt. Wait a second...
    • by Atario (673917) on Sunday December 28, 2008 @06:29AM (#26248081) Homepage

      When trying to hide something in plain sight, drown 'em in irrelevant crap.

      • by X0563511 (793323)

        It wouldn't take the general internet public of curious people very long to pick through 100TB.

    • by an.echte.trilingue (1063180) on Sunday December 28, 2008 @07:02AM (#26248151) Homepage
      For years I have been listening to people here rant about the potential for records to get lost due to proprietary formats. Nobody listened, and now it is happening:

      The contingency plan, quietly approved by the National Archives on Nov. 7, emphasizes the difficulties posed by large numbers of White House records created with proprietary commercial software.

      • by Anonymous Coward on Sunday December 28, 2008 @09:55AM (#26248903)

        Actually, there are advanced national and international programmes for digital preservation around the world. Quite a few people in those programmes have participated in these discussions.

        But the message hasn't entirely got through to all the other government departments, who are still stuck in a paper mentality. Most are willing but don't know quite what to do (and certainly aren't sure how much it will all cost), and a few are actively difficult to work with, for whatever reasons.

      • by mabhatter654 (561290) on Sunday December 28, 2008 @12:33PM (#26249801)

        Exactly, back in 2001 do you really have a way to restore, read or repair Office 97 and exchange 5 files? Especially if the data has been archived "according to law".. which means they wipe everybody's current mailbox every 6 months to a backup tape and start over.. now you have 8 years history of backup tapes of varying formats and varying versions of the software, none of it overlapping in time frames.

        This is the same game Microsoft pull when you sue them they can produce lots of "documents" but your ability to actually read thru them and get something meaningful is greatly diminished. With paper, every body expects boxes of 8 1/2 by 11... with a computer you can make every page take the secret decoder ring from your box of Cracker Jacks!

        • by NiceGeek (126629)

          To be fair, stuff like that is usually sent to "electronic discovery" firms, who have the means to convert/process data like that.

    • Re: (Score:3, Insightful)

      by StormyWeather (543593)

      Oh come on, now the Bush Administration is evil because they are giving too much data to the archives? That's complete garbage.

      Sure, Clinton only had 1 TB of data, but for the day that would have easily equaled 100TB now. I mean really, in 2008 I had like 10gb that more than held all of my entire world.

      in `2008 the family desktop machine in my living room has 2 TB of storage with all of my family movies, photos, music etc, and that's just one machine in my house. That doesn't count my linux box, my lapto

      • Re: (Score:3, Interesting)

        by kenh (9056)

        I agree, they are evil for witholding information, they are evil for dumping too much information - either pick a side OR admit your hatred of the Bush Administration isn't based on the free-flow (or lack ther-of) of information from this white house.

        In a recent article about Obama, they mentioned that George Bush decided to stop sending emails to his daughters while in office because he didn't want those private note part of a public record at some point. Link to the article [politico.com]. The collection of records know

        • by bussdriver (620565) on Sunday December 28, 2008 @08:40PM (#26253457)

          No need to draw attention by stopping something; flood the infonet with too much bad information and create a bigger haystack. Not to mention all the FUD that can be done much easier now. You could even put out almost true information with slight crazy distortions to make the truth look bad.

          The IMPORTANT INFORMATION WAS LARGELY DESTROYED ("mistakenly lost") and whatever might be useful will take forever to dig out and make any sense of.

          Criminal neglect never applies to politicians and don't think that they do not know this.

          • Hey I'm tired. I know I have typos in there. Hell, why does English need this old junk like the letter C anyhow? K and S fill the void just fine.

    • National Archivist: "What's a .gpg file and how do I open it?"
      Bush/Cheney: "heh heh heh..."

    • If they can't open on a typical machine...

      Like these [wikimedia.org] files? (all OGG)

  • Not much of a threat (Score:2, Interesting)

    by DrMrLordX (559371)

    While the National Archives obviously must catalog and make available all the data in some form or another (honestly I do not know their rules & regs for that sort of thing, and it seems a good bit is missing), the mere act of storing 100 terabytes hardly seems all that daunting. NewEgg has 1TB Samsung Spinpoint harddrives available for $100 with free shipping. You can't tell me the folks over at the National Archives couldn't afford 100 of those plus some additional hardware to oversee the transfer o

    • by MichaelSmith (789609) on Sunday December 28, 2008 @05:07AM (#26247835) Homepage Journal
      I think google have a free service where they lend you a Network Attached Storage box. You load it up with whatever then send it back to them. You still have online access to the data and google indexes it for you.
      • Wouldn't work here (Score:5, Informative)

        by an.echte.trilingue (1063180) on Sunday December 28, 2008 @06:27AM (#26248071) Homepage
        Problem is, some of this information is classified, and most of it isn't. Some of it legitimately contains private information that should not be made public yet (things like job applications). It contains personal information that should probably be permanently removed from what the public sees, such as employee social security numbers.

        What isn't classified or private needs to be available to the public, what is classified or private needs to be available to people with the proper credentials. Some of it will be automatically declassified in 5 years, some of it in 7 years.

        In short, somebody has to look at it before it is added to an index. Probably security will dictate that the classified information is stored on different machines, probably different networks, than the publicly available stuff. You might be able to write an algorithm that automates this process, but Google certainly isn't it.
        • by John Sokol (109591) on Sunday December 28, 2008 @08:20AM (#26248451) Homepage Journal

          >somebody has to look at it before it is added to an index.

          Sure, they can just outsource it to India...

        • Re: (Score:3, Insightful)

          by Meest (714734)

          A question that should be asked. Is why was there not a network admin/security offical managing this data durring the administration? Did all of this data just get dumped into a hard drive with no organization?

          Sounds like another bad excuse for not planning for the future... This should have been proactivly handled and organized. Now the "Oh Shit" factor is very large and daunting.

          Time to hire a security/system admin for the president administration...

          • Re: (Score:2, Interesting)

            by Anonymous Coward

            If you RTFA (I know, I know) you'd see that the information is principally in email systems and in a records management system storing all the textual material (e.g. documents, but also spreadsheets, pictures, audio, videa and everything else people like to plonk in those systems).

            By focussing on volume of data, the article is a bit misleading. It's 100Tb of stuff in several different email and records management systems, with no easy or obvious way to suck all the data out of them and preserve it and the

          • by lawpoop (604919)
            My guess is that what they're talking about is the data being federalized. Sure, this admin did this mail server, this admin did this webserver, etc. etc. Now they are getting *all* of that data, and it's no surprise it's a big unorganized cluster fuck. After all, who would be in charge of that? An IT Czar? Imagine trying to propose that and sell that to the American public and the special interest groups. Civil Libertarians would be concerned about a Big Brother controlling all government data. Conservativ
          • The problem is that the Executive branch hires the admin who has been managing this data and we happen to be getting to the end of an administration that does everything they can to prevent the office of the president from being held accountable to the good citizens of our nation.

            From the very beginning with Executive Order 13233 to the present day with Cheney claiming whatever he wants are personal papers. They just don't think we have any right to know what they are doing.

          • Security and Admin won't do any more good. If the mandated retention is 6 months on Inboxes, then the admin dutifully archives every 6 months to tape, wipes the mailboxes, and files it away for security. As long as that tape is "available" (remember, retention legally means GONE, daily and weekly backup tapes for operations are not "archive") for the archivist the admin has done his legal duty... anything more than that, like choosing formats or organizing is the President's job to tell somebody to do....

    • More to it than that (Score:5, Informative)

      by an.echte.trilingue (1063180) on Sunday December 28, 2008 @05:50AM (#26247971) Homepage
      They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.

      What's more, the interest is not just historical, they should be able to access this information immediately. For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why. Additionally, they need to know what overtures the North has made and what they could mean. It will save the new administration lots of time to read the old administration's analyses instead of having to generate their own. Theoretically, the transition team should be assuring that this kind of institutional knowledge is passed, but in reality something always gets missed.

      With this amount of data, you are looking at something a lot more complicated than a mysql database and a web based front-end. To be quite honest, I would be surprised if there is any off-the-shelf software capable of this task.
      • by story645 (1278106)

        For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why.

        But shouldn't all that data then (at the least) be archived in whatever database they already use for that stuff (probably kept at some intelligence agency) and/or in the Secretary of State's computer system or whatever databases he or she, and whoever else needs the info, already uses. Also at the archives for posterity, sure, but why keep the working copy there if it adds to the cost unnecessarily?

        I doubt there is any pressing need for the Barney cam (and probably some of the other date in the 100TB total

        • It should be in those databases, but sometimes it's not, simply because someone mentioned something in passing in an unrelated e-mail, a document was saved in the wrong directory, or a file got the wrong name and it wasn't opened to check the contents. A semi-manual review is about the only way to go through these documents to check them.

        • For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why.

          But shouldn't all that data then (at the least) be archived in whatever database they already use for that stuff (probably kept at some intelligence agency) and/or in the Secretary of State's computer system or whatever databases he or she, and whoever else needs the info, already uses. Also at the archives for posterity, sure, but why keep the working copy there if it adds to the cost unnecessarily?

          I doubt there is any pressing need for the Barney cam (and probably some of the other date in the 100TB total) to be protected or accessible at anywhere near the levels of some of the other stuff, and most of the data will mostly be accessed by different people.

          The impression I have of the way Governments operate is that only specific bits of information are preserved in the long run, and these specific bits were defined by legislation and convention hundreds of years ago. I think the secretary of state's computer system will get taken from the building and erased the minute she leaves office.

          I don't think there exists specific Secretary Of State groupware, just waiting to bring Hillary up to speed and pass on her next hundred tasks.

      • by TheRaven64 (641858) on Sunday December 28, 2008 @07:02AM (#26248149) Journal

        they need to know exactly what the old administration was doing and why

        The 'what' is probably relatively easy to answer. The 'why', I doubt even they know...

      • Re: (Score:3, Interesting)

        by MagusSlurpy (592575)

        They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.

        The cost for 100 TB of data storage from Newegg is currently $9322 [newegg.com]. So the cost for 100 backup systems, distributed geographically, would probably be less than a million dollars, after government bulk purchase rates are figured in. When we are giving 7,000 times that amount to the banks, etc., and hoping that they will be able to survive and pay the loans back, I don't think the cost is too much to worry about, when the figure calculated is an obscene amount of redundancy.

      • by X0563511 (793323)

        They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.

        We are talking about the U.S. Government. Even if we can't afford it, we will spend it anyway.

        You are also ignoring the possibility of an agency already having what's required.

        I'm sure the NSA could set up a colo for the Whitehouse if needed, for instance.

        Heck, I would feel a lot better with that data living under the NSA's roof than some random Whitehouse hodgepodge.

  • It's just unreal (Score:5, Interesting)

    by Weaselmancer (533834) on Sunday December 28, 2008 @04:59AM (#26247809)

    Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.'

    Thereby making what he was doing immune to FOIA requests. Nice.

    It's just unreal how unabashedly criminal Cheney is. Nobody ever calls him on it. Anyone in a position to do anything about him (other than Dennis Kucinich anyways) strangely...doesn't.

    Of course, he's also the same guy who shot a hunting buddy in the face. And had the victim apologize. [cnn.com]

    Far more dangerous than W. Will not be sorry to see him go. Good riddance. Go retire on your inflated Halliburton stock and please leave my country alone.

    • Re:It's just unreal (Score:4, Interesting)

      by Chris_Jefferson (581445) on Sunday December 28, 2008 @05:22AM (#26247891) Homepage
      Remember, this is the republican party. If they get back in any time before he dies off, he will be there in the background, pulling the strings of whatever new puppet president they choose. McCain would probably have stood up to him, but I suspect now the republicans blame him for their loss, it could well be back to the "Bush Box" to find their next candidate. Scary thought.
      • by Zorque (894011) on Sunday December 28, 2008 @06:18AM (#26248041)

        Sorry, Right-wingers, that wasn't a troll. Just because this guy said something you don't like doesn't make it any less true, or any less valid of an opinion to express.

        • Re: (Score:2, Insightful)

          by Firehed (942385)

          Does Cheney really follow the rules of typical partisan politics? (If so, someone should let him know - he refuses to follow any other rules) He's involved in too many places to go away just because we elected a democrat.

        • Perhaps, but funny how that only applies to one party here on Slashdot.
      • by mikael (484)

        If you change that to "if they go back in time before he dies off", you have the plot for a Summer Hollywood blockbuster movie.

      • by schon (31600) on Sunday December 28, 2008 @11:05AM (#26249277)

        McCain would probably have stood up to him

        Two points:

        1. McCain lost any credibility he had when he endorsed US torture of foreigners.

        2. If McCain was someone who would stand up to Cheney, he'd never have been chosen for the Republican ticket.

        • Probably true. I had real hope for McCain when he was first running, I'd vaguely followed his carrier before and thought he was one of the few people who had some backbone, but then it all fell away, shame really. Of course, the real question is what state Obama will be in after he has been in power for 4 years.
    • Re:It's just unreal (Score:4, Interesting)

      by martin-boundary (547041) on Sunday December 28, 2008 @05:26AM (#26247913)
      If particular records are classified as private, does that mean what's on them can be used in court to convict him of private crimes? (not that anybody would have the guts to try it)
    • by lanswitch (705539)

      senseless reply to undo accidental modding

    • Than go driving with Ted Kennedy.

      And where in the Constitution does it say that the Executive cedes power to Congress by a mere passage of a law (FOIA)? I thought the actual Executive was in charge of that branch.

      For all of you constitutional purists, I'd ask where in Article I Congress has the right to limit executive authority, whether it's FOIA, FISA, the War Powers Act, "congressional oversight," etc. I am looking but can't find in the Constitution where Congress can do this without ratifying a
  • Disgusting (Score:5, Insightful)

    by pnumoman (1348217) on Sunday December 28, 2008 @05:09AM (#26247845)

    "I'm told researchers like to come and dig through my files, to see if anything interesting turns up," Mr. Cheney said. "I want to wish them luck, but the files are pretty thin. I learned early on that if you don't want your memos to get you in trouble some day, just don't write any."

    This really says it all, doesn't it? I mean, wasn't this essentially Nixon's view on things? That if the president (or his puppet master, vice-president Cheney) deems it not for the public's purview, it's none of your damn business? I mean, what part of PUBLIC office does this numbskull not understand? (Excuse me, the mastermind understands, just doesn't care.)

    Sickening. What's even worse is that no one's gonna make this administration accountable for anything they've done. In fact, I'm sure no one's gonna really take a hard look at what exactly this administration has done until a looong time later; everyone's too preoccupied with moving on.

    • Politician is the only well paid position I know that requires neither any kind of credentials that you're able to do your job nor comes with any kind of responsibility if you do a bad job. What's the worst thing that could happen? You don't get reelected? Duh, if you don't manage to line your pockets sensibly in four years you're doing something wrong anyway.

    • Re:Disgusting (Score:4, Informative)

      by nurb432 (527695) on Sunday December 28, 2008 @09:55AM (#26248907) Homepage Journal

      Legally there are distinctions between public and private data release, even ( especially ) if you are an elected official.

      As along as the legal boundaries are followed, then the politician is completely in the right and your 'feelings' are null and void.

      Don't like this idea? Lobby and get the laws changed.

    • Re:Disgusting (Score:5, Interesting)

      by dbIII (701233) on Sunday December 28, 2008 @10:34AM (#26249083)

      I mean, wasn't this essentially Nixon's view on things? That if the president (or his puppet master, vice-president Cheney) deems it not for the public's purview, it's none of your damn business?

      Looking from overseas I find it sadly funny that people who call themselves "Republicans" really want to act like absolute monarchs instead. Hopefully future administrations will act a little bit more like the government George Washington and others wanted instead of like some 18th century German principality.

    • Actually Nixon's stance was that the president was above wrongdoing as long as he felt whatever he was doing was in the best interest of the nation. That may be the same thing Cheney thinks, but Nixon had no problem writing memo's and documenting everything because he felt that he was above the law.

    • by tobiasly (524456)

      I mean, what part of PUBLIC office does this numbskull not understand? (Excuse me, the mastermind understands, just doesn't care.) Sickening. What's even worse is that no one's gonna make this administration accountable for anything they've done. In fact, I'm sure no one's gonna really take a hard look at what exactly this administration has done until a looong time later; everyone's too preoccupied with moving on.

      Exactly, it's time to move on already. I mean, Clinton has been out of office for almost 8 years now. What's the point of stirring up the past again?

      Or, were you talking about Bush? It's hard to tell. Every time a president gets ready to leave office, the zealots on the other side start to complain about what criminals they were and how they should be prosecuted for their crimes and all of the dead bodies that were left behind and can't understand how it's not obvious to everyone what a snowjob they're pull

  • The Cheney is about if records he does as President of the Senate are covered [google.com] by the law. The position of the Vice President is that only work ordered by the President are covered those done as being a leader of the Legislative Branch are not. Also the group bring the suit are tryng to get access purely political conversations which the national archives have already ruled are not to be archived.
    If all you haters really want to look at something slimy how about the case was made that the NSA was "not an
    • by Yvanhoe (564877) on Sunday December 28, 2008 @06:15AM (#26248031) Journal
      Because, clearly, he has an archive folder as "president of senate" and "vice president". It is far from insane to suppose he also has a folder labeled "questionable documents" he will put in the archive less likely to go public.
    • by Boronx (228853)

      "Leader of the Legislative Branch"... that has a Limbaugh Horse Hockey smell to it. If the Files of the President of the Senate contain anything more than discussions about tie votes and parliamentary procedure, then we've got some fraud on our hands.

  • Who knew a Speak and Spell could amass that much data? Maybe it's the high scores from Minesweeper and Solitaire that did it? Perhaps Bush's tutorials from Mavis Beacon's typing?

    I'd really have thought with Bush's user level and Cheney's underhanded nature you'd have been able to archive their entire administration on a few floppies.
  • As a last addition, 1 TB of data was added to the archives. It contained all the news footage, replays, and parodies of the shoe-throwing incident.
  • by Anonymous Coward on Sunday December 28, 2008 @07:33AM (#26248229)

    100 TB! Wow! That's about 60 hard disks or so! And I don't even want to imagine how much money you'd have to pay for those drives - it'd be a *four digit sum*!

  • by acedotcom (998378) on Sunday December 28, 2008 @07:47AM (#26248279)
    100TB seems like a lot, but can you imagine how much data the Obama administration might leave? Considering that they seem more 'connected' then most other politicians.

    there is always talk of obsolete formats here on /. but can you imagine the baggage left by some of these administrations?
    • It's not really any administration's fault they use proprietary formats either. I have one of the first DVD video cameras. The format it records into is awful, and it's almost impossible to convert to anything else other than playing the movie off the camera, then recording that stream into another format. It's very aggravating.

  • The New York Times reports that the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data

    Does anyone else find it ironic this is the same administration that couldn't keep track of a few years worth of official emails? I seem to remember a lack of storage space being one of the excuses, too.

  • If you can't delete the records, just hide multiple pieces in soo much garbage data that you would be dead and gone by the time anyone was able to put it all together.
  • by nurb432 (527695) on Sunday December 28, 2008 @09:51AM (#26248883) Homepage Journal

    If you think this is bad today, this is only the tip of the ice burg. The national archives better ramp up for a drastic increasing curve of data to store as each new president is elected.

    Not that i have the answer, but i can see it happening. Just look at the exponential increases in personal information for the average citizen.

    • They're just a bunch of sniveling sots. Petabyte class storage solutions are common place and while pricey for the common man, they're well within reach of the corporate world and comparable to the cost of a single outing in Air Force one.
  • IOW, these guys have been cleaning anything relevant and creating lots of BS crap to put in there. By the time it is figured out, Cheney will be dead and W will be senile. Hopefully, these traitors are able to at least be caught on tax fraud.
  • by dbIII (701233) on Sunday December 28, 2008 @10:53AM (#26249191)
    Maybe we can find out how someone can have the goodwill of nearly every nation on earth (even Libya offered to help!), a tame congress, wartime expediency in letting anything you like get through, signing statements to change anything you don't like, a booming economy, a military grudgingly ready even to abandon the Geneva convention if ordered and access to experts on every subject yet STILL muck it up so badly.
  • Is there an 18 minute gap in the Barney Cam tapes?

  • Barney cam... FTW!

    I don't know if it was worth the 10 minutes of my life I just spent watching it, but it was cute.
  • I really do believe most people who post here are very smart and intelligent people involved in some of the most important fields we currently have: technology of all kinds.

    So why are we paying attention to this? Does this matter? Does it matter that some content is going to stored away (as IF the public will be able to access it easily and as IF the information will even matter)?

    To me, this is a sign of Slashdot turning into Fox News and any other mainstream media. Slashdot is supposed to be the other side

  • by belmolis (702863) <{billposer} {at} {alum.mit.edu}> on Sunday December 28, 2008 @08:06PM (#26253259) Homepage

    Government employees aren't supposed to be using government computers for non-government purposes. I can't complain if Cheney has a few shopping lists and personal emails lying around, but truly personal files should be few and small. If he claims that any substantial portion of his files are personal, he's either lying or he has been misusing government property.

  • "the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data"

    Does that include the lost emails [cnn.com], the ones that were on the non-existent backup tapes ?
  • This includes the Barney Cam and even 'formats not previously dealt with.' By way of comparison, the Clinton White House dumped less than a single terabyte into the archives.

    2^(8/1.5)=40, so the fact that Bush's term ended 8 years after Clinton's would automatically account for 40% of the increase. Reckoning that digital records are more widely used now than even a decade ago easily takes care of the remainder.

    Yeah, 100TB is a lot of data, but no more than expected. The National Archive better gear up for at least 10PB from President Obama, given those two factors alone.

Put your Nose to the Grindstone! -- Amalgamated Plastic Surgeons and Toolmakers, Ltd.

Working...