Obama Orders Federal Agencies To Digitize All Records 186
Lucas123 writes "President Obama this week issued a directive to all federal agencies to upgrade records management processes from paper-based systems that have been around since President Truman's administration to electronic records systems with Web 2.0 capabilities. Agencies have four months to come up with plans to improve their records keeping. Part of the directive is to have the National Archives and Records Administration store all long-term records and oversee electronic records management efforts in other agencies. Unfortunately, NARA doesn't have a stellar record itself (PDF) in rolling out electronic records projects. Earlier this year, due to cost overruns and project mismanagement, NARA announced it was ending a 10-year effort to create an electronic records archive."
Re:This shouldn't cost too much. (Score:5, Interesting)
Questions worth considering:
What are the savings for going digital? (Without a doubt, they exist; if not, we'd still all be filling out forms in triplicate at work.)
What is the up front cost to convert?
How long will it take the up front cost to be absorbed by the savings?
I suspect that it will pay for itself faster than you might think. Paper records searches are expensive to say the least. And they're extremely personal intensive, not to mention inefficient and error prone.
I realize that there are people out there who will condemn anything this administration does out of hand, but at least try to pretend that you think about things before you make a judgement.
Re:seriously, how hard is this? (Score:5, Interesting)
Is there some complication I don't understand?
Yes. More than one.
Nothing fancy, just a database of scanned forms in pdf format and the like.
There's the first problem. It's never simple.
First issue - if you're going to put documents in, you're going to want to get them out. How do you search for them? You're going to want to define the metadata, and that's a headache. Got lawyers? They'll want client and matter. But those fields are just about meaningless to anyone else. How do you resolve the incompatibility? Do you use different forms for different groups of users? How will the engineering department find the subpoena papers that the lawyers filed?
What fields are globally useful? Are they so generic that any search will retrieve hundreds of documents? Conversely, are they so specific as to make your metadata field selections horribly long and therefore ambiguous? (Free text metadata? Let's not go there.)
Remember that you've got to fill in that metadata any time you add a document. What's the balance between useful and annoying? Too many fields and nobody will want to fill it in. Too few, and you won't be able to find anything.
That's for new documents. When you first implement a DMS, you have a truckload of documents to be imported. You're not going to do it manually, you're going to use an auto-import. But how do you define the metadata for all those millions of documents you're importing? What if you have client/matter, for instance? Hopefully they're all already sorted, and you can use something like Kofax Capture, a seriously powerful and fast scanner, and separator sheets on which you can do forms recognition to define the metadata fields. But there's a lot of work involved up front to get that import working properly.
Don't forget the OCR. Hopefully all your paper documents are clean and will OCR nicely, so you can do full text indexing.
Security. Better get that set up right. Profile level security? It's more secure, but people will complain that they don't know if a document is there and they just need to request access because profile level security means if you don't have permissions to access a document it won't even show up in your search results. Groups. And by the way, remember to define the permissions on all those millions of documents you're importing.
Version control. How do you control check in and check out? Do you control check in and check out, or just audit it?
I've only just scratched the surface of a document management system. Then there's records management. You'll want to make sure your system is DoD 5015.2 compliant. Setting up the retention schedules...hopefully you've got a records retention policy already, otherwise that's months worth of work to define those policies and ensure you comply with all regulatory requirements while still balancing your need to purge/archive old records.
How does something even become a record? Hopefully you've already got knowledgeable librarians (yes, that's what they're called), and you just need to train them on your new RM system.
Are all your boxes already barcoded? Your RM system should be able to register where a record is - building, shelf, box.
You're probably getting the idea. The technology is easy. The processes are complicated, and they get exponentially more complicated as the size of your client base grows.
Private Industry Can Do This Better (Score:4, Interesting)
It's all about the formats! (Score:2, Interesting)
You missed the most important question worth considering - in what formats will these records be maintained?
And Obama missed it, too. I don't see anything in his directive about it.
Good archival practice entails preserving original documents, not just scanned copies.
And if the purpose is to place documents on the Internet, then it's a GIGO situation. If you allow garbage, closed formats like .doc or .docx or .xls or .xlsx to be put on the Web, you're not serving transparency very well, and you're defeating your whole purpose of wanting to make data accessible for Web 2.0 mashups and the like.
Why won't government ever "get" it? The prerequisite question is ALWAYS, what formats? If the formats aren't truly open, then the data isn't open, either.
Re:Private Industry Can Do This Better (Score:4, Interesting)
It was given out in a contract, so you are already getting your wish.
Though I think we could save money by having the government do something itself instead of having to pay for Lockheed's profit and overhead.
Re:seriously, how hard is this? (Score:3, Interesting)
Re:seriously, how hard is this? (Score:5, Interesting)
Free text metadata? Let's not go there.
Google and it's users seem to be doing a pretty good job of utilizing free text to locate documents.
Re:You've been smoking the hope (Score:5, Interesting)
Let's see. A difference of an order of magnitude in number of signing statements. The difference between putting the war costs in the budget - and insisting that they all be by special appropriation or would veto. The difference between starting multiple wars of occupation without a declaration and not. The difference between following the law as created by congress and accepting what congress passed (or didn't as law).
Bush was effective towards his goals. Because Obama doesn't play Bush's games, but the Republicans no longer play be the rules, Obama is not effective. That's part of my point.
No, I'm by no means happy with what Obama has (and hasn't) accomplished. But I'm sick to death of the Republicans and their Rovian games and of the charred earth policy of passing nothing that will help the country (see also abuse of cloture) and blaming Obama. The Republicans declared in 2008 that they had exactly one goal: to make sure that Obama failed. And everything that they've done during these years of crisis has been aligned with that goal, while America rots.
Finally, if you've something to say, say it for yourself as opposed to trying to spin what I'm saying into the opposite. You aren't very good at it.