NARA event: "A Conspiracy to Steal History"

Sorry for the late notice -- I found out about this event earlier today.

Tomorrow at noon EST, the U.S. National Archives and Records Administration (NARA) will hold a public discussion outlining how NARA and the U.S. Department of Justice investigated and prosecuted Barry Landau and Jason Savedoff, who stole archival records from the Franklin Delano Roosevelt Presidential Library, the Maryland Historical Society, the Connecticut Historical Society, and several other repositories.  The lead presenters are Mitchell Yockelson, Investigative Archivist with the NARA's Office of the Inspector General, which reviewed the masses of historical documents founded in Landau's New York City apartment, and Jim Warwick, Assistant U.S. Attorney for the Department of Justice, who was one of two U.S. attorneys who led the prosecution.

This event, which is free and open to the public, will be held in the William G. McGowan Theater of the National Archives Building in Washington, DC (aka "Archives I"). Attendees should use the Special Events entrance located on Constitution Avenue at 7th Street NW.

If you're not going to be in the Washington, DC area tomorrow afternoon, this event will also be streamed online.  If NARA's past practice is any indication, a recording will be made available online shortly after this event wraps up; if and when it does, I'll append it to this post.

Update 2013-03-15:  Here's the video, which is also available via NARA's Ustream page.

Video streaming by Ustream

"I lost archival perspective and made wrong choices"

Earlier today, Leslie Waffen, the former head of the Sound and Video Branch of the U.S. National Archives and Records Administration (NARA), was sentenced to eighteen months in prison and two years of post-prison probation and ordered to pay a $10,000 fine.  His crime?  Stealing the materials for which he was supposed to care and selling them on eBay.  In an odd twist of fate, the chain of events that culminated in his arrest and prosecution began when an item he put up for sale -- a 1937 recording of Babe Ruth -- was spotted by J. David Goldin, who had given the recording to NARA.

At his sentencing, Waffen tearfully apologized for his misdeeds, noting that he had "lost archival perspective and made wrong choices" and that he had "violated, personally, the archivist's code of ethics." His statement isn't accessible via Public Access to Court Electronic Records (PACER), but judging from various media reports, Waffen's criminal career began when he began taking recordings that were being considered for inclusion in NARA's holdings to his house so that he could work at home.  At some point, his passion for collecting old sound recordings overrode his professional and moral judgment.  Eventually, he apparently decided that selling the stolen recordings on eBay was a good idea.

Every archival security expert I've ever met has emphasized that collectors of and dealers in historical materials deserve special scrutiny. Waffen, who most assuredly got off easy, exemplifies why we need to be mindful of the fact that expertise and passion -- which are, by and large, wonderful things -- can lead some people horribly astray.

 I didn't have a PACER account at the time Waffen entered his guilty plea, but I've got one now.  For your reference, here's a copy of Waffen's October 2011 guilty plea, which states that he could have been imprisoned for ten years and fined up to $250,000 but broadly hints that he would likely receive an eighteen-month sentence, and a copy of the supplementary statement of facts that accompanied the plea. 
Waffen Plea 2011-10 Waffen Supplement 2012-10

NARA releases 2011 records management assessment

Since 2009, the U.S. National Archives and Records Administration (NARA) has conducted annual surveys of federal government agencies' records management practices.  All of these surveys have revealed that electronic records management is a particular challenge for the federal government, and the 2011 assessment, the results of which NARA released earlier this week, is no exception.  Although NARA identified some modest successes, most notably increased transfers of archival electronic records, it's plain that management of electronic records remains an area of particular concern.  NARA found that:

< snip >

• Many respondents do not know or understand key terms and concepts pertaining to electronic records;
• Many respondents consider various aspects of electronic records management to be the purview of information technology staff;
• A significant number of agencies do not have migration procedures in place to ensure that electronic records are retrievable and usable to conduct agency business;
• Many respondents believe that media neutral records schedules eliminate the need for records management policies and procedures specific to electronic records;
• A significant number of agencies use backup tapes, which NARA does not consider a recordkeeping system, to preserve electronic documents and e-mail records;
• A third of agencies are using an ERMS [Electronic Records Management System] or RMA [Records Management Application] to manage their electronic records;
• Over 40 percent of agencies use e-mail archiving applications to manage e-mail messages . . . .

< /snip >

These findings are depressing but not particularly surprising.  Electronic records management remains a real challenge for many public- and private-sector organizations.  I would be willing to bet that the feds are actually ahead of most (but by no means all) state and local governments, and I suspect that many corporations -- even those whose stock in trade is digital information -- are similarly challenged. Earlier this week, I blogged about the near-disaster that Pixar (which should be applauded for its candor) experienced, and Twentieth-Century Fox and Paramount have discarded or lost digital files that have monetary and artistic value.  A host of other corporations are probably hoping that their records and information management nightmares remain out of the public eye.

What does NARA propose to do about the sorry state of federal records management?  Appendix I of the recently released report offers a detailed plan of action, and I encourage you to read it -- and the rest of the report -- in its entirety.  However, I will say that I'm particularly pleased that NARA wants agencies to incorporate records management plans -- with benchmarks and resource allocations -- into their annual budget submissions to the Office of Management and Budget (OMB).  I'm also glad that NARA to work with OMB to ensure that records management and archival functions are incorporated into new electronic recordkeeping systems and into the federal "IT governance process."  When a fiscal control entity demands something, government agencies tend to listen.

NARA seeks your input

Last November, President Obama issued a memorandum that marked the start of a sweeping effort to reform federal records management practices. Among other things, the memorandum directs the U.S. National Archives and Records Adminstration (NARA) and the Office of Management and Budget (OMB) to consult with individuals and organizations -- government and non-government alike -- who have an interest in improving the management of federal government records and increasing government openness.

NARA, which particularly interested in ideas that will promote government openness, use records and information to enhance federal agency performance, and reduce unnecessary costs and burdens, is fulfilling this requirement in a variety of ways. It's hosting public meetings and has created a Managing Government Records IdeaScale community that enables archivists, records managers, vendors, and anyone else who cares to do so to offer their suggestions, read suggestions advanced by others, and comment and vote up or down on each suggestion. You'll need to create a free IdeaScale account in order to do so, but signing up is quick and painless.

The IdeaScale interface is pretty intuitive, but if you're new to IdeaScale (which is one of my favorite Web 2.0 tools), you may want to consult NARA's IdeaScale guide.

The Managing Government Records IdeaScale community will be open for comments until Friday, 6 April.

BPE 2011: ERA and the move to the cloud

This week, I’m spending a little time with my parents in Ohio and at the 2011 Best Practices Exchange (BPE) in Lexington, Kentucky. The BPE, which brings together state government, academic, and other archivists and librarians and other people seeking to preserve state government enduring information of enduring value, is my favorite archival conference. The Society of American Archivists annual meeting is always first-rate, but it’s gotten a little overwhelming, and I love the Mid-Atlantic Regional Archives Conference (MARAC), but nothing else has the small size, tight focus on state government records, informality, and openness that characterize the BPE.

Before I start detailing today’s highlights, I should say a few things about the content of these posts. For the past few years, those of us who have attended the BPE have tried to adhere to the principle that “what happens at BPE, stays at BPE.” This doesn’t mean that we don’t share what we’ve learned at the BPE (hey, I’m blogging about it!), but it does mean that we’re sensitive to the fact that candor is both essential and risky. The BPE encourages people to speak honestly about how and why projects or programs went wrong and what they learned from the experience. Openness of this sort is encouraging; all too often, we think that we’re alone in making mistakes. It's also helpful: pointing out hidden shallows and lurking icebergs helps other people avoid them. However, sometimes lack of senior manager commitment, conflicts with IT personnel, and other internal problems contribute to failure, and colleagues and supervisors occasionally regard discussion of internal problems as a betrayal. As a result, BPE attendees should exercise some discretion, and those of us who blog about the BPE should be particularly careful; our posts are a single Web search away. As a result, in a few instances I may write about the insights and observations that attendees have shared but obscure identifying details.

Moving on to this year's BPE itself, I'm going to devote the rest of this post to the insights and predictions offered up by U.S. National Archives and Records Administration (NARA) Chief Information Officer Mike Wash, who spoke this morning about the Electronic Records Archives (ERA), NARA’s complex, ambitious, and at times troubled electronic records system, and some changes that are on the horizon.

At present, ERA sort of works: staff use it to take in, process and store electronic records. At present, ERA holds approximately 130 TB of data. The Office of Management and Budget wants NARA to take in 10 TB of data per quarter, and NARA is working with agencies to meet this benchmark. However, ERA lacks an integrated access mechanism, and it contains multiple modules. The Base module handles executive agency data, the EOP module handles presidential records (and includes some internal access mechanisms), the Classified module holds classified records, and several other modules were built to deal with specific problems.

Building ERA taught NARA several lessons:

• Solution architecture is critical. ERA’s multiple modules are a sign of a failed system architecture. Anyone building such a system must carefully consider the business and technical architecture carefully during the planning stage and must manage the architecture carefully over time.
• The governance process must be clear and should start with business stakeholders. What do they really need the system to do, and how do you ensure that everyone stays on the same page throughout the process? Information technology invariably challenges control and authority, but if you set up your governance process properly, you should be able to retain control over system development.
• Over communicate. Funders and other powerful groups need frequent updates; failure to keep feeding information to them can be profoundly damaging.
• You must manage the project. The federal government tends to hire contractors to develop IT systems, and contractor relationships tend to deteriorate about six months after the contract is awarded. Most federal agencies cede authority to contractors because they are loath to be seen as responsible in the event that a project fails, but staying in control of the project increases your chances that you'll get the system you want.
• Watch costs closely. Cost-escalating provisions have a way of sneaking into contracts.
• Be mindful of intellectual property issues. The federal government typically reserves the right to all intellectual property created as a result of contracts, but this doesn’t always happen, and the vendor that built the first iteration of ERA has asserted that it controls some of the technology that now makes the system work; NARA will be much more assertive in working with future ERA vendors.

Wash also made some intriguing observations about some of the challenges that NARA and other archives are confronting:

• At present, our ability to acquire data is limited by bandwidth limitations. It takes more than three days to convey 20 TB of data over a 1 gbps data line and at least a month to convey it via the Internet. NARA recently took custody of 330 TB of 2010 Census data, and it did so by accepting a truckload of hardware; at present, there are no alternatives to this approach.
• The rate of data creation continues to accelerate. The administration of George W. Bush created 80 TB of records over the course of 8 years, but the Obama administration likely created more than 80 TB of data during its first year.

Wash indicated that NARA is starting to think that federal records should be created and maintained in a cloud computing environment and that transfer of custody from the creating agency to NARA should be effected by changing some of the metadata associated with the records being transferred.

Wash noted that the move to cloud computing will bring to the fore new preservation and authentication concerns. It also struck me that the transition that Wash envisions assumes the existence of a single federal government cloud that has adequate storage, security, and access controls and that, at least at this time, many states aren’t yet thinking of constructing such environments. Individual state agencies may be thinking of moving to the cloud, but most states don't seem to be preparing to move to a single, statewide cloud environment. Moreover, owing to its sheer size, the federal government is better able to negotiate favorable contract terms than state or local governments; the terms of service agreements that the feds hammered out with various social media providers are an excellent example. I have the uneasy feeling that some governments will accept, out of lack of knowledge, desperate financial straits, or inability to negotiate optimal terms, public cloud service contracts that prove problematic or outright disastrous.

Its nonetheless apparent that government computing will move into the cloud, that this transition offers both new challenges and new opportunities for managing and preserving records, and that archivists and records managers are going to have come to grips with these changes. The next decade promises to be most interesting.

The Lexington Laundry Company building on West Main Street, Lexington, Kentucky, 20 October 2011. This little gem was built ca. 1929, is an outstanding example of Art Deco architecture in the city, and is part of Lexington's protected Downtown Commercial District. It now houses an art gallery.

Lincoln document at NARA altered by researcher

U.S. National Archives and Records Administration, "National Archives Discovers Date Change on Lincoln Record," 25 January 2011.

Archivists devote a lot of attention to ensuring that researchers don't take records out of our facilities, but we don't always think about what they might be bringing in to our repositories. However, as recent events in the United States and the United Kingdom reveal, theft merely one of a host of malicious threats to the documentary record: fraudulent alteration and augmentation can be every bit as damaging.

Earlier today, the U.S National Archives and Records Administration (NARA) revealed that a researcher clandestinely changed the date on a 14 April 1864 pardon issued by President Abraham Lincoln by changing the "4" in "1864" to a "5" -- and thus giving the misleading impression that signing the pardon was one of Lincoln's last official acts. (Lincoln was shot at approximately 10:13 PM on 14 April 1865 and died nine hours later.)

Researcher Thomas Lowry snuck a pen into NARA's research room and the altered the date [images here] in an effort to bolster his scholarly reputation, and it worked: his "discovery" of the pardon garnered a substantial amount of media attention, resulted in the exhibition of the pardon in the rotunda of NARA's Archives I facility in Washington, DC, and helped to propel sales of his book.

Kudos to Trevor Plante, the eagle-eyed NARA archivist who suspected that something was amiss, contacted NARA's Office of the Inspector General, and did the historical research that confirmed his suspicions, and to NARA's Office of the Inspector General, which investigated the matter and secured Lowry's confession. Lowry's misdeed came to light well after the statute of limitations on prosecution expired, but he has been banned from NARA. Moreover, his scholarly reputation -- which is obviously of great importance to him -- is ruined.

NARA is not the only institution that has had to deal with a researcher seeking to alter the documentary record: in 2008, the National Archives of the United Kingdom discovered that researcher Martin Allen had clandestinely inserted 29 forged documents into 12 separate files documenting the activities of British secret agents during the Second World War. Allen, who used a laser printer to produce the forged records, then cited them in his books, which claimed, among other things, that the Duke of Windsor was a traitor, that Winston Churchill had ordered British agents to murder Nazi leader Heinrich Himmler, and that the British had entered into clandestine peace negotiations with the Nazis. Much to the dismay of many historians and archivists, the Crown Prosecution Service ultimately determined that prosecuting Allen, whose health was fragile, would not serve the national interest.

Let's start taking a closer look at the writing implements and research materials that our users bring to our research rooms -- doing so protects our researchers, protects us, and, most importantly, protects our records.

(A big tip o' the hat goes to Richard Pearce-Moses for alerting me to today's NARA story.)

80 TB of presidential records

Here's a not-surprising-but-nonetheless-mind-boggling snippet of news: the U.S. National Archives and Records Administration (NARA) reports that it has received approximately 80 terabytes of electronic records generated by President George W. Bush and his staff. As if this weren't astounding enough, consider these other facts:

• The transfer of the Bush records marks the first time that NARA has received more electronic than paper records from a presidential administration.
• If printed out, the Bush administration's e-mail -- which makes up a fraction of the electronic records transferred to NARA -- would comprise approximately 600 million sheets of paper. NARA estimates that all of the other presidential libraries that it administers hold a combined total of roughly 550 to 580 million sheets of paper records.

Is your head spinning yet? If not, just start contemplating the volume of electronic records that NARA will likely receive from the Obama administration.

The Bush administration records, which have been ingested into NARA's Electronic Records Archives (ERA) system, won't be fully processed for quite some time, but archivists are busy preparing for an anticipated deluge of researcher requests in January 2014, when some of the records will become accessible under the provisions of the federal Freedom of Information Act.

By the way, NARA and the federal Office of Management and Budget are halting further development of the ERA system earlier than planned. NARA will focus instead on getting federal agencies to use ERA and may "recompete" the operation and maintenance components of its contract with Lockheed Martin, which built the system. Interesting.

New NARA guidance re: Web 2.0 and social media records

Late last week, the U.S. National Archives and Records Administration released a long-anticipated bulletins concerning management of records created as a result of federal agency usage of Web 2.0 and social media tools to conduct government business. It details how to determine whether social media and Web 2.0 content meets the Federal Records Act definition of a government record, highlight the records management challenges particular to Web 2.0 and social media records, and outline how agencies can address some of these challenges. It also stress that agencies are responsible for managing records that are housed by third-party service providers.

If you're interested in managing or preserving government social media and Web 2.0 content, be sure to check out this bulletin. I'll be giving several of my colleagues a heads-up about it.

"Removing the Shroud of Secrecy": records management and open government

Last Tuesday (23 March), the U.S. Senate Subcommittee on Homeland Security and Government Affairs, Subcommittee on Federal Financial Management, Government Information, Federal Services, and International Security held a hearing on "Removing the Shroud of Secrecy: Making Government More Transparent and Accountable." Members of the subcommittee, which is chaired by Senator Thomas R. Carper (D-DE) heard testimony from, among others, Archivist of the United States David Ferriero and Federal Chief Information Officer Vivek Kundra.

In his written testimony and in the Webcast of the hearing, Ferriero stressed the centrality of records management to good government: "the government cannot be accountable if it does not preserve -- and cannot find -- its records." He went on to assert that heads of federal agencies and other senior agency personnel "need to understand that the records and information they and their organizations are creating are national assets that must be effectively managed and secured so that the public can be assured of the authenticity of the record [emphasis added]."

Well said. It's all too easy to forget that government records and information are, fundamentally, public property and that they warrant the same careful stewardship as other public assets, and we archivists and records managers need to keep reminding others of this essential fact.

Open Government in the Digital Age Summit: videos, slideshow, and other resources

On 19 March, the New York State Chief Information Officer/Office for Technology (CIO/OFT)and the New York State Archives jointly hosted the Open Government in the Digital Age Summit in Albany. Speakers included Archivist of the United States Ferriero, U.S. Deputy Chief Technology Officer for Open Government Beth Noveck, and e-Republic Inc. Chief Content Officer Paul Taylor.

CIO/OFT's Web site now features video recordings of all Summit speeches and panel discussions, Paul Taylor's PowerPoint presentation, and the final Summit agenda. It has also prepared a news release that captures many of the day's highlights and posted photos of the Summit on its Facebook page.

Over 150 people attended the Summit, which brought together information technology professionals, archivists and records managers, public policy experts, journalists, and others interested in the relationship between information technology, recordkeeping, and government accountability and transparency. If you're interested in these issues, be sure to check out the resources above.

"The Archivist of the United States Googled me . . . ."

Earlier today, National Public Radio's Weekend Edition program aired Liane Hansen's lengthy profile of David Ferreiro, the new Archivist of the United States, and the U.S. National Archives and Records Administration. It's a great piece, and it opens with news that Ferriero prepared for the interview by doing a little online research about Hansen and NPR and arranging to show Hansen records relating to her past work -- including a letter documenting President Eisenhower's opinions about Spam (the meat product nuisance, not the electronic communication nuisance).

Apart from a few minor errors (e.g., Hansen implies that "vault" items, most of which can be seen by researchers who document their need to see them and request access in advance, are completely off-limits to the public), Hansen avoids most of the journalistic clichés about archives. There's not a single reference to dust, and Hansen's description of the smell of a properly equipped and maintained repository -- "old paper and clean air" -- is elegantly simple and evocative.

Hansen's profile doesn't tackle the subject of electronic records, which Ferriero identified as one of his biggest concerns in a recent Federal Computer Week interview. However, it does emphasize one of his other top priorities: the digitization of NARA's non-electronic holdings. (Fortunately, it also stresses that digitizing each and every one of the 10 billion paper- and analog media-based items won't happen overnight.) It also stresses Ferriero's enduring passion for reference work, which will likely shape NARA's activities under his tenure.

My description of Hansen's profile is no substitute for actually listening to it. It's great publicity for Ferriero, NARA, and archives in general, and I wish that all media coverage relating to archives were this informative, entertaining, and perceptive.

Bush White House e-mail settlement

News that 22 million lost e-mail messages sent or received by the Bush White House have been recovered and that the National Security Archive (NSA) and Citizens for Responsibility and Ethics in Washington (CREW) have settled their 2007 lawsuit against the Executive Office of the President (EOP) has been all over the media for the past couple of days.

A lot of the media coverage is focusing on a couple of items in the settlement document. First, for reasons of cost, the White House will focus on recovering e-mails sent or received on select days, not every "missing" e-mail that the Bush White House created. Second, the U.S. National Archives and Records Administration (NARA) will take custody of the e-mails and manage them in accordance with the Presidential Records Act, which means that the e-mails won't be disclosed to researchers for years.

However, a quick review of the settlement document itself reveals that, with a handful of exceptions, the media isn't calling attention to a provision that ought to interest anyone who wants to know how the White House does business or how EOP manages its electronic records:

4. Description of Current EOP System: Defendants [EOP and NARA] will provide Plaintiffs [NSA and CREW] with a publicly releasable letter describing in as much detail as possible the current EOP computer system, including its email archiving and backup systems. This document will include a detailed description of the controls in the system that prevent the unauthorized deletion of records.

a. Prior to sending the letter, Defendants will review with Plaintiffs draft(s) of the letter and the Parties will agree upon a final version.

b. Defendants recognize that Plaintiffs intend to release the letter publicly, and Defendants do not object to such a release.

c. Defendants will produce this letter by January 15, 2010.

Although I'm sorry that the EOP and NARA personnel charged with producing this document will likely have to curtail their holiday breaks, I'm looking forward to the end result. It should make for interesting reading.

"So far, it's the best job in the country"

Last week, David Ferriero, the new Archivist of the United States, delivered his first State of the Archives address. I was particularly cheered by his continuing emphasis on the challenges posed by electronic records and electronic records management, which he likened to the problems faced by Robert Digges Wimberly Connor, the first Archivist of the United States, who fought valiantly to ensure that the nation's long-neglected records were properly housed:

. . . . It seems to me that we are at a similar crossroads in the history of the Archives in the challenges we face with the electronic records of the agencies we serve. Varieties of technology, platforms, software, practice, and lack of standards complicate the work of ingesting, preserving, and making available the records of the government. The work we have undertaken with Lockheed Martin is, of course, being watched closely by our funders, our stakeholders, and the rest of the archival community who is grappling with similar issues of born digital records. We have to get this right.

I also see the Electronic Records Archives initiative as a vehicle for reestablishing our oversight of the records management programs of each agency—working with agencies to establish protocols, practices, and annual audits.

I also like that Ferriero recognizes the larger archival community's interest in the Electronic Records Archives, and I hope that he continues predecessor Allen Weinstein's effort to bring the U.S. National Archives and Records Administration into closer alignment with archival professional organizations and other repositories throughout the nation.

If you want a sense of Ferriero's background and personality, check out the lengthy profile in today's Washington Post, which highlights his decades of work in libraries and includes video footage of him examining materials in the stacks of the Archives I facility in Washington, DC. The video's only 42 seconds long, but it reveals that the new Archivist has a puckish sense of humor:

Anthony Clark and NARA

Over the past few weeks, the Archives & Archivists listserv has been home to a lengthy series of postings chronicling the shabby manner in which the U.S. National Archives and Records Administration (NARA) has treated Anthony Clark, who has sought access to records concerning NARA's Presidential libraries. I've followed the discussion on the listserv and on Mr. Clark's blog, and I have to conclude that, to say the least, some folks at NARA have some explaining to do -- and some corrective actions to undertake.

Owing to the manner in which the discussion was initiated and the tone that it has taken, to date I've refrained from commenting; I simply assumed that the listserv was experiencing yet another heat wave and that cooler, more rational discussion would eventually prevail. Unfortunately, to date, there is nary a cold front in sight. The discussion, which now involves the blogosphere as well, remains as heated as ever. The latest hotspot: ArchivesNext. Kate T. did a little independent digging and dispassionately presented her conclusions, and received harsh and, in my view, unwarranted criticism from Clark and his archival champion, Richard Cox.

FWIW, Kate and I spoke for a few minutes at a conference several years ago, and our subsequent interactions have consisted of posting sporadic comments on a Facebook group and on each other's blogs. I'm defending Kate not because she's a friend but because I think her well-reasoned, deliberative post was met with unwarranted hostility -- and because this sort of hostility has, in my view, consistently inhibited honest discussion of Clark's case.

The discussion on the listserv was initiated with the desire to spur the Society of American Archivists to respond to the situation, and to do so in a very specific way. It was also animated by the belief -- sometimes explicit, sometimes implicit -- that anyone who did not immediately take to the streets and demand that SAA do exactly as Clark and Cox desired was guilty of bad faith, bad judgment, or both.

As Terry at Beaver Archivist aptly pointed out this morning:

It is clear that the constant public pressure from a vocal and respected archivist, Richard Cox, helped move NARA to finally act in a responsible way towards Anthony Clark. The importance of this kind of public advocacy cannot be understated and there is merit to the argument that a group like SAA should have gotten into the game earlier and should not feel constrained by its ethics code from taking public positions issues like this.

But there have been other efforts working from other angles to make this happen as well. It is disingenuous to claim that these other efforts are meaningless.

Unfortunately, Cox and now, judging from his most recent statements, Clark himself seem to be of the opinion that anyone proposes a different means of achieving the desired end -- NARA's prompt disclosure of the records that Mr. Clark has requested -- is simply sucking up to NARA or seeking to sweep an inconvenient problem under the rug. They are thus alienating many people who would otherwise be outspokenly supportive of efforts to hold NARA accountable for its conduct.

It really does seem that there are a few staffers at NARA who fully deserve whatever acid criticism comes their way. However, when it comes to SAA and the archival profession as a whole, you can, as the old saying goes, catch a lot more flies with honey . . . .

Postscript, 2009-03-31: I was mulling over the NARA-FOIA-Clark-Cox-SAA situation during a few minutes of downtime this morning, and it struck me that the above post might leave the impression that I view Clark's problem as isolated and that prompt review and disclosure of the records that are the subject of his FOIA request will solve everything. From the start, I've suspected some broader changes at NARA will likely be needed. I meant to make this point in my original post, which was written at the end of an intense day (more about that tomorrow), but didn't.

Bush e-records: an update

A few days ago, I noted that the New York Times had, in my view, done less than a stellar job of explaining the National Archives and Records Administration (NARA)'s implementation of a "contingency plan" for the transfer of electronic records created by the Bush White House. I'm happy to report that Julian Sanchez, who wrote a follow up piece on the Times story for Ars Technica earlier this week, is much more astute:

As the New York Times reported this weekend, many of those eager to get their hands on the outgoing administration's treasure trove of documents—like the folks at the National Security Archive and the watchdog group Citizens for Responsibility and Ethics in Washington—are skeptical of the Archives' ability to quickly absorb a data dump that the contingency plan estimates at "50-100 times the volume of electronic materials" left behind by the Clintons at the close of the millennium . . . .

Archives spokeswoman Susan Cooper argues that these concerns are exaggerated. She calls the Times story "misleading," noting that the contingency plan approved last month concerns only two types of data: the massive White House store of digital photos and the Records Management System that serves as the master index to most of the administration's text documents.

According to Sanchez, Cooper also challenged the Times' assertion that relations between the White House and NARA, which she characterized as "'cooperative and working'" were strained. This sort of cordiality isn't really consistent with the Bush White House's past interactions with NARA or handling of records-related issues, but I'm hoping that the imminent departure of President Bush has brought about a genuine change of attitude.

Sanchez provides an good overview of the "contingency plan," efforts to recover missing White House e-mails, and NARA's role in converting, at the time of transfer, all records created in proprietary formats (i.e., formats owned and controlled by software companies) to open formats (i.e., formats for which the full technical specifications are freely available to any software developer) that facilitate long-term preservation. He also wraps up with one of the smartest-- and most succinct -- media treatment of electronic records issues I've ever encountered:

. . . The eleventh-hour data migration may . . . make it harder to spot lacunae in the records before the president and his entourage are back in Crawford clearing brush. If the next administration wants to demonstrate its commitment to open, high-tech government, it can start by preserving its own records in open formats.

Here's hoping that the Obama White House takes this piece of advice -- and, better yet, uses open formats to create records, not just preserve them.

BTW, Sanchez's story is accompanied by a really cool nighttime photo of NARA's Archives I facility in Washington, D.C.; alas, there is no photo credit.

"Futuristic computer system"?

According to the New York Times, the National Archives and Records Administration (NARA) has "has put into effect an emergency plan to handle electronic records from the Bush White House." NARA had originally planned to transfer large quantities of digital images and e-mail messages into the Electronic Records Archives (ERA) on January 20.

Archives officials who disclosed the emergency plan said it would mean that the agency would initially take over parts of the White House storage system, freezing the contents on Jan. 20. Only later, after further study, will archivists try to move the records into the futuristic computer system they have devised as a repository for digital data.

Surely the article's authors could have come up with something better than "futuristic computer system," which leads one to suspect that they didn't do their homework.

Their subsequent use of quotation marks gives added weight to this suspicion:

The archives invoked its emergency plan to deal with problems in transferring two types of electronic files: a huge collection of digital photographs and the “records management system,” which provides an index to most of the textual records generated by Mr. Bush and his staff members in the last eight years.

I don't expect that reporters will acquire detailed knowledge of archival theory or the principles of records management, and blogging has made me increasingly appreciative of the time pressures and other limitations that they face. However, a little background information about the ERA project would have improved this article immeasurably and done a real service to the Times' readership.

The article is far better at highlighting the concerns of various government watchdog groups and explaining the challenges associated with assuming custody of an unprecedentedly large body of electronic records (NARA estimates it will receive 100 TB of data) and working with an administration that has not always managed its records properly and is not being particularly cooperative.

I'm hoping that as this saga continues, journalists start picking up on some of the basics of electronic records management and digital preservation. However, I'm not holding my breath . . . .

Bush e-mails

According to the Washington Post, the transfer of Bush administration e-mail messages to the National Archives might, for a variety of reasons, go less smoothly than desired:

Federal law requires outgoing White House officials to provide the Archives copies of their records, a cache estimated at more than 300 million messages and 25,000 boxes of documents depicting some of the most sensitive policymaking of the past eight years.

But archivists are uncertain whether the transfer will include all the electronic messages sent and received by the officials, because the administration began trying only in recent months to recover from White House backup tapes hundreds of thousands of e-mails that were reported missing from readily accessible files in 2005.

The risks that the transfer may be incomplete are also pointed up by a continuing legal battle between a coalition of historians and nonprofit groups over access to Vice President Cheney's records. The coalition is contesting the administration's assertion in federal court this month that he "alone may determine what constitutes vice presidential records or personal records" and "how his records will be created, maintained, managed, and disposed," without outside challenge or judicial review.

As the Post points out, previous administrations have also experienced technical difficulties or simply didn't want to transfer some electronic records to the National Archives. However, as Robert Blanton of the National Security Archive is quoted as saying, the quantity of electronic records created by the Bush Administration is much larger than that created by past administrations; as a result, both the cost of and the amount of time needed to recover improperly stored electronic records are going to be higher than they were in the past.

Another complicating factor: some White House aides improperly used Republican National Committee e-mail accounts to conduct official business. All such e-mail messages are subject to federal records laws, even if they were created and sent using non-governmental e-mail accounts. Administration officials are apparently negotiating with the RNC to secure copies of messages that meet the legal definition of a presidential record, but at present it doesn't seem that there is any definite timeframe for doing so.

The article notes that transfers of paper records from the White House to Dallas, where the George W. Bush Presidential Library will be located, and to NARA's Electronic Records Archives (identified in the story as "a remote Navy research center in the mountains of West Virginia") have already begun

At the Navy base, all the electronic data are supposed to be "ingested" by a new electronic system meant to allow such efficient cataloguing, indexing and searching that millions of documents can eventually be provided to researchers and citizens online.

"Ingested": I wasn't expecting to find Open Archival Information System terminology in a newspaper article. Maybe all of the attention being devoted to the Electronic Records Archives and the controversies surrounding presidential electronic records will gradually popularize the term.

The article goes on to explain some of the development and budget problems that the Electronic Records Archives project has experienced, and it includes a snippet of information about its e-mail searching capability:

The system, which has been under development for a decade by Lockheed Martin and other contractors at a cost of $67.5 million, will rely on software created after the collapse of Enron, when that company's creditors demanded new tools for quickly sorting its e-mail trove to find damaging information.

I wish the National Archives folks all the best as they take in truly mammoth quantities of electronic records, and I'll be interested in seeing what happens in the coming weeks, months, and years.

Archivist of the United States resigns

This afternoon, a couple of my colleagues received e-mails stating that Dr. Allen Weinstein had resigned, and now it's official:

Washington, DC…On December 7, historian Allen Weinstein, Archivist of the United States, submitted his resignation to the President, effective December 19, 2008. Professor Weinstein, who has Parkinson’s disease, cited health reasons for his decision.

Deputy Archivist of the United States, Adrienne Thomas, will serve as Acting Archivist until a new Archivist is appointed, in accordance with the National Archives governing statute, 44 USC 2103(c).

The circumstances surrounding Dr. Weinstein's nomination engendered quite a bit of controversy, but he seemed to take his responsibilities very seriously and made a real effort to reach out to the archival community. Shortly before his confirmation (if memory serves me correctly), he visited the New York State Archives and met informally with staff for about an hour. He seemed genuinely interested in learning about what we were doing, and he was shocked to learn that none of his predecessors had ever paid us a visit.

It's sad to learn that Dr. Weinstein is leaving because of ill health, not the desire to return to academic research or simply ease into a quieter life, and I wish him all the best.

Given the timing of Dr. Weinstein's resignation, I suspect that the next Archivist of the United States will be an Obama appointee -- and that the archival profession as a whole will be avidly interested in the selection and nomination process.