How the White House Archives Its E-mail

Last month, the National Security Archive (NSA) and Citizens for Responsibility and Ethics in Washington (CREW) settled their 2007 lawsuit against the Executive Office of the President (EOP) and the U.S. National Archives and Records Administration (NARA). In keeping with the terms of the settlement, earlier today EOP conveyed to CREW and NSA a letter outlining its e-mail archiving and backup practices. A few hours ago, CREW posted a copy of the letter and supporting information on its Web site.

So how's EOP doing? From the looks of it, a pretty good job:

• Since 20 January 2009, it has been using EMC's EmailXtender (now EMC SourceOne E-mail Management for Microsoft Exchange) to capture copies of all messages sent or received via its unclassified e-mail network. The EmailExtender system, which is centrally managed and housed in a secure offsite location, captures messages from EOP's central Microsoft Exchange Journal Servers (EOP's using Microsoft 2000 and will soon upgrade to Exchange 2010) immediately after they are sent or received by an EOP desktop computer or BlackBerry.
  
• EOP network operations staff continuously monitor the status and storage capacity of the system via "health-check dashboard reports."
• The system produces full backups on the second Tuesday of each month and incremental backups on every Monday, Wednesday, Friday, and Sunday.
• Although some users of the e-mail network can search the EmailXtender system and view archived messages, they can search and view only those messages created by their own offices and do not have the ability to alter or delete messages.
  
• Only a select handful of people have the ability to delete messages from the system, and only those messages that were subsequently found to contain classified information are deleted. EOP seems quite serious about preventing inappropriate deletions: messages are deleted only after the Office of Security and Emergency Preparedness and the National Security Council have been consulted and EOP's Office of the CIO, Office of the General Counsel, and the Director of Office Administration have granted permission. Moreover, record copies of deleted messages and records documenting adherence to the deletion protocols are maintained separately
  
• The system produces weekly audit reports that identify individuals who conducted searches, the search terms they used, and whether they opened any messages in connection with their searches. The audit reports also document the deletion of messages, thus ensuring that unauthorized deletions will not go undetected.
  

From an archival/records management perspective, there's a lot to like about EOP's approach:

• Mindful of some of the problems that confronted the previous administration, EOP has configured its e-mail network so that access to "all known Web based external e-mail systems" is blocked and neither the e-mail network nor EOP-issued BlackBerries can access "known instant messaging systems. Of course, secrecy-minded White House personnel could conduct official business via personal cell phones or PDA's -- and I would like to know how EOP is combating this practice -- but EOP seems to be doing whatever it can to ensure that its own hardware is locked down.
  
• The EmailXtender system somehow determines whether a given message is subject to the Presidential Records Act or the Federal Records Act, which ought to make it a lot easier for NARA staff to manage these records after transfer. I wouldn't mind knowing more about this neat trick, which is probably based on analysis of the message's content, the account holder's role within EOP, or some combination of the two.
  
• Although the EmailXtender system stores the messages and their attachments in their native formats, they can be extracted in .eml (Microsoft Outlook Express Electronic Mail) format for transfer to NARA. I would be happier if the messages could be exported in some sort of optimal preservation format, but the archival profession is just starting to figure out precisely what an optimal e-mail preservation format would look like. If, as EOP's letter implies, NARA can take in .eml files and convert them to a preservation format, .eml is okay.
  

This sounds all well and good, but a caveat is in order: we almost certainly don't have the whole picture. The very first sentence of EOP's letter indicates that the system and protocols described above relate only to "unclassified White House e-mails," which suggests that there is a separate system and set of protocols for messages containing classified information. Getting information about the workings of the classified system -- which I really hope isn't based on Exchange or any other off-the-shelf application -- is doubtless going to be a lot more difficult, and there is legitimate reason for not disclosing at least some of it. The care with which EOP is handling deletions from the unclassified system suggests that there is a substantial degree of commitment to doing the right thing, but we may not know with certainty just how well the Obama administration is managing classified e-mails until well after President Obama leaves office.

"The Archivist of the United States Googled me . . . ."

Earlier today, National Public Radio's Weekend Edition program aired Liane Hansen's lengthy profile of David Ferreiro, the new Archivist of the United States, and the U.S. National Archives and Records Administration. It's a great piece, and it opens with news that Ferriero prepared for the interview by doing a little online research about Hansen and NPR and arranging to show Hansen records relating to her past work -- including a letter documenting President Eisenhower's opinions about Spam (the meat product nuisance, not the electronic communication nuisance).

Apart from a few minor errors (e.g., Hansen implies that "vault" items, most of which can be seen by researchers who document their need to see them and request access in advance, are completely off-limits to the public), Hansen avoids most of the journalistic clichés about archives. There's not a single reference to dust, and Hansen's description of the smell of a properly equipped and maintained repository -- "old paper and clean air" -- is elegantly simple and evocative.

Hansen's profile doesn't tackle the subject of electronic records, which Ferriero identified as one of his biggest concerns in a recent Federal Computer Week interview. However, it does emphasize one of his other top priorities: the digitization of NARA's non-electronic holdings. (Fortunately, it also stresses that digitizing each and every one of the 10 billion paper- and analog media-based items won't happen overnight.) It also stresses Ferriero's enduring passion for reference work, which will likely shape NARA's activities under his tenure.

My description of Hansen's profile is no substitute for actually listening to it. It's great publicity for Ferriero, NARA, and archives in general, and I wish that all media coverage relating to archives were this informative, entertaining, and perceptive.

American Falls, Niagara Falls, from below, ca. 1900. New York State Education Dept., Division of Visual Instruction, Instructional lantern slides, 1911-1925, A3045-78, Lantern slide D47_NiG81, Box 13. Image courtesy of the New York State Archives -- and available via the New York State Archives' Flickr photostream.

Apologies for the light posting as of late. The unrelenting cold afflicting the eastern half of the United States has sapped my initiative and soured my mood, and I've become a bit of a hermit.

Winter in New York State can be really beautiful, and I'm yearning for the temperatures to rise a bit and the sunshine to return so that I can get out and enjoy it. In the meantime, I'm reading Archives Power -- Kate Theimer's online discussion group gets underway on 11 January -- and getting ready to start blogging again. Look for more posts during the coming days . . . even if the cold sticks around.

A little end-of-year electronic records housekeeping . . . .

Governor Nelson A. Rockefeller and First Lady Margaretta "Happy" Rockefeller with guests on New Year's Eve, 31 December 1970. At noon on this day, Governor Rockefeller was sworn in as governor for the fourth and final time. New York (State). Governor. Public information photographs, 1910-1992. Series 13703-83, Box 3, Number 4408_23. Image courtesy of the New York State Archives.

I'm planning to devote part of the long weekend to tackling some long-neglected domestic chores, and I'm going to start by tidying up some loose ends on this blog. Owing to a combination of time pressures and a slow-to-heal (but steadily improving) injury, I let a few significant electronic records developments slide by without comments. All of them will remain relevant in 2010, so this is a good time to draw attention to them.

• In October, the Arizona Supreme Court ruled (Lake v. City of Phoenix) "that if a public entity maintains a public record in an electronic format, then the electronic version, including any embedded metadata, is subject to disclosure under [the state's] public records laws." This ruling may seem a bit obvious to any archivist or records manager, but it's actually quite significant: earlier this month, an attorney who works for New York State's Committee on Open Government noted that, until now, neither case law nor legislation has really specified whether metadata is covered by state and federal freedom of information laws. As a result, Washington State's Supreme Court, which is set to hear a similar case, and courts in other jurisdictions will likely devote a lot of attention to this ruling. If you're curious about some of the potential implications of this ruling, check out what Ars Technica and Inside Counsel have to say about it.
  

• Since 2003, ARMA and Cohasset Associates (and, sometimes, AIIM) have conducted annual surveys of electronic records management practices. The results of the 2009 survey were released in October, and as you might expect, the results are a mix of good and bad -- very bad -- news: organizations are starting to take action to correct their electronic records and information management problems, but most of them still have a long, long way to go before all of their problems are solved. Moreover, today's electronic records and information management shortcomings are so severe that they may well "jeopardize the future reliability, availability, and trustworthiness of many records. " If you want to figure out how your organization's policies and practices compare to those of others or have any sort of interest in electronic records management, the report (executive summary here, full text here) is an interesting, sobering read.
  

• AIIM regularly offers free Webinars focusing on records, content, and business process management, and archived Webinars are available via its Web site; registration is required. Recently archived Webinars focus on the current state of electronic records management, determining responsibility for records/content management, managing government content in the cloud, and other topics relating to electronic records management. If you're finding it harder and harder to get permission to travel to conferences or training sessions or simply want to keep pace with new developments, you might want to check out these Webinars.

• The New York State Historical Records Advisory Board recently launched a new Web resource, 9/11 Memory and History, that is designed to help survivors and people who lost friends or family on 11 September 2001 preserve photos or letters, drawings or paintings, scrapbooks, sound or video recordings, computer files or digital images, articles of clothing, or other objects. In addition to text-based instructions, the site also includes a number of short videos. Hofstra University archivist Geri Solomon and family member Margie Miller's discussion of what to save and how to save, and staff from the New York State Archives discuss other preservation-related concerns: Director of Operations Kathleen Roe talks about donating materials to a repository, Paper Conservator Sue Bove details how to care for photographs, drawings, and newspapers, and Electronic Records Archivist Bonita Weddle (i.e., Yours Truly) discusses preservation of digital files. Although this site is really targeted to the 9/11 community, other people interested in preserving family history materials and other personal materials should also find it useful.

• Andrew Sniderman has written a reflective, thought-provoking piece about the psychic cost of GMail and other services that unwittingly lead users to document their lives more fully than ever before: digital archives of e-mails, texts, etc., may make it harder for users to deceive themselves about their motives and actions, but they also make it easier for them to fixate on old wounds and regrets. An ever-growing number of people will no doubt agree with Sniderman's assertion that "preservation gives the past more weight than it sometimes deserves," and many professional archivists will no doubt regard their own personal digital archives with at least some ambivalence. However, as Cal Lee pointed out at SAA earlier this year, archivists are ethically obligated to furnish guidance to people who are struggling to care for their personal digital records, so we really should start thinking about what we'll say to friends, relatives, prospective donors, and others who come to us for help.
  

• Finally, some fun stuff. Building upon the success of its first animated short, Digital Preservation and Nuclear Disaster: An Animation, the good people at Digital Preservation Europe have produced two additional videos. Enjoy, and have a Happy New Year!

A Christmas Carol: crowdsourcing analysis of the mss.

Christmas tree being cut down at Cherry Plain, New York, for President Roosevelt and Governor Lehman, December 1934. 57L275REF. Image courtesy of the New York State Archives.

A little while ago, the New York Times worked with the Morgan Library and Museum, which owns the original manuscript copy of Charles Dickens's A Christmas Carol, to digitize the manuscript and invited its readers to identify the changes Dickens made while writing and revising it. Over 100 Times readers rose to the challenge, and their findings are featured today. The most interesting finding concerns Tiny Tim's name . . . .

Lots of repositories have manuscripts or records that would lend themselves to this sort of crowdsourcing analysis. However, right now we should probably focus on our families and friends. Whatever you're celebrating or not celebrating, I hope that you're with people you love and that you're having a great time.

Reading Archives reading group

Kate T. at ArchivesNext is a font of superb ideas about archival use of Web 2.0 technology, and her latest initiative is an online "group read" of Rand Jimerson's Archives Power: Memory, Accountability, and Social Justice. Kate's started a new blog, Reading Archives Power, that will serve as the vehicle for the group's discussion, and so far approximately 35 people -- a multinational mixture of new graduates and seasoned professionals -- have publicly expressed interest in participating. Rand Jimerson will also participate, so the discussion ought to be really stimulating.

There is no formal signup for the group, but if you are interested -- and I hope you are -- you will need to obtain a copy of Archives Power and start reading it before the discussion begins on 11 January 2010. (Tip from Kate T.: the Society of American Archivists bookstore is charging a lot less than Amazon and other retailers.) Although it's not absolutely necessary, you should also consider introducing yourself to the other members of the group and reviewing Kate's proposed discussion schedule.

I'm really looking forward to this discussion, and I hope to see you over at Reading Archives Power.

Bush White House e-mail settlement

News that 22 million lost e-mail messages sent or received by the Bush White House have been recovered and that the National Security Archive (NSA) and Citizens for Responsibility and Ethics in Washington (CREW) have settled their 2007 lawsuit against the Executive Office of the President (EOP) has been all over the media for the past couple of days.

A lot of the media coverage is focusing on a couple of items in the settlement document. First, for reasons of cost, the White House will focus on recovering e-mails sent or received on select days, not every "missing" e-mail that the Bush White House created. Second, the U.S. National Archives and Records Administration (NARA) will take custody of the e-mails and manage them in accordance with the Presidential Records Act, which means that the e-mails won't be disclosed to researchers for years.

However, a quick review of the settlement document itself reveals that, with a handful of exceptions, the media isn't calling attention to a provision that ought to interest anyone who wants to know how the White House does business or how EOP manages its electronic records:

4. Description of Current EOP System: Defendants [EOP and NARA] will provide Plaintiffs [NSA and CREW] with a publicly releasable letter describing in as much detail as possible the current EOP computer system, including its email archiving and backup systems. This document will include a detailed description of the controls in the system that prevent the unauthorized deletion of records.

a. Prior to sending the letter, Defendants will review with Plaintiffs draft(s) of the letter and the Parties will agree upon a final version.

b. Defendants recognize that Plaintiffs intend to release the letter publicly, and Defendants do not object to such a release.

c. Defendants will produce this letter by January 15, 2010.

Although I'm sorry that the EOP and NARA personnel charged with producing this document will likely have to curtail their holiday breaks, I'm looking forward to the end result. It should make for interesting reading.