The Hudson River, as seen from Walkway Over the Hudson State Historic Park, Poughkeepsie, New York, 4 June 2010.This post concerns something that happened 11 days ago, which I suppose makes me a full-fledged slow blogger . . . .
I noted a little while ago that I thought that Session 1, which focused on the Archivists’ Toolkit, was the highlight of the recent joint meeting of the New York Archives Conference/Archivists Roundtable of Metropolitan New York. However, Session 7, Management and Preservation of E-mail, was a very, very close second.
Nancy Adgent of the Rockefeller Archive Center (RAC) got things rolling by discussing one of my favorite preservation projects, the Collaborative Electronic Records Project (CERP) undertaken by the RAC and the Smithsonian Institution Archives (SIA).
Along the way, they learned a host of lessons:
- Transfers from active systems go most smoothly when an archivist and an IT person work together.
- Minor problems will arise. Some attachments that should have accompanied messages were missing, most likely because they were stored in a central server and the messages were copied from individual users’ desktops. The dates of some individual messages were replaced by the date they were bundled into a .pst file, and the name of the author was sometimes changed to that of the archivist who examined the file. Most strikingly, the installation of new GIS software on testbed equipment changed the display of some message fonts.
- E-mail requires some processing before it’s converted to XML. The CERP team used a variety of off-the-shelf tools in order to do so.
- Different anti-virus tools sometimes yield different results. The CERP team used Kaspersky and Symantec, each of which detected a few viruses that the other didn’t find.
- Searching file names is not a foolproof means of identifying “sensitive” materials. Although both repositories conducted such searches and either removed sensitive information from access copies or documented its existence in finding aids and metadata, they realized that some sensitive information was still lurking in the messages.
Prompted by recent changes in the Federal Rules of Civil Procedure, ORPS decided that staff had to take responsibility for their own information. It first reviewed, updated, and consolidated its e-mail, the Internet, and IT equipment use policies so that they harmonized with the state’s information security policy, ethics law, and relevant executive orders. It then amended its e-mail retention policy, which now mandates that users who want to keep their e-mail longer than 120 days must move it into a centralized archive.
ORPS already owned EMC’s DiskXtender archiving product and purchased the EmailXtender (now SourceOne) component (which the Obama White House also uses) for e-mail archiving. Instead of keeping every message, the agency opted to create folders based on length of retention period and rely upon staff to file messages appropriately. Folder access is customized by unit, so units that create records with longer retention periods can manage them properly and those that don’t can’t keep records forever.
After the system was installed, ORPS began providing training and guidance to staff. Staff had eight weeks to manage their existing e-mail, and Lotus Notes’ save-to-local-drive option was disabled. Project leaders sent out reminders to staff and stressed that managers were responsible for ensuring that unit records were managed properly and that unit staff knew how to do so.
Paul identified a number of key success factors:
- Backing of senior management. (In a brief conversation we had after the session, he indicated that senior managers’ support was the single most important factor. If only figuring out how to secure the backing of senior management were as simple.)
- Policy development preceded implementation. Business needs should drive IT investment, not the other way around.
- Staff educated themselves via State Archives workshops and discussions with other agencies.
- Availability of funding
- Records management liaisons served as a test group, which facilitated identification of problems and prepared the liaisons to handle questions from other staff
- All tutorials, videos, and communications relating to the project were placed online
- Turning a “save everything” organization into an organization that manages its information requires a lot of effort
- Some people saw e-mail management as a distraction from their “real” work
- Everyone wanted more time to review and sort messages
- Upper managers retired, resulting in loss of momentum
- Networking staff had other responsibilities thrust upon them
- People save e-mail for easy reference, and don’t necessarily think of it as a record
- Don’t assume people are paying attention. Despite repeated warnings and reminders, one person did not review and organize his/her messages and as a result lost all of them.
- Elicit concerns, and do so upfront if at all possible (As Fynette Eaton has pointed out, this is a key principle of change management)
- Weigh overhead against policy. On several occasions, ORPS had to tweak its policies because they were placing an undue burden on network personnel.
- Legal risks and discovery. New products can provide consistent, rapid search across email archives, apply litigation holds by message (and apply multiple holds to a given message), and manage e-discovery cases so that teams can access only those messages responsive to the case they’re working on.
- Expense. New products can apply retention schedules, streamline costs via outsourcing storage to a cloud environment (my take: the cloud isn’t ready for state and local government records), and consolidate archiving, business continuity, security, anti-virus, etc. functions into a single product.
- Data loss. Technology can provide a tight, documented chain of custody, capture complete delivery information, and consolidate or eliminate message files stored on individual users’ hard drives.
- Privacy. Software can now block or quarantine e-mail that contains prohibited or suspect content (e.g., Social Security Numbers) and provide role-based access to e-mail (whole message, metadata only, no access)
- Productivity. New products offer “continuity” features that minimize e-mail outages, eliminate e-mail quotas, and automate application of retention policies.