John Kitzhaber's e-mail

It's been quite a week in re: gubernatorial e-mail.

Yesterday, Oregon's governor, John Kitzhaber, announced that he would resign from office. Last October, an Oregon newspaper reported that the governor's fiancee, Cylvia Hayes, had served as an unpaid energy and economic policymaker at the same time as she was running a private green-energy consulting business. In November, the Government Ethics Commission opened an investigation into Kitzhaber and Hayes. Last week, the state's attorney general announced that a criminal investigation was underway.

Earlier this week, the Willamette Week, the Portland alternative paper that broke the news regarding  Hayes' potential conflicts of interest, reported that a Kitzhaber staffer had requested that the state Department of Administrative Services delete all e-mails from Kitzhaber's personal e-mail accounts that were stored on departmental servers. Kitzhaber's camp maintained that the request covered personal messages that had mistakenly been auto-forwarded to state servers and would not result in the destruction of public records. However, Oregon law specifies that personal e-mail messages that discuss government business are considered public records, and Department of Administrative Services staff were keenly aware that multiple media organizations had filed freedom of information requests that included Kitzhaber's and Hayes's e-mail. They sent the administration's request and their concerns about it up the department's chain of command and the department's director decided that the e-mail should not be deleted.

A few hours after Governor Kitzhaber resigned, U.S. Attorney Amanda Marshall revealed that a federal grand jury investigation was underway and issued a sweeping subpoena seeking e-mail, memoranda, and other records that documenting Kitzhaber's environmental and economic policy initiatives, and Hayes' state government work, consulting business and clients, personal and corporate tax returns, and use of state credit cards. The subpoena covers eleven state government agencies and includes records that the state's Justice Department and Government Ethics Commission created or collected during the course of their investigations. The Federal Bureau of Investigation, which never publicly comments upon investigations in progress, also seems to have taken an interest in Hayes's affairs.

What a mess. I imagine that Salem, Oregon is currently experiencing the sort of surreal standstill that Albany, New York experienced in March 2008, but things are going to start moving again, and very quickly. Eleven state agencies will have to devote a lot of time and effort to responding to the federal subpoena and a host of freedom of information requests. The State Archives must scramble to document the administration of a governor who was re-elected a few months ago, who left office with little advance notice, and whose records are of abiding interest to the feds. Moreover, it must do so as it loses the head of its parent agency: when Kitzhaber's resignation takes effect next Wednesday, Secretary of State Kate Brown will become the state's next governor. However, given the clear-eyed, resolute manner in which the Department of Administrative Services responded to the Kitzhaber administration's e-mail deletion request, the Oregon State Archives' recent history of innovation and effectiveness, and Oregon's tradition of (relatively) clean governance, I suspect that these challenges will be met head-on.

Jeb Bush's e-mail, continued

Earlier this week, Jeb Bush made available online hundreds of thousands of emails he sent and received during his tenure as Florida's governor. As I noted yesterday, the emails Bush's organization placed online contained Social Security numbers, home addresses and phone numbers, and a wealth of other personal information about private citizens. In the wake of the controversy, the Bush camp pledged to review and redact the e-mails, which are identical to the unredacted e-mails held and made accessible to researchers by the Florida State Archives.

Earlier today, Fortune reported that the e-mails approximately 13,000 Social Security numbers and that roughly 12,500 of these numbers were housed within a spreadsheet embedded within a PowerPoint presentation attached to a message that Governor Bush and approximately 50 other people received in October 2003. The other 500 are scattered throughout the correspondence. The Bush team has been able to use software to identify and redact approximately 400 of them, but as of earlier today approximately 100 were still available online because they don’t conform to the usual XXX-XX-XXXX pattern and thus can’t be easily found.

Fortune also reported that a spokesperson for the Florida Department of State, of which the Florida State Archives is part, stated that: “the Department of State is currently reviewing our process for redacting confidential information from documents given to the State Archives.” Ouch.

To add insult to injury, ComputerWorld notes that the Microsoft Personal Storage Table (PST) versions of the Bush e-mails that the Florida State Archives disclosed to researchers and that were, for a short time, made available for downloading on the Bush e-mail site contain a number of old viruses and Trojan Horse applications. Most of them pose little threat to anyone who has a newer computer and up-to-date anti-virus software, but they might cause problems for people who have older machines or don't have anti-virus software installed.

Jeb Bush's e-mail

On Monday, former Florida governor Jeb Bush placed online copies of hundreds of thousands of e-mails he sent and received while in office. Bush is actively exploring the possibility of running for President and has stated that he released the messages to show his commitment to transparency and his embrace of information technology; many political observers have concluded that the release is also meant to prove that he's a dedicated, responsive, and effective executive. Things did not go quite as planned, and the resulting uproar ought to be of interest to any government archivist who might accession electronic records that contain legally restricted information, respond to FOI requests for born-digital or digitized records, or confront the sweeping records requests that invariably occur whenever a former official seeks higher office.

As soon as the e-mails were released, tech journalists and bloggers began exploring the search interface that Bush's staff created and the contents of the messages their searches yielded. They found thousands of Social Security numbers, home addresses, and tons of other personal data that had not been redacted. The Verge, Ars Technica, Buzzfeed, and a host of other media outlets quickly redacted and published copies of numerous e-mails that contained such information, and Bush and his staff quickly promised that they would remove Social Security numbers and other personal data. However, the e-mails – in searchable database form as well as downloadable Microsoft Personal Storage Table (PST) files – were freely available online for almost a day before the Bush team decided to take action.

Bush and his staff were also quick to point fingers. Yesterday, Bush told reporters in Tallahassee that the messages were public records held by the Florida State Archives (which is part of the state's Department of State) and that he and his staff had merely "released what the government gave us." The Bush team also revealed that in May 2014, an attorney representing Bush sent a letter to an unidentified state official asserting that the state was responsible for redacting any legally restricted information found within the e-mails:

We hope these emails will be available permanently to the public, provided the records are first reviewed by state officials in accordance with Florida Statute to ensure information exempt from public disclosure is redacted before release, including social security numbers of Florida citizens who contacted Governor Bush for assistance; personal identifying information related to victims of crime or abuse; confidential law enforcement intelligence; and other information made confidential or exempt by applicable law.

The Florida State Archives holds 26.2 gigabytes of Bush's gubernatorial e-mail, and the catalog record describing the correspondence indicates that the records consist of "PST files" that "must be loaded onto user's hard drive and opened using MS Outlook software." The catalog record makes no mention of access restrictions, and unredacted copies of the files have evidently been disclosed to other researchers. Yesterday, National Public Radio (NPR) reported that many of the e-mails Bush released on Monday had first been disclosed to reporters shortly after they were created or received and that several media organizations, NPR among them, had previously obtained copies of the full set from the Florida State Archives.

At this point in time, I am not going to second-guess or condemn the Florida State Archives. I simply don't know enough about Florida's Sunshine Law, which is more expansive than many other state freedom of information laws, or the Florida State Archives' disclosure protocols to come to any sort of informed conclusion. I do know that the Sunshine Law for the most part bars the disclosure of Social Security numbers, but many freedom of information laws mandate that previously disclosed information cannot be withheld for any reason; given that many of these e-mails had been disclosed to reporters while Bush was in office, the Florida State Archives might have no choice but to release them without redacting them. To date, no one from the Florida State Archives or Florida Department of State has commented upon this matter, but I hope that some sort of explanation will eventually be made.

I am more willing to second-guess Jeb Bush and his associates. As the Miami Herald has pointed out, the May 2014 letter written by Bush's attorney strongly suggests that Bush has been seriously thinking about running for president for quite some time. To my way of thinking, it also suggests that Bush or, at the very least, his lawyers knew that the e-mail contained legally restricted information, decided that the State of Florida was solely responsible for redacting it prior to disclosure, and figured that it was ethically okay to make information that Florida couldn’t or wouldn’t redact a lot easier to find. Requesting a PST file from the Florida State Archives and importing it into Microsoft Outlook doesn’t require a ton of effort or technical know-how, but at least some of the people who are now idly rummaging through the searchable Web database of e-mails created by the Bush campaign probably wouldn’t feel the need to make the effort. Manual redaction and review of e-mail is a pain – trust me on this – but there are numerous tools that will flag and facilitate redaction of Social Security numbers, telephone numbers, and other consistently formatted data. Why didn't the Bush camp make even a modest attempt to weed out the Social Security numbers?

Finally, I must be a bit skeptical about the Bush camp's claims of transparency: the Tampa Bay Times recently reported that Bush used a private e-mail account to conduct all state business and transferred only some of the messages associated with this account to the archives when he left office. Specifically, all messages relating to “politics, fundraising, and personal matters” were removed prior to transfer. I have no problem with purging messages relating to purely personal matters, but the removal of messages relating to political affairs and fundraising efforts raises a few questions in my mind. How were these messages identified? Were they identified as they were sent or received, or was there a massive end-of-term review effort? If the latter, who was involved in the review and what criteria were employed? And, of course, why didn't Bush use a state government e-mail account to conduct state business?

CoSA SERI PERTTS Portal

If you know what the above means, feel free to skip this post. If you don't, here's an explanation:

• "CoSA" is the Council of State Archivists.
• "SERI" is CoSA's State Electronic Records Initiative, which seeks to improve the management of state government electronic records and is funded by the Institute for Museum and Library Services and the National Historic Records and Publications Commission.
• "PERTTS" refers to -- look at the image.
• "Portal" refers to a new, Web-based collection of resources that will be of interest not only to state government archivists and records managers but also to other archivists who work with electronic records.

This afternoon, I attended the first half of a two-part CoSA workshop focusing on the new portal, which was developed by the SERI Best Practices and Tools Subcommittee. I've been aware of its development -- I'm a member of the SERI Education Subcommitee, which has developed some content for it --but I haven't had the chance to check it out until today. It's still something of a beta build and will be expanded considerably in the coming months, but it already contains a wealth of information:

• Information about CoSA's electronic records webinars, including a schedule of upcoming sessions and links to recordings and slides from past webinars. 
• Handouts and slides prepared by instructors of the July 2013 SERI Introductory Electronic Records Institute: Mike Wash (U.S. National Archives and Records Administration), Doug Robinson (National Association of State Chief Information Officers), Pat Franks (San Jose State University), and Cal Lee (University of North Carolina - Chapel Hill).
• How-to guides and short videos that explain how to complete various processes or use specific tools. Areas covered: file authentication and integrity processes, detection of duplicate files, file format conversions, identifying file properties, renaming files, and ingest/accessioning processes.
• Links to electronic records training opportunities offered by other organizations.
• Information about the State Electronic Records Program Framework, which is based upon the Digital Preservation Capability Maturity Model and enables state archives (and anyone else interested in doing so) to assess their preservation infrastructure and identify areas for improvement. If you're employed by a state archives and took the SERI self-assessment, you'll be particularly interested in the portal's discussion of the tangible steps needed to advance from Level 0 to Level 4 within each of the framework's 15 components and in its practical tips for completing the self-assessment the next time it's offered.
• An ever-expanding and keyword searchable database of summary information about and links to resources relating to virtually every aspect of electronic records management and preservation. If you create a free PERTTS portal account, you'll be able to comment upon these resources; if you would prefer not to create an account, you'll still be able to access them. CoSA will also develop a simple form that will enable you to suggest resources that should be added to the portal.
• An electronic records glossary that draws from a wide array of sources.
• Brief case studies and examples of real-world implementations of metadata standards, security protocols, Archival Information Package construction, and other facets of electronic records work.

This is a great resource, and I think it's going to expand and evolve in some really interesting ways. Check it out.

Best Practices Exchange, day two

Yesterday was the second day of the 2012 Best Practices Exchange, and the sessions I attended were delightfully heavy on discussion and information sharing. I had some problems accessing the hotel's wifi last night and the BPE is still going on, so I'm going to post a few of yesterday's highlights before turning my attention back to this morning's discussion:

• Arian Ravanbakhsh, whose morning plenary speech focused on the Presidential Memorandum - Managing Government Records and U.S. National Archives and Records Administration (NARA) efforts to implement it, made an important point that all too often gets overlooked: even though an ever-increasing percentage of records created government agencies are born-digital, government archivists will continue to accession paper records well into the future. Substantial quantities of paper federal records have extremely long retention periods and won't be transferred to NARA until the mid-21st century, and, judging from the nodding heads in the audience, most state government archivists (l'Archivista included) anticipate that they'll continue to take in paper records for at least several more decades. Sometimes, I forget that we as a profession will have to ensure that at least two future generations of archivists have the knowledge and skills needed to accession, describe, and provide access to paper records. At the moment, finding new archivists who have the requisite interest and ability isn't much of a challenge -- sadly, archival education programs still attract significant numbers of students who don't want to work with electronic records -- but things might be quite different in 2030.

• Butch Lazorchak of the Library of Congress highlighted a forthcoming grant opportunity: the Federal Geographic Data Committee, which is responsible for coordinating geospatial data gathering across the federal government and coordinates with state and local governments to assemble a comprehensive body of geospatial data, plans to offer geospatial archiving business planning grants in fiscal year 2013. The formal announcement should be released within a few weeks.

• Butch also highlighted a couple of tools about which I was aware but haven't really examined closely: the GeoMAPP Geoarchiving Business Planning Toolkit, which can easily be adapted to support business planning for preservation of other types of digital content, the GeoMAPP Geoarchiving Self-Assessment, which lends itself to similar modifications, and the National Digital Stewardship Alliance's Digital Preservation in a Box, a collection of resources that support teaching and self-directed learning about digital preservation.

• This BPE has seen a lot of discussion about the importance and difficulty of cultivating solid relationships with CIOs, and this morning one state archivist made what I think is an essential point: when talking to CIOs, archivists really need to emphasize the value added by records management and digital preservation. As a rule, we simply haven't done so.

• This BPE has also generated a lot of ideas about how to support states that have yet to establish electronic records programs, and in the coming months you'll see the Council of State Archivists' State Electronic Records Initiative start turning these ideas into action. As a particularly lively discussion was unfolding this morning, it struck me that most of the people taking part in established full-fledged programs only after they had completed several successful projects; in fact, intense discussions about the challenges associated with transforming projects into programs took place at several early BPEs. If you don't have any hands-on electronic records experience and are facing resource constraints, it makes sense to identify a pressing but manageable problem, figure out how to solve it, and then move on to a few bigger, more complex projects.  After you've accumulated a few successes and learned from a few surprises or failures, you can focus on establishing a full-fledged program.

Image:  storm drain marker on Randall Street at City Dock, Annapolis, Maryland, 3 December 2012.

Future of the Georgia Archives is still uncertain

Today is the first day of American Archives Month, and the news out of Georgia remains deeply worrisome.  Two weeks ago, Georgia Secretary of State Brian Kemp announced that, in response to a directive to cut his agency's expenditures by 3 percent, the Georgia Archives will be closed to the public effective 1 November.  Shortly afterward, news that Kemp was planning to lay off 7 of the Georgia Archives' 10 employees ricocheted around various listservs and social media sites; appalling as this news is, it takes on added resonance when one remembers that that as recently as 2006, the Georgia Archives had a staff of 47.

A few days after Secretary Kemp made this announcement, Governor Nathan Deal raised the hopes of family historians, scholars, attorneys, land surveyors, scientists, and other users of the Georgia Archives when he stated that the Georgia Archives would remain open.  However, in Georgia the office of the Secretary of State and the office of Governor are constitutionally separate, and Governor Deal doesn't have the authority to rescind layoff notices issued by the Secretary of State or to dictate how the Secretary of State expends its allocated funds.  Secretary Kemp opted to make the Georgia Archives bear the entire brunt of the 3 percent budget cut; other divisions overseen by the Secretary of State will continue to operate as they did before the budget cut was announced.

Georgia's legislature is ultimately responsible for approving the proposed budget cuts, but the next legislative session won't begin until January -- well after the layoffs go into effect.

As of 1 November, anyone seeking access to the holdings of the Georgia Archives will have to make an appointment in advance -- and might not be able to secure one in a timely manner.  At the time of this writing, the Web site of the Secretary of State indicates -- unsurprisingly -- that "the number of appointments could be limited based on the schedule of the remaining employees."

What a wretched state of affairs.  As noted above, the holdings of the Georgia Archives are used by a wide array of users.  Some are interested in the histories of their families.  Some are scientists trying to figure out how to best to reintroduce the American chestnut in Georgia.  Some are local historians conducting research in connection with the Civil War sesquicentennial -- and helping to pave the way for an influx of tourist dollars into the state's coffers.  Some are lawyers -- in many instances representing the State of Georgia or one of its local governments -- preparing for trial.  If they can't obtain the records they need in a timely manner, the state and its localities will likely be forced to enter into costly out of court settlements even if they are squarely in the right.

Moreover, reducing the staff of the Georgia Archives to a mere three people will have devastating effects.  There's simply no way three people can respond to more than a handful of inquiries at any given time.  Moreover, the Georgia Archives, which has already suffered crippling hemorrhages of expertise, is going to lose even more of its institutional knowledge.  We archivists struggle mightily to pour our knowledge of our holdings into finding aids, MARC records, and accession files, but as yet there's no substitute for deep familiarity with the content and quirks of one's own holdings.  This familiarity comes slowly and once lost it's incredibly hard to reconstruct. 

Finally, one can't help but wonder about the long-term effects of the closure and layoffs upon Georgia's historical record.  When people think of historical records, they think of linen or cotton paper bearing elaborate handwriting, ornately bound record books, and manuscript maps.  They don't think of the masses of paper records created ten years ago or the ever-increasing number of digital files that Georgia's state agencies and local governments create in the course of conducting the people's business.  However, some of these records are every bit as valuable as those record books and manuscript documents, and the Georgia Archives is responsible for working with records creators to identify records of enduring value, ensure that they are managed properly, and arrange for their eventual transfer to the archives.  There's simply no way that three people can simultaneously provide access to the existing holdings of the Georgia Archives, provide records management guidance to local governments and state agencies, and continue to take in new accessions of archival records and make them accessible to researchers.

(By the way, Georgia is not the only state affected by such challenges.  Last week, Kim Severson of the New York Times asserted in a must-read article that "an amalgam of recession-driven budget cuts and fast-moving technological changes could result in a black hole of [state] government information whose impact might not be understood for decades.")

In honor of American Archives Month, I encourage you to do the following:

• Sign the online petition Leave Our State Archives Open to the Public.  You do not have to be a Georgia resident to do so.
• "Like" the Facebook group Georgians Against Closing State Archives, which has been a consistently excellent source of up-to-date information about the impending closure and the struggle against it.  (It's also a great source of protest cartoons, among them the cartoon featured at the start of this post).
• Check out the Web site of Friends of Georgia Archives and History, which has been instrumental in coordinating the campaign against the Georgia Archives' closure.  (Pay particular attention to the slideshow presentation outlining the importance of the Georgia Archives -- its clarity, coherence, and visual attractiveness make it a useful model for other advocacy efforts.)
• If you're going to be in Atlanta on 3 October, attend the "Support the Archives / Save the Seven" rally that will be held in the State Capitol Rotunda at noon.
• Write letters to Governor Deal and Secretary of State Kemp or call their offices and explain why you object to the closing of the Georgia Archives.  Letters and phone calls still mean a lot to politicians.  If you need some help composing your letter or preparing your comments, be sure to check out the action alert issued by the National Coalition for History.
• If you're a Georgia resident, call or write your state senator and assembly representative.  All of Georgia's legislators serve two-year terms, and there's an election coming up in a few weeks.  Now really is the time to make your concerns known to them.  The Society of Georgia Archivists has put together a series of handy tips for legislative contacts.

Help save the Georgia Archives!

On 13 September, Georgia Secretary of State Brian Kemp announced that the Georgia Archives will be closed to the public as of 1 November 2012.  Georgia agencies have been instructed to reduce their budget expenditures by 3 percent, and Secretary Kemp has opted to take the required cut of $750,000 entirely and only from the State Archives.  Staff reductions will be announced soon.

The Georgia Archives was among the first state archives established (1918).  It has won many awards for its programs and state-of-the-art archival facility and has been a respected leader in archives, government records programs, and research use.  It's also done some important electronic records work.  However, in recent years, the Georgia Archives has repeatedly suffered budget cuts, staffing reductions, and reductions in public hours.  At present, the repository is open to the public only two days a week.  Secretary Kemp now wants to make it virtually impossible for Georgia's citizens to access their own history.

Georgians appalled by this proposal have started an online petition protesting against these cuts and are sharing information via a new Facebook group, Georgians Against Closing State Archives.  Please sign the petition and "like" the group, but keep in mind that online activism simply isn't enough.  Elected officials pay more attention to paper letters, faxes, telephone calls, and in-person visits than to e-mail messages or online petitions.  Given the seriousness of this situation, I urge you to write, call, or visit Georgia's Governor, Lieutenant Governor, and Secretary of State and urge them to reverse this devastating decision.  If you're a resident of Georgia, you should also write, call, or visit your state Representative and your state Senator.\
When you call, write, or visit the Governor, Lieutenant Governor, Secretary of State, or legislator, ask him or her to:

• Restore a minimum of $1 million to the Georgia Archives budget.  Doing so will enable the repository to open its doors five days a week and avoid projected staff reductions.
• Reverse the Secretary of State’s proposed budget cuts to the Archives to ensure uninterrupted service to the public after 1 November.

You should also focus on a few of the points below, putting them in your own words and using your own examples -- particularly if you are a Georgia citizen:

• The Secretary of State was directed to reduce his budget expenditures by 3%.  The entire sum needed to accomplish that has been taken from the Archives budget alone and will result in the termination of all public hours.   The proposed "access by appointment…limited based on the schedule of the remaining employees" effectively denies access based on "reasonable time and place" required by Georgia law.
• Access to government records promotes government accountability and safeguards the legal rights of citizens:

• The proposed closure deprives citizens of regular and predictable access, as mandated in the Georgia Public Records and Open Records Act which states that all public records "shall be open for a personal inspection by any citizen of this state at a reasonable time and place, and those in charge of such records shall not refuse this privilege to any citizen."
• It is contrary to the practice of government transparency by depriving citizens of predictable and ready access to the records that are essential to providing evidence of government accountability.
• It deprives citizens, as well as Georgia’s own government, of access to records needed to support due process of law.   The Georgia Archives holdings have been used in a range of court cases, including land claims, boundary disputes, utility right-of-way, and claims against state agencies.
• Access to records is essential to avoid costly litigation that will result if records cannot be located or accessed.

• The proposed closure will also hamper efforts to research the history of the state and its citizens:

• As the Civil War Sesquicentennial begins, researchers need access to the historical record in the Georgia Archives to provide accurate, factual evidence of that experience.  Many of Georgia’s governmental records were destroyed during Sherman’s March.  Closing the Archives similarly deprives Georgians of access to their heritage—but this time the fault does not lie with an invading army, but with Georgia officials themselves.
• The Georgia Archives holds records actively sought by genealogists and family historians; in particular, they provide essential evidence for African-American history and genealogical research not available in many private historical collections.
• The Georgia Archives has been an essential resource for environmental research and activities, including efforts to reintroduce the American chestnut tree in the state and issues relating to pollution.
• The Georgia Archives has been the site of research for television and films, including episodes of the popular NBC series Who Do You Think You Are featuring Paula Deen and Spike Lee, as well as Emmy award-winner Ben Loeterman’s documentary The People v. Leo Frank.

Here's where to direct your letters, calls, and visits:

Governor Nathan Deal
203 State Capitol
Atlanta, GA 30334
Phone:  404-656-1776
Fax:  404-657-7332
E-mail ("Contact Us" form):   http://gov.georgia.gov/contact-us-0

Lieutenant Governor Casey Cagle
240 State Capitol
Atlanta, GA 30334
Phone:  404-656-5030
Fax:  404-656-6739

Secretary of State Brian Kemp
214 State Capitol
Atlanta, GA 30334
Phone:  404-656-2881
Fax:  404-656-0513

The Web site of the Society of Georgia Archivists provides detailed contact information for individual Representatives and Senators and legislative committee heads as well as tips for communicating effectively with them.

Georgia archivists are trying to keep track of advocacy efforts relating to the Georgia Archives, so please be sure to send copies of any letters you send or summaries of any contacts you make to Kaye Lanning Minchew [kaye-at-troupcountyarchives-dot-org] of the Troup County Archives.

The text of this post is based upon a draft action alert developed by representatives of the Society of Georgia Archivists, the Council of State Archivists, the Society of American Archivists, the National Coalition for History, and other organizations.  The image was created by Georgians Against Closing State Archives.

SAA 2012: electronic records in political collections

I spent most of yesterday afternoon contending with a migraine, so my memories of yesterday's “Share a Byte! A Practical, Collaborative Approach to Electronic Records in Modern Political Collections” session are a bit vague in spots, but I was so impressed with both presentations that I feel compelled to write about them. I only hope I can do them justice.

The first presenter, Jennifer Huebscher of the Minnesota HistoricalSociety, discussed her repository's quick-and-dirty but highly effective approach to increasing access to electronic records. She focused on the electronic records of former Governor Timothy Pawlenty, who was exploring a presidential run at the time the records were transferred, and on the records of a gubernatorial redistricting commission.

From the start, the Minnesota Historical Society's collaborative relationship with the Office of the Governor smoothed the way. The records were covered by a retention schedule that was devised during the tenure of Pawlenty's predecessor, and as a result Pawlenty's staff knew that certain types of records should be kept. Toward the end of Pawlenty's second term, his staff contacted the Minnesota Historical Society and then arranged an in-person meeting to discuss the impending transfer.

At the end of the year, the Office of the Governor placed the records on portable media and gave them to the Minnesota Historical Society. Owing to its discussions with the governors' staff, the archivists had a clear sense of what to expect, were able to compare the files it had in hand with the list of files it anticipated receiving, and were able to obtain a missing set of files from Pawlenty's staff

The files consisted of image files, sound files, and one moving image file. The image files, which consisted of digital photographs of Governor Pawlenty and the First Lady, were transferred on two DVD-R discs and consisted of 1,740 files, most of which were in JPEG format but also included some TIFF, PDF, and BMP files. Images of the Governor were placed on one disc, and images of the First Lady were placed on the other, and each disc contained nine folders – one for each year the Governor was in office. The file names ranged from descriptive to vague, and the naming conventions used for images of the Governor differed from those used for images of the First Lady.

Some of the sound files were transferred on CD and DVD, and Minnesota Historical Society harvested others from the Web using HTTrack. Most of the 410 files were in MP3 format, but others were WAV or CDA files, and one was an MP4 file.

Minnesota Historical Society's electronic records archivist copied the files onto a secure Storage Area Network maintained by the state's Enterprise Technology department, and staff continue to run checksums periodically; however, a full-fledged framework for preserving these files has yet to be developed.

Internal collaboration made the records broadly accessible. The electronic records archivist produced a set of copies that cataloging staff processed and described, and the two worked together to figure out how best to provide access to them. Owing to significant public and media interest in the files, Huebscher and her colleagues sought to apply the principles of More Product, Less Process processing. They didn't alter file names or the overall arrangement of the files unless duplication or other problems made doing so absolutely necessary, they didn't create a set of preservation masters in normalized formats, they didn't add any extra metadata, they didn't do any additional research that would have enhanced description of files that had non-descriptive or undated file names. They created the finding aids describing the records by using a simple template, extracting file names, and using matching the hierarchical arrangement of the finding aids to the hierarchical arrangement of the files themselves.

The finding aids also facilitate access to the files themselves. The sound recordings finding aid covers a mix of born-digital files and physical cassettes and CDs, the display is simple and uncluttered, and the access copy of each born-digital file is hyperlinked in a field so users can easily download the files; the finding aid also includes file sizes to that users could estimate download times. The photographs finding aid includes a thumbnail illustration for each photograph (housed within a tag), and the amount of description varies depending upon information provided with each photo.

The Minnesota Historical Society took a similar approach to making accessible geospatial data created by a gubernatorial redistricting commission, and plan to use the procedures they developed when processing the Pawlenty records and the redistricting commission files to make other records transferred on disc accessible via the Web.

I was in pretty bad shape by the time Jim Williams of Middle Tennessee State University's Albert Gore Research Center began discussing his institution's efforts to rescue two U.S. Representatives' constituent service files, so my notes and my memory of his presentation are both deficient; as a result, I'm limiting my comments to the portions of the presentation I remember semi-clearly.

The constituent services files Williams and his colleagues sought to preserve were created using Lockheed Martin's Intranet Quorum (IQ) application. The offices of many U.S. Representatives use IQ to track correspondence, store constituent contact information, and track the progress of constituent cases. IQ is proprietary, and each office that uses it pays roughly $60,000 per year to do so. Lockheed can convert the data in IQ systems to a more user-friendly format, but there is a cost associated with doing so.

Middle Tennessee State University was able to persuade the U.S. Representatives who donated their records to pay for the conversion of their IQ data, but other repositories may find themselves forced to pay for conversion or to convert the data themselves. As a result, the university hopes to take the lead in developing ways to reconstruct IQ databases and to collaborate with other archives seeking to to the same thing. Anyone interested in participating in a consortium devoted to preserving IQ data should contact Williams at Jim.Williams-at-mtsu.edu.

Image:  Light fixtures in Sapphire Room OP, Hilton San Diego Bayfront, 10 August 2012.

Electronic records roundup

In no particular order, some electronic records news that may be of interest:

• The thoughtful and hard-working folks at the South Carolina Department of Archives and History have explained some of the challenges of preserving the state's digital history. (As you'll recall, gubernatorial e-mail management practices recently gave rise to controversy in the Palmetto State.)
• The archivists at Queens University (Canada) are grappling with similar issues.
  
• David Pogue and CBS Sunday Morning drew attention to "data rot" -- the problems associated with hardware and software obsolescence. (N.B.: Pogue thinks that the word "archivist" contains a long "i."
• If you're interested in the evolution of cybersecurity, be sure to check out the short films that were shown at the annual conferences attended by Bell Labs executives. You'll find them on YouTube courtesy of the AT&T Archives. (And if you're interested in the history of hacking, be sure to check out Ron Rosenbaum's fascinating 1971 article on "phone phreaking," which captured the imagination of a generation of computer enthusiasts -- Steve Jobs among them.)
• A computer science Ph.D. student has found that, less than a year after the revolution in Egypt, approximately 10 percent of the social media posts documenting it have vanished from the live Web. A variety of factors account for this situation. People sometimes post things, regret doing so, and then delete them. Others get tired of maintaining their accounts and delete or deactivate them. Others were almost certainly the target of government repression and either removed content under duress or had content removed without their consent. The student's overarching conclusion: we need to become a lot more proactive about capturing Web content that documents the unfolding of historically significant events. (He'll get no argument from me.)
  

Catching up

A few things you might have missed:

• Late last month, President Obama issued a memorandum directing each federal government agency to perform a comprehensive review of its records management program and then prepare a report for the Archivist of the United States and the Director of the Office of Management and Budget that outlines its plans to maintain and improve its program, "particularly with respect to managing electronic records, including email and social media, deploying cloud based services or storage solutions, and meeting other records challenges." These reports are due on 27 March 2012.

• Paper records created during an internal military investigation of a November 2005 massacre of civilians in the Iraqi city of Haditha were slated for destruction. However, the records, many of them marked as being secret, ended up in trailers purchased by a local businessman, who hauled the trailers to a Baghdad junkyard. Several weeks ago, a New York Times reporter covering the American withdrawal from Iraq inadvertently found them there. At present, it is unclear whether the military will open an investigation into the handling of these records.
  

• After a legal review, the Massachusetts State Archives has decided to open approximately 460 boxes of paper records of former Governor and current Presidential candidate Mitt Romney to researchers. Staff will review the files prior to disclosure and either remove or redact legally restricted information. The repository initially restricted access to the records as a result of a court ruling stating that gubernatorial records were exempt from the state's freedom of information law. As you'll recall, during the last days of the Romney administration, all of the files on its e-mail servers were deleted, several high-ranking officials were allowed to purchase the state-owned hard drives they used, and leased computer equipment was replaced.
  

• The administration of South Carolina Governor Nikki Haley routinely deletes internal e-mails. The administration claims that it does so in order to free up storage space on its server, but Erik Emerson, Director of the state's Department of Archives and History, asserts that it violates state records laws.
  

• OccupyArchive is George Mason University's Roy Rosenzweig Center for the History of New Media effort to capture digital items documenting Occupy Wall Street and other Occupy movements throughout the world. As Rosenzweig Center director Sharon Leon notes, they're "documenting a post-print movement" -- something that archivists must do if they want to ensure a complete and accurate documentary record.
  

• Finally, on a lighter note, here's why we need to caution teens about sexting: sooner or later, their sexts will be all over the Internet for everyone to read.
  

State government electronic records in the news

Two stories relating to the management and continued accessibility of state government records popped up on my radar screen earlier today. Both of them warrant watching; it doesn't seem as if either situation will be resolved any time soon.

The first involves gubernatorial records, an ever-present matter of interest and concern. Earlier today, the Boston Globe reported that during the last days of Republican presidential candidate Mitt Romney's tenure as governor of Massachusetts, eleven of his high-ranking staffers used personal funds to purchase their state-supplied hard drives and laptops, staff replaced all of the other computers in the governor's office, and all Romney-era e-mail was deleted from the office's e-mail servers. When Deval Patrick, a Democrat, took office, he and his staffers found an electronic blank slate.

Romney's position is that staffers who purchased hardware did so openly and that he and his staffers complied with all records laws. It does seem that the Romney administration did transfer a substantial body of records to the Massachusetts Archives: according to the Globe, the the repository holds 700-800 boxes of paper records documenting the Romney administration. However, it's not clear whether these records include print copies of the e-mails. The Globe doesn't provide detailed information about them, and the Massachusetts Archives doesn't have an online catalog or detailed online finding aids.

Secretary of State Bill Galvin, who oversees the Massachusetts Archives, told the Globe that the hardware purchases strike him as odd and that the gubernatorial e-mail should have come to the archives: "Electronic records are held to the same standard as paper records. There’s no question. They’re not in some lesser standard."

Romney's campaign manager asserts that the Patrick administration is making a stink about the hardware purchases, computer replacement, and e-mail deletion because it is acting as "an opposition research arm of the Obama reelection campaign." After the Globe story appeared this morning, he filed a state Freedom of Information Act request seeking "all email correspondence, phone logs, and visitor logs" documenting contacts between Patrick administration staffers and prominent Obama political advisers David Plouffe, David Axelrod, and Jim Messina. Governor Patrick’s chief legal counsel has stated that staff will "be happy to fulfill" this request.

I'm not an expert on Massachusetts records laws, so I'm going to have wait for the experts to weigh in on whether the actions of Governor Romney and his staff were legal. Do I wish that the e-mail had been preserved? Of course I do. I'm an archivist, and my job is to preserve records of enduring value and to provide access to them. Gubernatorial correspondence and internal memoranda, regardless of format, do have enduring value. Do I think that Governor Romney should be pilloried for destroying the e-mail? If he violated the law, I hope he gets what's coming to him. If he didn't, I hope that Governor Patrick and other Massachusetts politicians focus on strengthening laws concerning the retention and disposition of gubernatorial records.

Do I think that Governor Patrick brought up these issues in an effort to give President Obama a boost? I don't know. Patrick and Obama are close allies, so it's possible. However, I'm also under the impression that Governor Patrick has his own reasons for disliking Governor Romney, and I'm open to the possibility that he and his staffers are discussing the matter because they keep getting freedom of information requests for Romney-era records. I must admit that I am curious as to how well the Patrick administration is managing its own records.

The second relates to an outrage. As anyone who's been paying even the slightest attention to the American news media knows, former Penn State assistant football coach Jerry Sandusky was recently arrested on charges that he sexually molested eight young boys. Two university administrators have been charged with perjury, and the university's president and football coach have lost their jobs.

Questions as to precisely what the president, the coach, and other university administrators knew about Sandusky and when they knew it are rampant. However, Pennsylvania's Right to Know Law, which was extensively revised in 2008, explicitly exempts most records created by Penn State, Lincoln University, the University of Pittsburgh, and Temple University. As a result, there is a distinct possibility that only those e-mails, phone records, and other Penn State records introduced in open court will be disclosed to the public -- unless, as the New York Times urged earlier today, the Pennsylvania legislature and governor move to lift this exemption.

Publicly funded universities in many other states -- New York included -- are subject to freedom of information laws. For what it's worth, I really don't see why Penn State, Pitt, Temple, and Lincoln should be granted such sweeping exemptions, and I hope that Pennsylvania's law changes. At the very least, I hope that Penn State's new administrators recognize that openness and honesty are essential to restoring the university's good name and start releasing records of their own accord as soon as prosecutors permit them to do so.

Yes, I know that Penn State is going to be hit with civil lawsuit after civil lawsuit and that its lawyers would probably jump for joy if a fire or flood destroyed a ton of university records. However, the lawsuits will come and the cost of settling them will be staggeringly high no matter what the university does.

Of course, Penn State is not the only entity that has relevant records: Sandusky met the boys he is accused of sexually assaulting through The Second Mile, a charitable organization that he founded. However, earlier today, the New York Times reported that investigators have yet to locate some important Second Mile records:

Officials at the Second Mile . . . reported that several years of the organization’s records were missing and had perhaps been stolen. The missing files, investigators worry, may limit their ability to determine if Sandusky used charity resources — expense accounts, travel, gifts — to recruit new victims, or even buy their silence . . . .

Much of the [charity's] older paperwork was stored at an off-site records facility. The travel and expense records, for instance, had been sent over several years earlier. But select members of the charity’s board of directors were alarmed to learn recently that when the records facility went to retrieve them, some of those records — from about 2000 to 2003 — were missing.

. . . . Subsequently, the [Second Mile] foundation located apparently misfiled records from one of the years, but the rest seem to have disappeared.

As awful as the Sandusky-Penn State situation currently appears, I can't help but think that we've seen only the tip of the iceberg. All the more reason to be as honest and as open as possible. The sooner the truth comes out, the sooner the victims can focus on rebuilding their lives and the sooner Penn State can focus on rebuilding itself.

September 11: electronic records, service, and remembrance

The New York State Archives has just published Ground Zero from the Air, an online exhibit that features aerial photographs, thermal images, and flyover simulations of the World Trade Center site created in September and October 2001. These records were created by EarthData, a mapping firm working under contract to the New York State Office for Technology, which ultimately transferred them to the State Archives for long-term preservation.

I helped to put this exhibit together, and I have to say that the experience was, in some respects, profoundly rewarding. The records document an event of profound significance and are visually compelling (the level of detail in the aerial photographs is nothing short of astounding). I got to work closely with several colleagues whose work typically doesn't overlap with mine, and I am once again in awe of their talent and dedication.

Moreover, all of these records were born digital, and this is the first time that electronic records have been featured in one of our online exhibits -- and incorporated into the Digital Collections section of our Web site. Electronic records can be every bit as haunting, fascinating, and visually arresting as paper records, and it's good to remind people -- archivists and researchers alike -- of this fact every now and then.

At the same time, the experience of putting together this exhibit was extremely difficult. Anyone who spends any time with these records will instantly be transported back to the days immediately following September 11. Magnify one of the aerial photographs of the World Trade Center site, and you'll understand instantly why the first responders who worked there always referred to it as "the pile." In some of the September 2001 images, you can see bucket brigades of emergency personnel removing debris by hand. In some of the October 2001 images, you can see tractor trailers carrying debris away from the site.

As glad as I am that these records exist and are in our holdings, sometimes I had to get up and walk away from them for a while. I wish with all my heart that the circumstances that led to the creation of these records had never come to pass, and I don't think that this wish will ever fade away.

There is, of course, nothing any of us can do to change what happened on September 11, 2001. However, we do have the power to determine how we respond to it. This morning, National Public Radio aired a quietly and profoundly moving story about Father Mychal Judge, the Fire Department of New York chaplain who died at the World Trade Center site. The story featured a substantial excerpt from the homily that Father Michael Duffy delivered at Father Judge's funeral, and I couldn't help but think that we should all strive to live as Father Judge did:

And he would say to me once in a while, “Michael Duffy,” he always called me by my full name, “Michael Duffy, you know what I need?” And I would get excited because it was hard to buy him a present or anything. I said, “No, what?” “You know what I really need?” “No, what Mike?” “Absolutely nothing. [MURMURING] I don’t need a thing in the world. I am the happiest man on the face of the earth.” And then he would go on for ten minutes, telling me how blessed he felt. “I have beautiful sisters. I have nieces and nephews. I have my health. I’m a Franciscan priest. I love my work. I love my ministry.” And he would go on, and he would always conclude it by looking up to heaven and saying, “Why am I so blessed? I don’t deserve it. Why am I so blessed?” But that’s how he felt all his life.

Father Judge knew that service, not self-aggrandizement, is the way to fulfillment and that meaningful work is a gift. The families of many of the men and women who were killed on September 11, 2001 also know these things: in 2002, they began pressing to have September 11 designated a National Day of Service and Remembrance focused on honoring the dead, helping the living, and recapturing the spirit of unity, generosity, and compassion that prevailed in the weeks following the attacks.

If you have a few hours to devote to community service on Sunday, Serve.gov can help you find an organization that could use a helping hand. If you live in the Northeast -- which has just suffered yet another round of catastrophic floods -- your help is particularly needed; if you do a Google search for "Hurricane Irene volunteer opportunities [your state]" you'll find ample opportunities. Of course, there are countless community organizations that can use your help not only on Sunday but also throughout the year; September 11 should be merely one day in a lifetime of service and purposeful work.

I'm going to spend Sunday morning sorting donations at an area food bank, and as I shift the canned goods, bottled water, and toiletries around, I'm going to reflect upon my good fortune: I have a home, my friends and family are safe, and I have work that gives me purpose and direction -- in large part because the archival profession is, on every conceivable level, a service-oriented profession.

I'm also going to think about the most important passage in Father Michael Kelly's funeral homily for Father Mychal Judge:

And so, this morning … we come to bury Mike Judge’s body but not his spirit. We come to bury his mind but not his dreams. We come to bury his voice but not his message. We come to bury his hands but not his good works. We come to bury his heart but not his love. Never his love.

Government social media records

Local, state, and federal governments are increasingly using social media to convey important information and to solicit feedback from citizens. However, governments and officials are still struggling to adapt to a Web 2.0 world. Michigan, for example, is actually taking down some social media content as a result of legal considerations and resource limitations. U.S. Representative Patrick McHenry (R-NC) and the U.S. Food and Drug Administration have recently learned that citizen feedback isn't limited to "likes" or approval. At this moment, U.S. Representative Anthony Weiner (D-NY) is probably wishing that he had never gotten a Twitter account. And Kent County, Delaware has recently issued a policy that bars employees from posting negative comments about their colleagues or county government -- whether via county-owned computers while at work or via their own computers and cell phones while off the clock .

What to do? I can't help you strike the right balance between the need to present an appropriate face to the public and the free speech rights of your employees -- the courts will probably do that -- but if you're a government official or employee contemplating using social media, be sure to check out the following resources:

• Government Technology's Five Social Media Missteps to Avoid
• iDisaster 2.0's Top 36 Items to Include in Social Media Policies
• The Center for Technology in Government's Designing Social Media for Government: Eight Essential Elements, which contains links to a wide array of state and local government policies

Even a cursory glance at these resources will underscore the fact that, in most jurisdictions, social media content typically meets the statutory definition of a "public record" and must thus be managed properly. For tips on how to do so, consult the following:

• Patricia C. Franks's How Federal Agencies Can Effectively Manage Records Created Using New Social Media Tools
• The U.S. National Archives and Records Administration's NARA Bulletin 2011-02, Guidance on Managing Records in Web 2.0/Social Media Platforms

At present, just about everyone seems to agree that most social media content has a short retention period. Unfortunately -- but not surprisingly -- there is no consensus regarding how best to capture and preserve content that has enduring value. There are lots of tools out there, and all of them have different features and save content differently. Given that most social media content will likely be destroyed within a relatively short timeframe, this isn't as big a problem as it might be. However, I suspect that those of us charged with capturing and preserving content deemed archival may run into some preservation problems a few years down the road -- and I hope that the federal government's 2009

My own experience is limited to capturing content created by others, which poses some additional challenges: some social media capture tools are expressly designed to help people preserve their own content and require full login rights. In such a situation, use of a Web crawler may be the best approach. I've experimented -- with decidedly mixed results -- with using OCLC's Heritrix-based Web Harvester to capture Facebook, IdeaScale, Twitter, and YouTube content, and I know several people have had somewhat greater success with Heritrix-based Archive-It service. If you're interested in exploring Web crawling of social media content, check out this nice list of Web crawling software and services.

If you're looking to preserve your own content, other options are available:

• Several low- and no-cost tools that support capture and archiving of one's own Facebook and Twitter content are out there. For more information, consult April Edmonds's superb overview.
• TwapperKeeper enables you to capture and preserve tweets (i.e., individual Twitter posts) that contain specified hashtags or keywords. Using this tool to capture all of the tweets created by a specific office may be a challenge, but it can be used to capture all of the tweets related to a specific subject or event. Sadly, the "download and export" and "API" components features present within the Web-based version of TwapperKeeper were recently removed at the behest of Twitter. However, it's still possible to install an open source version of the software that still includes these features on your own server.
  
• A growing number of software companies, among them Arkovi, Backupify, LiveOffice, Smarsh, Sonian, and Symantec, are creating social media archiving tools or incorporating them into larger e-mail archiving products. If you're already using an e-mail archiving product, investigate whether it also supports social media archiving. If you're not, a stand-alone commercial product or service may meet your needs.

Finally, please note that, at present, the imperative to manage state and local government social media records may conflict with the terms of service agreements governing usage of social media services such as Facebook or Twitter; in many instances, these agreements limit the extraction or repurposing of content. The federal government has negotiated special agreements with many social media service providers, and the National Association of State Chief Information Officers has negotiated a model Terms of Service agreement for state and local government Facebook users and is currently seeking to develop similar agreements with other social media service providers, but it's likely going to be some time before the legal issues that might affect our ability to manage social media records are resolved conclusively.

U.S. Federal Cloud Computing Strategy released

Last week, the Chief Information Officer of the United States released the Federal Cloud Computing Strategy, which outlines how the federal government anticipates saving money, increasing the efficiency of its IT operations, and delivering better service to the public via cloud computing.

This strategy, which will help federal agencies migrate at least some of their IT infrastructure to commercial or government cloud environments, is intended to:

• Articulate the benefits, considerations, and trade-offs of cloud computing
• Provide a decision framework and case examples to support agencies in migrating towards cloud computing
• Highlight cloud computing implementation resources
• Identify Federal Government activities and roles and responsibilities for catalyzing cloud adoption (p. 2)
  

I haven't had the chance to give this document a close reading and likely won't have the chance to do so for a couple of weeks, but I have skimmed it and was pleased to note the following:

Storing information in the cloud will require a technical mechanism to achieve compliance with records management laws, policies and regulations promulgated by both the National Archives and Records Administration (NARA) and the General Services Administration (GSA). The cloud solution has to support relevant record safeguards and retrieval functions, even in the context of a provider termination (p. 14) [emphasis added]

As a state government employee, I'm also intrigued by this statement:

Federal Government contracts will also provide riders for state and local governments. These riders will allow all of these governments to realize the same procurement advantages of the Federal Government. Increasing membership in cloud services will further drive innovation and cost efficiency by increasing market size and creating larger efficiencies-of-scale (p. 29) [emphasis added].

And this one:

To effectively manage these governance issues in the long-term, the Federal Government needs to lay a stable governance foundation that will outlast single individuals or administrations. To the best extent possible, individuals or committees should have explicitly defined roles, non-overlapping responsibilities, and a clear decision-making hierarchy. These steps will empower the government for action, minimize unnecessary bureaucracy, and ensure accountability for results.

The following bodies will therefore have these roles and responsibilities:

• National Institute of Standards and Technology (NIST) will lead and collaborate with Federal, State, and local government agency CIOs, private sector experts, and international bodies to identify and prioritize cloud computing standards and guidance . . . . (p. 31) [emphasis added]

I'm looking forward to seeing how all of this plays out.

Missouri State Capitol fire centennial

On the evening of 5 February 1911, a bolt of lightning struck the dome of the Missouri State Capitol and started a fire that ultimately destroyed the building. As the flames ate away at the structure, government officials, residents of Jefferson City and nearby communities, and inmates from a nearby prison formed a human chain and salvaged what they could. Thanks to their efforts, the state seal, important land records, and the official record that abolished slavery in the state were rescued. However, other important records were lost and many of those that were saved were badly amaged; you can see an example here.

I haven't been able to find any photographs of the fire and its aftermath on the Missouri State Archives Web site, but the Kansas City Star has posted copies of nine State Archives photographs on its site. Given the extent of the damage -- the dome ultimately collapsed into the lower floors of the building -- it's amazing that no one perished and that so many important records were saved.

My own repository suffered similar losses fifty-two days later, when flames swept through the third and fourth floors of the New York State Capitol's western wing, which then housed the New York State Library. The building survived, but one person was killed and most of the State Library's holdings, which then included government records as well as books, periodicals, and manuscript collections, were either destroyed completely or suffered extensive damage.

A quick Google search reveals that until the second half of the 20th century, state capitol fires were depressingly commonplace events. Given the heavy usage of oil lamps and gaslights in the 19th century, the lack of safety standards during the first decades of the electric era, and the absence of smoke detectors, automated sprinkler systems, and other modern safety technologies for much of the 20th century, it's not surprising that so much of our documentary record has literally gone up in smoke.

The infopocalypse is upon us

Last week, the Boston Phoenix published an article by Chris Faraone highlighting how local, state, and federal governments are struggling to manage ever-increasing amounts of digital data. Provocatively titled "Infopocalypse: The Cost of Too Much Data," Faraone notes that:

The United States Census Bureau alone maintains about 2560 terabytes of information -- more data than is contained in all the academic libraries in America, and the equivalent of about 50 million four-drawer filing cabinets of documents.

Other federal agencies have similarly mind-boggling quantities of data, and state and local governments are also amassing vast stores of digital information.

Not surprisingly, "public data remains, by and large, a disorganized mess." Governments don't know precisely what they have or how to make best use of it, and old, paper-centered ways of responding to freedom of information requests and performing other essential functions persist.

Why does this situation exist? In my humble opinion, Faraone has nailed the root causes:

There is too much data. Digital storage is not a natural resource. The amount of information that government agencies may be required to keep — from tweets and e-mails to tax histories — is growing faster than the capacity for storage.

There's not enough manpower to manage all this data. The Obama administration hopes to make more information freely available online. But in the meantime, the old method of requesting data from the government -- filing a FOIA request -- is bogged down due to an insufficient workforce and long request backlogs.

Private companies are storing public data. This trend in outsourcing, largely the result of too much data and too little manpower, is a potential threat to both access and security, as resources that belong to the people are entrusted to outside vendors, raising new privacy concerns.

What to do about this situation? As Faraone notes, the data center consolidation strategy being pushed by Vivek Kundra, the Chief Information Officer of the United States, may help, but it's only a start. Faraone also suggests -- correctly -- that hiring additional staff who can process freedom of information requests and making readily available online data that doesn't contain legally restricted or, in the federal environment, classified information would also improve things a bit.

However, none of these things will solve the problem, which, as Sunlight Foundation policy director John Wunderlich pointed out to Faraone, is in many ways akin to that posed by the explosive growth of paper government records during the first two-thirds of the 20th century:

"Back then [government agencies] didn't know what to throw out, what to standardize, or how to organize. The challenges we face in data are in similar scope -- that's why it's so important that these issues are addressed head-on before it's too late."

Surprisingly, Faraone makes no mention of the U.S. National Archives and Records Administration (NARA), which works with agencies to figure out how to standardize and organize their records and how and when to dispose of records that have reached the end of their useful life, or of the role that agency records managers have -- or, as is all too often the case, should have -- in ensuring that all agency records are properly managed. Hiring some records management personnel -- at NARA, the 50 state archives, larger local governments, and larger government agencies -- would no doubt help to reduce agency storage pressures.

However, the more I work with electronic records, the less convinced I am that simply hiring a few more records managers will make everything better. We forget sometimes that formalized records management theory and practice are not mere outgrowths of common sense. They were practical responses to the challenges posed by the deluge of paper records created by ever-larger and ever more complex organizations. The infopocalypse that we face is in some respects quite similar to that which confronted our mid-20th century predecessors, but it is also, at least in some respects, unique. Addressing the challenges associated with our infopocalypse successfully will likely mean a shift in thinking no less monumental than that which propelled the rise of records management as a discipline and More Product, Less Process archival processing.

What will this shift in thinking look like? I don't know. I anticipate I that we're going to focus less on one-on-one guidance and more on standards development and automation of tasks now performed by humans. I also expect that our definitions of "record" and "records series" will be altered significantly and I suspect that, at some point in the future, be discarded altogether.

Yeah, I'm scared, too. However, our mid-20th century predecessors were as shaken by the changes in their record-keeping environment as we are by the changes in ours. They chose to meet those challenges head-on, and, after a lot of hard work and mistakes along the way, eventually developed workable solutions to complex problems. If we have any interest in surviving -- which may well mean evolving from "archivists" and "records managers" into "digital preservationists" or "data curators" or somesuch -- we'll take our lumps and do the same.

State archives and state archivists in the news

Icicles and blowing snow at the Cultural Education Center, which houses the State Archives, State Library, and State Museum, Albany, New York, 27 December 2010, 5:43 PM.

Just in case you missed them in the holiday whirlwind, here are some recent news stories relating to American state archives and state archivists:

• A lengthy feature highlights the work of Kentucky state archivist Barbara Teague and the other archivists at the Kentucky Department of Libraries and Archives.
• The Vermont State Archives now has a renovated and expanded facility that incorporates a host of energy efficiency measures.
• Outgoing New York Governor David Paterson vetoed a bill mandating transfer of gubernatorial records to the New York State Archives but signed an executive order propelling creation of a gubernatorial records management program.
• A descendant of one of the nation's first Supreme Court justices is suing the North Carolina Department of Cultural Resources; he claims that the justice's papers were lent, not given, to the North Carolina State Archives more than a century ago.
• A broken water pipe damaged 10 boxes of paper records held by the Alaska State Archives. Fortunately, staff expect that the records can be salvaged.
• South Carolina state archivist Eric Emerson discusses the lingering impact of the Civil War and North-South cultural divisions.

Quiet times ahead . . .

Governor-Elect Nelson A. Rockefeller and Margaretta "Happy" Rockefeller arrive at the Executive Mansion for a luncheon with Governor W. Averill Harriman and First Lady Marie Harriman, 10 December 1958. New York (State). Governor, Public Information Photographs, 1910-1992, accretion 13703-83. Image courtesy of the New York State Archives.

. . . at least on this blog. Posting will likely a little light during the next few weeks: New York State will undergo a gubernatorial transition on January 1, 2011, and I'm going to be busy overseeing a comprehensive Web crawl and helping with transition-related transfers of records from multiple offices and agencies. I'll pop in whenever I can, but things will likely be pretty quiet around here until January. Have a nice December.

More catching up

Happy Monday. I'm not at the office today -- I worked on Saturday, so this is the second day of my "weekend" -- and I've rounded up some stuff that may interest you:

• Vermont Public Radio's Vermont Edition interviewed Vermont State Archivist Gregory Sanford and Terry Cook and Wendy Smith of the University of Manitoba about the value of government archives, the Vermont State Archives' efforts to document the perspectives of citizens as well as the workings of state government, functional appraisal, and the new archival challenges of the digital era. You can catch this excellent episode, which aired on 18 October, here.

• Last year, Google began working with archivists to add digitized aerial photographs of major European cities that were taken in 1945 into its popular Google Earth application, which allows users to view present-day aerial images of the entire planet. Last week, Google added additional historical photographs, including photographs of London taken in 1945, to Google Earth. As a result, users can easily see London, Paris, Warsaw, and several other major European cities -- some of which were heavily bombed during the Second World War -- looked in 1945 and how they look today. Cool.

• On 1 October, George Mason University hosted an Archiving Social Media conference that addressed the following topics: potential uses of archived social media content, institutional responsibility for preserving social media content, the ethics of archiving social media, capture and preservation tools, types of content that are being overlooked, and copyright issues. Notes are available on the Archiving Social Media conference Web site, Travis Kaya at Wired Campus and Kate Theimer at ArchivesNext have posted about it, there's an Archiving Social Media Zotero group, and all of the conference-related tweets (#asome) are here. (NB: the #asome hashtag apparently has multiple uses, so you'll need to zero in on tweets sent on or around 1 October.)

• The state of Texas recently recovered an 1858 state Supreme Court document that concerned a slave-related case and that somehow fell into private hands -- and one of my own colleagues at the New York State Archives helped to make this recovery possible. She traveled, on her own time and at her own expense, to the upstate New York home of the man who held the document and calmly explained how she knew it was a Texas government record. The collector, who had reacted angrily when a police officer aggressively sought to recover the document, quickly agreed to turn over the record. There's a lesson here, folks: most collectors want to do the right thing, and civility and a willingness to explain the value of government records will often result in the return of an alienated record. Calling in law enforcement probably shouldn't be the first step.