John Kitzhaber's e-mail

It's been quite a week in re: gubernatorial e-mail.

Yesterday, Oregon's governor, John Kitzhaber, announced that he would resign from office. Last October, an Oregon newspaper reported that the governor's fiancee, Cylvia Hayes, had served as an unpaid energy and economic policymaker at the same time as she was running a private green-energy consulting business. In November, the Government Ethics Commission opened an investigation into Kitzhaber and Hayes. Last week, the state's attorney general announced that a criminal investigation was underway.

Earlier this week, the Willamette Week, the Portland alternative paper that broke the news regarding  Hayes' potential conflicts of interest, reported that a Kitzhaber staffer had requested that the state Department of Administrative Services delete all e-mails from Kitzhaber's personal e-mail accounts that were stored on departmental servers. Kitzhaber's camp maintained that the request covered personal messages that had mistakenly been auto-forwarded to state servers and would not result in the destruction of public records. However, Oregon law specifies that personal e-mail messages that discuss government business are considered public records, and Department of Administrative Services staff were keenly aware that multiple media organizations had filed freedom of information requests that included Kitzhaber's and Hayes's e-mail. They sent the administration's request and their concerns about it up the department's chain of command and the department's director decided that the e-mail should not be deleted.

A few hours after Governor Kitzhaber resigned, U.S. Attorney Amanda Marshall revealed that a federal grand jury investigation was underway and issued a sweeping subpoena seeking e-mail, memoranda, and other records that documenting Kitzhaber's environmental and economic policy initiatives, and Hayes' state government work, consulting business and clients, personal and corporate tax returns, and use of state credit cards. The subpoena covers eleven state government agencies and includes records that the state's Justice Department and Government Ethics Commission created or collected during the course of their investigations. The Federal Bureau of Investigation, which never publicly comments upon investigations in progress, also seems to have taken an interest in Hayes's affairs.

What a mess. I imagine that Salem, Oregon is currently experiencing the sort of surreal standstill that Albany, New York experienced in March 2008, but things are going to start moving again, and very quickly. Eleven state agencies will have to devote a lot of time and effort to responding to the federal subpoena and a host of freedom of information requests. The State Archives must scramble to document the administration of a governor who was re-elected a few months ago, who left office with little advance notice, and whose records are of abiding interest to the feds. Moreover, it must do so as it loses the head of its parent agency: when Kitzhaber's resignation takes effect next Wednesday, Secretary of State Kate Brown will become the state's next governor. However, given the clear-eyed, resolute manner in which the Department of Administrative Services responded to the Kitzhaber administration's e-mail deletion request, the Oregon State Archives' recent history of innovation and effectiveness, and Oregon's tradition of (relatively) clean governance, I suspect that these challenges will be met head-on.

Jeb Bush's e-mail

On Monday, former Florida governor Jeb Bush placed online copies of hundreds of thousands of e-mails he sent and received while in office. Bush is actively exploring the possibility of running for President and has stated that he released the messages to show his commitment to transparency and his embrace of information technology; many political observers have concluded that the release is also meant to prove that he's a dedicated, responsive, and effective executive. Things did not go quite as planned, and the resulting uproar ought to be of interest to any government archivist who might accession electronic records that contain legally restricted information, respond to FOI requests for born-digital or digitized records, or confront the sweeping records requests that invariably occur whenever a former official seeks higher office.

As soon as the e-mails were released, tech journalists and bloggers began exploring the search interface that Bush's staff created and the contents of the messages their searches yielded. They found thousands of Social Security numbers, home addresses, and tons of other personal data that had not been redacted. The Verge, Ars Technica, Buzzfeed, and a host of other media outlets quickly redacted and published copies of numerous e-mails that contained such information, and Bush and his staff quickly promised that they would remove Social Security numbers and other personal data. However, the e-mails – in searchable database form as well as downloadable Microsoft Personal Storage Table (PST) files – were freely available online for almost a day before the Bush team decided to take action.

Bush and his staff were also quick to point fingers. Yesterday, Bush told reporters in Tallahassee that the messages were public records held by the Florida State Archives (which is part of the state's Department of State) and that he and his staff had merely "released what the government gave us." The Bush team also revealed that in May 2014, an attorney representing Bush sent a letter to an unidentified state official asserting that the state was responsible for redacting any legally restricted information found within the e-mails:

We hope these emails will be available permanently to the public, provided the records are first reviewed by state officials in accordance with Florida Statute to ensure information exempt from public disclosure is redacted before release, including social security numbers of Florida citizens who contacted Governor Bush for assistance; personal identifying information related to victims of crime or abuse; confidential law enforcement intelligence; and other information made confidential or exempt by applicable law.

The Florida State Archives holds 26.2 gigabytes of Bush's gubernatorial e-mail, and the catalog record describing the correspondence indicates that the records consist of "PST files" that "must be loaded onto user's hard drive and opened using MS Outlook software." The catalog record makes no mention of access restrictions, and unredacted copies of the files have evidently been disclosed to other researchers. Yesterday, National Public Radio (NPR) reported that many of the e-mails Bush released on Monday had first been disclosed to reporters shortly after they were created or received and that several media organizations, NPR among them, had previously obtained copies of the full set from the Florida State Archives.

At this point in time, I am not going to second-guess or condemn the Florida State Archives. I simply don't know enough about Florida's Sunshine Law, which is more expansive than many other state freedom of information laws, or the Florida State Archives' disclosure protocols to come to any sort of informed conclusion. I do know that the Sunshine Law for the most part bars the disclosure of Social Security numbers, but many freedom of information laws mandate that previously disclosed information cannot be withheld for any reason; given that many of these e-mails had been disclosed to reporters while Bush was in office, the Florida State Archives might have no choice but to release them without redacting them. To date, no one from the Florida State Archives or Florida Department of State has commented upon this matter, but I hope that some sort of explanation will eventually be made.

I am more willing to second-guess Jeb Bush and his associates. As the Miami Herald has pointed out, the May 2014 letter written by Bush's attorney strongly suggests that Bush has been seriously thinking about running for president for quite some time. To my way of thinking, it also suggests that Bush or, at the very least, his lawyers knew that the e-mail contained legally restricted information, decided that the State of Florida was solely responsible for redacting it prior to disclosure, and figured that it was ethically okay to make information that Florida couldn’t or wouldn’t redact a lot easier to find. Requesting a PST file from the Florida State Archives and importing it into Microsoft Outlook doesn’t require a ton of effort or technical know-how, but at least some of the people who are now idly rummaging through the searchable Web database of e-mails created by the Bush campaign probably wouldn’t feel the need to make the effort. Manual redaction and review of e-mail is a pain – trust me on this – but there are numerous tools that will flag and facilitate redaction of Social Security numbers, telephone numbers, and other consistently formatted data. Why didn't the Bush camp make even a modest attempt to weed out the Social Security numbers?

Finally, I must be a bit skeptical about the Bush camp's claims of transparency: the Tampa Bay Times recently reported that Bush used a private e-mail account to conduct all state business and transferred only some of the messages associated with this account to the archives when he left office. Specifically, all messages relating to “politics, fundraising, and personal matters” were removed prior to transfer. I have no problem with purging messages relating to purely personal matters, but the removal of messages relating to political affairs and fundraising efforts raises a few questions in my mind. How were these messages identified? Were they identified as they were sent or received, or was there a massive end-of-term review effort? If the latter, who was involved in the review and what criteria were employed? And, of course, why didn't Bush use a state government e-mail account to conduct state business?

Freedom of information laws throughout the world

Things are going to be quiet around here this week: my modem abruptly ceased working on Sunday afternoon. Owing to the holiday I expect that the new one won't arrive until Friday at the earliest and that the coffeeshop in which I'm writing this post and all of my other usual wifi hotspots will be closed.

However, I wanted to draw your attention to a recent Associated Press article highlighting the results of its first-ever test of freedom of information laws throughout the world. At present, 105 countries have such laws, but the experience of the AP, which submitted requests for information to all of these nations and to the European Union, reveals that the extent to which these laws are observed varied widely.

• Only 14 countries supplied all the information requested within the time frame specified in their laws, and 38 more eventually complied. More than half ignored the AP's requests altogether.
• Newer democracies often complied more swiftly than mature democracies. Moreover, newer democracies' laws, which tend to reflect the existence of the Internet, are often better suited to today's world than the laws that mature democracies enacted decades ago, when the overwhelming majority of records were created on paper and the Xerox machine was the height of technological sophistication.
  
• Many countries adopted freedom of information laws as a condition of securing foreign assistance, and most of these countries ignore or seek to circumscribe these laws as quickly as possible.
• In some countries, citizens who file freedom of information requests may be targeted for retaliation. In India, where activists are using such requests to expose and combat entrenched governmental corruption, at least a dozen people who have filed freedom of information requests have been killed and dozens more have been violently attacked.
  

It's an interesting, thought-provoking piece, and it bears close reading.

SAA 2011: Skeletons in the Closet

This just about beats the record for tardy posting, but below you'll find the slides from my Society of American Archivists presentation, which was part of Session 101, "Skeletons in the Closet: Addressing Privacy and Confidentiality Issues for Born-Digital Materials." In it, I outline the current climate in which government archives operate, discuss how my repository responded to two sweeping freedom of information requests, and detail some of the lessons we learned as a result of these experiences.

Personal Privacy and Freedom of Information in the Digital Age: Challenges and Strategies for Government A...

I'll have a post concerning session 705,"Theft Transparency in the Digital Age: Stakeholder Perspectives," up later this weekend.

The infopocalypse is upon us

Last week, the Boston Phoenix published an article by Chris Faraone highlighting how local, state, and federal governments are struggling to manage ever-increasing amounts of digital data. Provocatively titled "Infopocalypse: The Cost of Too Much Data," Faraone notes that:

The United States Census Bureau alone maintains about 2560 terabytes of information -- more data than is contained in all the academic libraries in America, and the equivalent of about 50 million four-drawer filing cabinets of documents.

Other federal agencies have similarly mind-boggling quantities of data, and state and local governments are also amassing vast stores of digital information.

Not surprisingly, "public data remains, by and large, a disorganized mess." Governments don't know precisely what they have or how to make best use of it, and old, paper-centered ways of responding to freedom of information requests and performing other essential functions persist.

Why does this situation exist? In my humble opinion, Faraone has nailed the root causes:

There is too much data. Digital storage is not a natural resource. The amount of information that government agencies may be required to keep — from tweets and e-mails to tax histories — is growing faster than the capacity for storage.

There's not enough manpower to manage all this data. The Obama administration hopes to make more information freely available online. But in the meantime, the old method of requesting data from the government -- filing a FOIA request -- is bogged down due to an insufficient workforce and long request backlogs.

Private companies are storing public data. This trend in outsourcing, largely the result of too much data and too little manpower, is a potential threat to both access and security, as resources that belong to the people are entrusted to outside vendors, raising new privacy concerns.

What to do about this situation? As Faraone notes, the data center consolidation strategy being pushed by Vivek Kundra, the Chief Information Officer of the United States, may help, but it's only a start. Faraone also suggests -- correctly -- that hiring additional staff who can process freedom of information requests and making readily available online data that doesn't contain legally restricted or, in the federal environment, classified information would also improve things a bit.

However, none of these things will solve the problem, which, as Sunlight Foundation policy director John Wunderlich pointed out to Faraone, is in many ways akin to that posed by the explosive growth of paper government records during the first two-thirds of the 20th century:

"Back then [government agencies] didn't know what to throw out, what to standardize, or how to organize. The challenges we face in data are in similar scope -- that's why it's so important that these issues are addressed head-on before it's too late."

Surprisingly, Faraone makes no mention of the U.S. National Archives and Records Administration (NARA), which works with agencies to figure out how to standardize and organize their records and how and when to dispose of records that have reached the end of their useful life, or of the role that agency records managers have -- or, as is all too often the case, should have -- in ensuring that all agency records are properly managed. Hiring some records management personnel -- at NARA, the 50 state archives, larger local governments, and larger government agencies -- would no doubt help to reduce agency storage pressures.

However, the more I work with electronic records, the less convinced I am that simply hiring a few more records managers will make everything better. We forget sometimes that formalized records management theory and practice are not mere outgrowths of common sense. They were practical responses to the challenges posed by the deluge of paper records created by ever-larger and ever more complex organizations. The infopocalypse that we face is in some respects quite similar to that which confronted our mid-20th century predecessors, but it is also, at least in some respects, unique. Addressing the challenges associated with our infopocalypse successfully will likely mean a shift in thinking no less monumental than that which propelled the rise of records management as a discipline and More Product, Less Process archival processing.

What will this shift in thinking look like? I don't know. I anticipate I that we're going to focus less on one-on-one guidance and more on standards development and automation of tasks now performed by humans. I also expect that our definitions of "record" and "records series" will be altered significantly and I suspect that, at some point in the future, be discarded altogether.

Yeah, I'm scared, too. However, our mid-20th century predecessors were as shaken by the changes in their record-keeping environment as we are by the changes in ours. They chose to meet those challenges head-on, and, after a lot of hard work and mistakes along the way, eventually developed workable solutions to complex problems. If we have any interest in surviving -- which may well mean evolving from "archivists" and "records managers" into "digital preservationists" or "data curators" or somesuch -- we'll take our lumps and do the same.

MARAC Fall 2010, day one

Market Street Bridge over the Susquehanna River, Harrisburg, Pennsylvania, 12 November 2010. The Ionic columns at the entrance of the bridge were salvaged from the old State Capitol building, which burned down in 1897.

The Fall 2010 meeting of the Mid-Atlantic Regional Archives Conference got underway today. I'm offering only a few highlights from a jam-packed and rewarding day:

• Colgate University, the Rockefeller Archive Center, and Syracuse University are investigating the possibility of developing a New York State EAD consortium, with particular emphasis on assisting repositories that have some EAD knowledge and experience but are having problems with publishing their finding aids and securing adequate technical support. If you're interested in seeing how this project proceeds or in contributing your expertise, contact Colgate University Archivist Sarah Keen at skeen - at - colgate.edu

• Kathleen Roe (New York State Archives) delivered a great plenary address on the importance of advocating for archives. Noting that we all need to explain -- to administrators, boards of directors, or local, state, and federal politicians -- the value of archives and what we need to do our jobs as effectively as possible, she offered some practical words of advice:
• Learn the rules of engagement and accept them for what they are. You don't have to compromise yourself or your principles, but you do need to learn how to find your way through established channels. For example, if you're seeking Congressional support for legislation, you simply have to accept that you'll be making your case to the incredibly bright twenty-somethings who run Congressional offices.
• Archival issues are generally poorly understood, and you need to explain, clearly and succinctly, the value of archives: records safeguard rights and benefits, influence major policy decisions, enable people to connect to family and community history, help to document and correct longstanding injustices, and, in some instances, help to save lives. When dealing with legislators, make the story local -- how do records help their constituents? Have records helped constituents secure benefits to which they're entitled? Are archives attracting tourist dollars to their districts?
• Archivists have substantial competencies and qualifications that can be of use to legislators and other stakeholders. We can help legislators manage the ever-increasing volume of records that they create and can help all stakeholders care for electronic materials.
• Don't listen to people who tell you that you can't do what you need to do; just go ahead and do it. You may be pleasantly surprised by the results.
• Don't forget to state plainly what you want. Legislators, administrators, and board members aren't mind readers.

• In "Replevin: Pros and Cons," Joseph Klett discussed the New Jersey State Archives' new Document Recovery and Amnesty Web pages, which encourage holders of alienated state government records to convey them to the State Archives without penalty, lists records known to be missing, and lists records that have been returned to the State Archives. Most of the missing records listed are enrolled laws of the Royal Colony of New Jersey (1703-1775) and the State of New Jersey (1776-1804), which were alienated from state custody a long time ago and which have been sold openly for decades; in fact, the listing on the Web site is based upon auction catalogs from the 1950s onward. Making these lists, which have been shared with law enforcement, readily accessible alerts dealers and members of the public to the fact that the listed records are the property of the State of New Jersey. This is a good thing -- after these Web pages went live, several people contacted the State Archives and voluntarily returned listed records that they held -- and I hope other states follow New Jersey's lead.

• In "Compulsory Candor? Open Records Laws and Recordkeeping," Pennsylvania State Archivist David Haury noted that new ways of doing government business can eliminate documentation of how things are done. For example, press releases, which were once issued and retained in paper format, are now issued electronically -- and even the electronic master copies may be deleted after the releases are posted on the Web. Archivists and records managers have yet to come to grips with the transitory nature of modern recordkeeping.

Documenting Leadership: media and records

One of the benefits of blogging is its real-time nature: stuff happens, and posts follow almost almost immediately afterward. One of the drawbacks of blogging its its real-time nature: a lone blogger can only do so much. At the end of an intense week, I “desymposed” by watching part of the first season of The X Files on DVD and playing Google’s Pac Man game, the disappearance of which is both an annoyance and a relief. As a result, I’m running behind. However, I hope that this post and its successors will be better for having been written by someone took some time to reflect on what transpired.

One of the most entertaining sessions at the Documenting Leadership symposium centered upon journalists’ use of government records. The panelists were sharp, funny, and expressed a range of perspectives. A number of themes came to the fore:

A reporter’s background knowledge is indispensable. Mickey Carroll (Quinnipiac University Polling Institute) emphasized that most reporters don’t have time to sift through records. They do broad reading, cultivate reliable sources, and draw upon their knowledge of the players and their context-- which takes young reporters a long time to develop -- in order to make sense of the tidbits of information they find.

Rex Smith, Ethan Riegelhaupt, and Mickey Carroll

Records are the lifeblood of investigative journalism. Sandra Peddie (Newsday) asserted that as investigative reporter, she has both the time to file freedom of information requests and the obligation to “get things right.” Government records were at the coreof her exposes of state pension system abuses and the inner workings of some of Long Island’s “special districts,” and she’s currently fighting to obtain records documenting the inner workings of the office of Suffolk County Executive (and gubernatorial candidate) Steve Levy. Ethan Riegelhaupt (New York Times), whose employer has the resources needed to sustain in-depth investigative journalism and mine government databases, concurred, as did Peter Elkind (Fortune), whose biography of Eliot Spitzer relies heavily upon records that illuminated the complexities of his subject’s political decisions and motives.

The needs of government and the press sometimes conflict. Riegelhaupt noted that as former government counsel, he understood that officials need space to determine policy and that closed-door meetings are sometimes necessary; as a newspaper attorney, he also recognized the need for access to information. However, Elkind stressed that although we seem to have reached a point where day-to-day, intense coverage of politics has gotten in the way of getting things done, officials are responsible for running the government and reporters are responsible for unearthing and disseminating information about government.

Proactive disclosure is a good thing. Noting that President Obama has directed federal agencies to disclose federal records in anticipation of receipt of freedom of information requests, Mark Mahoney (Glens Falls Post-Star) asserted that he wanted to see state and local government do the same. Moderator Rex Smith (Albany Times-Union) also endorsed this practice, which may sometimes displease journalists: the city of Chicago has just started posting summary information about all of the freedom of information requests it receives, and as a result the Chicago Tribune and the Chicago Sun-Times can no longer hide their probes from one another.

Peter Elkind, Mark Mahoney, and Sandra Peddie

Freedom of information laws aren’t perfect. Elkind was stunned to discover that the New York State Freedom of Information Law (FOIL) permitted the withholding of records that, in his view, should have been disclosed and that the law doesn’t cover legislative records. Caroll noted that the law itself may have had an inhibiting effect: prior to FOIL’s enactment in the 1970s, many government officials were actually more willing to share records with reporters than they are now.

Journalists are still trying come to terms with the new information ecosystem. Riegelhaupt noted that everyone is still grappling with the implications of having ready access to vast quantities of information and to the perspectives not only of officials and reporters but also of millions of other people. Mahoney noted that in this environment, engaging readers on a personal level -- via blogs such as his (highly recommended) Your Right to Know -- is essential. Riegelhaupt emphasized that the changes are going to be even more profound: newspapers and other media outlets can now publish the texts of officials’ speeches, copies of government databases, and large quantities of other government records. As a result, their Web sites will both serve as the first draft of history. Peddie noted that when newspapers put electronic government records on their Web sites, these sites become important public access points. As a government archivist, I have concerns about ceding this role to the media, but I also understand that the media also has an interest in providing authentic information and that most people will view media sites as sufficiently trustworthy.

Archivists must also come to grips with this new ecosystem. Smith and Riegelhaupt mentioned in passing that archivists are going to have to grapple with whether and how to preserve Web sites, blogs, and other new information resources. As New York State Archivist Chris Ward pointed out, the New York State Archives is working on it -- and so is the profession as a whole. Not surprisingly, journalists also want faster, easier access to the records that archivists have. Mahoney stressed that reporters really appreciate ready access to digitized records: they’re a big help to people who have tight deadlines, and they enable readers to evaluate the quality of reporting. We’re working on that, too -- even if scarce resources force us to move slowly than anyone would like.

Documenting Leadership: keynote and first session

Today was the first day of Documenting Leadership: A Symposium on Public Executive Records in the 21st Century. About 100 journalists, current and former public officials, policy researchers, archivists, and records managers were present, and more will be coming tomorrow.

I’ve had a very long day -- it began at 8:00 AM and ended at 8:30 PM, and the festivities begin again tomorrow at 8:00 AM -- so this post is going to focus on today’s keynote address and the first panel discussion. I’ll be playing catch-up tomorrow and over the weekend.

Richard Norton Smith

Richard Norton Smith, who has written biographies of several presidents and New York governors Theodore Roosevelt, Thomas Dewey and who is currently working on a biography of Nelson Rockefeller, got the day off to a great start: Noting that 16 U.S. Presidents have had gubernatorial experience, he asserted that the fact that 4 of these presidents came from New York really shouldn’t be surprising: New York’s governors have reformed the civil service, altered industrial and labor practices, spearheaded mammoth public work projects, oversaw the development of an effective public health system, championed civil rights, and fought water and air pollution. For decades, New York functioned as the laboratory for federal reform.

He then discussed how his research has led him to value both archival records, which shed unique light on the personal qualities of executive leaders and the nuances of their thoughts and deeds, and oral histories, which may be flawed but help to fill in the gaps in the documentary record. I was particularly struck by his assertions concerning executive leaders’ attitudes toward their records: in his experience, presidents think they did a pretty good job, and they don’t lie awake trying to figure out how to cover up their mistakes or misdeeds. However, they do lie awake thinking of how to prove the Washington Post and the New York Times wrong. Noting that history is always more generous than headline writers, he expressed the hope that New York’s governors would eventually recognize that history is not a continuation of the New York Post.

Robert Ward and Robert Sink

Some of the themes that Smith touched upon returned during the first panel discussion of the day, “Public Policy and the Public Interest.” Robert Ward (Rockefeller Institute of Government) highlighted several factors that may help to account for the divergence between federal and New York State laws governing executive records: declining gubernatorial and legislative interest in tackling the large-scale issues that commanded the attention of the state’s governors for much of the 20th century, the “broad degrading” of political discourse, the State Legislature’s efforts to limit executive power, and growing disbelief in the value of public institutions. Moderator Rex Smith (Albany Times-Union) noted that the relatively new but widely held belief that government is inherently ineffective has also contributed to the situation.

Rex Smith and Ned Regan

Former State Comptroller Ned Regan (Baruch College, CUNY), who transferred his official records to the State Archives when he left office and periodically consults them, asserted that New York’s political circumstances may also be to blame. New York is a “strong governor” state, and its current “three men in a room” approach to politics -- only the governor, the Senate majority leader, and the Assembly speaker wield any real power -- emerged because it’s the only way that the legislative branch can stand up to the executive. However, this system breeds legislative dysfunction, and the fate of executive records is one of many important matters that legislators have not addressed. Robert Ward, who asserted that the “three men in a room” phenomenon is rooted in legislators’ self interest and their constituents’ failure to hold them to account, agreed that legislative dysfunction helps to account for the state’s failure to modernize laws relating to executive records.

Gerald Benjamin (SUNY New Paltz), who will participate in one of tomorrow’s panels, offered an interesting comment from the floor: unlike Smith, whose work centers upon people who have been out office for some time, he’s found that executives dealing with the day-to-day grind of politics often have a short-term interest in tampering with the historical record. Moreover, New York is facing an even more unsettling problem: loss of institutional capacity to deal with records. Responding in part to this comment, Ned Regan offered an incisive observation: “systems are more important than competence.” Indeed: while at the reception, a colleague and I concluded it’s better for an organization to have a long-lived, workaday records management program than to have a records management dynamo whose influence fades after his or her departure.

In retrospect, I’m kind of surprised that none of the panelists devoted much attention to Watergate. I’ve always suspected that one of the reasons that New York lacks a strong executive records law is that, to date, it hasn’t had a governor who has abused the powers of the office as egregiously as Richard Nixon abused those of the presidency. However, all of the other factors identified by the panelists -- the narrowed focus of gubernatorial and legislative effort, the strident tone of current political discourse, cynicism about government, and legislative dysfunction -- are certainly in play.

Robert Sink (formerly of the New York Public Library) highlighted how the problems identified by the other panelists fed a scandal centering upon municipal executive records. New York City had a weak public records law and a weak archival program and Mayor Rudolph Giuliani, who was intent upon reorganizing city government and had a “defiant attitude” toward the concept of freedom of information, slashed funding for the Department of Records and Information Services (DORIS), appointed a commissioner who had no archival or records management experience, and sought to rob it of its independence.

As Mayor Giuliani was preparing to leave office, he asserted that DORIS, which lost half of its staff as a result of budget cuts that he himself had made, couldn’t properly care for his records. He wanted to transfer them to a non-profit organization that bore his name, allow the organization to limit access to certain materials, and engage an archival consulting firm to process them; it soon came to light that he had offered his records to several non-governmental repositories, all of which turned him down.

These actions prompted a public outcry, and the records were returned to DORIS after processing. However, the city has not substantially increased support for DORIS, and its public records law, while strengthened, is still relatively weak. Moreover, even though a reputable archival consulting firm processed the records, the possibility remains that the records were “cleansed” while they were outside public control. If New York City had a strong public records law and a strong government archival program that was insulated from political pressure, the integrity of the records wouldn’t be subject to question.

A lively discussion of the merits of preserving public records in public archives ensued. Bob Sink noted that if a gubernatorial administration works with state archives staff to manage the transfer of records, the records’ chain of custody will be firmly established. Moreover, New York’s state archivist is not a political appointee, and the State Archives itself is not directly controlled by the governor. Archivists in the audience pointed out that state archives are repositories of other government records and knowledge of the workings of state government and that, unlike some private repositories that mandate that the archivists who process a politician’s records share that politician’s political views, they do not factor political beliefs into their hiring decisions.

All in all, a great start. More tomorrow . . . .

Open Government in the Digital Age Summit: David Ferriero

Approximately 150 people -- IT professionals, archivists and records managers, public policy experts, and journalists -- attended the Open Government in the Digital Age Summit jointly sponsored by the New York State Office of the Chief Information Officer/Office for Technology and the New York State Archives.

In lieu of a lengthy recap, I’m going to put up two or three shorter posts -- one centering on the keynote address delivered by David Ferriero, the 10th Archivist of the United States, and at least one other post highlighting the key threads of the day’s discussion. I have several reasons for doing so:

• I expect that all you archivists out there are at least a little curious about the new Archivist.
  
• Certain points and themes kept coming to the fore throughout the day, and it makes more sense to take a little time and tease them out than to do a session-by-session summary.
• I’m on vacation. I have some pretty intense sightseeing plans for the next few days -- which means that I need to break things up a bit and that posts about the Summit are likely going to alternate with posts about my travels. (Where am I? Come back tomorrow, and you’ll find out.)

First, however, a few words about “open government.” As a number of speakers and panelists pointed out, “openness” is commonly seen as being synonymous with transparency of operations and decision-making processes, accountability to citizens, and promotion of citizen participation in the creation of policy and development of services. Moreover, as several Summit participants noted as the day went on, open government requires an ongoing commitment to transparency and accountability -- and to ensuring that essential government information remains uncorrupted and accessible over the long term.

David Ferriero really helped to bring into focus the relationship between records and openness by stressing that if open government is your goal, you should focus on records management. In many respects, his point is obvious, but it’s all too often overlooked: how can you promise transparency and accountability if you can’t find the records that provide insight into government operations or policy development or can’t guarantee their completeness, accuracy, or integrity?

He then focused on the management of electronic records and the immense amount of work awaiting the federal government: a recent U.S. National Archives and Records Administration (NARA) survey of federal agencies revealed that agency records management and information technology staff, who really need to work together in order to manage electronic records appropriately, rarely do so. Moreover, almost every agency surveyed had records subject to a moderate or high risk of loss.

(Sadly, this situation isn’t unique to the federal government: the records management-IT disconnect is a big problem in New York State government and, judging from what I’ve heard from colleagues elsewhere, most other state and local governments are in the same boat. We’re all falling far short of the mark.)

Ferriero then discussed NARA’s own efforts to become more open and citizen-centered. Its Open Government Working Group, which is responsible for enhancing NARA’s ability to interact and collaborate with agency personnel and the general public, will post a formal plan for doing so on NARA’s Web site on 7 April. However, Ferriero gave us a preview of some of the recommendations that may appear in it:

• Establish an ongoing group charged with figuring out new ways of doing business and increasing openness
• Retool NARA’s strategic plan to include open government
• Create staff-only Web 2.0 tools that will enable NARA personnel to share ideas and collaborate more effectively
• Seek to engage the public via Facebook and other popular social media sites -- in essence, go where users are instead of waiting for users to come to NARA’s site
• Redesign NARA’s Web site with end users in mind, update the records management section, incorporate an “Ask an Archivist” interactive feature, and set up a wiki so that researchers can share the results of their research
• Sponsor an Apps for Archives competition (akin to the Apps for Army and other federal competitions) competition for development of applications that will improve access to NARA’s holdings
• Realign digitization priorities and give users ability to see what’s in the digitization queue
• Continue publishing high-value datasets in open formats on www.data.gov -- in essence, move from providing services to providing a platform for others to develop services
• Investigate the possibility of developing a Freedom of Information Act (FOIA) Web dashboard on FOIA services and responsiveness
• Actively declassify records and streamline declassification protocols
• Create an open government Web portal

Ferriero also took questions from the audience and offered up a number of fascinating tidbits:

• NARA is working with the secretaries of the Senate and the House of Representatives to ensure that draft versions of bills are captured and preserved appropriately
• NARA holds about 1 million e-mails from the Reagan administration, 250 million e-mails from the administration of George H.W. Bush, and anticipates getting approximately 1 billion e-mails from the Obama administration. Several of the people sitting around me gasped audibly.
• In an effort to breach the records management-IT divide, he’s convening the first-ever joint meeting of the federal CIO Council and the federal Records Management ??
• Like everyone else, NARA is still trying to figure out how to preserve Web sites, which change constantly; however, it’s plain that sporadic crawls such as those performed by the Internet Archive aren’t sufficient. (I agree; however, it’s just about the only practical approach available to most archivists at this time.)
• When developing new online tools and services, you really need to focus on users’ circumstances and preferences, not your own internal uses of technology; if you don't, you're not really committed to openness. For example, only fifty percent of New York City residents have home Internet connections, but most of them have cell phones -- and want mobile services and applications.
  

The last couple of questions centered on e-mail, BlackBerries, and cell phones, all of which pose particular challenges for archivists and records managers.

• Even the most conscientious employees occasionally use them to send and receive personal messages. Ferriero noted that, in the absence of automated tools that can pick out personal messages, it’s probably easier to keep everything than to conduct a manual review of messages. As an archivist who works with a small but growing volume of e-mail “archives,” I agree wholeheartedly: weeding these archives would be a time-consuming, soul-sucking task, and the resulting reduction in storage costs simply wouldn’t justify the investment of staff time.
• Determining which messages to keep is a particular challenge. I disagree with the approach that Ferriero advocated -- keeping everything -- but I understand why someone charged with upholding the Presidential Records Act and acutely cognizant of the research potential of routine correspondence would advance this argument. There has to be a way to preserve an adequate historical record -- sampling (by individual or by agency), targeting the accounts of key personnel -- without committing to saving every message that passes through an agency’s e-mail servers.

Ferriero’s speech was really well received, and I’m really looking forward to seeing how NARA evolves under his direction. The next few years ought to be really interesting.

Last call: Open Government in the Digital Age Summit

If you're interested in electronic records, open government, and the relationship between the two and will be in (or can arrange to be in) New York State's Capital District this Friday, you'll definitely want to attend the Open Government in the Digital Age Summit jointly sponsored by the New York State Chief Information Officer/Office for Technology and the New York State Archives.

We've got a first-rate group of speakers. David Ferriero, the 10th Archivist of the United States, will deliver the opening address, and Cal Lee of the University of North Carolina-Chapel Hill's School of Information and Library Science will take part in the discussion on the archival implications of government openness. Panelists representing state government, the news media, software companies, and the open source community will explore the meaning of government openness in the digital age and citizen expectations for access to government records. Beth Simone Noveck, the U.S. Deputy Chief Technology Officer for Open Government, will discuss the federal government's openness efforts, and Paul Taylor of e.Republic, Inc. will deliver the closing address. I'm really looking forward to hearing each and every one of them.

The Summit agenda, registration information, parking information, and directions to the Empire State Plaza and a map showing the location of the Cultural Education Center, where the Summit will be held, are all available online.

If you can join us in Albany this Friday, we would love to see you! If you can't, recordings of all of the speeches and panel discussions should be available via the Web shortly after the Summit takes place; once the links are up, I'll be sure to post them. I'm also planning to blog about the Summit, but I may be a little tardy in doing so: for reasons I'll explain later, the end of this week is going to be really, really hectic -- but in a good way!

Open Government in the Digital Age Summit: tentative agenda

Last week, I posted registration information for the Open Government in the Digital Age Summit, which the New York State Chief Information Officer/Office for Technology and the New York State Archives are hosting in Albany on Friday, 19 March 2010 in Albany.

If you're interested in electronic records, government openness, and the relationship between the two and can arrange to be in New York State's Capital District on 19 March, I strongly encourage you to attend this free event. We're still confirming a couple of speakers, but those we have lined up really are first-rate. Archivists and records managers will be particularly pleased to note that David Ferriero, the 10th Archivist of the United States, will deliver the opening address and that Cal Lee of the University of North Carolina-Chapel Hill's School of Information and Library Science will take part in the discussion on the archival implications of government openness. Moreover, I've heard several of the other confirmed speakers at other events and can promise that they'll have interesting things to say on 19 March.

Here's the tentative agenda for the Summit, which is also available online:

8:30 a.m. – 9:00 a.m.

• Registration

9:00 a.m. – 9:20 a.m.

• Susan E. Beaudoin, Counsel, CIO/OFT: Introduction of the State CIO
• Dr. Melodie Mayberry-Stewart, New York State Chief Information Officer/Director of Office for Technology: Introduction of U.S. Deputy Chief Technology Officer

9:20 a.m. – 10:00 a.m.

• Guest Speaker, U.S. Deputy Chief Technology Officer

10:00 a.m. – 10:10 a.m.

• Break
  

10:10 a.m. – 10:30 a.m.

• Geof Huth, Director of Government Record Services, New York State Archives: Introduction of the State Archivist
• Christine Ward, New York State Archivist: Remarks and Introduction of David S. Ferriero

10:30 a.m. - 11:10 a.m

• Guest Speaker, David S. Ferriero, Archivist of the United States

David S. Ferriero was confirmed as 10th Archivist of the United States on November 6, 2009. Previously, Mr. Ferriero served as the Andrew W. Mellon Director of the New York Public Libraries (NYPL) where he developed the library’s digital strategy. Before joining the NYPL, Mr. Ferriero served in top positions at the Massachusetts Institute of Technology and Duke University libraries.

11:10 a.m.– 11:20 a.m.

• Break

11:20 a.m.– 12:30 p.m.

• Panel Discussion: "Meaning of Open Government in the Digital Age"

• Susan E. Beaudoin, Counsel, CIO/OFT, moderator
  
• Andrew Hoppin, CIO, New York State Senate
• Sam Litt, Deputy CTO, New York City DoITT
• Stuart McKee, National Technology Officer, Microsoft Corporation
• Patrick Toole, CIO, IBM (invited)

12:30 p.m.– 1:30 p.m.

• Lunch on your own

1:30 p.m. – 2:40 p.m.

• Panel Discussion: "Citizen Expectations for Access in the Digital Age"

• Michael Ridley, Director of High Performance Computing, NYSTAR, moderator
• John Wonderlich, Policy Director, Sunlight Foundation
• Dr. Steve Sawyer, Associate Professor, School of Information Studies, Syracuse University
• Dr. Jerry Mechling, Vice President, Gartner Research (invited)

2:40 p.m. – 2:50 p.m.

• Break

2:50 p.m. – 4:00 p.m.

• Panel Discussion: "Open Government Implications of Sunshine Laws and Archival Responsibilities"

• Robert Freeman, Executive Director, New York State Committee on Open Government, moderator
  
• Dr. Cailin Brown, Assistant Professor Communications Department, College of St. Rose
• Dr. Cal Lee, Assistant Professor, School of Information and Library Science, University of North Carolina, Chapel Hill
• Robert Port, Senior Editor for Investigations, Albany Times Union

4:00 p.m. – 4:10 p.m.

• Break

4:10 p.m. – 5:00 p.m.

• Closing Address, Paul Taylor, Chief Content Officer, e.Republic, Inc.

Paul W. Taylor, Ph.D., is the Chief Content Officer for e.Republic, Inc. Previously, Dr. Taylor served as the Chief Strategy Officer for the Center for Digital Government. Prior to joining the Center, Dr. Taylor served as Deputy State CIO in Washington State.

5:00 p.m. – 6:00 p.m.

• Reception - State Museum

Save the Date: Open Government Summit, Albany, New York, 19 March 2010

The New York State Office of the Chief Information Officer/Office for Technology and the New York State Archives are hosting an:

OPEN GOVERNMENT SUMMIT
EMPIRE STATE PLAZA, ALBANY, NY
March 19, 2010
9 AM - 5 PM

This one day summit will feature keynote speakers, panel discussions and a reception addressing “hot-button” issues in the "open government" dialogue including:

• The meaning of “open government” in the digital age
• Operationalizing digital openness
• Meeting citizen expectations for access
• What the future holds for openness
• Best practices
• Sunshine Law and archival implications of digital records

This summit is designed to bring government officials, technologists, archivists, and open government advocates together with speakers from across the country to discuss the issue of open government in the digital era.

Detailed program and location information will be available shortly -- and I can promise you that the roster of speakers will be impressive!

A little end-of-year electronic records housekeeping . . . .

Governor Nelson A. Rockefeller and First Lady Margaretta "Happy" Rockefeller with guests on New Year's Eve, 31 December 1970. At noon on this day, Governor Rockefeller was sworn in as governor for the fourth and final time. New York (State). Governor. Public information photographs, 1910-1992. Series 13703-83, Box 3, Number 4408_23. Image courtesy of the New York State Archives.

I'm planning to devote part of the long weekend to tackling some long-neglected domestic chores, and I'm going to start by tidying up some loose ends on this blog. Owing to a combination of time pressures and a slow-to-heal (but steadily improving) injury, I let a few significant electronic records developments slide by without comments. All of them will remain relevant in 2010, so this is a good time to draw attention to them.

• In October, the Arizona Supreme Court ruled (Lake v. City of Phoenix) "that if a public entity maintains a public record in an electronic format, then the electronic version, including any embedded metadata, is subject to disclosure under [the state's] public records laws." This ruling may seem a bit obvious to any archivist or records manager, but it's actually quite significant: earlier this month, an attorney who works for New York State's Committee on Open Government noted that, until now, neither case law nor legislation has really specified whether metadata is covered by state and federal freedom of information laws. As a result, Washington State's Supreme Court, which is set to hear a similar case, and courts in other jurisdictions will likely devote a lot of attention to this ruling. If you're curious about some of the potential implications of this ruling, check out what Ars Technica and Inside Counsel have to say about it.
  

• Since 2003, ARMA and Cohasset Associates (and, sometimes, AIIM) have conducted annual surveys of electronic records management practices. The results of the 2009 survey were released in October, and as you might expect, the results are a mix of good and bad -- very bad -- news: organizations are starting to take action to correct their electronic records and information management problems, but most of them still have a long, long way to go before all of their problems are solved. Moreover, today's electronic records and information management shortcomings are so severe that they may well "jeopardize the future reliability, availability, and trustworthiness of many records. " If you want to figure out how your organization's policies and practices compare to those of others or have any sort of interest in electronic records management, the report (executive summary here, full text here) is an interesting, sobering read.
  

• AIIM regularly offers free Webinars focusing on records, content, and business process management, and archived Webinars are available via its Web site; registration is required. Recently archived Webinars focus on the current state of electronic records management, determining responsibility for records/content management, managing government content in the cloud, and other topics relating to electronic records management. If you're finding it harder and harder to get permission to travel to conferences or training sessions or simply want to keep pace with new developments, you might want to check out these Webinars.

• The New York State Historical Records Advisory Board recently launched a new Web resource, 9/11 Memory and History, that is designed to help survivors and people who lost friends or family on 11 September 2001 preserve photos or letters, drawings or paintings, scrapbooks, sound or video recordings, computer files or digital images, articles of clothing, or other objects. In addition to text-based instructions, the site also includes a number of short videos. Hofstra University archivist Geri Solomon and family member Margie Miller's discussion of what to save and how to save, and staff from the New York State Archives discuss other preservation-related concerns: Director of Operations Kathleen Roe talks about donating materials to a repository, Paper Conservator Sue Bove details how to care for photographs, drawings, and newspapers, and Electronic Records Archivist Bonita Weddle (i.e., Yours Truly) discusses preservation of digital files. Although this site is really targeted to the 9/11 community, other people interested in preserving family history materials and other personal materials should also find it useful.

• Andrew Sniderman has written a reflective, thought-provoking piece about the psychic cost of GMail and other services that unwittingly lead users to document their lives more fully than ever before: digital archives of e-mails, texts, etc., may make it harder for users to deceive themselves about their motives and actions, but they also make it easier for them to fixate on old wounds and regrets. An ever-growing number of people will no doubt agree with Sniderman's assertion that "preservation gives the past more weight than it sometimes deserves," and many professional archivists will no doubt regard their own personal digital archives with at least some ambivalence. However, as Cal Lee pointed out at SAA earlier this year, archivists are ethically obligated to furnish guidance to people who are struggling to care for their personal digital records, so we really should start thinking about what we'll say to friends, relatives, prospective donors, and others who come to us for help.
  

• Finally, some fun stuff. Building upon the success of its first animated short, Digital Preservation and Nuclear Disaster: An Animation, the good people at Digital Preservation Europe have produced two additional videos. Enjoy, and have a Happy New Year!

SAA 2009: Government Records Section meeting

Austin's Congress Street Bridge, which spans Lady Bird Lake, is home to the nation's largest urban bat colony. During the summer months, up to 1.5 million Mexican free-tailed bats live in the crevices under the bridge. At dusk, the bats fly out in waves and search for food; each night, they eat between 10,000-20,000 pounds of insects.

After a brief business meeting, SAA members who attended the Government Records Section got to hear two great speakers address freedom of information issues at the state federal and local level.

David Mengel of the U.S. National Archives and Records Administration (NARA), Special Access/FOIA Staff unit, which handles Freedom of Information Act (FOIA) requests concerning archival materials created by executive branch agencies, discussed recent changes in the federal government’s handling of FOIA requests and recent trends affecting the National Archives.

Immediately after taking office, President Obama issued statements pledging his commitment to transparency, openness, and collaboration within government and instructing agencies that records should be reviewed with a presumption of openness. Shortly afterward, the U.S. Attorney General issued a memo instructing agencies to review FOIA requests with a presumption of openness and issued comprehensive guidelines relating to each FOIA exemption.

Mengel emphasized that these developments constitute a significant shift in the federal government’s commitment to openness. However, this shift also complicates NARA’s work in ways that ought to be increasingly familiar to other government archivists:

• NARA gets a lot of FOIA requests for open records, in part because people don’t know that records are open and in part because they want NARA to do their research for them. The volume of requests covering very large quantities of records is steadily increasing.

• NARA is also trying to figure out how to review large quantities of e-mail and other electronic textual documents. For example, it has yet to find any workable search strategy for locating Social Security Numbers within State Department cables, which have been kept electronically since 1973. NARA ended up sampling the records and discovered that SSNs were typically bundled with dates and places of birth; however, the abbreviations associated with these elements of information have changed over time.

• Federal agencies are running out of storage space, so NARA is getting records that are more recent -- and thus have more privacy issues and other problems (e.g., proprietary information) than older records. It doesn’t help that agencies don’t always identify these issues prior to transfer. In an effort to [identify problems,] Mengel’s unit is trying to get more involved in the appraisal process and looking at records as they come in. However, NARA’s need to review newer records may lead the public to conclude that it’s hiding information.

Mengel concluded by noting that although his unit is trying to improve its handling of FOIA requests, it’s being overwhelmed; there simply aren’t enough people to do the work. Moreover, future developments that NARA views as positive will likely bring new challenges: it looks as if President will soon establish a National Declassification Center under the supervision of the Archivist of the United States, and NARA’s workload may increase dramatically as a result.

Sean Malone, the Records Service Manager of Travis County, Texas, then discussed openness and transparency issues from a local government perspective. He emphasized that in some respects, the situation in Texas is better than that at the federal level: Texas has a strong tradition of openness and transparency, and the Attorney General determines statewide procedures for reviewing and disclosing records. However, local governments in Texas face persistent and worsening fiscal constraints; when times are bad, it’s especially hard to make the case for spending money on records and archives instead of aid to needy families or other essential services.

Malone also highlighted two records issues that pose particular problems for him and for other records managers:

• Resolving conflicts between employees’ First Amendment rights and the public’s right to access government information. Information created on BlackBerries, etc., is a particular problem: people are deeply resistant to the idea that the personal e-mails and text messages on their employer-supplied devices are public records and that work files on their home computers must be turned over.

• The need for better ways to search electronic records and identify materials requested by researchers. Unfortunately, the technology is not forthcoming. For example, Novell’s GroupWise e-mail application, which the county uses, does not have the market share needed to spur development of good e-mail management solutions. Managing e-mail is thus a real problem for the county, and there is a real tension between what the state says it should be doing and what it can do given its fiscal situation.

Malone closed by discussing e-mail issues at the state government level: Governor Rick Perry has ordered that his staff’s e-mail be purged after seven days, and staffers are responsible for printing out records that have longer retention periods. Owing to the sheer volume of e-mail that users receive and the complexity of the state’s records schedules, gubernatorial staffers automatically forward all messages to their personal accounts so that they can retrieve it later if necessary. However, happens to this e-mail when people no longer work for the state?

Great presenters, great presentations, lots to think about.

Government e-records

On the eve of the 4th, here are a couple of examples of the heady new opportunities and horrific challenges that governments face in the digital era:

• The City of New York's NYC Big Apps contest exemplifies how a little creative thinking can make government more open and enhance citizen access to government information. The city is inviting software developers to create applications that enhance access to and use of one or more of roughly 80 datasets created by 32 city agencies and commissions. Officials are still determining which datasets will be released -- and are soliciting public comment -- but some possibilities include restaurant inspection data, recreational facility directories, and citywide event schedules. The developers who produce the applications most useful to New Yorkers will get a cash prize, dinner with Mayor Michael Bloomberg, and opportunities to market their work.

• On the other hand, the plight of Alaska illustrates just how damaging deficiencies in government records systems and recordkeeping practices can be, particularly in an era of hiring freezes and scant funds for upgrades. Alaska officials have announced that, to date, they have devoted over 4,000 hours of staff time (roughly $450,000 in salaries) to attempting to fulfill freedom of information requests for e-mails sent or received by Governor Sarah Palin, who a few hours ago announced her intention to resign from office on 26 July. This unenviable situation is what happens when an unprecedented number of sweeping requests intersects with an electronic recordkeeping system that doesn't facilitate electronic search, retrieval, and disclosure of e-mail.
  

Stonewall riot police reports available online

This month, OutHistory.org features a new online exhibit that was made possible by the New York State Freedom of Information Law (FOIL).

The Stonewall riots, a series of spontaneous demonstrations that erupted after police raided a Greenwich Village gay bar and became the founding symbol of the modern gay, lesbian, bisexual and transgender rights movement, erupted during the wee hours of 28 June 1969. In honor of the fortieth anniversary of this event, Jonathan Ned Katz, the pioneering scholar of LGBT history, has created an online exhibit that features digital images and transcriptions of nine New York City Police Department (NYPD) records documenting the protests.

With the assistance of historian David Carter, Katz obtained copies of seven of the documents by submitting a FOIL request to the NYPD in May 2009. When responding to his request, the NYPD opted not to redact the names of the people who were arrested in connection with the Stonewall riots. As Katz notes, these records identify protesters and police officers whose involvement has not been documented in other sources and suggest avenues for further research.

They also highlight how the NYPD's attitudes have changed as time has passed (or, perhaps, because time has passed): the other two documents at the center of the exhibit were released in 1988 to another researcher who filed a FOIL request (and ultimately sued the NYPD), and at that time, the NYPD blacked out the names of arrestees.

Although I visit OutHistory.org from time to time, I learned about the addition of these records to the site via a New York Times City Room blog post highlighting the new exhibit. This post also includes a brief interview with David Carter, whose 2004 account of Stonewall and its immediate aftermath is widely regarded as definitive.