Archivist of the United States Colleen Shogan says the National Archives and Records Administration needs to prepare now for a future where her agency manages potentially trillions of digital records.
Earlier this fall, NARA released a “framework” document that lays out the tenets and goals of the agency’s forthcoming update to its strategic plan. At the top of the list of goals is building “our digital future,” emphasizing the need for NARA to build out its technology infrastructure and embrace artificial intelligence.
Late last week, I interviewed Shogan about NARA’s approach as more and more federal records – the future history of the nation — are managed digitally.
“Right now, we are still a predominantly analog records agency,” Shogan said. “We have more physical or paper records than we have digital records. But that is going to change quickly within the next decade.”
Shogan talked about AI at the Archives, some of NARA’s key digitization projects, and how the agency is using surveys to drive improvements in customer experience.
Days before our conversation, the Wall Street Journal also released a lengthy story reporting that Shogan and her top advisors “have sought to de-emphasize negative parts of U.S. history” as part of a forthcoming update to the National Archives Museum and Discovery Center in Washington, D.C.
I asked Shogan for her response to the story. She pushed back on the Journal’s report, while pointing to some examples of how the National Archives has sought to tell a “a more complete story of American history.”
You can see Shogan’s full response in the transcript of our interview below.
Interview transcript:
Colleen Shogan In part, what’s driving this strategy is preparing for NARA’s dominant digital future. Right now, we are still a predominantly analog records agency. We have more physical or paper records than we have digital records. But that is going to change quickly within the next decade, because we are moving to the transmission of electronic records coming to us from federal agencies. So once those record schedules play out and we start to receive those born electronic records, we are not just going to receive billions of electronic records. We will likely receive trillions of electronic records. And so this strategy, the beginning of a full strategic plan, is really trying to prepare NARA for its predominantly digital future, to make sure that our mission and our values and our goals are aligned with that future.
Justin Doubleday In terms of some of the key actions that NARA needs to take to prepare for this digital future, I know there’s been a lot of effort around electronic archiving, around electronic transferring. What does NARA need to do to really drive toward that future where most records are indeed digital instead of paper?
Colleen Shogan I think the biggest challenge we have ahead is building what I call a system of systems. It won’t just be one single system, but it’ll be a number of systems that will enable us to execute on our mission of preserving, protecting and sharing our nation’s records. That will involve, as you mentioned, the transmission of electronic records from agencies. That will also involve the preservation of those records and the ability for archivists to work with those records in an efficient manner. That’s including trying to work with declassification as well as answering Freedom of Information Act requests.
And then the third part of the system has to be an application that enables all Americans to be able to access those records. To search, like I said, not just billions of electronic records, but trillions of electronic records. To be able to search those records and locate the records that are responsive to their request.
Justin Doubleday And I know that a big piece of the strategic plan, and this was highlighted in a blog that the Archives put out, is artificial intelligence, artificial intelligence, machine learning, automation, all those things. How do you view AI as critical to the future of the archives, and how will it change how Nara carries out its work?
Colleen Shogan I think it’s necessary for the future of the National Archives, because we are going to be taking in so much information, it will be impossible for any number of archivists to be able to have a handle on that volume of records. We will need to have human beings working alongside and utilizing those technologies and refining artificial intelligence so that we can make those records discoverable for the American people. There’s no way that we would be able to do that efficiently and effectively without the assistance of artificial intelligence. I know that there are concerns about artificial intelligence, so we are following the guidelines that have been established by the Biden administration and the federal government about the use of AI. And we are utilizing those guidelines to make sure that we proceed in a cautious but effective manner.
Justin Doubleday And are there specific use cases of AI that you’re already working on or have in your future plans that you think are particularly important?
Colleen Shogan We have a couple examples that I’d love to talk about. So in 2022 the National Archives released the 1950 census. Of course, the 1950 census was handwritten. The Archives very successfully utilized the form of artificial intelligence in order to be able to index the 1950 census. Artificial intelligence has progressed to the point where now it can read human beings handwriting, and obviously a diversity of handwriting, because the census was taken by many, many thousands of census takers across the country, and it’s been very successful. I was able to locate both of my parents in the 1950s census, including my mother, who had a very common name in 1950 — Patricia Smith — and I was able to locate her in the census.
Also, I think there are a lot of applications, yes, for improving declassification and hopefully reducing the FOIA backlog that we have at NARA and in other agencies, but also for our veterans records. We hold all of our nation’s military service member records in our facility in St Louis, Mo., and we did do a pilot project to reduce the backlog of requests that we had previously, and it was a very exciting [pilot] that used robotic technology to look at a very digitized, pretty complex military service record. And being able to look at that, those hundreds of pages, but pull out the DD-214, which is the one page summary of a military service member’s service record. And a DD-214, doesn’t answer all of requests that veterans require, but it does actually answer the vast majority of the requests that that we have that come into our facility in St Louis.
So as we digitize those military records, and we’re able to use artificial intelligence, we can actually answer these easier requests much more effectively and efficiently, which means our archivists and our archivist techs can then spend more time on the complex requests and getting those answered and back into the hands of the American citizens and service members and veterans who’ve requested them.
Justin Doubleday Yeah, that’s a really interesting example, and of course, it talks to how digitization of analog records is also such an important part of what narrow is doing and to make them digitally accessible. I know that that records backlog for veterans was a huge focus for NARA when you first came in. Is there any update on where you are in digitizing those records with the VA and are there other digitization projects that you think are particularly impactful for NARA going forward?
Colleen Shogan We are working with the Veterans Administration to digitize those records. We are starting with the most recent military service Records and moving backwards. Because you can imagine that the most recent records are the ones that are the most requested by families and service members themselves. It’s still going to take a quite a bit of time to be able to digitize all of those records going back to the beginning of the country. But I think actually starting from the most recent, going backward, will really have a big impact I would say, in the next four or five years.
We have a new digitization center in College Park, Md., that we opened in the past six months that will increase our digitization capacity at the National Archives by ten-fold. So we’re very excited about that. We have 50 new employees that work every day at the National Digitization Center preparing documents for digitization. And one of the records groups that we are working on right now is our Alaska Native records.
A long time ago, the National Archives had a facility in Anchorage, Alaska, but that facility was closed well before my tenure, and those Alaska Native records were moved to our nearest facility, which is in Seattle, Wash. Now that’s great. We try to keep the records near to the populations and communities that use them, but that’s still quite a bit of distance for Alaska Natives to have to travel to make use of those records. So what we did is we are moving those records to the digitization center in College Park, Md., piece by piece, and we are going through and digitizing them. As we’re digitizing them, we’re then returning them back to Seattle and bringing more records forward. So we’re making real progress with that record series, which I think is very important.
Justin Doubleday That’s fascinating. And that gets to another pillar of this strategic plan framework, which is improving user experience, customer experience. What do you think the approach here is going forward for NARA? Are there other areas where you think there are strides to be made in improving, researcher access, public access, the experience overall of kind of interfacing with the Archives?
Colleen Shogan One thing that we’ve done since I’ve been the Archivist of the United States, I’ve really challenged some of our senior leaders for some of these experiences that we want to find out exactly how we’re doing, is that we need information and data about that. So for the first time I think ever in the history of the National Archives, we have done two really important user surveys. One for users of our catalog online. How is that experience? Are you finding what you’re looking for? And we’re doing repeated surveys – very important from a data perspective – so that we have some longitudinal data, so we can see, you know, are the findings that we’re coming up with, are they consistent across time, as those user groups might change over time? So we’re starting to gather data there.
We are also doing wholesale surveys of people who come into our research rooms, not just here in Washington, DC or in College Park, but really across the NARA system of all of our archival facilities and our presidential libraries, and we’re getting really interesting feedback about, yes, it’s really helpful that we have pools ahead of time, before we show up. Is there a way that we can maintain service, even through lunch hours? That’s one piece of feedback that we’ve gotten. Is there any way that we can provide evening service or weekend service? Which I would really like to do, but that is truly a resource issue. So we have to constantly evaluate about how where we are spending our money, our appropriation, and are there ways in which we need to maybe reallocate so that we can make sure we’re meeting the needs of that user community.
Justin Doubleday And so changing gears here a little bit, but I think this is still connected to the strategic plan, and it’s in the news this week, so we need to ask about it. But the Wall Street Journal’s report on, in their words, you and your senior staff have sought to deemphasize negative parts of US history as part of a forthcoming update to the Archives Museum and Discovery Center. We’re talking about the strategy, the future here for NARA. What is your response to this report and its characterization of your approach?
Colleen Shogan Well, I do not agree. I do not believe that the Wall Street Journal has an accurate depiction of our exhibit program at the National Archives. We are building a new exhibit that will be open next fall, a long standing exhibit at the National Archives. A much needed update that is being done, but we are including many, many stories from the National Archives in that exhibit from communities all across the United States, and we are still in development for that exhibit, and all the decisions and choices have not been made yet.
We are very, very confident that when Americans come to visit the National Archives, as we lead into the 250th anniversary of the United States, we anticipate we will have increased visitor-ship at the National Archives. We are convinced that people will very much enjoy this exhibit and will see their stories reflected in their experience and also in the totality of American history. So I encourage all listeners, it’s still going to take us a little while to open it. It’s going to be open around this time next year, in 2025, I urge your listeners to come and see the exhibit and check it out.
Justin Doubleday Totally understand that these decisions are still being made, and it’s a balance that you need to strike. One aspect of the current strategic plan that you’re now in the process of updating is a huge emphasis on equity and increasing access to records that are important to underserved communities. Just within the context of this conversation, are those still priorities for you under this new strategy going forward? Will we see that reflected in the strategy and the decisions that NARA makes going forward?
Colleen Shogan Absolutely. In fact, we had just previously talked about our digitization efforts. We have 13.5 billion records here, and we have about 350-360 million that are online and digitized. So we still have a ways to go. That means we have to prioritize what records we’re going to be digitizing in the near future. We talked a little bit about the Alaska Native communities. One of the driving criteria that we use to make decisions about prioritization of digitization is to digitize records of previously underserved communities. We also are really very sensitive about records that get repeated use in our research rooms, so that we can put those online so more Americans can use those records. But that is a driving factor in how we make those decisions for digitization, and certainly a factor about how we make decisions for exhibits as well.
Justin Doubleday And just last question on this, and I just want to make sure we asked about this, I think the Journal report also cast NARA making decisions to potentially appease certain lawmakers. Is that how you’re making decisions about how the Archives puts out certain materials in museums and things like that?
Colleen Shogan Absolutely not. In fact, I have never had a lawmaker comment to me in many of the meetings that I’ve had on Capitol Hill, any complaints about our exhibits. In fact, the rotunda where we house our founding documents – the Declaration of Independence, the Constitution, the Bill of Rights – that rotunda has been really in the same those three documents have been featured in the rotunda since the 1950s it’s really a terrific experience to come and see those documents in person. But I’ve made the decision that we’re going to be adding two additional documents to the rotunda for permanent display: the Emancipation Proclamation and the 19th Amendment, which removes restrictions on voting for women, the largest single enfranchisement in American history.
And when I go around, I’m going to be honest with you, when I go around and talk to lawmakers about these two editions of the Emancipation Proclamation and the 19th Amendment, I hear nothing but support for these additional documents being on permanent display because lawmakers want the National Archives to tell a more complete story of American history. So, no, that is not accurate.
Justin Doubleday And obviously we are in the height of election season, and NARA plays a critical role in transitioning from one administration to the next. I know you put out an update just this week to agency heads on the importance of federal records law through this transition. Can you just talk about what NARA is doing to really make sure that records are preserved as staff come in and depart?
Colleen Shogan Absolutely. We have a very good presidential transition team that is in place. In fact, I met with the leaders of that team yesterday, and they gave me a very in depth brief. We are on track. Yes, we have leadership changes potentially in federal agencies, so that’s why that letter went out. And of course, we know that there’s the end of a presidential administration, because President Biden is not running for reelection. So we’re preparing for that time period, January 20 at noon, when his term will end, and there will be the transfer of both the electronic records from the Biden administration, which is the vast majority of the records, and any paper or analog records. And we are on track for that transfer and transition. There’s been a lot of good work between the National Archives and the White House. And of course, if there are any questions from federal agency leaders about questions about transition for record keeping, we are here and poised and ready to answer and help and assist.
Justin Doubleday We started out talking about the strategic plan framework, which is launching this new forthcoming strategy. What do you see as the plan going forward into 2025 as you pull together this new strategy and start to implement it potentially in the future?
Colleen Shogan The reason why we released the framework first, which is really just the outline of the strategic plan, is that we wanted to get input from, of course, employees here at the National Archives, and those listening sessions are already beginning. We’re collecting information and feedback in a number of ways from staff. We’re also actively soliciting input from a lot of our stakeholder communities across the United States, whether that’s user communities, or from the library and archival world and Congress. We’ll be soliciting that information. And depending on the feedback we get, we may need to make adjustments to that framework. Maybe we’ve missed something, maybe emphasize something too much. I don’t know what the feedback is going to be that we get in, but we’re poised to make those adjustments.
And once we’re satisfied with that feedback, then of course we’ll be outlining the specific goals that will be underneath these major points in the strategic plan. I will rely upon my senior leadership team to help outline what those specific goals are going to be. And as any good strategic plan will have. We will need to have benchmarks for those goals to make sure, not just by certification that we’re fulfilling the goals, but empirically, that we know that we’re doing what we’re supposed to do so we can start the transformation once again of the National Archives from a predominantly analog Archive to a predominantly digital Archive.
Copyright
© 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.
