More old SD newspapers to be digitized

In a news release last week, the South Dakota State Historical Society announced it had received another NEH grant – this one for $280,200 – to digitize old newspapers that were published in the state.

The newspapers will be digitized from microfilm already housed at the state archives. The images and the digital text – which will allow people to search across the entire set of newspapers – will become a part of a U.S. Library of Congress initiative called Chronicling America.

The website for Chronicling America allows visitors to search newspapers filtered by state or a single newspaper. One of the search features allows a user to specify separate words that are within 5, 10, 50 or 100 words of each other.

State Archives librarian Kimberly Smith told the Capital Journal it's a feature she uses a lot, to search first and last names, for example, that are within 5 words of each other. It generates some false positives, she said, but will find a person's name with or without middle initials and even if the typesetting put two spaces instead of one between the names.

Using that technique on the South Dakota newspapers already available through Chronicling America, a search of the separate terms "governor", "charles" and "sheldon" turned up the September 22, 1893 edition of the Mitchell Capital, which lists out a schedule of events for the Cornbelt Exposition:

"2 p.m. dedicatory oration by His Excellency Governor Charles H. Sheldon, accompanied by his entire staff in gorgeous uniform."

A search of the "fort," "horse" and "racing" within 10 words of each other produced a snipped from the Pierre Weekly Free Press from July 26, 1906:

"The Presho baseball nine will come to Pierre to cross bats with the Pierre nine Sunday at the Buffalo park up the river. It promises to be a good game and if the Presho boys can beat the Pierre crowd they will do more than any other club has done this year, with the exception of one game won by Fort Pierre. Horse racing and other attractions will also be a feature at the park."

The previous NEH grants that put pages from South Dakota newspapers into accessible and searchable form came in 2014 ($294,665) and 2016 ($240,000).

This year's grant funding will pay for the digitization of 100 rolls of microfilm, or about 100,000 images, archivist Chelle Somsen said.

Somsen said a committee will meet this fall to select the next round of newspapers to digitize. The committee will weigh several factors, she said, including the quality of the microfilm. If the images are blurred they can be hard to read, and hard for the optical character recognition (OCR) software to convert the images accurately to text.

Another consideration is whether the microfilmed copies have a lot of missing issues. They'd like to have a good run of a publication without a lot of gaps, Somsen said. They'd also look at geographic coverage. They'd like to include as broad a geographic area as possible.

Only newspapers published through 1922 are eligible to be digitized under conditions of the NEH grant. Librarian Kimberly Smith said that makes copyright issues simpler.

A member of the newspaper selection committee, historian Jon Lauck, told the Capital Journal that he'll have an eye on other available digitized resources after 1922 that could complement the South Dakota collection on Chronicling America. Organizations like newspapers.com are digitizing lots of newspapers, Lauck said.

It might make sense, Lauck said, to pursue the pre-1922 part of a big newspaper's run if that would, in combination with resources on newspapers.com, complete the paper's run through the 1980s. Even if the complete run had to be cobbled together from different resources, having it all available on some digital resource would be valuable for scholars, Lauck said.

Having a complete run of a major newspaper that's searchable and reviewable without traveling to Pierre is important for what he sees as an overdue project – a full-fledged monographic history of South Dakota.

That kind of history of the state has not been attempted since the Herbert Schell's "History of South Dakota," published in 1961, Lauck said. In that respect, South Dakota is lagging behind other states, like Indiana, where two years ago, James Madison published "Hoosiers: A New History of Indiana."

For the first round of NEH funding, Smith told the Capital Journal, the focus was on weekly publications, because that maximized geographic area for the amount of funds available.

In the second round of funding, some foreign language titles were included – Sud Dakota Nachrichten, Sud Dakota Nachrichten und Herold, Nachrichten-Herold, Deutscher Herold. The OCR software was trained to recognize the old fraktur German font, Smith said.

One possibility for the third round of funding is a newspaper out of Tabor, in Bon Homme County, that was primarily in English, but which included but a column written in Czech, Smith said. A newspaper that includes some Lakota language would also be a part of the mix put in front of the selection committee, Smith said. The State Archives staff is trying to assemble as much information as possible for the committee to review.

The goal of the project, Smith said, is not to "preserve" the newspapers. That's basically been done already for many of the titles. The Capital Journal has bound printed volumes dating back to the late 1800s – and those have already been microfilmed, Smith said. Microfilm will last for 500 years in a proper environment, Smith said and takes up very little space compared to paper copies.

What Chronicling America adds, Smith said, is access to the collection. People won't need to travel to Pierre to look at the old newspapers. And they will be able to search for just what they're looking for.

The digitization work won't take place in Pierre. It'll be handled by the Minnesota Historical Society.

The two previous rounds of grant funding allowed for the digitization of a total of 30,753 editions from 37 different newspapers: The Canton Advocate, Dakota Farmers’ Advocate, The Dakota Farmers’ Leader, The Daily Dakota Farmers’ Leader, Lincoln County Advocate, Canton Daily Leader, The Mitchell Capital, Daily Press and Dakotaian, Yankton Daily Press and Dakotaian, Press and Daily Dakotaian, Forest City Press, The Hot Springs Star, Hot Springs Weekly Star, Turner County Herald, Kimball Enterprise, Saturday News, The Kimball Graphic, The Black Hills Union, The Black Hills Union and Western Stock Review, Pierre Weekly Free Press, The Sisseton Weekly Standard, The Citizen-Republican, Wessington Springs Herald, The State Democrat, The Aberdeen Democrat, Sud Dakota Nachrichten, Sud Dakota Nachrichten und Herold, Nachrichten-Herold, Deutscher Herold, The State-Line Herald, The Lemmon Herald, The Sully County Watchman, The Grant County Herald, The Herald, The Advance, The Herald-Advance and The Madison Daily Leader.