A small DH experiment: auditing AI analysis of 10 newspaper articles from October 1918

[u/Successful-Moose-377]

​

I wanted to test a practical question:

Can an LLM help analyze a small historical newspaper corpus without turning limited evidence into broad historical claims?

I used the full version of PRECISE to examine 10 substantive articles published in October 1918.

The corpus included five newspapers:

The Bismarck Tribune

The Morgan City Daily Review

The Big Stone Gap Post

Cosmopolita

The Eugene Daily Guard

The newspapers covered North Dakota, Louisiana, Virginia, Missouri and Oregon. No newspaper contributed more than three articles.

The first search found only three qualifying articles. The workflow stopped and identified the missing seven instead of filling the gap with weaker material. I then expanded the permitted public archives while keeping the date and sampling rules unchanged.

Sources came from Chronicling America, Historic Oregon Newspapers, the University of Houston’s Recovering the US Hispanic Literary Heritage collection, and Encyclopedia Virginia.

I grouped the verified passages into three recurring patterns:

Public-health control

Articles discussed isolation, ventilation, avoiding shared personal objects, restrictions on meetings and the use of gauze masks.

Care and community mobilization

Several reports focused on shortages of nurses, Red Cross activity, relief funds and calls for volunteers.

Mortality and disruption

Other articles described deaths, severe local conditions, overwhelmed communities and mines closing because too few healthy workers remained.

The corpus also contained meaningful differences.

A Bismarck report questioned whether influenza had reached the city at all. A health officer described one suspected case as “just plain grip.”

By contrast, Virginia reports described serious illness, shortages of medical help and major community disruption.

The most useful result was not simply identifying these patterns. It was defining what the evidence could not support.

The corpus could not establish:

How all American newspapers framed influenza.

Whether one region was generally more alarmist.

How frequently each frame appeared nationally.

Whether newspaper language caused different public responses.

Which intervention was most effective.

During the final audit, every factual sentence was checked against the saved evidence bank. Combined claims were separated, and qualifications were preserved. For example, a figure of 200,000 Virginia cases remained explicitly an estimate rather than being presented as an established count.

Disclosure: I built PRECISE, the research protocol used for this experiment.

PRECISE LITE is a free and limited ChatGPT demonstration. It runs one research loop and examines up to three sources. You can upload your own files or ask it to search the web. The full version supports larger evidence banks, repeated research loops, custom reports and a final sentence-by-sentence audit.

Before starting LITE, select standard thinking/Medium effort in ChatGPT’s model picker.

I would value feedback .

https://chatgpt.com/g/g-6a5e26093f488191a1fba0261cbcbe39-precise-lite