Honors Project
Chatbots from Major News Publishers
About the Project
This honors project explores the chatbots created by four major news publishers, examining how AI-powered conversational interfaces are being integrated into journalism and news delivery.
As artificial intelligence improves, nearly every sector has had to make tough choices about how - or whether - to incorporate it into its operations. In journalism, that conversation is especially prickly. LLMs tend to hallucinate information and not cite its sources. The mission of a journalist is the exact opposite: tell the truth and how you know it's true. However, avoiding generative AI altogether is another gamble. News organizations know they must keep pace with the latest technology to stay relevant. And the power of AI offers alluring efficiency to newsrooms struggling for resources.
Some outlets, like the New York Times, have sued LLM creators for drawing from their content without permission. For other outlets, the solution is to make their own in-site chatbots trained off their own reporting. Below, I look at the differences between the LLMs of four major news outlets: The Washington Post, The Financial Times, The Atlantic, and Business Insider.
News Publishers Studied
Ask the Post AI
This is The Washington Post's chatbot, which draws from articles published since 2016. It uses the ArcXP model.
Its answers were short and concise, and avoided being overly-confident. When it didn't have an answer - and it often didn't - it said so. When it did, it cited five articles with every answer.
Ask FT
This is the Financial Times' chatbot, which gets its answers from material published on the site since 2004. It uses Claude 3.
Its answers were longer and more detailed, often citing upwards of a dozen articles.
Atlantic Companion
This is The Atlantic Monthly's chatbot, which uses OpenAI.
This chatbot is clearly labeled as an experiment, and it acts as an advanced search engine. Its primary purpose isn't to answer questions on its own, but instead refer users to articles that might.
Business Insider
This is the Business Insider's chatbot, which also uses OpenAI.
This, like the Atlantic's chatbot, seems essentially act as a search engine. It gives short bulleted answers to queries, then recommends relevant articles from the publication.
Comparing Outputs
On March 11, 2026, I asked each news LLM the same question: What has happened to the price of oil in the US since the start of the Iran war?
I choose this subject because I knew each outlet had covered it. But I was curious whether the LLM would reflect the difference in how each outlet covered it. Below is how their answers stacked up.
Ask the Post AI: The price of oil in the US has increased significantly since the start of the Iran war. The average cost of a gallon of regular gasoline in the US increased by 34 cents in the past week to $3.32, and then to $3.48, with diesel prices surging even faster, reaching $4.78 a gallon. The price for a barrel of Brent crude spiked to nearly $120 at one point, before dropping to under $90. The conflict has disrupted the flow of oil through the Strait of Hormuz, leading to a surge in oil prices.
Ask FT: I cannot include Ask FT's answers in this project. When I first copied and pasted the answer into a private Google doc, this warning also printed: Copying the full text of Ask FT answers to share with others is a breach of FT's T&Cs and Copyright Policy. If you publish or redistribute any "extracts" or "abstracts" (as per the Copyright Policy), you must state that AI generates Ask FT answers. And that FT staff have not checked or moderated them.
Atlantic Companion: The selected articles delve into the dynamics of the oil market in relation to geopolitical tensions involving Iran, including the impacts of conflicts and sanctions on global and U.S. oil prices. They cover the implications of such events on the economy, discuss factors behind price changes, and explore the strategic and economic interactions between the U.S. and Iran. These articles collectively provide a comprehensive overview of how the price of oil has been influenced by the geopolitical situation surrounding Iran, fulfilling the reader's inquiry into the effects of the Iran war on oil prices.
Business Insider: - Business Insider reports that oil prices have surged past $100 due to the Iran war. - Recent coverage highlights oil price volatility as it fluctuates around $80 amidst ongoing conflict. - Stocks have fallen as rising oil prices threaten inflation, indicating a turbulent market outlook.
Key Findings
For this project, I wanted to explore Ask the Post AI in more depth. I found that it was wary of being overly confident - a common pitfall of LLMs. In fact, if anything Ask the Post AI was too hesistant to answer questions. It responded "I don't have the details on that. Is there another topic you'd like to explore?" to even simple inquiries.
When it did provide summaries of The Washington Post's reporting, it was sometimes a little sloppy. It's not that anything was innaccurate, per se, but I found that when it came to attributing information, it did not always match the thoroughness of the articles it was citing.
For example, I asked it on 4/16/26 about the former Virginia Lieutenant Gov. Justin Fairfax, who fatally shot his wife and then himself. The Washington Post reported that in the weeks leading up to the incident, Fairfax was acting increasingly erratic. They clearly attribute this information to "court records" in the headline of their story. But when I asked the chatbot about it, it wrote the following: "He had become increasingly isolated and was behaving erratically, with records showing 'heavy daily alcohol' use."
It might not seem like a big difference, but in the world of journalism, it is. "Records" are a different thing than "court records." And it's not totally clear whether the statement "he had become increasingly isolated and was behaving erratically" is attributed to the same records. In the actual article, these distinctions are clear. But do journalistic standards around attribution still apply to newspaper chatbots? Are newspaper liable for their chatbots' hazy sourcing? I suspect these questions will have to be answered as the news media's relationship to artificial intelligence continues to develop.
This honors project was completed as part of the DH340 course requirements.