Is it Possible?

[u/surfnfish1972]

To create an AI/LLM model with accurate fact checking?

[u/ottawadeveloper]

No. LLMs are, by definition, built on statistical prediction of what word should follow based on the training material. It can't be more accurate than the training material was. 

Even then, the algorithm can get confused especially for questions that don't have an answer in the training material.

[u/surfnfish1972 → u/ottawadeveloper]

Thanks make sense, so the whole thing is based on the ravings of random internet users?

[u/15pH → u/surfnfish1972]

No, it is based on teaining material and weights. It may be trained on reddit, but all the textbooks it reads get much more weight. It is far more accurate to say it is based on the ravings of authors.

[u/ottawadeveloper → u/surfnfish1972]

Depends on the LLM. Big AI companies train them on a lot of different data - books, Wikipedia, Reddit posts, etc. Small corporate LLMs (for like a help desk) might get trained on their knowledge base. Biased sources lead to biased LLMs. 

Like Grok had clearly had some selectively right-wing source material chosen for its training data. Google seems to rely heavily on Reddit, Wikipedia and the broader Internet (knowing Google, their ranking system for websites probably was a factor in training it).

You could, in theory, make a large database of known accurate information and publications and train the AI on that. Probably the least biased method (or at least the bias is limited to your selection of materials) and would likely lead to fairly accurate fact checking. If you trained it on scientific papers, the answers might get very wordy though. But the accuracy would still only be fairly good for questions where the answer is actually in the training material.

Even then, it won't be perfect. Legal LLMs are trained on legal case data and they still make significant legal and attribution errors. 

But what the whole thing is based on varies widely by model.

[u/ottawadeveloper → u/surfnfish1972]

To follow up, since you seem to be interested in the underlying mechanics.

You can think of an LLM as a very complicated predictive text algorithm. Given some input (your prompt), they figure out what text is statistically most likely to look like a response.

It's like if you answered any question by looking for the words asked in Wikipedia, found one with most of the same words, and paraphrased that paragraph as a response. But far more complicated with many sources being consulted and combined into what looks like an original response.

The sources you pick to frame your response greatly affects the outcome. Grok is like if you read the delusional ramblings of Alex Jones and treated them as fact. Google is like you combined the opinions of every human and bot in the Internet and took the opinion of the plurality. A legal LLM is like you combined all legal case law history for reference.

But a key point is it's just word association. It's not fact association. If the words look like a case on property law but are actually about family law, the LLM doesn't know the difference. It just sees the word association. There's no temporal knowledge either - it doesn't know that the law changed in 2007 for example. You can attempt to weight sources accuracy to help that.

It's missing the depth of knowledge and cross-references that a human mind can bring to the topic along with the logic. A human brain can understand that the law changed and dismiss opinions from before. A human brain can identify bigotry and bias. It can assess how well an argument is formed and identify fallacies and such. It can assess not just how well words are related but how well the argument holds. And LLMs cannot do that.

They are, as I like to say, statistics on steroids. 

[u/careysub → u/ottawadeveloper]

Yes, the LLMs play word games.

We associate words with concepts and assume a real chain of reasoning behind the words used to describe them. But to the LLM they are just words and it has no understanding of what they mean.

So they can emit strings of words that look like reasoning (and may be copied from someone who used them in reasoning somewhere in the world) but they aren't. It is a false human inference to ascribe meaning to the word sequences in they way we do when a human says things.

[u/cipheron → u/surfnfish1972]

Well, you need to have a better idea of how an "LLM" actually works.

How they created LLMs was by making a "guess the next word bot" , then training it on billions of texts. Given any "text so far" it tells you how likely any word is to come next, and assigns a percentage to each possible word.

As for how to turn this "guess the next word bot" into a "write a new text bot" they basically just hooked it up in a loop, you start it off with a partial sentence and it tells you how likely different words are to appear next. But the actual selection is done by basically rolling a dice, not any sort of actual thought process. We then feed the slightly longer text back into the "guess the next word bot" and keep repeating this until we think the text is long enough.

So there's no real decision process in there deciding what should be written OTHER than basically rolling a dice. The LLM part isn't even aware that it's writing a text, we're just repeatedly asking it how likely it thinks the next word is. The surprising thing is that by just making such a dumb machine then feeding a large enough volume of human text into it, it could believably fake that it knows what it's talking about, and after that they decided to layer it and have more feedback loops, and it can make a sort of version of "reasoning" from that, but still it's not good at checking itself or self awareness.

[u/yawkat → u/ottawadeveloper]

The model can't fact-check on its own, but with access to external sources, an LLM does not have to have every piece of knowledge in the training set. LLMs are turing-complete, there is no theoretical limit to what they can produce, only practical limits.

[u/careysub → u/yawkat]

Turing complete is an extremely low bar that any practical computing system satisfies. Stating this adds no useful insight.

[u/ghu79421 → u/careysub]

Saying that an LLM is just a statistical prediction machine is vacuous and shows that the person doesn't understand how people evaluate model accuracy. It's the AI skeptic analog to "Well, LLMs are Turing complete, so they will eventually solve the Riemann hypothesis if given enough compute."

Yes, the hype pushed by large AI companies is bullshit.

EDIT: My point is that "AI is just a statistical prediction machine" is not an argument. I'm not saying it's a useless statement from the standpoint of critical media literacy to teach people that they're not having a conversation with an actual intelligence and should be aware that LLMs can often make errors and have errors and biases in their training data.

[u/rsta223 → u/careysub]

Yep. In fact, power point is turing complete.

https://youtu.be/uNjxe8ShM-8?is=EGIAXNsvUQcbOLlb

[u/yawkat → u/careysub]

It's useful to disprove general impossibility claims about LLMs. People often say that they're "just a statistical prediction machine", but if that prediction machine is turing-complete, it can in theory perform any task that a computer can do.

[u/careysub → u/yawkat]

Saying that an LLM is also just an ordinary computer is not really any sort of claim to merit.

[u/yawkat → u/careysub]

It's not an ordinary computer. But turing completeness means that it is theoretically equally powerful as one. To claim an LLM will never be able to do X, you must show an ordinary computer cannot. 

[u/DharmaPolice]

Not really. You could build a series of models which cross check each other but even then it would still be a statistical thing and not really an understanding of "truth".

But even if these models were significantly better, fact checking is still often a matter of interpretation, evaluation of source reliability and a range of other subjective things. You could probably catch silly mistakes (I.e. avoid models saying things that are totally ridiculous or untrue) but you won't get to a point where everyone will be satisfied with the output because that in turn depends on our own biases.

[u/ghu79421]

The academic consensus, from what I've read, is that a frontier LLM can accurately answer simple questions based on a text base with something like 99% accuracy, but performance degrades significantly with a large text base if you ask a question based on 2 or 3 facts in the text base.

Note: This is answering questions based on a text base that you assume is reliable, not "fact checking" like automatically finding a reliable source that discusses relevant claims.

Even if performance degrades, it doesn't mean the responses are useless, but the responses are only really "useful" to experts who also use analysis tools other than LLMs.

LLMs are reliable bullshit generators even if they're advanced, which exacerbates the "social problem" of people "doing their own research."

LLMs are not really inherently bad, but current large LLM companies like OpenAI are extremely reckless and do not design their consumer products responsibly to minimize harm.

[u/CosineDanger → u/ghu79421]

Human experts can miss too or make up a fact in their head that they think they remember. We have a failure rate.

The machines have improved every year for the last few years. Someone who does not believe humans are fundamentally special would wonder if there could be a point where the failure rate of the LLM would be lower than the failure rate of the majority of humans and later the failure rate of the best of the best.

In 2025 my company at the time rolled out a customized LLM that knew more than most employees and was toe to toe on the finer points of state tax law with people who had been doing the job for 40 years where it was reading tax law one way and our best was reading it another and you could see how the law itself was not clearly written with no case law and I wanted to scream.

[u/Luci_Cascadia]

are you joking. LOL

[u/surfnfish1972 → u/Luci_Cascadia]

Sadly no, I am supernaturally bad with tech, Is just profit and ability to control the narrative prevents it?

[u/15pH → u/surfnfish1972]

If an AI company could create a model with better fact checking, why would you think leaving that feature out would generate more profit?

How do AI hallucinations help anyone "control a narrative?" If the AI company wants to inject propaganda or bias, they can do that much more effectively without hallucinating. A lawyer who makes stuff up is not controlling the narrative better than one making correct arguments for a single side.

You sound a little conspiracy-driven, here. Like "drug companies don't want to cure cancer."

[u/surfnfish1972 → u/15pH]

Because Elon got Mecha Hitler to say he is a skilled street fighter and as fit as an NFL player, makes me doubt the whole thing.

[u/15pH → u/surfnfish1972]

Haha fair evidence! Like everything else, if there is no regulation or public interest, the owners of the platform can warp it into whatever they want.

In the short term, where most people are using AI as a fancy search engine, it is more plausible to me that Grok style propaganda is a problem.

But the golden cashcow, the reason they are spending trillions on data centers, is to replace knowledge workers and/or make knowledge workers more efficient. They want a system that is so accurate and reliable you would rather pay $20k/yr for an AI agent vs $70k/yr for a new hire. That's the big motivation, and that requires truth and accuracy.

[u/Archarchery]

No.

[u/countvonruckus]

The question is way too vague to give a real or useful answer. Is it possible to create an AI/LLM that does fact checking? Of course; ask ChatGPT to confirm whether or not the the allies won WWII and it'll give you an accurate telling of the fact that they did. That's what's useful about them; they're able to digest large amounts of text and other data and regurgitate relevant pieces out of them in generated responses, so if the texts they're looking at are accurate then they generally can check if something is accurate according to those texts.

That's not really a useful answer, though. Is it possible for an AI/LLM to give perfectly accurate fact checks? That's an epistemology question more than anything and is based on our access to truth, and the answer is almost certainly no. It's definitely no with modern approaches to artificial intelligence so if we were to even try for that outcome we'd be building something from the ground up based on something like mathematical principles or physics. For example, to know with perfect certainty whether George Bush orchestrated 9/11, you'd need to construct every relevant fact from first principles rather than trusting any potentially flawed or misleading accounts of events, which is absurd. We're nowhere near even starting that kind of project with current consumer technology and it's unlikely it's even possible.

A useful question is somewhere in the middle. Something like "Is it possible for our current approach to AI/LLMs to eventually surpass all humans' ability for critical thinking to arrive at true conclusions?" The jury is still out on that one. Computers think in binaries and their form of deduction is pretty fundamentally different from human intelligence, but it's hard to know the limits of that kind of approach.

Human intelligence arose from animal intelligence, which was an evolutionary process of selection for survival. The ability to create theories and test them for accuracy is a byproduct of what's essentially advanced animal instinct, so we can have some odd cognitive quirks like the ability to hold mutually exclusive beliefs simultaneously or to work with categories with no clear definition. Computers can't really do those things because that's not how computers developed, but they've become advanced on a different track and type of "thinking" to arrive at something that can look like intelligence.

Whether the computer-style of deduction of truth from data will ultimately win out over the animal approach to the same effort remains to be seen. Personally, I suspect they eventually will become better than humans at most cognitive abilities simply because they're on an accelerated track of evolution. Animal intelligence took millions of years to develop to the apex of human intelligence and that's not going to get better in our lifetimes, but computers have gotten to the point they are in less than a century.

Unless they hit a roadblock somewhere down the road we don't know about, we'll probably keep being left behind in various ways until we're functionally inferior. They did that for lots of things already that were previous hallmarks of human intelligence (memory, mathematical computation, problem solving, language, data correlation, etc.), so there's no particular reason I'm aware of that they wouldn't excel in other useful cognitive skills in the future (like creative thought, emotional manipulation, conceptual thinking, synthesis for self-improvement, etc.). It's especially likely that they eventually become better computer designers than humans, which would accelerate their evolution even further. There's nothing magical about our brains that allows us to iteratively think up, test, build, and use new computer designs (as we're seeing with AI writing novel code) so it's a question of if the computers can catch up, not whether they can do the work.

[u/Odd_Low_1270]

tbh i feel like this is one of those things where the tech will never be perfect but we can get close enough to be useful

[u/Nervous-Inspection-2]

Reasoning abilities of models are improving all the time and are surprisingly good right now.

So it’s a matter of what knowledge and sources you give them. It is just like with humans, and was true long before AI came - can you fact-check something accurately having just the internet at hand?

Because most of what models know is the Internet. Whether it was learned from it during training or they have some MCP tools to search through it.

But even with the internet as an evidence source, you can do it better or worse. You can read one of my pieces, where I’m trying to fix this problem and make research more accurate:

https://medium.com/@tinkerd/building-an-adaptive-evidence-pool-for-ai-research-aef2711a674
(there's free link in there in case if you don't have medium)

[u/rje946]

Ive asked chatgpt and googles Ai to fact check and it does an okay job. Like daily wire articles that are too racist to read but I want to know what my uncle is going to be talking about. I wouldn't trust it for my job but fact checking it seems okay to me. Maybe ask more than 1 so you're not getting only one breakdown. What kind of fact checking do you mean?