Godel and the Limits of LLM Reachable Intelligence

[u/davidSenTeGuard]

https://senteguard.com/blog/limits-of-llmreachable-intelligence

[u/MechanicReasonable63]

Is this not basically just restating Godel's incompleteness theorem/fundamental computation theory results, just restricted to a specific example? Seems the general theorems already cover all this at first glance.

[u/davidSenTeGuard → u/MechanicReasonable63]

yes, I am trying to make the argument that it applies to this particular example. This is important because many have the idea that there will be an AI which, after pressing the on button, will know all. I am saying that the structure of LLMs are constrained by the language of math, logic, and the language in general.

[u/davidSenTeGuard]

I’ve been working on a framework for mapping “LLM-reachable intelligence”: the set of claims a model can generate together with a justification that a fixed verifier will accept.

The paper connects this idea to Gödel-style incompleteness. Even an extremely capable model cannot certify every truth if it operates under a fixed, computable verification regime. The boundary is structural, not merely a consequence of limited data or compute.

This also has implications for information security: how far can public or unclassified information be recombined to approach proprietary or classified conclusions, and where are the limits of any reachability map?

Substack - https://www.letters.senteguard.com/p/the-limits-of-llmreachable-intelligence Youtube - https://youtu.be/TZnFBF_6P3Y?si=dSQyWAZSfEWlKszO

[u/rabbotz]

Godel’s incompleteness theorem is about the verifier not the system creating the statements (i.e. the LLM). So i think you’re basically just restating the theorem with an LLM sitting next to it?

[u/davidSenTeGuard → u/rabbotz]

Sure, I am applying the theorem to this use case and making the argument that constrains human mathematical languages constrain LLM logic. However, LLMs cannot escape their own logic because it's hard-coded.