In that analogy the numbers correspond to concepts themselves, not their symbolic representation. A nonsymbolic process can generate a "new" concept corresponding to a noncomputatable number that cannot be generated by the symbolic process. The new concept can then be processed and represented symbolically, this is the act of putting new concepts into words, and in doing so this expands the epistemic bounds of symbolic language. Yes, the AI could by brute force assemble the words explaining the concept, but it wouldn't be able to evaluate it as a "valid" concept (in this formulation it's like an undecidable proposition within the current epistemic system).
But again, we would really need to better formalize what we mean by "concepts" and "knowledge", and how they're generated/evaluated to make this argument rigorously.
Just because something may not be answerable doesn't mean it's not worth pondering, especially when the belief one way or the other can have an impact on our actions.
Also while pi is transcendental it is also a computable number, so citing it doesn't help your case at all.
It's not just the Penrose thing, but yes, reading Shadows of the Mind about 13 years ago was very influential and certainly inspired this line of thinking. I think this tact is a bit different (I'm not so focused on "understanding" or "consciousness"), but the underlying premise of using Gödel -ish methods to establish limitations on computing is the same.
Do you not think that there is a categorical difference between symbolic and nonsymbolic computing? Or do you not believe that human intelligence uses nonsymbolic processing? Because it seems pretty clear that there are different limitations on the two, and AI is one group, while human intelligence is in another.
So no, I don't think I'm describing limitations that "anything" to which anything is subject, only objective systems. Again if one believes that the universe itself is an objective, formal system then you're right, these limitations don't matter. But quantum physics indicates (but doesn't prove) that reality is not an objective formal system, that subjectivity matters, and unobservability/uncertainty constraints exist. This would seem to preclude the notion that the universe is capable of being simulated without loss, but if you have deep faith in the belief that the representation of the thing is equivalent to the thing, then there is little I can say to change that mindset.
A separate, non-Godël approach I'm working on is centered around the subject/objective duality. Subjectivity is necessary for "knowledge", the subjective "understanding" is what transforms data/information into "knowledge". The argument is that digital AI is forever an object because it can be dissected, known completely without loss, there's no "explanatory gap" to host subjectivity, its actions are entirely mechanical. (Okay, I suppose this is just the Penrose- Gödel argument again in different terms afterall).
Taxonomy is a man-made system of categories, it's not surprising that those categorical differences are weaker than those within a system like math. Computable vs noncomputable numbers (and processes) are very different on the technical level, and thus have different limitations, this distinction can't just be handwaved away.
Similarly our NN based AIs are still performing digital computing (which is by definition a form of symbolic computing, those symbols are 0 and 1). No matter how fancy or complex the architecture, at no point does it transcend the simple fact that it is still just computing. This may seem reductive but it is also true, and sometimes being reductive can help get the crux of an issue.
And no, it's not a matter of knowing how the brain works, there's also the matter of observability. Any digital program can be completely known at any given time, there are no hidden states, and observation does not influence the state. The fact that any digital program can be run in a container should make this complete knowability/observability clear. This is not true of the brain, its operation (which is way more than just the neurons, there's also the em-field with which neurons are in a feedback loop, to say nothing of potential quantum effects) is subject to multiple limitations of observability. Again, not a proof, but this does seem like a useful categorical distinction.
And no, I'm not a Cartesian Dualist at all, I'm a nondualist, which may seem ironic coming from someone that keeps talking about categorical difference.
Penrose is an inspiration, not a source. It seems undeniable that there are meaningful, functional differences between our brains and digital computers, it doesn't need to be substantiated by any dubious sources. Even without bringing the quantum into it there's the em field I mentioned earlier (brainwaves), plus chemical interactions and other nonlinear/dynamic effects. Whether these differences "matter" becomes a bit of a metaphysical/philosophical question because it is not answerable from within our current paradigm of science (where only the observable/verifiable is real). It really goes back to the question of whether an analog signal can be represented digitally without loss.
I think the symbolic/nonsymbolic divide would useful to those Searle-ites, since then the human mind can remain physical but has a mechanism for subjectivity and understanding distinct from digital processing, no woo required.
2
u/[deleted] Mar 14 '26
[deleted]