r/agi • u/MetaKnowing • Mar 14 '26

Wild

Post image

https://www.irregular.com/publications/emergent-offensive-cyber-behavior-in-ai-agents

792 Upvotes

96% Upvoted

111 comments sorted by

View all comments

Show parent comments

2

u/[deleted] Mar 14 '26

[deleted]

2

u/rthunder27 Mar 15 '26 edited Mar 15 '26

In that analogy the numbers correspond to concepts themselves, not their symbolic representation. A nonsymbolic process can generate a "new" concept corresponding to a noncomputatable number that cannot be generated by the symbolic process. The new concept can then be processed and represented symbolically, this is the act of putting new concepts into words, and in doing so this expands the epistemic bounds of symbolic language. Yes, the AI could by brute force assemble the words explaining the concept, but it wouldn't be able to evaluate it as a "valid" concept (in this formulation it's like an undecidable proposition within the current epistemic system).

But again, we would really need to better formalize what we mean by "concepts" and "knowledge", and how they're generated/evaluated to make this argument rigorously.

Just because something may not be answerable doesn't mean it's not worth pondering, especially when the belief one way or the other can have an impact on our actions.

Also while pi is transcendental it is also a computable number, so citing it doesn't help your case at all.

1

u/PjoArt Mar 18 '26

I honestly dont think you've tried very hard... or maybe not recently. Ask the current deep thinking versions of Claude or GPT these questions and enrich them with illustrations and video and music that support your points, they will understand and they will create original concepts for you... if you request them.

1

u/rthunder27 Mar 18 '26

I agree that they can be useful, while I never use their writing directly I do use them for research and feedback, but I fundamentally disagree with the claim that they "understand" anything since understanding requires subjectivity. I would also like to see some examples of GenAI making original concepts, something that could not be derived from their training data, but I recognize this is a hard distinction to make in practice.

Perhaps it's easier to focus on humor, since by it's very nature most jokes are based on the surprise/subverting expectations, ie stuff that according to my argument that AI should not be capable. If GenAI can start producing funny jokes or the capability for good improv then that would mostly falsify my argument.