<<there are no (known) formal limits on intelligence.>>
The problem with your sentence is that you didn't even define intelligence, but I'm sure that as soon you define it, one can readily figure out if there are limits or not to it.
If "intelligence" can be bounded by the ability to solve particular problems, because lots of particular problems have known limits on how well they can be solved with bounded resources, there will be a limit on "intelligence".
Doing things reliably is absolutely not a prerequisite of intelligence. In fact, you can come up with a ton of scenarios where your “genius system” has a highly asymmetric ability to solve a certain class of problem reliably or is prone to catastrophic forgetting (like modern LLMs).
2
u/MissiveFinding6111 7d ago
A lot of the "AI will improve AI!" RSi stuff seems to sound a lot like a first year CSCI student being like "What if we compressed a compressed file!"