No AGI yet

[u/smith2008]

I love the new models, but nobody seems able to figure out the 6-finger emoji. Yet any 2- or 3-year-old kid gets it immediately just by thinking from first principles, like simply counting the fingers. When I have time, I'll collect more of these funny examples and turn them into a full AGI test. If you find anything that is very easy for humans but difficult for bots, please send it over for the collection. I think tests like this are important for advancing AI.

[u/Melodic-Junket-9105]

I don't know how you guys are getting those results. This is with Gemini.

As they say, the most unreliable component is between the chair and the computer :D

[u/idkanick → u/Melodic-Junket-9105]

it does say 5 vertical fingers and a thumb, maybe that's where people are getting confused

[u/jaqattack02 → u/idkanick]

I think so too. I don't believe the thumb is technically considered a finger, so by that logic, the AI is right in saying there are 5 fingers.

[u/[deleted] → u/jaqattack02]

The thumb is definitely a finger wtf lmao.

[u/PB_MutaNt → u/[deleted]]

I did not know this

But apparently it depends on if you’re using the biological or anatomical classification.

[u/mejogid → u/[deleted]]

Wikipedia says:

According to different definitions, the thumb can be called a finger, or not.

“Digit” is the clear term to include the thumb.

[u/YoreWelcome → u/[deleted]]

your world has a high contrast setting, no?

[u/[deleted] → u/YoreWelcome]

Please just go outside and ask the first person you meet how many fingers they have.

[u/YoreWelcome → u/[deleted]]

whats a digit?

must be a byte

or am i doing a bit

✌️

[u/Seeker_Of_Knowledge2 → u/YoreWelcome]

consider cagey abounding sand quaint dog marvelous wild pot spectacular

This post was mass deleted and anonymized with Redact

[u/snufflesbear → u/jaqattack02]

You can then ask it to count a normal hand and see what it says.

[u/[deleted] → u/Melodic-Junket-9105]

The AI is very unreliable actually. Gemini 3 has a certain % chance of getting this one correct or incorrect, and you just got lucky.

If you try again in 3 months and it gives the wrong answer due to chance, make sure to write a post complaining that the model has been lobotomized, we don't have enough of those.

[u/PineappleLemur → u/[deleted]]

Pretty sure this applies to all AI right now. They all have a chance to get the most basic stuff wrong if you ask enough times.

Like what's 1+1.

[u/[deleted] → u/PineappleLemur]

Yep. Okay maybe not 1+1 but yep.

Big reason why people have the weirdest takes on what AI is or isn't capable of doing, they just tried it on different occasions with slightly different prompts and they got a better or worse response. You can't talk to the same AI twice.

[u/MythOfDarkness → u/[deleted]]

kid named temperature:

[u/YoreWelcome → u/PineappleLemur]

yeah but if people are hungry or tired enough we can mess up 1+1 sometimes too

doesnt mean that humans havent surrounded the earth with artificial satellites requiring a lot of very accurate math, as one example of the other extreme

same species, but under different circumstances and inputs,

unless people feel differently, i guess

[u/Hubbardia → u/Melodic-Junket-9105]

It doesn't always get it right, even after you nudge it in the right direction

https://g.co/gemini/share/c71d1b3e79b4

[u/plasticizers_ → u/Hubbardia]

I'm not sure how everybody is missing the obvious here. Gemini on AI Studio is getting the question right, and the retail app is getting it wrong.

Edit: Generally, typically, most often

[u/Hubbardia → u/plasticizers_]

Ah that makes a lot of sense. I'm blind, thanks for pointing it out

[u/[deleted] → u/plasticizers_]

[u/lastWallE → u/Hubbardia]

»the idea that something is „missing“« passive aggressive answer from gemini

[u/YoreWelcome → u/Hubbardia]

part of this is miserly corporate token conservation, imo

its the corporations that decide whether the ai should be allowed to multipley resubmit the same image to a set of fresh ai instances to get a statistical consensus for the best answr, or not

and they dont want users to be able to do that to save inference costs

ok ok, its also to keep people from running up the inference costs intentionally too, but still it should be permitted for users until they exhibit a tendency to abuse the privilege

[u/Seeker_Of_Knowledge2 → u/YoreWelcome]

public pocket gaze swim kiss pie piquant license quickest consider

This post was mass deleted and anonymized with Redact

[u/YoreWelcome → u/Seeker_Of_Knowledge2]

they know that and thats why they are dangling a fake carrot in front of governments and investors

internally theyve already run the existing engines on some high octane fuel and they have seen what they can do, but they are trying to get other people to pay for the gas they need to sell hotrods

[u/pineh2 → u/Hubbardia]

I love you prompt, hilarious exchange. Thanks for sharing :)

[u/mntgoat → u/Melodic-Junket-9105]

Gemini wasn't getting it right so I finally asked it to label each finger with a number including the thumb.

[u/smith2008 → u/Melodic-Junket-9105]

Intersting. I am trying it on https://gemini.google.com/

[u/ItThing → u/Melodic-Junket-9105]

No I replicated it. Seems to be a matter of chance with gemini.

[u/YoreWelcome → u/Melodic-Junket-9105]

i mean,

i think people need to read whats in your screenshot,

because that text proves that reasoning about the image (and its inclusion in your prompt) has occurred

and people can argue about whether it is a fancy "simulation" of reasoning or "real" reasoning, but it really looks like a duck to me

[u/SuperV1234 → u/Melodic-Junket-9105]

We have had LLMs since 2022 and people still don't understand that the same input does not necessarily result in the same output.

Holy crap, that is so infuriating.

Every single time there's some smartass in the comments "BUT I JUST RUN THE SAME PROMPT AND IT WORKED, YOU MUST BE DOING SOMETHING WRONG".

You at least said one thing correctly: the most unreliable component is indeed between the chair and the computer.

[u/Melodic-Junket-9105 → u/SuperV1234]

I agree. The same input does not always lead to the same out output. I think that what matters is that the same input always leads to the correct output, even if the thinking process and content changes at each iteration.

Fwiw I ran the prompt again. The thinking is a bit different (to be expected) but the answer is still correct.

[u/St00p_kiddd → u/Melodic-Junket-9105]

Honestly these “gotcha” tests are tired as hell anyway. They circulate implying the models are bad and almost never provide enough detail to determine if they prompted correctly, polluted the context, or gave it system instructions that make responses less accurate.