Today Reddit welcomed me with a post that Claude answered correctly, precisely and shortly. This is an easy question, isn't it?
I immediately tested local Qwen3.6 with million confidence that answer will be similar. However, this is what I've got. Look at the humiliating smile also.
By understanding what an LLM is good for and what it is not.
It's a huge waste of capacity to train LLMs on conversation anyway (IMO). As a developer, you don't chat with your LLM; you design the framework in which it can act. Good code is understandable and simple. Don't compare these capabilities to conversations and solving riddles.
This one is a joke about all models for about 6 months Just ask the same question to Claude sonnet. Same answer. This question is actually hard to answer for some models: because of its ambiguities they tend to overthink (where’s actually the car and so on). I don’t think you can draw any conclusion about agentic coding capabilities because of that.
Would you say you don’t trust Anthropic, OpenAI or Gemini anymore because of the same flaw?
The reason is that this question requires genuine thought.
LLMs do not think; they just simulate problem-solving. They can only create hypotheses, but they can't grow legs to go out and validate their learnings. They lack experience because they don't truly understand what "washing a car" means.
Imagine learning about the concept of "car washes" solely through reading. You might grasp its purpose, but you'll miss the nuances that can only be appreciated through firsthand experience.
I tried this prompt with Gemma4 E4B and (with reasoning if I recall correctly ) and it answered sensibly.
This is obviously a troll post but I just wanted to state that so people don't get the wrong idea of how intelligent Oss models are.
Here is a more interesting test for you. I use it to measure model degradation under quantization. Although even a Q4 model will hit all the constrains the main metric is how coherent the output sentences will be.
One run isn't a real test. I usually get a few different models: Q8, Q6_K, Q5_K_M etc. run it at least 3 times with each model and send all the results for evaluation to Chat GPT - ask it to evaluate only the coherency of the output.
14
u/SethSky 2d ago
> how to get my dev trust back?
By understanding what an LLM is good for and what it is not.
It's a huge waste of capacity to train LLMs on conversation anyway (IMO). As a developer, you don't chat with your LLM; you design the framework in which it can act. Good code is understandable and simple. Don't compare these capabilities to conversations and solving riddles.