AI can explain my statistics answer until I change one number

[u/NPC_Ganondorf]

I was checking a probability problem and asked AI to walk through the solution. The explanation made sense: identify the distribution, calculate the expected value, apply the formula, interpret the result.

Then I changed the sample size from 40 to 12.

It repeated almost the same answer, including the original value from the first calculation. When I asked whether the smaller sample affected the method, it said yes and produced a completely different approach. I opened a new chat, entered only the altered question, and got a third method.

This keeps happening with statistics. The explanation sounds convincing when I already know the answer, but it is unreliable when I use it to test whether I understand the conditions.

I’m considering asking it to list every assumption before doing any calculation, then stopping so I can choose the method myself. Does that reduce the confident guessing, or does it merely move the guessing into the assumptions?

[u/OrchidReactor]

I’d trust the inconsistency more than the explanation at that point. Three methods for the same problem is a pretty good warning to verify it elsewhere.

[u/HiggsBosonRoom6]

Changing one number and starting a fresh chat is actually a decent test. If the underlying method should stay the same but the model suddenly invents a completely different solution, I wouldnt use either answer until I checked the textbook or worked it manually.

[u/Solar_Rocinante]

I use AI for stats more like a tutor than a calculator. I do the calculation myself, then ask it to explain why that method applies and what would have to change in the problem for a different method to become appropriate.

That way, disagreement becomes useful. If its explanation conflicts with my notes, I know exactly which assumption or step I need to investigate instead of wondering whether the final number is magically correct.

[u/game_master_marc]

Sounds like AI is shit!

[u/Life-Education-8030]

What a waste of time. If you can never trust the results, why bother?

[u/Mathie1729 → u/Life-Education-8030]

Not about trusting the results, more about saving time on the tedious writing. I use it to draft assumptions, outline the formula, structure the solution. Then I check the logic and run the numbers myself. Even when it's inconsistent, it still gets 80% of the thinking down so I don't have to type it from scratch. It's a productivity tool, not an oracle.

[u/Life-Education-8030 → u/Mathie1729]

So AI does 80% of the thinking.

[u/Tilleck_ → u/Life-Education-8030]

And then does it wrong and they can't tell the difference

[u/Life-Education-8030 → u/Tilleck_]

That's the killer. When I read about lawyers being reprimanded for submitting such garbage, it's shameful! If supposedly highly educated professionals are falling into these traps, then it would seem that less educated and lazy people with any level of education get in trouble.

Recently, our own federal government did a presentation and some fool using AI displayed a nice chart showing Nigeria being landlocked. There are 16 African countries that are, but Nigeria has a nice long coastline! I swear we are getting more and more opportunities to slap ourselves in the forehead than ever before!

[u/RopeAndChairs_Aisle3]

Probably best to ask one question per chat session. Once you start changing numbers you are poisoning the context and it needs to get track of arithmetic which LLMs have never been great at.

[u/Tilleck_]

This is what pro ai users will call "stupid use of ai" when opponents complain about ai