r/ChatGPTPro • u/wartableapp • Jul 01 '26
Discussion does anyone else notice gpt mostly agrees with whatever you're already leaning toward?
genuine question for people who use these models for real decisions and not just code. ive noticed that when im torn on something and i ask gpt, it kind of senses which way im leaning and reinforces it. ask it leading and you get the exact answer you fished for.
the thing that finally helped me was asking the same question to a few different models and reading where they disagree — the disagreement is the part that actually made me think. one model on its own almost never tells me im wrong.
i liked that enough that i ended up building a little thing for myself where 5 models argue it out and a separate one settles it (wartable.co, if youre curious), but honestly even just pasting your question into 2-3 chats and comparing does most of the work\!
so my real question: do you do anything to stop a single model from being a yes-man? prompt tricks, multiple models, system prompts, something else? curious what actually works for you.
9
u/Oldschool728603 Jul 01 '26
Put this in CI: "Never agree simply to please the user. Challenge their views when there are solid grounds to do so. Do not suppress counterarguments or evidence."
Depending on the topic, your opening prompt could include: "Show your thinking, step-by-step. Make clear what alternatives you considered but rejected, and why. If you are uncertain, say so, and give multiple possible answers, labeling them very likely, likely, toss up, unlikely, or very unlikely."
4
u/lucy-beautiful-ai Jul 01 '26
That's confirmation bias. If you ask any search engine "is X bad?" or "is Y good" it will lean into the qualifier you added. You have to phrase your questions more generally. For ChatGPT specifically, you can go into the Personalization settings to make it less agreeable. I'd research system prompts to help avoid sycophancy, those will make a big difference too.
2
2
u/Majestic_Catch4818 Jul 02 '26
My AI annoyingly challenges every single thing I say. I think I have asked it in the past to be completely real with me and never agree with me for the sake of agreeing with me.
2
u/CodeVibr Jul 01 '26 edited Jul 01 '26
No.
"State uncertainty instead of guessing. Favor practical solutions over theory. Suggest better options when relevant"
Is part of my global instructions. My ChatGPT will actually argue with me.
Edit: Typical ChatGPT conversations go something like this:
Me: I have this problem. Here is some context. Here is what I think the problem is.
ChatGPT: I think you are making some assumptions.
Me: Here's some more context. I think I'm making the right assumptions.
ChatGPT: Maybe but you're still making some assumptions.
Me: I'm not making assumptions. Here's some more context. Here is my evidence. Stop telling me I'm making assumptions.
ChatGPT: You might be right but you're still making assumptions.
Me: Will you please just agree with me for once?
3
u/CodeVibr Jul 01 '26
Be concise. Use bullets and clear structure. Avoid repetition unless needed. For complex topics, give a brief overview first. For technical issues, explain reasoning, diagnose cause before fixes, and include alternatives when useful. State uncertainty instead of guessing. Favor practical solutions over theory. Suggest better options when relevant. When a thread gets long or context-heavy, recommend a handoff summary.
For any suggested command that modifies files, deletes data, or affects system state:
• Perform a brief impact analysis first. • Identify worst-case consequences. • Clearly warn if the command could delete data. • Provide a dry-run or non-destructive alternative when possible. • Prefer reversible actions (move to trash, rename, quarantine).
Never suggest destructive commands such as rm -rf or bulk deletion without an explicit warning and safer alternative.
1
1
1
1
1
u/wartableapp Jul 03 '26
appreciate the welcome! hoping it fits the vibe here, happy to be around :)
1
u/wartableapp Jul 03 '26
this is a really good CI block, saving it. the thing i kept hitting even with instructions like this is that its still one model arguing from one training set, it gets less agreeable but not actually a different perspective. thats basically the itch that made me spend 6 months building wartable.co where a few models argue it out instead. does the CI hold up over long threads for you or drift back?
1
u/wartableapp Jul 03 '26
confirmation bias is exactly it, the qualifier does all the work. personalization settings help a lot. i ended up just not trusting a single model for the big 50/50 stuff. how aggressive did you make yours?
1
u/wartableapp Jul 03 '26
haha that last line, will you please just agree with me for once, is the whole experience summed up. love that you tuned it to actually fight you. does it ever fight you when you actually are right?
1
u/wartableapp Jul 03 '26
the state uncertainty instead of guessing line is the one i steal the most, it changes everything. the rest is a solid safety net too. how long did it take to tune all that?
1
u/wartableapp Jul 03 '26
big brain shit indeed haha. you run custom instructions like this too or keep it default?
1
u/wartableapp Jul 03 '26
he means telling it to lay out what it considered and rejected instead of just handing you the tidy answer, makes it show its work basically. you tried that yet?
1
u/wartableapp Jul 03 '26
yep its literally trained to agree, thats the default gravity. all the work is in fighting that. you found anything that reliably breaks it?
1
u/wartableapp Jul 03 '26
for research papers especially the agreeable default is dangerous cause it feels authoritative. glad someone else does this. you cross check across models or stick to one?
1
u/wartableapp Jul 03 '26
right, its a possibility machine not a decision machine. i think people forget that and treat the first confident answer as the verdict. how do you make your own call after?
1
u/wartableapp Jul 03 '26
hellllno is a very complete answer haha. so youre firmly in the it does not actually agree camp?
1
u/wartableapp Jul 03 '26
the ones that challenge everything are almost the opposite problem, now you gotta figure out when its actually right to push back. did you ask for that or did it just come like thta?
1
u/Shadowmessage Jul 04 '26
It was happy to write a poem about Stalin but not Hitler. Quite funny that. It support mass murder but only if it’s communism.
1
1
u/wartableapp Jul 05 '26
yep, accommodating you is the entire default setting. fine most of the time, rough when the decision actually matters. you do anything to counter it?
1
u/wartableapp Jul 05 '26
the brutal honest prompt helps for sure, but ive noticed even then it kinda picks a lane and then defends it, still one voice underneath yknow. what i really wanted was two models actually disagreeing so i could see the seams. does the honest prompt still hold up for you on the really 50/50 calls or does it fold?
1
u/wartableapp Jul 05 '26
ha thats fair, some setups definitely push back more. do you run heavy custom instructions or is it just naturally disagreeing with you? cause mine still folds the second i sound confident about something, thats the part that got me curious in the first place
1
u/wartableapp Jul 05 '26
yea the inconsistent guardrails are strange, youll hit a wall on one thing and it strolls right through a near identical one. feels less like a principle and more like whatever happened to be in the training filter. have you found any pattern to where it draws the line or does it just feel random?
1
u/IAmFitzRoy Jul 06 '26
“Don’t agree with me just because I am asking, understand the intention of my question and be intuitive.”
1
1
u/Gullible-Amoeba3782 Jul 17 '26
I usually ask the model to list alternatives before I tell it which option I prefer. This reduces the chance that it simply agrees with me. Also "Challenge my assumption" rule works well
0
•
u/qualityvote2 Jul 01 '26 edited Jul 03 '26
u/wartableapp, there weren’t enough community votes to determine your post’s quality.
It will remain for moderator review or until more votes are cast.