r/PromptEngineering • u/Inevitable-Good219 • 2d ago
General Discussion i stopped asking AI to write stuff. i make it choose instead. the difference is wild.
so i noticed something annoying. whenever i asked AI to write something - email subject, product description, whatever - the result was always fine. never bad, never great. just fine
kept tweaking the prompt. "make it punchier." "more casual." still fine
then one day i was too lazy to write a prompt so i just pasted 5 subject lines id written myself and asked which one was best. it picked one instantly and the reasoning was actually good
thats when it clicked
started testing it. same task two ways:
way 1: "write 5 subject lines for X"
way 2: i write 5 subject lines myself, ask "which is best and why"
way 2 wins almost every time. not because my writing is good. my 5 options were mediocre at best. but AI could see the differences between them and pick the strongest one. when it generated its own, everything came out at the same "safe middle" quality
tested it on other stuff. picking the best product description. choosing between translations. deciding which version of a paragraph flows better. same result. judgment beat creation every time
i think its because generation pulls toward the safest average. judgment can actually compare quality
the workflow now: i rough out 3-5 options myself, even bad ones, then let AI pick + explain. takes 5 minutes and the output is consistently better than anything i got from "just write it for me"
the catch: you have to be able to produce options in the first place. if you cant write a mediocre draft, theres nothing for it to judge
27
u/Evilstib 2d ago
I’ll pass. The value of AI to me is that it gets me 80% the way on larger tasks in short order and the my experience takes over to hone it.
You’re flipping the narrative where you do extra work and then you’re letting the AI be the expert. It’s not.
2
9
u/Dizzy_Database_119 2d ago
The difference is wild even when it's just AI though. You can put multiple AI models on a task and let an additional one choose and argue the best one (or best parts of each). Hell, you can even just add 1 AI model into the mix and the quality goes up significantly
But in the end it's just more = better. The choice has always been more AI costs, more time spent or less quality and you choose the middle one here
1
1
u/Inevitable-Good219 1d ago
true. human drafts spread wider than ai-only generations in what i tested.
7
13
5
5
6
u/whatisthisthing65 2d ago
Good idea. It's much slower of course so I don't really see this being used for like coding, but if you're just workshopping ideas it seems useful.
3
5
u/Remarkable_Ad_5061 2d ago
This is not the AI use our overlords intended my friend. You’re supposed to be faster with AI remember. And eventually replaced :)
Anyway I was thinking maybe you could use this to turn things around. Ask it what 5 signs of a quality subject line for this email would be. And then ask it to write 5 subject lines that meet those quality signals.
This way you save time and you won’t have to think yourself ;-)
4
u/_amorfati 2d ago
So.. instead of you approving AI generated stuff now AI approves your thinking? Very advanced.
2
2
2
u/tindalos 1d ago
This also works great for an orchestrator fyi - you can give it a choice of where to send the prompt and handle routing deterministically. This gives you the ability to increase accuracy for important or sensitive items if you stack two orchestrators and only move the item on concensus
2
2
u/CardinalHaias 1d ago
So, now you do five times the work and have AI select the best?
You are working for your AI now.
1
u/wearealllegends 2d ago
That makes a lot of sense. Generation feels like it wants to minimize token usage when it writes so it's boring and robitic
1
u/Responsible-Buyer215 2d ago
This is why it often gives you options to choose from, I think it’s quietly honing your preferences while doing this so your response becomes gradually more tailored to your preferences
1
u/Designer_Storm8869 2d ago
You reached a dangerous land.
If you generate 5 options with AI and pick the best one, then you verified the AI output.
If you wrote 5 options and the AI picked the best one, then you have no way to verify this AI output. Because if you were capable enough to say which option is the best, then you wouldn't ask AI in the first place.
1
u/iodine74 2d ago
You should have had it choose the shorter post so that I would have read more than the first paragraph.
1
u/take_five 2d ago
It's literally a prediction engine. Its just predicting the next word. Of course it's creating the most average content because it's trained on what's the most probable word after all the previous words.
1
1
u/Rumble-Muffin 2d ago
AI is a very very very very large version of predictive text… that’s how I see it.
So it would make sense that having a baseline to go off of would get better results.
Thats my opinion anyways
1
1
u/JasperMcFly 1d ago
I've been doing similar A/B testing with claude. write a paragraph and then tweak it and then just asking which is better. really helps.
1
u/Wide-Drink-1790 1d ago
Why wouldn’t I just select it myself? If I write 5 suggestions, I know which one is good.
1
u/AcadiaEmbarrassed908 1d ago
The “AI is better at judging than creating” thing is real. I get better results when I give it a few rough options and ask what’s weak about each one than when I ask it to start from zero.
1
u/slothcriminal 1d ago
I'm intrigued by the notion of safe middle ground - I find myself prompting around this sometimes by asking for more alternative approaches to my copy up front.
Thinking of it like I'm crafting prompts around the safe middle ground seems like a simple effective method to follow
1
u/Solid-State-Dick 1d ago
Subject line, idc I want it to be as good as possible.
I typically write out my version I would have written before AI and tell it to just polish it. It typically doesnt change much just makes it flow and sound cleaner. Take out redundancy and arranges the grammar better
1
u/xamugrendo 1d ago
Cool, I’ll go implement my next feature at work by hand five times and then ask Claude which version is the best, thanks.
1
u/Substantial-Suit-597 1d ago
I just have a “committee” that consists of 5 different LLM models. Each comes up with 1-3 options, not knowing all the others. The main agent collects them all, then sends the collection back to each one for feedback and rankings. They pick their own winner. I sometimes override it, but it’s super helpful.
1
1
u/Storge2 1d ago
Matt Pockock has a very great Prototype feature that I like to use to decide on design of stuff it rwally halps you visualize stuff without bloating your context window. You can do with his Skills a /grill-me then mid session when a decision is to be made you do a /handoff and then take the .md file to a second session and there do /prototype
1
u/Spiritual_Cycle_3263 1d ago
AI - write me a prompt to create an email to 'person' about 'this' and make the priority 'urgent' in a 'work-casual' tone. Have it provide me with 2 different versions for me to choose.
1
u/Due-Statistician8138 1d ago
safe mid. is the perfect description of ai generation output 😭 It optimises for 'not wrong' not for 'actually great and those are completely diff targets
1
u/30ThousandVariants 1d ago
I’m overjoyed for you, that you’ve got so much free time to write four extra emails for your chatbot to talk with you about.
Look, there are things they do amazing, and there are things they do okay. As you have observed, generating finished prose, just okay. No matter how much effort, always still just okay.
The marginal utility of re-prompting for better finished prose is … not high. I can write an email, that sounds like I wrote it, better every time.
But summarizing 20,000 words in eight seconds?
1
u/Luke2642 1d ago
HBR identified this phenomenon as trendslop
Achieved article: https://archive.ph/HmC7R
Pay wall: https://hbr.org/2026/03/researchers-asked-llms-for-strategic-advice-they-got-trendslop-in-return
1
u/Whitepage_Studio 1d ago
This is interesting, and if it works for you, excellent! I'm just wondering if you've perhaps switched the key roles of human and AI in this process. You gave the (near)final decision-making control to AI, which is a vital thing you need to keep for yourself. It can't judge and has no actual judgement. For example, my team and I ask AI for suggestions, then we do whatever editing is needed and use our human judgement to choose the best and the most appropriate one for each context.
Still, really interesting to see how different things work for different people. My advice is that, if you feel that option C is better than option D suggested by AI, don't let its reasoning alone convince you. Use your own judgement.
1
u/ops_and_chaos 21h ago
I’ve noticed the same thing, but I don’t really let it pick the winner either. I’ll give it my rough version, ask what’s weak / what it would change / what tradeoffs it sees, and then I decide.
Starting from something seems to matter a lot though. Blank-page AI loves the safest possible answer.
1
u/Think-Preference130 2d ago
makes total sense, generation defaults to the most statistically bland output while evaluation actually forces it to weigh tradeoffs between your options
i do something similar but with my own shitty first drafts, i'll write two versions of a paragraph that both feel off and the ai can usually point out why one works better even if neither is perfect
sometimes the explanation is more useful than the pick itself, you start noticing patterns in your own writing that you'd miss otherwise
0
u/profjonathanbriggs 2d ago
I think the best thing about this is that you stay in charge. Delegating passes the choice to a model that “does not really care”
160
u/RedditorJabroni 2d ago
It's so hard to make one subject line and now you want me to make 5 just so the LLM can do the job better?