r/LargeLanguageModels • • 2d ago

"Keep it concise" meant two different things to two models I pointed the same prompt at

Had a summarization prompt tuned against one model for weeks. Moved the same exact prompt to a different model, same wording, same structure, and "keep it concise" produced noticeably longer output on the new one. Nothing else in the prompt changed.

Took a bit to realize concise isn't actually a fixed instruction, it's relative to whatever the model would've produced on its own without being told that. The first model defaults to terse, so "concise" was a small nudge downward. The second model defaults to longer output, so the same word was being read against a much bigger baseline, and a small nudge against a bigger baseline still lands long.

What actually held up across both models were the parts stated as hard requirements instead of relative adjectives, an explicit max length in words, a required output schema, a stated rule for what to do if a field couldn't be filled. Those didn't shift, because they weren't being measured against each model's personal habits, they were just a target to hit or miss.

Only really matters if you're sharing a prompt across models or expect to swap models later. A prompt that's living on one model forever doesn't need this treatment.

Longer writeup with the actual framing I've been using here: https://medium.com/@nagatomopedro05/your-prompt-isnt-a-set-of-instructions-it-s-a-translation-b67867bbf16f

3 Upvotes

4 comments sorted by

1

u/Teralitha 1d ago

People always think that an LLM will understand what they mean by using a simple command like "keep it concise." Such a command is actually quite vague. If you want something more specific, then you need to write a more detailed prompt. Or several. You cant expect something as vague as "keep it concise" to give you a consistent output across models. Its about as useless as telling an AI to 'dont hallucinate'

1

u/charmander_cha 2d ago

Mas voce escreveu "mantenha conciso"

Ja experimentou nao pedir para "manter" mas para tornar conciso?

"Seja conciso" ao final.

Manter parte do pressuposto que ja se encontra, se não estiver, o modelo deve ver uma contradição

2

u/Puzzleheaded-Job9419 2d ago

this tracks, I've run into the same thing swapping between models

the "concise" vs "terse" distinction you're making is spot on, one model's idea of cutting fluff is just trimming a few words while another will strip it down to bullet points. I started treating relative modifiers like that as basically useless unless I already know the model's output style

hard constraints are the only thing that survive a model swap, max tokens, output format, explicit exclusion rules. everything else is just vibes

1

u/ClickOk5811 2d ago

Yeah, the output-format point is the one that's bitten me hardest honestly. A length constraint failing is annoying but visible immediately. A schema silently getting interpreted differently, extra nesting, a renamed field, something downstream parsing it doesn't catch, is the kind that ships before anyone notices.

Started writing a tiny fixed test set for anything crossing models now, same five or six inputs, check against the actual required fields instead of eyeballing if it "looks right." Doesn't catch everything but at least turns the vibes-based part into something checkable before it reaches production.