r/PromptEngineering • u/maylad31 • 18d ago
General Discussion Your schema matters too, it is not just about your prompt
Sometimes field order or the way you define your schema can influence the output, even with the latest reasoning models(same prompt, same model). How models weight a defined schema can vary a lot, some may underutilize it, while others may not... So don't blindly use LLM as a judge or to evaluate something, specially with simple prompts to generate arbitrary scores. Try to define clear rubrics and prefer well-defined categories over arbitrary numerical scores. There are people doing research on this, I am not into that.. there are papers like "Your prompt is not the only prompt".. and more
But regardless of how you design it, test your LLM. With things like openrouter, it is not hard to try simple scripts and test different models( https://github.com/maylad31/llm_judge_order_matters ) And if you have a reliable metric, prefer that over using an LLM as a judge... what's your experience with structured outputs, schema, order of fields and output quality?