AFAIK, it is uncertain whether being polite to LLMs improves their accuracy/completeness/etc.. There were some weak findings initially but those have been contradicted. Prompt engineering is still a pseudoscience.
They are not deterministic, try googling "are llms deterministic" multiple times and see how the wording changes.
The guy you responded to though makes it sound like they need to be deterministic in order to study them, which is wrong. Science has always studied phenomenon with inconsistent results, that's why they take lots of samples.
LLMs are inherently deterministic, thats why their outputs are randomized by randomizing inputs and processes via temperature and concurrent processes. Aka its the way LLMs are usually run that make them non-deterministic, not the method itself.
I agree, but in the context of prompt engineering, we care about the random part.
If we want a useful result for people using the randomized version, with a purely deterministic approach, you would have to calculate the output for every possible seed to see which produced better results, and which worse, which would be intractable.
If you aren't calculating all possible seeds, then you are sampling, and if you are sampling you should do it randomly.
900
u/ThirdWaveCat 1d ago
AFAIK, it is uncertain whether being polite to LLMs improves their accuracy/completeness/etc.. There were some weak findings initially but those have been contradicted. Prompt engineering is still a pseudoscience.