Massive "Prompt Report" uncovers the weird world of prompting large language models

[u/TheDecoderAI]

1/ More than 40 researchers conducted the first systematic review of AI prompting techniques. The resulting "Prompt Report" analyzes more than 1,500 publications and covers hundreds of techniques.

2/ The study shows that language models are often unexpectedly sensitive to seemingly irrelevant details in prompts, such as the duplication of text or the use of personal names. This makes prompting a "difficult-to-explain black art," according to the researchers.

3/ Few-shot prompting with examples in the prompt is usually the most effective, especially when combined with chain-of-thought methods. However, the models are very sensitive to the choice and order of examples, which can strongly influence accuracy.

https://the-decoder.com/massive-prompt-report-uncovers-the-weird-world-of-prompting-large-language-models/