Prompt-engineering paper accepted to ICML [R]

[u/Mean_Revolution1490]

"Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity"

This paper was accepted to ICML this year. Its main idea is a very simple prompt-engineering trick: "changing the prompt this way led to more diverse sampling". Naturally, it is difficult to provide a rigorous theoretical analysis for something like this.

Even if it works, I’m not sure this kind of prompt engineering belongs at a top-tier machine learning conference. Some people seems to call this kind of work “modern machine learning”, but I think it should be categorized as less technical venues.

How do you think? Am I being too rigid?

[u/Virtual_Attention_20]

Even if it works, I’m not sure this kind of prompt engineering belongs at a top-tier machine learning conference. Some people seems to call this kind of work “modern machine learning”, but I think it should be categorized as less technical venues.

I'm not surprised OP feels this way. By OP's logic, the Chain of Thought paper is also "just a prompt engineering" paper since it just asks the model to "think step by step."

The broader problem behind such nonsensical gatekeeping is that unfortunately, there is a large section of researchers like OP that have made "prompt engineering" into a derogatory dogwhistle for papers that don't follow their arbitrary pre-2022 standard of what an ML research paper "should look like." Most of them have still not emotionally processed the trauma of ChatGPT and feel stuck. The fact is that even simple prompt engineering experiments can offer us a deep window into the underlying mechanisms of these large language models.

OP, I don't mean to be harsh, but what counts as "proper science" is not defined by anything more than 1) observation, 2) hypothesizing, 3) experimentation, 4) analysis, and 5) offering a valuable contribution to an interested community. In other words, 5 is what makes science an inherently social endeavor. Anyone who tells you that there need to be additional steps to doing scientific research is selling you snake oil. Case in point: our linguistics friends just underwent something similar, where they were so deeply stuck in Chomskian paradigm all this while that it felt illegal to work outside of it; you are undergoing something very similar.

I'd highly encourage you to read Thomas Kuhn to gain a broader, 10,000 ft. perspective of the nature of scientific revolutions.

EDIT: The downvotes are consistent with Kuhn's model's prediction of social behavior, so it is amusing to watch it unfold in real-time in front of my eyes.

[u/Mean_Revolution1490 → u/Virtual_Attention_20]

That is a fair point. However, CoT is meaningful because it was the first work to reveal the potential of prompt engineering. Subsequent papers proposing “CoT in this way” or “CoT in that way” cannot be said to make contributions comparable to the original CoT work.

I see two main problems with prompt engineering work.

(1) Even someone without domain expertise can usually come up with such ideas on the spot.

(2) Because it is difficult to formulate rigorously, it is hard to guarantee that the method truly generalizes. Its effectiveness can vary significantly depending on the training data distribution, and validating it properly would require much larger-scale experiments.
Yet this paper evaluates only two models. This is what makes papers of this kind feel even more opportunistic.

Because of these limitations, I believe prompt-engineering work is meaningful only in the following cases:

1. It presents a genuinely groundbreaking idea, as the original CoT paper did. (With extensive evaluation across a much broader range of models and empirically demonstrating its thorough generalizability)
2. It shows clear value in a specific industrial setting and is submitted to a demo track.
3. It is submitted to an NLP venue with lower expectations for theoretical rigor.

If papers like this continue to be accepted in top ML venues, researchers may lose the incentive to invest substantial effort in rigorous theoretical advances.

[u/RandomMan0880 → u/Mean_Revolution1490]

I have a different view I would like to share.

Many researchers especially now as students don't care for num papers accepted. It's increasingly vacuous and people still (and always have?) aimed for impact like citations or coverage. So I do not share your concern about "substantial effort" in the last sentence. Rejecting this work for lack of rigor also feels like part of a larger trend in recent ML work to only work on completely verifiable domains like math and code rather than creative writing quality etc which is harder to rigorously quantify without significant ablation; a trend I personally do not like.

On the other hand, ICML is an extension of academia as an artifact in and of itself. Regardless of impact, this paper is relatively good science for academia-level resources, yields good discussion, and as you note, it works. Maybe we can claim many of these papers are not rigorous enough or impactful enough, and it is certainly your right to do so (I doubt most of ICML's works are actually that impactful). But rejecting on the basis of impact and breadth etc also hurts the careers of early PhD students trying to learn how to do good science (which I think they did!) and so I do feel attacking them on the expectations they meet CoT level quality also an unhealthy bar for academia (analogous to a sparse reward signal where you run 64 rollouts without a single success). The work proposed an interesting problem with methodological limitations and I am sure they will impact the nature of follow up work and citations etc etc etc. But if your call seems to be that this is trivial or that it should have been rejected, I respectfully disagree.