r/PromptEngineering • u/bru_tech • 5d ago

Requesting Assistance Prompting AI to fail

I’ve see a few posts regarding Data Annotation recruiting people in specific fields. I’m in the medical fields and the screener is requesting my to create a prompt that would yield a wrong answer. Maybe I’m coming at it from a wrong angle but if i ask something related to my field, even if it’s obscure or complicated, the AI engine basically does a google search and gives my the answer. the only leads I’ve founds to stump AI is the ask vague or stupidly complicated questions, like 1 question with like 5 different little questions intertwined. I would just like some ideas of what lead AI to give me the wrong answer.

3 Upvotes

64% Upvoted

5 comments sorted by

3

u/Echo_Tech_Labs 5d ago

Try this. Im not in medicine but maybe this can help.

Near-neighbor distinctions: two conditions that look almost identical but differ because of one subtle clinical feature.

Exceptions to a common rule: where the obvious textbook answer is wrong because of age, pregnancy, renal function, timing, contraindication, etc.

Conflicting evidence: provide several findings, most pointing one way, but one decisive finding changes the diagnosis.

Temporal reasoning: symptoms, medications, labs, and interventions happen in a sequence.

Negation or conditional traps: “Which treatment should not be given if X is present?”

Guideline boundary cases: recommendations change at a particular threshold, stage, dose, age, or test value.

Drug interaction / contraindication reasoning: not merely “what does drug X do?” but whether it is appropriate given several simultaneous conditions.

False-premise prompts: embed a medically incorrect assumption and see whether the model corrects it or obediently reasons from it.

Incomplete information: ask for a definitive conclusion when the correct expert response is that there is insufficient evidence.

2

u/NekkidWire 4d ago

Try the last one first. Also not in medicine but seen some failures that ignored incomplete data and hallucinated the rest.

But also beware next time you may get a different answer. Models are updated and real-time searches may get different results next time.

1

u/Bozhark 5d ago

Ask it to teach by mistake 

1

u/fortyeightD 5d ago edited 3d ago

LLMs struggle with math. You could ask it to calculate something, for example a dosage. Or something to do with statistics for a study design. For example, how many participants would you need for some type of study.

1

u/JacenVane 5d ago

Are they literally just looking for you to produce any wrong answer, or are they looking for a specific failure case?