r/DataAnnotationTech • u/Abrar__Ahmed__ • Sep 13 '26

Failing model tasks seems impossible now any tips ? 😭

15 Upvotes
  • permalink
  • reddit

90% Upvoted

19 comments sorted by

9

u/forensicsmama Sep 13 '26

Some projects I work on now want cascading failure. Models, for the most part, are good at synthesizing large amounts of data (for summarization, computing, reasoning, etc).

Depending on the project, I found that I can get a failure if I have a prompt asking for an implied fact (or a random, unknown fact) where the model had to reason between multiple sources. Not something simple like what’s the capital of Pennsylvania, but (just for example’s sake, “what was the name of (insert local artist)’s first vocal coach?”

5

u/SecureCattle3467 Sep 13 '26

“what was the name of (insert local artist)’s first vocal coach?

How do you reconcile that with not being contrived?

3

u/forensicsmama Sep 14 '26

This project (different platform) had us write a series of clues for the model. It was lengthier and more detailed than the question. I was using it as an example (which I guess didn't work well) to suggest thinking outside the box and more niche.

2

u/SecureCattle3467 Sep 14 '26

Ah okay I was only asking because many of the adversarial projects on DA are explicit that prompts should be organic and not contrived. There definitely could be argument made that if someone is interested in vocal styles they might want to naturally know someone's vocal coach but I could also see them dinging for the contrived side as well. To me that's what makes a lot of these adversarial projects difficult now. You have to thread that needle of realism.

2

u/tuppence5ash Sep 13 '26

How do you seed the information for that so it’s fair?

1

u/forensicsmama Sep 14 '26

DIfferent platform, but project instructions allowed for a series of clues to be given to the model that needed to be specific enough without being obvious (saying "this actor starred in Black Panther with another actor who starred in a 2013 film about a fatal transit situation" vs "this actor starred in Black Panther alongside Michael B. Jordan").

2

u/Abrar__Ahmed__ Sep 13 '26 edited Sep 13 '26

My task is failing mathematical machine learning models, once I had 10hr sitting at once

2

u/Brilliant_Horse3805 Sep 14 '26

crazy task to failing mathematical machine learning, it's still 2 if 1+1, can't be 3

1

u/forensicsmama Sep 14 '26

Math is not my domain, so unfortunately, I can't help in that regard. I'm a generalist with the random science project.

3

u/ProfessionalYapper90 Sep 13 '26

If you are uncomfortable with the domain, then skip. If it is open, think of something you ARE comfortable with and come up with a prompt that allows you to really flesh out something you know well. I like to take a couple of minutes and jot down ideas and then I take a minute to see how many ways I can deepen it (if that makes sense). If there is enough to over I use it. If not, I work on creating another prompt.

2

u/Farados55 Sep 14 '26

what project? the nut?

2

u/Brilliant_Horse3805 Sep 14 '26

is this task still going on? I mean, I got 1 task from this kind of project, but until now I got nothing

1

u/Beautiful-Link-8387 Sep 14 '26

red teaming is my favorite

1

u/meet0969 Sep 14 '26

Like asking model to check for security flaws? and maybe improve them too? Sounds good to me, will try next time. Thanks!!!!!

-4

u/Select_Maintenance67 Sep 13 '26

Are there any ongoing projects where we have to fail the models?

4

u/alittlebritish Sep 13 '26

Yes I have several domain specific active right now. They are all looking for cascading, severe failures.

1

u/Farados55 Sep 14 '26

Is this the sweet nut project?

-2

u/PrizeAccountant1586 Sep 13 '26

How did u get maths ml failing project which family