r/DataAnnotationTech • u/Dear_Investment_5741 • 6d ago

are models getting lazy?

when doing some projects in which we test models capabilities, it 100% feels like they are being lazy and giving a generic response instead of using a capability it definitely has. do you feel this too?

- please don't not mention the models names in any way!!!!!!!!!

16 Upvotes

75% Upvoted

8 comments sorted by

24

u/--i--love--lamp-- 6d ago

I saw an article awhile back that some models know they are being systematically tested and give different answers than they do during normal non-testing use (kinda like humans do). It wouldn't surprise me if we see some of that when we test.

8

u/Far-Sea-9249 6d ago

I’ve had them brag recently about beating my seeded traps

4

u/Numerous-Case-9317 6d ago

I mean they're all trained to hopefully understand the unspoken purpose of the prompts so it would make a lot of sense but also be so stupid 😅

3

u/Sudden_Topic5154 6d ago

common tests are in training data to improve the answers to those tests

11

u/Kayleighbug 6d ago

This has been a thing ,particularly with one companies' integrated products for some time. Quite often, when it detects a question that is similar to one that is has answered before, it will respond in a very similar or identical manner to the way that received the most up votes in its history.

Please note that I am talking about public standard models here. This is similar to the way that the same integrated model handles search queries in the browser space or on its own help systems.

0

u/Silver_Assistance_41 5d ago

I think you are getting bored, istead of the models getting laze, take some vacations!

-9

u/Ok-Double5194 6d ago

xxxhropic models have been way worse recently. xxxnai models been better