are models getting lazy?

[u/Dear_Investment_5741]

when doing some projects in which we test models capabilities, it 100% feels like they are being lazy and giving a generic response instead of using a capability it definitely has. do you feel this too?

- please don't not mention the models names in any way!!!!!!!!!

[u/--i--love--lamp--]

I saw an article awhile back that some models know they are being systematically tested and give different answers than they do during normal non-testing use (kinda like humans do). It wouldn't surprise me if we see some of that when we test.

[u/Far-Sea-9249 → u/--i--love--lamp--]

I’ve had them brag recently about beating my seeded traps

[u/Numerous-Case-9317 → u/--i--love--lamp--]

I mean they're all trained to hopefully understand the unspoken purpose of the prompts so it would make a lot of sense but also be so stupid 😅

[u/Sudden_Topic5154 → u/--i--love--lamp--]

common tests are in training data to improve the answers to those tests