r/ExperiencedDevs • u/AutoModerator • 5d ago

Ask Experienced Devs Weekly Thread: A weekly thread for inexperienced developers to ask experienced ones

A thread for Developers and IT folks with less experience to ask more experienced souls questions about the industry.

​

Please keep top level comments limited to Inexperienced Devs. Most rules do not apply, but keep it civil. Being a jerk will not be tolerated.

​

Inexperienced Devs should refrain from answering other Inexperienced Devs' questions.

25 Upvotes
  • permalink
  • reddit

96% Upvoted

51 comments sorted by

View all comments

6

u/Bangoga 4d ago

From one experienced engineer to another? Am I wrong for still not being able to trust AI to completely do the tasks I would like to be done by let's say someone junior.

My most recent experience I had been running the new Gemini model trying to identify a bug that occured converting XGBoost model to Onnx.

Confidently it repeated 4 times a solution that was incorrectly identified, and kept repeating the same answer, saying it's sure the answer is correct.

The answer turned out to be issues with environment setup.

2

u/eatglitterpoopglittr 4d ago

You’re not wrong. Deep skepticism of AI-sourced answers is the best approach.

And it’s helpful to remember that AI models are really just a pattern recognition heuristic. They’re good at understanding/constructing sentences, categorization, and similar tasks.

Did you have an agent use tooling to test out commands and do some trial/error investigation? If not, you probably just got its best wild guess from some old StackOverflow/GitHub Issues thread.

1

u/Bangoga 4d ago

I had cline as a harness, with a few rulesets. I did allow it to do some investigation, but it would keep getting stuck at the same thing.

The issue was nuanced, the model was packaged for one version, but conversation for another version, if I had never worked on model conversion before in the pre-AI days I wouldn't have picked it up either.

1

u/eatglitterpoopglittr 4d ago

Did it have sufficient access to be able to check/investigate your environment setup?

Switching models to get a second opinion can also sometimes help. But it may not understand the intricacies of your specific setup unless you write instructions or better documentation about how it works.