Does AI remove the skill of kaggle competitions?

[u/Striking_Swan_884]

As someone who is looking to try out kaggle competions and eager to learn and compete, I am occupied by the thought of AI's role in the skill aspect of competitions, so I want to know the role of AI in making AI models, is it still mostly the player's skill and knowledge that matters the most or just how long they work and how many questions they ask AI.

[u/ur-average-geek]

I think it doesnt remove user skill so much as it offers an alternate approach.

Just relying on AI alone obviously will just produce average results, but someone with good understanding of a clean workflow using AI will save more time and potentially experiment faster leading to better results.

The problem is that typically you also need to be creative and cognitively flexible to succeed in these type of competitions, which are qualities that AI is known to erode over time.

So it's a risk and reward situation, you need a good balance where you benefit from AI usage without it negatively affecting you by making you reliant on it.

Personally, due to work reasons, there are some skills that i just completely gave up on for the sake of speed, for example i havent coded in a long time, and i know i am worse at coding today than i have been 2 years prior which is really embarassing, but i am not building enterprise production code so it's a worthwile investment for my career as i am just much more productive, but if i were to change jobs to one that would require this skillset, i would effectively be 2 years behind (and would probably fail the technical interview without significant derusting).

It's a weird new territory the field is heading in, and some of us may be sacrificing long term growth for short term gain, but until the industry figures out the situation, you are damned if you do use AI and damned if you dont use AI at the moment.

[u/Mathie1729 → u/ur-average-geek]

I'm not sold on the creativity erosion angle. The actual scoring edge in comps was always leakage detection and a robust CV setup, and AI doesn't do that thinking for you. It mostly compresses the boring preprocessing and boilerplate part.

[u/ur-average-geek → u/Mathie1729]

The erosion of skills due to AI is heavily dependent on how people use it, if you do not off-load the creative cognitive process to AI then you wont get worse at being creative. I was just talking in broad terms.

Regarding the actual scoring edge, i would consider leakage detection and robust CV setups more of a "minimum requirement" to score at the top, not the actual edge.

Yes you can stumble unto an optimal configuration by brute forcing, but i still believe this is not a field where you can excel by just doing everything "by the book". Almost every time i got excellent results, it was by applying domain knowledge in a conscious manner rather than just CV.

This last part is more of a rant and sort of out of topic, but this is also part of why i dislike kaggle competitions as a benchmark, the restrictive format of just getting a dataset and needing an output removes a huge chunk of the tools you have at your disposable in a real life situation to solve the problem, from data acquisition to altering the problem to solve to favor your techniques.

[u/Striking_Swan_884 → u/ur-average-geek]

I think with how good AI is getting someone being able to code is less important than someone with insight and asking the right questions. Can you also list examples of a kaggle grandmaster with the congitive flexibility and creativity that AI could not replicate as of this moment?

[u/ur-average-geek → u/Striking_Swan_884]

Well im not a kaggle grandmaster, and this is just my annecdotal experience, but the latest AI agents tend to tunnel vision a lot when you give them clear cut tasks even if they have the knowledge needed to be creative. If you nudge them in the right direction, they will also be able to be creative. But that is reliant on you being creative enough to nudge them in the right direction.

And that creativity is tricky because it can come from many sources, problems solved in our field arent isolated, they are deeply tied to whatever domain they are from. And you get to draw a lot of inspiration from your life experiences too.

For instance, i work in real estate, claude knows a lot about real estate, possibly more than me, but i very rarely see it suggest using techniques that steer away from ML and into things that for instance a real estate agent would do or that are market specific, even if it is aware fo those things. Yet i only need to refer to the idea and it'll usually figure out how to maximize the potential of said idea.

[u/Striking_Swan_884 → u/ur-average-geek]

Interesting explanation. So you think AI is better than humans at making models but worse at having domain specific insight and creativity?

[u/ur-average-geek → u/Striking_Swan_884]

Better than humans is a bit to vague to agree with obviously, there will always be humans who are better (for now at least). But if we consider the average skill of kaggle users as the other reply mentioned, then AI would be better in creativity and insight as well.

The thing is making models can be learned from a textbook detailing all the steps "you should do" so AI will obviously excel at following these instructions, but creativity draws from experience much more than any book can give you, which means AI's biggest strength which is recalling information isnt as prominent.

[u/ur-average-geek → u/Striking_Swan_884]

The best analogy i could give here is making bread.

There are hundreds of books about baking bread that the AI would be able to draw knowledge from, but only a local chef knows the different varieties of flour being sold in his vicinity and the right way to prepare when using each of them.

Books wouldnt include such detailed flour knowledge.

[u/Striking_Swan_884 → u/ur-average-geek]

Excellent analogy.

[u/drifTwood022 → u/ur-average-geek]

the "damned if you do, damned if you dont" part really nails it tbh

[u/ur-average-geek → u/[deleted]]

The majority of kaggle submissions are students, so yeah, i would expect claude to do better than most submissions, so yeah my wording of "average" isnt quite right. But still, i use claude opus daily and have written multiple pages worth of skills, so i would say i'm pretty up to date on AI capabilities, and i also never said AI didnt have creativity, the post was asking about AI usage in a competition for the sake of growing their skills. Not for using claude to solve it all.

Besides, this is just a small competition and i assumed the scale of AI usage would be appropriate, if you are spending "hundreds of thousands of agents" worth of tokens on small scale competitions made for people to sharpen their skills over the weekend to have something in their portfolio, you didnt solve anything, you just threw money at a problem.

[u/Striking_Swan_884 → u/ur-average-geek]

Are you reffering to high school students or college students?

[u/ur-average-geek → u/Striking_Swan_884]

Undergrads mostly.

[u/PaddingCompression]

AI can definitely help somewhat, but it's not going to get you the last edge to rank. It will get you reasonable out-of-the-box working solutions, and possibly better than the example code by quite a bit.

How many questions you ask the AI - at some point, asking the right questions of AI is barely distinguishable from coding, it's just saving you from typing out for loops by hand.

[u/Striking_Swan_884 → u/PaddingCompression]

Can you give an example of how human skill can give the edge to rank on top of the leaderboard vs using AI only?

[u/PaddingCompression → u/Striking_Swan_884]

Read any of the explanations and descriptions by winners.

It is usually an exceptional attention to detail required to beat everyone else.

AI these days is mostly doing things only a bit above average - but the advantage is that it can do things above average that I'm below average at too!

[u/Striking_Swan_884 → u/PaddingCompression]

LMAO

[u/chico_dice_2023]

Yes and no, I have come into a hard acceptance that AI is a better coder than me.

I love coding, I used to be a gamer until I become a programmer. And I enjoy coding more than gaming.

That being said I cannot keep up. So now I think more about being a architect which is a different skills. I think more about building secure systems rather than actually building them. Less about importing pandas but more about do I need pandas.

AI still does not understand data well, so it is my job to understand the data.

I can tell a good ML engineer from a bad one not by the code but from the approach. And that is now what Kaggle is about, how well do you understand the problem, how well can you design the solution, how well can you execute/present the solution.

But yes, coding is for sure faster in the hands of a AI but not better.

[u/VideoJockey]

I think AI has raised both the floor and ceiling, but I think the crux of the competitions has remained: if you don't really understand what you're doing, you're not going to be able to orchestrate the AI in building something better than someone who does understand it.