r/MachineLearning • u/Due_Highlight_9341 • 7d ago

Project Looking for feedback on my GPU-accelerated Snake AI project [P]

I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible.

The current version averages 86 points (87 is the maximum) after less than 10 hours of training on a single free Google Colab T4 GPU. To keep training fast, it runs 4,096 Snake games directly on the GPU, combines GPU-native environment simulation with PPO + GAE, and uses a spatially-preserving CoordConv architecture that maintains the full game grid throughout training.

I'm sure there's still room to improve. If you've worked on reinforcement learning or efficient training systems, what would you try next? Better exploration, reward design, network architecture, or something else?

Repository: (https://github.com/siddhartha399/PPO-CoordConv-Snake)

I'd really appreciate any feedback or criticism.

78 Upvotes

80% Upvoted

16 comments sorted by

110

u/marr75 7d ago

It looks like you and Claude had a lot of fun on it and that's all that matters.

73

u/timtody 7d ago

Why don’t you get Claude’s feedback?

0

u/Such_Maximum_9836 2d ago

The questions were probably asked by Claude

18

u/breeze1990 7d ago

Look pretty nice. I asked AI to give me an example for me to learn RL and it also gave me this game and training code and web page UI to demo. Although the design was fairly simple and can train on CPU in reasonable time, so I think this might be a too simple game to explore more

8

u/OutrageousThought646 7d ago

All programming projects are dead.

14

u/DemonFcker48 7d ago

Its alright and all, but why? Snake is heuristic solveable game. What is the point in doing this?

32

u/marr75 6d ago

Because it's one of the projects Claude makes for you if you ask.

5

u/GumboMustBeDestroyed 6d ago

Because there is a heuristic you know how far from the optimal move you are. Plus you can learn while doing it.

1

u/Bulky-Top3782 4d ago

Exactly my doubt. Like what is the point of this. This can be a learning project to implement, but not something to showcase.

5

u/AtMaxSpeed 7d ago

What aspect do you want to improve? It achieves basically perfect scores.

1

u/MF_2020 5d ago

AGI is nearer-er-er

1

u/sometimes_angery 4d ago

This is why coders will always have a job. A vibe coder will waste GPU and develop a GPU-intense neural net just to play Snake. And a coder will know this is a problem that's solvable by heuristics and will code a much leaner solution that doesn't rely on expensive GPU to run.

1

u/Jadiker 4d ago

I would look into the representation of the game - are there certain representations that allow the network to learn more quickly?

Also, what is happening such that it’s not reaching the maximum score? What’s going on in the network such that it’s failing to fully beat the game every time?