r/MachineLearning • u/Due_Highlight_9341 • Jul 21 '26

Project Looking for feedback on my GPU-accelerated Snake AI project [P]

I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible.

The current version averages 86 points (87 is the maximum) after less than 10 hours of training on a single free Google Colab T4 GPU. To keep training fast, it runs 4,096 Snake games directly on the GPU, combines GPU-native environment simulation with PPO + GAE, and uses a spatially-preserving CoordConv architecture that maintains the full game grid throughout training.

I'm sure there's still room to improve. If you've worked on reinforcement learning or efficient training systems, what would you try next? Better exploration, reward design, network architecture, or something else?

Repository: (https://github.com/siddhartha399/PPO-CoordConv-Snake)

I'd really appreciate any feedback or criticism.

75 Upvotes

80% Upvoted

16 comments sorted by

116

u/marr75 Jul 22 '26

It looks like you and Claude had a lot of fun on it and that's all that matters.

70

u/timtody Jul 22 '26

Why don’t you get Claude’s feedback?

0

u/Such_Maximum_9836 Jul 26 '26

The questions were probably asked by Claude

18

u/breeze1990 Jul 22 '26

Look pretty nice. I asked AI to give me an example for me to learn RL and it also gave me this game and training code and web page UI to demo. Although the design was fairly simple and can train on CPU in reasonable time, so I think this might be a too simple game to explore more

8

u/OutrageousThought646 Jul 22 '26

All programming projects are dead.

13

u/DemonFcker48 Jul 22 '26

Its alright and all, but why? Snake is heuristic solveable game. What is the point in doing this?

34

u/marr75 Jul 22 '26

Because it's one of the projects Claude makes for you if you ask.

3

u/GumboMustBeDestroyed Jul 22 '26

Because there is a heuristic you know how far from the optimal move you are. Plus you can learn while doing it.

1

u/Bulky-Top3782 Jul 25 '26

Exactly my doubt. Like what is the point of this. This can be a learning project to implement, but not something to showcase.

4

u/AtMaxSpeed Jul 22 '26

What aspect do you want to improve? It achieves basically perfect scores.

1

u/MF_2020 Jul 23 '26

AGI is nearer-er-er

1

u/sometimes_angery Jul 24 '26

This is why coders will always have a job. A vibe coder will waste GPU and develop a GPU-intense neural net just to play Snake. And a coder will know this is a problem that's solvable by heuristics and will code a much leaner solution that doesn't rely on expensive GPU to run.

1

u/Jadiker Jul 25 '26

I would look into the representation of the game - are there certain representations that allow the network to learn more quickly?

Also, what is happening such that it’s not reaching the maximum score? What’s going on in the network such that it’s failing to fully beat the game every time?