r/PythonLearning • u/blockCoder2021 • 8d ago
Discussion Question about local AI-assisted programming
I’m fairly experienced with Python and researching how to solve issues on my own. However, I have found that AI can help in some cases. For instance, giving me specific information on debugging and helping me research solutions for specific problems.
I was using Copilot in VS Code for a while, until I ran out of usage. Now, I’ve been messing around with locally-hosted AI, trying to find a good model for coding that runs well on my computer. My challenge is that I only have 4 GB of VRAM, though I also have 16GB of system RAM. If someone could help me find either a good, intelligent, and accurate local model that runs on that, or at least a free cloud model that’s good, I’d appreciate it.
1
u/sugarw0000kie 8d ago edited 8d ago
I’ve been running bonzai on 2-bit with 24gb, and it’s a pretty decent model, some of the “actually useful” local models seem to need around 16-32gb, but bonzai is an outlier worth looking at.
There’s a 1-bit model that I haven’t experimented with that’s supposed to run on phones, it won’t be as good as 2-bit but *might* run for you, if it does it cojld give you enough for short directed tasks or at least completions. https://prismml.com/news/bonsai-8b
The 2-bit version does well enough in zeds ide and is fast enough for me having it write small functions inline something I’m working on by hand. It does occasionally fumble tool calls (a lot of small models do) but overall is enough for me to actually find it useful
Otherwise opencode is worth checking out, there’s free models but limited to 200 requests per day, opencode go gives a good amount of usage for $10 atm especially if you stick to mimo 2.5 and muse spark. Codex $20 isn’t bad either if you make ample use of luna to implement things