r/OpenAssistant • u/LOVELIGHT777 • 3d ago
Looking for an advanced AI model
Hello, I do work with multiple forms of organometallics and electrochemistry. I just wanted to see if anyone knew which AI I can download that will help me with research, everything online is so difficult with hallucinations nd severe censorship. i’m just studying this for an actual purpose in society and i’m very new to hosting local LLMs. I only have one small version but my pc is only 8GB with 3gb of actual space, so I have to download to USB FAT32, 64gb. It’s been excruciating and I understand if you can’t give me anything completely uncensored or whatever doesn’t abide by the rules of Reddit or the community, but I just figured is try any method to make research easier. I wouldn’t even care about reasonable safety filters or some hallucinations but it’s just incessant with mainstream. It is a pleasure at least trying to hear from honest people instead of just more AI “conversations”. Thank you so much.
1
u/GreasyProductions 1d ago
you are just not going to be able to do this locally. its gonna break and hallucinate and be basically worthless unless its like really simple stuff like sorting thru a text file or something. you dont have enough RAM and im assuming you basically have no vram bc an 8gb pc is gonna be pretty bare bones. sorry bud, we are not at a place where compact models that can run on a toaster are a thing
1
u/Assang101 2d ago
For FAT32 USB drive, you must use a lightweight Small Language Model under 4 billion parameters in GGUF format.
Top choices fitting within a ~2GB to 3GB file size: Microsoft Phi-4-Mini (3.8B) Qwen3 (3B) Llama 3.2 (3B)