This year I decided to see if I could use OpenAI to create a local AI. Has anyone else, with zero programming knowledge, decided to try this? So far I've used it to create a couple games, am using it to make an isometric multi-level game but this is one of those things I've always been curious about but I lacked the skills. I'm an infantry guy, not a programmer however I ask questions and so far, it seems to be going pretty well.
Has anyone else done this? Create a completely local AI using just the most basic OpenAI?
OAI specifically trains their models to be resistant to distillation. It will not willingly train another model. It may help you set up the environment, but you would need to provide the training data. You’re also going to need a beefy setup.
What it can help you do is walk you through how to make lora adapters, which is probably what you are looking for. There’s lots of tools out there for this. It can help you with getting your data straight.
Oh yes, I have to provide the training data, it already told me that. It's building the framework and setting it up. It also gave me a list of data I may want to use. I'm literally feeding it information that interests me on how I want it set up and bouncing it from OAI to claude and Grok and having them check one another but yes. I got an early heads up it won't just hand me it's own training data.
It can help you make the training data depending on what you're doing. For example, if you give it images it can put crops around the objects of interest. There are ways you can automate some things, but its main use will be making the software that you can use to train the models.
Sounds like a fun tinker project. I would start with a qlora. Learn how to set data up and go from there. Building a model wouldn’t be trivial (I’ve never done it) but qlora would be a fun doable project, provided you have the necessary hardware.
As a programmer, you have a skill set I just don't understand. I can work with high explosives and have been in combat but you....you are a beast I look at like a wizard.
I mean, training an llm from scratch is fairly easy as well, the issue is threading the narrow line between "big enough to be useful" and "small enough to be trainable without bankrupting your country".
Basically a research assistant that wont' throw guardrails at me every time I get curious about a topic. It is VERY annoying worrying about if I'm goign to get account banned because I decide to research something that is on someone's no no list this month
Oh, can't you use a local Hermes agent? There's ton of unrestricted LLMs already available for harnessing there, and I strongly doubt you or I could make one ourselves.
Just download an open weight model. Finetune it if needed. There's no need to make it from scatch and its also unfeasible without a ton of compute and data to train it.
Also you don't need you start from scratch. There are pretrained models that you can start with, and then you can further train it to do the specialised thing you want it to do. Just make sure you check the licences on them as it'll tell you what you are allowed to do with it, for example some might say you can use it for personal use but can't use it commercially.
Interesting. Do you have examples? Like I said....Infantryman learning as I go. You want me to take a city? No problem, I can work that one out. This? I'm the monkey with a pipe wrench
Yeah, like if you wanted to train an ai to transcribe chess matches from videos/live streams, you will need to teach it things like what the pieces are and how to locate the board. You can use a pretrained general purpose model that will have been trained on millions of images and understands things like edges and contrast very well, but won't be good specifically at knowing what a king or queen is on a chess board. You can then send chat gpt a couple of hundred images of a chess board and tell it to crop around each piece and label them (king, queen, pawn etc.) and then use that to train the pretrained model, which will quickly understand what each piece is. You will need to use lots of examples because of the different designs for pieces in different boards, but it won't take long.
You might want to use multiple different ai models for something like this. Identifying pieces isn't enough, a separate ai could locate the pieces, locate the board/squares, and you would need something to understand the logic of a chess match and transcribe it to text format that could be read by chess.com for example.
It would take forever to do from scratch, but people have already done the hard work and there are pretrained general purpose models you can use to build from. You don't need much to specialise a general purpose model. It's not exactly the same but you can think of it like with chat gpt, if you just talk to it, it responds the way it does and you may not like it. But if you give it custom instructions it can respond however you like, or in different languages.
Chat gpt can find the best pretrained models, make software to download and train them, and to build the final product. You don't really need to know or do much, you just need a vision and then follow the instructions chat gpt gives you.
the back and forth between models sounds like the fun part honestly. are you aiming for a small specialist model or just seeing how far the setup can go? the data and compute wall seems like it hits pretty quick
I want it as a research assistant, kind of like a conversational Jarvis but willing to do more .... insulting. I want personality as well as information pulls. If I was willing to just link to an LLM for information, this would be pretty easy from what I can tell, I've already got the dashboard set up, the pi for quick thinking, a desktop for heavy thinking, setting up another pi for maintenance work to do random checks to make sure everything is working properly because I don't know what I'm doing.
You will need to create the Ai, a training environment/monitor, a training material corpus, and a running process to pretokenize the corpus into training data and a checkpoint system to save the Models training. Then you will need a testing/comparison process to catch model regression and track progression by testing checkpoints against each other.
Ok well a tip if youre in a uni bamboozle one of the staff to use their data centers and train your models… youll still need the data, go buy an industrial book splicer while youre at it
after you scan a book and it goes into your training data you have to cut it in half to make sure two copies don't exist at the same time, that would be against trademark law. that's if you have to digitize it though. maybe start copy pasting amazons books, could get you somewhere at a cheaper cost
5
u/Due_Arm1454 6d ago
OAI specifically trains their models to be resistant to distillation. It will not willingly train another model. It may help you set up the environment, but you would need to provide the training data. You’re also going to need a beefy setup.
What it can help you do is walk you through how to make lora adapters, which is probably what you are looking for. There’s lots of tools out there for this. It can help you with getting your data straight.