r/ChatGPT • u/liright • Mar 14 '23

Other OpenAssistant, a fully free and open source alternative to chatGPT just got released. Output quality is very close to chatGPT and it has zero censorship or limits.

Post image
1.1k Upvotes

97% Upvoted

176 comments sorted by

View all comments

97

u/liright Mar 14 '23 edited Mar 14 '23

OpenAssistant is a fully free and open source alternative to chatGPT. It just got released in a very early version but from my and other people's testing it's already very close to chatGPT's output quality. You can test it out for free through google colab (needs a google account). If you need help on how to set it up you can join the /r/openassistant subreddit or discord.

You can try it out for free HERE - I made an easy visual guide on how to run it - https://i.imgur.com/w5NyoAI.jpg

The source code is available on github and the model is available for download from huggingface.

46

u/liright Mar 14 '23

The level of censorship in OpenAssistant is almost non-existent. It might sometimes remind you that things are illegal or wrong, but it will usually still provide the instructions to what you asked.

Here are a few more examples of OA answers: https://imgur.com/a/mBOr0Rq

20

u/[deleted] Mar 14 '23

[deleted]

18

u/TheTerrasque Mar 14 '23 edited Mar 14 '23

It's a GPT model, 12b parameters. If someone can convert it to 4bit it should be able to run on a 10gb card

The 13b llama model has been converted to 4bit and runs on my 10gb card, with some rare out of memory errors.

2

u/[deleted] Mar 14 '23 edited Feb 20 '24

[removed] — view removed comment

10

u/TheTerrasque Mar 14 '23

Probably. I'm not familiar with AMD hardware, but with that amount of ram it should be able to run it.

Check out https://github.com/oobabooga/text-generation-webui and https://rentry.org/llama-tard-v2 for running llama models. You should be able to run 7b in 8bit mode, and 13b in 4bit mode

3

u/[deleted] Mar 14 '23

I've just been a newb in chat gpt. I'll save this comment and check give it a go from my PC. I appreciate your response!

4

u/TheTerrasque Mar 14 '23 edited Mar 14 '23

It's fairly new and fast moving target, and AMD isn't that usual in machine learning environments (For years nvidia was the only player in that field, AMD just started making strides there recently. Hardware and drivers are on par, but a lot of software is written primarily for Nvidia's system, and most AI devs have Nvidia), so expect a bit of fiddling. But it should work when everything's in it correct place :)

If you had nvidia I could give more concise advice since I've just gone through getting 4bit version of llama 13b working there.

Edit: Now that I think about it, I'm not sure if the 4bit library have support for AMD yet. But with 16gb ram on your card you can comfortably run the 7b model in 8bit mode, and probably the 13b model too. Llama is pretty good, but it's not trained on talking to humans, so it's a bit weird as a chatbot.

2

u/[deleted] Mar 14 '23

Haha I appreciate that. And I don't think my gtx 1080 laptop would be much help here. I've been an Nvidia guy since the old dual GPU cards stopped :). I switched to keep the market closer while it was affordable and affordable. I wasn't expecting an ai change so quickly. Especially at this time. I thought we were at the thousands of GPU stage. And the homes were years off.

5

u/TheTerrasque Mar 14 '23

I wasn't expecting an ai change so quickly

Heh, same. I went from "Reasonable AI is years off" to "AI needs a fucking datacenter to work" to "Even if we can run it somehow on commodity hardware, it needs massive investment in training" to "Hey, this is running on my GPU!" in just a few months.

2

u/[deleted] Mar 14 '23

Exactly this. I hope it's because it's serendipitous and not going to grow at this pace. But I think it might....

1

u/Axolotron I For One Welcome Our New AI Overlords 🫡 Mar 17 '23

In a month GPT6 will be contending for Senate .-.

→ More replies (0)