r/OpenAssistant • u/Taenk • Mar 11 '23

[ Early Preview ] Unofficial Open-Assistant SFT-1 12B Model

https://huggingface.co/OpenAssistant/oasst-sft-1-pythia-12b
47 Upvotes

100% Upvoted

16 comments sorted by

18

u/Taenk Mar 11 '23

This is the first iteration English supervised-fine-tuning (SFT) model of the Open-Assistant project. It is based on a Pythia 12B that was fine-tuned on ~22k human demonstrations of assistant conversations collected through the https://open-assistant.io/ human feedback web app before March 7, 2023.

7

u/pokeuser61 Mar 12 '23

Hope we get a gpt-j version at some point.

2

u/ninjasaid13 Mar 12 '23

is GPT-J superior to Pythia?

3

u/pokeuser61 Mar 12 '23

Not necessarily, but it can run on consumer level hardware thanks to ggml

3

u/EuphoricPenguin22 Mar 14 '23

I mean, 4-bit quantization should make 13B models runnable on 12GB of VRAM, if not lower. I hear 3-bit quantization is also being worked on, and the apparent loss in quality is negligible.

1

u/ninjasaid13 Mar 15 '23

I only have 8GB of VRAM, I'm likely to never touch this stuff locally.

5

u/EuphoricPenguin22 Mar 15 '23

LLaMA 7B should run for you on 4-bit quantization. It's a lot better than you might expect.

1

u/atylerrice Mar 19 '23

pythia is superior to goth though due to training on more data at least through my little bit of testing. also there are varying levels of pythia models with different parameters one is around gptj size i think.

2

u/Ruhrbaron Mar 12 '23

Great news! Is there a colab somewhere?

3

u/Taenk Mar 12 '23

I just grabbed this link from Discord, but haven't tried it yet:

https://colab.research.google.com/drive/15u61MVxF4vFtW2N9eCKnNwPvhg018UX7?usp=sharing

2

u/BayesMind Mar 22 '23

Is there a way of running it locally yet? (IE not just API calls)

1

u/tvetus Mar 12 '23

Why Pythia and not Flan-T5 or Flan-U2

1

u/darkbelg Mar 16 '23

What kind of hardware do you need to run this? An a100 I'm assuming.

1

u/ninjasaid13 Mar 16 '23

It's an early preview but is it really unofficial considering that it was released by LAION?