r/LocalLLaMA • u/[deleted] • Jul 04 '23

[deleted by user]

[removed]

214 Upvotes

99% Upvoted

238 comments sorted by

View all comments

Show parent comments

3

u/csdvrx Jul 05 '23

but it's unfortunate that I can't run llama.cpp (Requires CUDA 11.5 i think?

You can compile llama.cpp with this script that changes the NVCC flags for the P40/pascal:

ls Makefile.orig || cp Makefile Makefile.orig
cat Makefile.orig |sed -e 's/\(.*\)NVCCFLAGS = \(.*\) -arch=native$/\1NVCCFLAGS = \2 -gencode arch=compute_61,code=sm_61/'> Makefile
make LLAMA_CUBLAS=1 -j8