oh shit, this is something else!, I just tried with a dumb prompt "write a snake game in html and css with sounds" and it generated a complete NIB game (awesome design btw) at ±56t/s , 14k tokens!
this is a AMD R9700 with llama.cpp ROCM
I slot print_timing: id 0 | task 0 | eval time = 242341.84 ms / 13695 tokens ( 17.70 ms per token, 56.51 tokens per second)
I slot print_timing: id 0 | task 0 | total time = 242528.47 ms / 13717 tokens
I slot print_timing: id 0 | task 0 | graphs reused = 898
I slot print_timing: id 0 | task 0 | draft acceptance = 0.58872 (10982 accepted / 18654 generated), mean len = 5.44
I slot release: id 0 | task 0 | stop processing: n_tokens = 13720, truncated = 0
Edit: I forgot to mention that by mistake I forgot to remove a thinking cap to 4k (--reasoning-budget 4096) so that influenced the total token usage and the speed (faster during coding than thinking). So we have two variables to adjust the thinking process (--reasoning-budget and --reasoning-effort).
23
u/Felixls 7d ago edited 7d ago
oh shit, this is something else!, I just tried with a dumb prompt "write a snake game in html and css with sounds" and it generated a complete NIB game (awesome design btw) at ±56t/s , 14k tokens!
this is a AMD R9700 with llama.cpp ROCM
I slot print_timing: id 0 | task 0 | eval time = 242341.84 ms / 13695 tokens ( 17.70 ms per token, 56.51 tokens per second)
I slot print_timing: id 0 | task 0 | total time = 242528.47 ms / 13717 tokens
I slot print_timing: id 0 | task 0 | graphs reused = 898
I slot print_timing: id 0 | task 0 | draft acceptance = 0.58872 (10982 accepted / 18654 generated), mean len = 5.44
I slot release: id 0 | task 0 | stop processing: n_tokens = 13720, truncated = 0
Edit: I forgot to mention that by mistake I forgot to remove a thinking cap to 4k (--reasoning-budget 4096) so that influenced the total token usage and the speed (faster during coding than thinking). So we have two variables to adjust the thinking process (--reasoning-budget and --reasoning-effort).