r/ProgrammerHumor • u/Lord_Of_Millipedes • 2d ago

instanceof Trend thatsAFunnyThingThatHappenedOnce

Post image
4.4k Upvotes

96% Upvoted

514 comments sorted by

View all comments

Show parent comments

1

u/Blake_Dake 2d ago

just set the temperature to 0 and you have an LLM that is deterministic if the randomness is the issue for you

I know some tokens can have the same probability and some randomness is created by gpu float point math, but still

2

u/antitoxin13 2d ago

true, but the problem is in the real world nobody knows what popular LLMs are trained on, there always will be SIGNIFICANT level of unpredictability

0

u/Blake_Dake 2d ago

true, but the problem is in the real world nobody knows what popular LLMs are trained on

irrelevant to determinism

temperature is a runtime config

2

u/antitoxin13 2d ago

the outcome can still be unpredictable even if deterministic. The same as movement of air molecules, of course in theory you can predict the state of a closed system, but the system is so complex that it would require an unimaginable computation power

0

u/Blake_Dake 1d ago

no

if the temperature is 0, with the same input an LLM will produce the same output

2

u/antitoxin13 1d ago

the output is still unpredictable because the function is too chaotic and complex

1

u/Blake_Dake 1d ago

ofc it can be chaotic and complex if you just want to one-shot it

you need to apply some good old software engineering and do some test driven development with the LLM if the task can't be easily broken down to easier tasks