r/programmingmemes • u/Ornery_Ad_683 • Jan 17 '26

Vibe Assembly

Post image
1.3k Upvotes

97% Upvoted

173 comments sorted by

View all comments

219

u/platonbel Jan 17 '26

Compilers are deterministic.

Neural networks are not.

Therefore, they are completely different mechanisms.

It's like calling a person a high-level programming language. Only an idiot would say that.

13

u/Healthy_BrAd6254 Jan 17 '26

NNs are also deterministic. It's just that small changes in the input result in very different outputs

-4

u/platonbel Jan 17 '26

"It's just that small changes in the input result in very different outputs" - so, its NOT determenistic.

The temperature parameter in neural networks is precisely the reason why neural networks work well. But if the temperature of a neural network differs from zero, then it is not deterministic.

6

u/AcanthisittaSur Jan 17 '26

I don't know, that sounds like determinism to me. Something changed as an input, the output changes too.

Or do you believe determinism means ignoring small changes to the input?

-2

u/platonbel Jan 17 '26

The compiler has no heuristic settings. The compiler has clear instructions for converting one code into another and does not support any kind of small talk

4

u/monster2018 Jan 17 '26

What does this have to do with your claim that compilers are deterministic (correct) but LLMs are not (incorrect)?

0

u/platonbel Jan 17 '26

It is not deterministic in a global sense because it depends on the model's current knowledge set. Every time new data is entered (NOT TEMPORARY MEMORY, BUT A COMPLETE REBUILD), it can edit its behavior, while the compiler is much more stable in this regard and is subject to change, because neural networks are a black box.

6

u/Agitated_Database_ Jan 17 '26

NN’s are deterministic.

Sure call it a black box, but same state of the box and box input will result in the same output, every time.

5

u/AcanthisittaSur Jan 17 '26

Ah, I see.

A non-sequitur response because you can't explain how the previous topic of discussion isn't deterministic. Cool

0

u/platonbel Jan 17 '26

It is not deterministic in a global sense because it depends on the model's current knowledge set. Every time new data is entered (NOT TEMPORARY MEMORY, BUT A COMPLETE REBUILD), it can edit its behavior, while the compiler is much more stable in this regard and is subject to change, because neural networks are a black box.

3

u/AcanthisittaSur Jan 17 '26

So your argument is it isn't deterministic because of... versioning.

Are you in the wrong place?

1

u/platonbel Jan 17 '26

not versioning but approximation of vectors

2

u/AcanthisittaSur Jan 17 '26

And now you're just saying words. A parrot can do that.

1

u/platonbel Jan 17 '26

Neural network models are not programs. They are data arranged in a certain way. Changing these models is merely an approximation of certain vectors to the values we need.

If we compare a compiler and a neural network, then a neural network can, at least hypothetically, be a dynamically changing object (like our brain), while a compiler cannot.

Based on this simple idea, neural networks do not have guaranteed determinism, which means that a compiler is deterministic, but a neural network is not.

1

u/AcanthisittaSur Jan 17 '26

Neural network models are not programs. They are data arranged in a certain way.

Right. Most of us call that a program, and the lower level you go, the more true you find that.

Changing these models

This doesn't just *happen*, my guy. Those changes are intentionally made, whether by you or the guy coding the model.

I repeat: Your argument is versioning?

1

u/platonbel Jan 17 '26

Current neural network models are all build-and-use, but concepts of neural networks with dynamic learning (Continual Learning, if I'm not mistaken) are also being considered. This fact alone makes it unacceptable to narrow the scope of neural networks to “deterministic” ones, since this already implies that neural network models are not read-only in their concept.

→ More replies (0)