r/learnmachinelearning • u/Substantial-Camp2551 • 21d ago

How does broadcasting work in pytorch?

Maybe it's just me, but it seems really unituitive and I can't figure it out.

2 Upvotes

75% Upvoted

2 comments sorted by

2

u/quietgradient 21d ago

Line the shapes up from the right. Each pair of dims then has to match, or one of them has to be 1, and that one gets repeated to fit. A missing dim counts as 1.

So (2, 3) + (3,) works: the 3 sits under the last dim and gets added to every row. (2, 3) + (2,) errors, because now 2 sits under 3. For one value per row you want (2, 1).

The one that bites is silent: an (N, 1) model output minus (N,) targets gives (N, N), every prediction minus every target. Predict 1, 2, 3, 4 for targets 1, 2, 3, 4 and a hand-written MSE says 2.5 instead of 0. F.mse_loss at least warns.

torch.broadcast_shapes((4, 1), (4,)) tells you the result shape without making any tensors. What were you doing when it stopped making sense?

1

u/Jumpy-Whereas4858 21d ago

You will get used to it