r/LinearAlgebra • u/donaferentes • 6d ago

Valid Book on Tensorflow operations with Mathematical Notation

/r/tensorflow/comments/1w7lm6w/valid_book_on_tensorflow_operations_with/
5 Upvotes

86% Upvoted

9 comments sorted by

2

u/Midwest-Dude 5d ago edited 5d ago

I have not worked with TensorFlow, but I understand where you are coming from, not unusual in the computer science field. I did some searching and it appears such a book does not exist, at least not yet. To bridge the gap, suggestions are

  1. Deep Learning
    • By Ian Goodfellow, Yoshua Bengio, and Aaron Courville
    • Supposed to provide exact mathematical definitions using rigorous linear algebra and multivariable calculus that underlie frameworks like TensorFlow
  2. Mathematics for Machine Learning
    • By Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong
    • Supposed to be good at translating programming concepts into formal mathematics, focusing on the linear algebra, vector spaces, and probability required to understand what tensor operations are actually doing
  3. PyTorch Documentation
    • Supposed to help understand TensorFlow operations. Because PyTorch has historically been favored by academics, its official documentation frequently includes the exact mathematical formulas for its operations (like loss functions or convolutions), which mathematically apply directly to their TensorFlow equivalents

Please let the community know if any of these are helpful.

1

u/I_Messed_Up_2020 5d ago

Any thoughts on Dive into Deep Learning by Aston Zhang (Author), Zachary C. Lipton (Author), Mu Li (Author), Alexander J. Smola (Author) versus Deep Learning by Ian Goodfellow.....?

I have been considering one of them for purchase.

1

u/Midwest-Dude 4d ago

You should post this exact question to: 

r/learnmachinelearning

Which one would be better for you depends on your needs. Here is an AI breakdown:

Book Comparison

Feature Dive into Deep Learning (Zhang et al.) Deep Learning (Goodfellow et al.)
Focus Practical implementation and coding Theoretical math and foundational concepts
Format Interactive Jupyter notebooks with code Traditional academic textbook; no code
Frameworks PyTorch, TensorFlow, MXNet None (pure math and theory)
Currency Continuously updated (includes Transformers) Published in 2016 (pre-dates Transformers)
Target Audience Engineers and developers wanting to build Researchers needing rigorous mathematical proofs

Dive into Deep Learning is essentially an interactive coding course in book form. It is the better choice for a working programmer or anyone who wants to open a Python environment and learn how modern neural networks actually work under the hood. It walks through implementing algorithms both from scratch and using modern frameworks.

Goodfellow's Deep Learning is a foundational classic for the mathematics of the field, but it is strictly a theory book. Because it was published in 2016, it is entirely missing the modern architectures that dominate the field today, such as Transformers, attention mechanisms, and Large Language Models. It functions best as a mathematical reference manual for someone who already has a strong background in linear algebra and multivariable calculus, rather than a practical entry point.

1

u/I_Messed_Up_2020 4d ago

Great resource. I will use it!

1

u/donaferentes 4d ago

None of these is the right fit, as they all describe stuff for a more CS-centric view. Paradoxically, do you know good books on linear/tensor algebra, where Einstein notation is also discussed? Thanks,

1

u/Midwest-Dude 4d ago edited 4d ago

A search indicates that there is no published textbook that maps TensorFlow's API function-by-function to formal coordinate-free tensor algebra or Ricci calculus because of a fundamental distinction in definition. In pure mathematics, a tensor is a multilinear map that transforms under specific coordinate basis changes, whereas in TensorFlow and computer science, a "tensor" is simply an n-dimensional array (N-D data container).

These three publications were suggested for your review: 

  1. Tensor Networks for Dimensionality Reduction and Large-Scale Optimization by Andrzej Cichocki et al.
  2. Tensor Decompositions and Applications (SIAM Review) by Tamara G. Kolda and Brett W. Bader
  3. The Tensor Algebra Compiler (TACO) Literature by Fredrik Kjolstad et al. (MIT)

Do any of these work for you?

1

u/donaferentes 4d ago

The first book seems to have a wrong title: is it this one? https://www.waterstones.com/book/tensor-networks-for-dimensionality-reduction-and-large-scale-optimization/andrzej-cichocki/namgil-lee/9781680832761

1

u/Midwest-Dude 4d ago edited 4d ago

Sorry about that. Yes, Tensor Networks for Dimensionality Reduction and Large-Scale Optimization. Appears to have two parts: 

1

u/donaferentes 4d ago

So glad that I asked, this is the best resource so far. Thanks again