The Tensor Cookbook

What are Tensor Diagrams?

Machine learning involves a lot of tensor manipulation, and it's easy to lose track of the larger structure when manipulating high-dimensional data using notation designed for vectors and matrices.

Graphical notation (first introduced by Roger Penrose in 1971) reduces the mental overhead and makes the connections "come alive":

In short, each edge is the index of a tensor, and connecting two edges contracts the tensors over this dimension. After a bit of practice, this becomes incredibly intuitive.

The Tensor Cookbook aims to popularize tensor diagrams by rewriting the classical "Matrix Cookbook". You can think of it as a reference book, skip around for some cool diagrams, or a crash course full of exercises to practice your skill.

Tensorgrad

is a python library for symbolic tensor manipulation and derivatives using tensor diagrams. Try it here:

@book{ahle2024tensorcookbook,
  title={The Tensor Cookbook},
  author={Ahle, Thomas Dybdahl},
  year={2024},
  url={https://tensorcookbook.com},
  note={Version: September, 2024}
}