Thanks! And now the links I've promised.

First, I want to apologize for the manner of pitching things in the most objectionable form one can think of at a moment

It's more fun this way though

Second, I'd like to clarify, that I find that the subject actually is relevant: the word "tensor" is horribly overloaded and causes a lot of confusion even now.

I'm not particularly familiar with the history of mathematics, but I suspect the word used to be used predominantly by physicists, and in the same sense as formalized later with differential geometry. Of course, the many loose possible meanings physicists may conjure up in their heads don't always coincide exactly with the formalization, and that's already one source of confusion.

As the century of breaking the grounds of physics becomes more of a century of computation, it also makes more sense to talk just about "multidimensional arrays". Especially for computer scientists, or people dealing with discrete mathematics. Today we seem to have what one could call the "numerical (linear) algebra" community with their "tensor trains" and other "decompositions"^[1] who tend to purposefully forget where the structure of their "tensors" is coming from. I recall chatting to a student of a professor-not-unrelated-to-tensor-trains who, rather violently, denied that their "multidimensional arrays" have got anything to do with "tensors from physics, or with multilinear maps, or anything like it". It seems that to many physicists "tensors" really "are things that transform like tensors" (meaning they aren't anything concrete at all!), and they cannot quite make one step further. To these people I say: "why do you contract?".

Things gets infinitely worse with the rise of "deep learning", but this also demonstrates the need for a consistent-but-comfortable language. Accidentally, this is also where things get infinitely more complicated, because people start discussing dependent types for testing shapes, and linear types for some imperative algorithms.

Then, of course, we have the algebraists, who would often talk about "tensor products" before deciding what a "tensor" is… This must include all sorts of Bourbakists, this probably includes those who found in categories, because they like "universal properties", et cetera

Personally, I'd rather prefer any resolution, where all of "tensors that transform like tensors", and multilinear maps, and fields of multilinear maps, and multidimensional arrays, and arrays treated as functions on finite domains, and what not: really are derivatives of a single unified concept. This is pretty much orthogonal to what we talked about today, i.e. what foundations of mathematics you choose: with sets, and with types, and with categories you end up with the same building blocks, namely "arrows" that map "values" and "arrows" that construct "types". Formalizing arrays as functions ("arrows") is obvious. The relation between arrays and multilinear maps or sections of whatever-bundles on manifolds is also obvious: the former are simply the coordinate representations for the latter, when one chooses some coordinates. "Tensors that transforms like tensors" is what differential geometry formalizes anyway

In the previous paragraph I hoped to make a convincing argument, that it should not be impossible to talk about all of these "tensors" from different peoples within a single language. The next question is whether we can come up with a concrete consistent and usable notation? Well, here come the two (pretty much unrelated) attempts that I wanted to share: one belongs in the realm of programming languages, and the other is more classical stock.

One: the Dex language by some folks from Google.^[2] Very briefly, the core idea is to extend the typed lambda-calculus to numerical arrays: an array is a (memoized) function on a finite domain (the "index set"). The array types are constructed from index sets and a value type using a special "arrow" operator, much like classical functions are constructed using the classical "arrow" from their domain and range spaces. The array-values are constructed using a special kind of lambda-expression (which they dub "for"-expressions), much like classical functions. This results in a formal language, that can be actually implemented in a computer^[3] and even do some useful stuff.

Two: an entirely unrelated PhD thesis^[4] by someone Tai-Danae Bradley, the relevance of which to our conversation consists in the fact, that she connects the notions of "arrays as functions" to the linear algebraic notion of tensors ("multilinear maps", also carefully handling questions like duality), and to the "tensor network diagrams", a notation used a lot by the "numerical algorithms" people. This piece of writing is beautiful in how new concepts are introduced gradually, in a natural consequention. We don't invent tensor contraction because "then we'd get some nice structure to study". The nice structure arises naturally, and we discover the tensor contraction as we attempt to formalize it!

P.S. Yes, the index-sets in Dex are type-level natural numbers, separate from "values". I went into the set-theoretic version just to enjoy the conversation more

---

1. https://en.wikipedia.org/wiki/Tucker_decomposition#References ↩︎

2. Dex-lang Arxiv: https://arxiv.org/abs/2104.05372 ↩︎

3. Dex-lang GitHub: https://github.com/google-research/dex-lang ↩︎

4. "At the Interface of Algebra and Statistics": https://arxiv.org/abs/2004.05631 ↩︎