Why Amplect Should Replace Broadcasting for Tensor Operations

The core of all tensor algorithms is matrix multiplication, and matrix multiplication relies heavily on broadcasting under the hood, the mechanism that allows operations between tensors of different shapes. While broadcasting works in many cases, its arbitrary rules are inelegant and error-prone.

There's a better approach. Following the pattern of Einop notation, Amplect provides a clearer, more consistent way to specify these element-wise tensor operations.

Why Broadcasting Is Weird

Let's say you want to add two tensors:

T1 with shape [a b c] and T2 with shape [c].

Broadcasting handles this elegantly. It automatically stretches T2 to match T1's shape for the addition.

But what if you want to do the same addition when T2 has shape [b] instead? Now broadcasting fails, declaring these tensors "unbroadcastable." To make it work, you'd need to manually reshape T2 into [b 1], but why? The issue is broadcasting's arbitrary rules.

Rules for Broadcast

To understand why broadcasting behaves so inconsistently, let's look at exactly how it works.

Broadcasting compares tensors dimension by dimension, starting from the rightmost:

1. If both dimensions have the same size -> Continue
2. If either dimension has size 1 -> Stretch it to match the other -> Continue
3. If a tensor runs out of dimensions -> Treat missing dimensions as size 1
4. Otherwise -> Broadcasting fails

Example 1: T1 [a b c] and T2 [c]

• Compare c vs c -> Equal, continue
• Compare b vs (missing) -> Treat T2 as size 1, stretch to b, continue
• Compare a vs (missing) -> Treat T2 as size 1, stretch to a, continue

Result: Broadcasting Succeeds. T2 effectively becomes [a b c].

Example 2: T1 [a b c] and T2 [b]

• Compare c vs b -> Different sizes, neither is 1 -> Broadcasting fails

Result: Broadcasting Fails.

Example 3: T1 [a b c] and T2 [b 1]

• Compare c vs 1 -> T2's dimension is 1, stretch to c, continue
• Compare b vs b -> Equal, continue
• Compare a vs (missing) -> Treat T2 as size 1, stretch to a, continue

Result: Broadcasting Succeeds. T2 effectively becomes [a b c].

A Common Frustration with Broadcasting

Suppose you’re working with a batch of sequences:

data with shape [batch, seq_len, features] and a mask with shape [batch, seq_len]

You want to zero out padded tokens by multiplying the mask with the data.

masked = data * mask

But because of our above rules, these two tensors are unbroadcastable.

To make it work, you have to reshape the mask:

masked = data * mask[..., None] 

That [..., None] may be second nature to experienced users, but it’s a workaround for broadcasting’s arbitrary dimension-matching rules.

Why should you have to remember which axis to stretch? The intent is obvious, align batch and seq_len, and perform element-wise multiplication across features. Yet broadcasting forces you to manually manipulate dimensions just to express something that’s mathematically straightforward.

Broadcasting Forces Us Into a Box

The examples above reveal broadcasting's fundamental flaw: its rules are arbitrary. Why should [a b c] and [c] be compatible, but [a b c] and [b] incompatible? Both operations make mathematical sense.

In fact, any two tensors can be combined through their outer product.

Take tensors A with shape [a b c] and B with shape [d e f].

Broadcasting calls these "unbroadcastable," but if we add enough singleton dimensions to A so that it has shape [a b c 1 1 1], they can suddenly broadcast to shape [a b c d e f] which is the outer product.

The issue isn't that certain operations are impossible, it's that broadcasting forces us to guess which specific operation we want, based on arbitrary positioning rules.

We need a way to be explicit about our intent.

Amplect

Amplect solves broadcasting's guessing problem by letting us explicitly specify which dimensions correspond to each other. Instead of relying on position-based rules, we declare our intent directly. This is a theoretical approach that should eventually replace broadcasting.

Remember our problematic case?

We wanted to add tensors with shapes [a b c] and [b].

With Amplect, we simply write:

a b c, b -add> a b c

This tells the computer: "Take the b dimension from the second tensor and align it with the b dimension in the first tensor for addition."

For each position [i, j, k] in the output with shape [a b c], the operation becomes clear:

Output[i, j, k] = A[i, j, k] + B[j]

The outer product from our earlier example becomes:

a b c, d e f -multiply> a b c d e f

For each position [i, j, k, l, m, n] in the output with shape [a b c d e f]:

Output[i, j, k, l, m, n] = A[i, j, k] * B[l, m, n]

We can even combine more than 2 tensors:

a b, a c, b c, c d -multiply> a b c d

For each position [i, j, k, l] in the output with shape [a b c d]:

Output[i, j, k, l] = A[i, j] * B[i, k] * C[j, k] * D[k, l]

For multi-tensor operations, amplect is less error-prone if the function is associative.

This is because on parallel hardware, for n tensors, we can perform O(log(n)) parallel steps instead of O(n) sequential steps.

Associativity is a property of functions where the order of evaluation doesn't matter.

So:

f(f(x, y), z) = f(x, f(y, z))

For example, multiplication:

(a * b) * c = a * (b * c)

Making Our Common Frustration Less Frustrating

We can compare the current and proposed approaches to the frustration I shared earlier.

Broadcasting

masked = data * mask[..., None]

Amplect

masked = amplect("batch seq features, batch seq -multiply> batch seq features", data, mask)

You can see that not only do we avoid the [..., None] workaround, but the intent is much clearer at a glance.

Conclusion

Broadcasting's position-based rules create unnecessary friction in tensor programming. When [a b c] and [c] work together but [a b c] and [b] don't, we're not dealing with mathematical limitations; we're fighting arbitrary syntax.

Amplect eliminates this guessing game. By explicitly declaring which dimensions align, we write code that expresses our intent clearly. No more mental gymnastics about dimension ordering, no more surprise "unbroadcastable" errors, no more reshaping tensors just to satisfy arbitrary rules.

The result is tensor code that reads like what it actually does, a step toward making tensor programming more intuitive and less error-prone.

As tensor programming grows more complex, moving toward explicit, intention-driven tools like Amplect will reduce bugs and make code easier to read and maintain.