Theorem 6

Topological Organization: Neural Fiber Bundles

๐Ÿง’

Explain Like I'm 5

Imagine you're organizing your toys. You could put them in boxes with straight lines dividing them (like a grid), but that's boring and doesn't match how toys naturally group together.

Instead, imagine each toy has a magnetic field around it that attracts similar toys. These fields create swirling, vortex-like territories where similar toys naturally gather.

The โตŸ-product creates these "vortex territories"! Each neuron creates a swirling field around its weight vector, and inputs are classified based on which territory they fall into. It's like having smart, organic boundaries instead of boring straight lines.

Theorem: โตŸ-product classification creates a fiber bundle structure over the input space, where each neuron defines a "fiber" (territory) with vortex-like boundaries. The decision boundaries are smooth, localized, and topologically organized.

๐ŸŽฏ The Problem This Solves

Traditional linear models create unbounded, half-space partitions:

  • Decision boundaries are hyperplanes (straight lines/planes)
  • Each class occupies an unbounded region
  • No natural "territories" or localization

The โตŸ-product creates localized, vortex-like territories with smooth, organic boundaries that better match natural data distributions.

๐Ÿ“ The Mathematics In Depth

In a classification setting with $K$ classes, we have $K$ weight vectors $\{\mathbf{w}_1, ..., \mathbf{w}_K\}$. The decision rule is:

$$\text{class}(\mathbf{x}) = \arg\max_k \text{โตŸ}(\mathbf{w}_k, \mathbf{x})$$

This creates a Voronoi-like partition, but with smooth, curved boundaries instead of straight lines. Each region is:

  • Localized: Bounded around the weight vector (due to inverse-square decay)
  • Smooth: Infinitely differentiable boundaries (due to Cโˆž property)
  • Vortex-like: Creates swirling patterns due to the interaction of alignment and proximity

The fiber bundle structure comes from viewing each class as a "fiber" over the input space, with the โตŸ-product defining the connection.

๐Ÿ’ฅ The Consequences

๐ŸŒ€
Vortex Territories

Each neuron creates a localized "vortex" territory around its weight vector, with smooth, curved boundaries that naturally adapt to data distribution.

๐Ÿ“
Geometric Interpretability

The weight vectors act as "prototypes" or "centers" of their territories, making the learned representations geometrically interpretable.

๐ŸŽฏ
Better Generalization

Localized territories prevent overfitting to distant outliers and create more robust decision boundaries that generalize better.

๐Ÿ”ฌ
Topological Insights

The fiber bundle structure provides a rich mathematical framework for understanding how NMNs organize and partition the input space.

๐ŸŽ“ What This Really Means

This theorem reveals the geometric organization of NMNs. Unlike linear models that create arbitrary half-spaces, โตŸ-product neurons create natural territories โ€” regions of space that "belong" to each neuron.

This organization is:

  • Localized: Each territory is bounded and centered around a prototype
  • Smooth: Boundaries are infinitely differentiable, creating organic shapes
  • Interpretable: Weight vectors directly represent the "center" of each territory
  • Topologically Rich: The fiber bundle structure provides deep mathematical insights

This is why NMNs learn sharp, coherent prototypes (as seen in MNIST experiments) rather than diffuse, blurry representations โ€” each neuron naturally organizes into a well-defined territory.