Soap Bubbles and Attention Sinks: The Theory and History of the HALO-L
www.pisoni.ai
The standard Cross-Entropy loss has a well-known flaw: it forces neural networks to push their features toward infinity just to reach full confidence. The result is a messy latent space and models tha