Sakana AI Proposes Error Diffusion Training Without Backpropagation, Achieving 96.7% on MNIST and 61.7% on CIFAR-10

Sakana AI introduced Error Diffusion, a training method for Dale's principle two-stream excitatory/inhibitory networks. Dale's principle states a neuron releases only one type of neurotransmitter, but backpropagation's weight transport is biologically implausible. Error Diffusion routes error signals through modulus error routing to excitatory and inhibitory pathways, enabling training without backpropagation. It achieves 96.7% on MNIST and 61.7% on CIFAR-10, and can be extended to reinforcement learning. This method is significant for researchers in bio-inspired learning and neuromorphic computing, demonstrating that backpropagation-free training can achieve practical accuracy on larger tasks. For those seeking alternatives to backpropagation or more biologically consistent algorithms, Error Diffusion provides a concrete approach with known performance. The work also highlights the role of task-specific ablation studies in understanding network behavior.