Winner-take-all is a computational principle applied in computational models of neural networks by which neurons compete with each other for activation. In the classical form, only the neuron with the highest activation stays active while all other neurons shut down; however, other variations allow more than one neuron to be active, for example the soft winner take-all, by which a power function is applied to the neurons.

In the theory of artificial neural networks, winner-take-all networks are a case of competitive learning in recurrent neural networks. Output nodes in the network mutually inhibit each other, while simultaneously activating themselves through reflexive connections. After some time, only one node in the output layer will be active, namely the one corresponding to the strongest input. Thus the network uses nonlinear inhibition to pick out the largest of a set of inputs. Winner-take-all is a general computational primitive that can be implemented using different types of neural network models, including both continuous-time and spiking networks.

Winner-take-all networks are commonly used in computational models of the brain, particularly for distributed decision-making or action selection in the cortex. Important examples include hierarchical models of vision, and models of selective attention and recognition. They are also common in artificial neural networks and neuromorphic analog VLSI circuits. It has been formally proven that the winner-take-all operation is computationally powerful compared to other nonlinear operations, such as thresholding.

In many practical cases, there is not only one single neuron which becomes active but there are exactly k neurons which become active for a fixed number k. This principle is referred to as k-winners-take-all.

Consider a single linear neuron, with inputs ${\displaystyle x_{1},\dots ,x_{n}}$. Each input has weight ${\displaystyle w_{i}}$, and the output of the neuron is ${\displaystyle \sum _{i}w_{i}x_{i}}$. In the Instar learning rule, on each input vector, the weight vectors are modified according to ${\displaystyle \Delta w_{i}=\eta (x_{i}-w_{i})}$ where ${\displaystyle \eta }$ is the learning rate. This rule is unsupervised, since we need just the input vector, not a reference output.

Now, consider multiple linear neurons ${\displaystyle y_{1},\dots ,y_{m}}$. The output of each satisfies ${\displaystyle y_{i}=\sum _{j}w_{ij}x_{j}}$.

In the winner-take-all algorithm, the weights are modified as follows. Given an input vector ${\displaystyle x}$, each output is computed. The neuron with the largest output is selected, and the weights going into that neuron are modified according to the Instar learning rule. All other weights remain unchanged.

The k-winners-take-all rule is similar, except that the Instar learning rule is applied to the weights going into the k neurons with the largest outputs.