In machine learning and statistical classification, multiclass classification or multinomial classification is the problem of classifying instances into one of three or more classes (classifying instances into one of two classes is called binary classification). For example, deciding on whether an image is showing a banana, peach, orange, or an apple is a multiclass classification problem, with four possible classes (banana, peach, orange, apple), while deciding on whether an image contains an apple or not is a binary classification problem (with the two possible classes being: apple, no apple).

While many classification algorithms (notably multinomial logistic regression) naturally permit the use of more than two classes, some are by nature binary algorithms; these can, however, be turned into multinomial classifiers by a variety of strategies.

Multiclass classification should not be confused with multi-label classification, where multiple labels are to be predicted for each instance (e.g., predicting that an image contains both an apple and an orange, in the previous example).

From the confusion matrix of a multiclass model, we can determine whether a model does better than chance. Let ${\displaystyle K\geq 3}$ be the number of classes, ${\displaystyle {\mathcal {O}}}$ a set of observations, ${\displaystyle {\hat {y}}:{\mathcal {O}}\to \{1,...,K\}}$ a model of the target variable ${\displaystyle y:{\mathcal {O}}\to \{1,...,K\}}$ and ${\displaystyle n_{i,j}}$ be the number of observations in the set ${\displaystyle \{y=i\}\cap \{{\hat {y}}=j\}}$. We note ${\displaystyle n_{i.}=\sum _{j}n_{i,j}}$, ${\displaystyle n_{.j}=\sum _{i}n_{i,j}}$, ${\displaystyle n=\sum _{j}n_{.j}=\sum _{i}n_{i.}}$, ${\displaystyle \lambda _{i}={\frac {n_{i.}}{n}}}$ and ${\displaystyle \mu _{j}={\frac {n_{.j}}{n}}}$. It is assumed that the confusion matrix ${\displaystyle (n_{i,j})_{i,j}}$ contains at least one non-zero entry in each row, that is ${\displaystyle \lambda _{i}>0}$ for any ${\displaystyle i}$. Finally we call "normalized confusion matrix" the matrix of conditional probabilities ${\displaystyle (\mathbb {P} ({\hat {y}}=j\mid y=i))_{i,j}=\left({\frac {n_{i,j}}{n_{i.}}}\right)_{i,j}}$.

The lift is a way of measuring the deviation from independence of two events ${\displaystyle A}$ and ${\displaystyle B}$ :

${\displaystyle \mathrm {Lift} (A,B)={\frac {\mathbb {P} (A\cap B)}{\mathbb {P} (A)\mathbb {P} (B)}}={\frac {\mathbb {P} (A\mid B)}{\mathbb {P} (A)}}={\frac {\mathbb {P} (B\mid A)}{\mathbb {P} (B)}}}$

We have ${\displaystyle \mathrm {Lift} (A,B)>1}$ if and only if events ${\displaystyle A}$ and ${\displaystyle B}$ occur simultaneously with a greater probability than if they were independent. In other words, if one of the two events occurs, the probability of observing the other event increases.

A first condition to satisfy is to have ${\displaystyle \mathrm {Lift} (y=i,{\hat {y}}=i)\geq 1}$ for any ${\displaystyle i}$. And the quality of a model (better or worse than chance) does not change if we over- or undersample the dataset, that is if we multiply each row ${\displaystyle R_{i}}$ of the confusion matrix by a constant ${\displaystyle c_{i}}$. Thus the second condition is that the necessary and sufficient conditions for doing better than chance need only depend on the normalized confusion matrix.