In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables. More specifically, it quantifies the "amount of information" (in units such as shannons (bits), nats or hartleys) obtained about one random variable by observing the other random variable. The concept of mutual information is intimately linked to that of entropy of a random variable, a fundamental notion in information theory that quantifies the expected "amount of information" held in a random variable.

Not limited to real-valued random variables and linear dependence like the correlation coefficient, MI is more general and determines how different the joint distribution of the pair ${\displaystyle (X,Y)}$ is from the product of the marginal distributions of ${\displaystyle X}$ and ${\displaystyle Y}$. MI is the expected value of the pointwise mutual information (PMI).

The quantity was defined and analyzed by Claude Shannon in his landmark paper "A Mathematical Theory of Communication", although he did not call it "mutual information". This term was coined later by Robert Fano. Mutual Information is also known as information gain.

Let ${\displaystyle (X,Y)}$ be a pair of random variables with values over the space ${\displaystyle {\mathcal {X}}\times {\mathcal {Y}}}$. If their joint distribution is ${\displaystyle P_{(X,Y)}}$ and the marginal distributions are ${\displaystyle P_{X}}$ and ${\displaystyle P_{Y}}$, the mutual information is defined as

where ${\displaystyle D_{\mathrm {KL} }}$ is the Kullback–Leibler divergence, and ${\displaystyle P_{X}\otimes P_{Y}}$ is the outer product distribution which assigns probability ${\displaystyle P_{X}(x)\cdot P_{Y}(y)}$ to each ${\displaystyle (x,y)}$.

Expressed in terms of the entropy ${\displaystyle H(\cdot )}$ and the conditional entropy ${\displaystyle H(\cdot |\cdot )}$ of the random variables ${\displaystyle X}$ and ${\displaystyle Y}$, one also has (See relation to conditional and joint entropy):

Notice, as per property of the Kullback–Leibler divergence, that ${\displaystyle I(X;Y)}$ is equal to zero precisely when the joint distribution coincides with the product of the marginals, i.e. when ${\displaystyle X}$ and ${\displaystyle Y}$ are independent (and hence observing ${\displaystyle Y}$ tells you nothing about ${\displaystyle X}$). ${\displaystyle I(X;Y)}$ is non-negative, it is a measure of the price for encoding ${\displaystyle (X,Y)}$ as a pair of independent random variables when in reality they are not.

If the natural logarithm is used, the unit of mutual information is the nat. If the log base 2 is used, the unit of mutual information is the shannon, also known as the bit. If the log base 10 is used, the unit of mutual information is the hartley, also known as the ban or the dit.