In information theory, the information content, self-information, surprisal, or Shannon information is a basic quantity derived from the probability of a particular event occurring from a random variable. It can be thought of as an alternative way of expressing probability, much like odds or log-odds, but which has particular mathematical advantages in the setting of information theory.

The Shannon information can be interpreted as quantifying the level of "surprise" of a particular outcome. As it is such a basic quantity, it also appears in several other settings, such as the length of a message needed to transmit the event given an optimal source coding of the random variable.

The Shannon information is closely related to entropy, which is the expected value of the self-information of a random variable, quantifying how surprising the random variable is "on average". This is the average amount of self-information an observer would expect to gain about a random variable when measuring it.

The information content can be expressed in various units of information, of which the most common is the "bit" (more formally called the shannon), as explained below.

The term 'perplexity' has been used in language modelling to quantify the uncertainty inherent in a set of prospective events.^{[citation needed]}

Claude Shannon's definition of self-information was chosen to meet several axioms:

The detailed derivation is below, but it can be shown that there is a unique function of probability that meets these three axioms, up to a multiplicative scaling factor. Broadly, given a real number ${\displaystyle b>1}$ and an event ${\displaystyle x}$ with probability ${\displaystyle P}$, the information content is defined as the negative log probability:$${\displaystyle \mathrm {I} (x):=-\log _{b}{\left[\Pr {\left(x\right)}\right]}=-\log _{b}{\left(P\right)}.}$$The base ${\displaystyle b}$ corresponds to the scaling factor above. Different choices of b correspond to different units of information: when ${\displaystyle b=2}$, the unit is the shannon (symbol Sh), often called a 'bit'; when ${\displaystyle b=e}$, the unit is the natural unit of information (symbol nat); and when ${\displaystyle b=10}$, the unit is the hartley (symbol Hart).

Formally, given a discrete random variable ${\displaystyle X}$ with probability mass function ${\displaystyle p_{X}{\left(x\right)}}$, the self-information of measuring ${\displaystyle X}$ as outcome ${\displaystyle x}$ is defined as:$${\displaystyle \operatorname {I} _{X}(x):=-\log {\left[p_{X}{\left(x\right)}\right]}=\log {\left({\frac {1}{p_{X}{\left(x\right)}}}\right)}.}$$The use of the notation ${\displaystyle I_{X}(x)}$ for self-information above is not universal. Since the notation ${\displaystyle I(X;Y)}$ is also often used for the related quantity of mutual information, many authors use a lowercase ${\displaystyle h_{X}(x)}$ for self-entropy instead, mirroring the use of the capital ${\displaystyle H(X)}$ for the entropy.