A word n-gram language model is a statistical model of language which calculates the probability of the next word in a sequence from a fixed size window of previous words. If one previous word is considered, it is a bigram model; if two words, a trigram model; if n − 1 words, an n-gram model.

Special tokens are introduced to denote the start and end of a sentence ${\displaystyle \langle s\rangle }$ and ${\displaystyle \langle /s\rangle }$. To prevent a zero probability being assigned to unseen words, the probability of each seen word is slightly lowered to make room for the unseen words in a given corpus. To achieve this, various smoothing methods are used, from simple "add-one" smoothing (assigning a count of 1 to unseen n-grams, as an uninformative prior) to more sophisticated techniques, such as Good–Turing discounting or back-off models.

Word n-gram models have largely been superseded by recurrent neural network–based models, which in turn have been superseded by Transformer-based models often referred to as large language models.

A special case, where n = 1, is called a unigram model. Probability of each word in a sequence is independent from probabilities of other word in the sequence. Each word's probability in the sequence is equal to the word's probability in an entire document.

$${\displaystyle P_{\text{uni}}(t_{1}t_{2}t_{3})=P(t_{1})P(t_{2})P(t_{3}).}$$

The model consists of units, each treated as one-state finite automata. Words with their probabilities in a document can be illustrated as follows.

Total mass of word probabilities distributed across the document's vocabulary, is 1.

$${\displaystyle \sum _{\text{word in doc}}P({\text{word}})=1}$$