In mathematics, the term undefined refers to a value, function, or other expression that cannot be assigned a meaning within a specific formal system.

Attempting to assign or use an undefined value within a particular formal system, may produce contradictory or meaningless results within that system. In practice, mathematicians may use the term undefined to warn that a particular calculation or property can produce mathematically inconsistent results, and therefore, it should be avoided. Caution must be taken to avoid the use of such undefined values in a deduction or proof.

Whether a particular function or value is undefined, depends on the rules of the formal system in which it is used. For example, the imaginary number ${\displaystyle {\sqrt {-1}}}$ is undefined within the set of real numbers. So it is meaningless to reason about the value, solely within the discourse of real numbers. However, defining the imaginary number ${\displaystyle i}$ to be equal to ${\displaystyle {\sqrt {-1}}}$, allows there to be a consistent set of mathematics referred to as the complex number plane. Therefore, within the discourse of complex numbers, ${\displaystyle {\sqrt {-1}}}$ is in fact defined.

Many new fields of mathematics have been created, by taking previously undefined functions and values, and assigning them new meanings. Most mathematicians generally consider these innovations significant, to the extent that they are both internally consistent and practically useful. For example, Ramanujan summation may seem unintuitive, as it works upon divergent series that assign finite values to apparently infinite sums such as 1 + 2 + 3 + 4 + ⋯. However, Ramanujan summation is useful for modelling a number of real-world phenomena, including the Casimir effect and bosonic string theory.

A function may be said to be undefined, outside of its domain. As one example, ${\textstyle f(x)={\frac {1}{x}}}$ is undefined when ${\displaystyle x=0}$. As division by zero is undefined in algebra, ${\displaystyle x=0}$ is not part of the domain of ${\displaystyle f(x)}$.

In some mathematical contexts, undefined can refer to a primitive notion which is not defined in terms of simpler concepts. For example, in Elements, Euclid defines a point merely as "that of which there is no part", and a line merely as "length without breadth". Although these terms are not further defined, Euclid uses them to construct more complex geometric concepts.

Contrast also the term undefined behavior in computer science, in which the term indicates that a function may produce or return any result, which may or may not be correct.

Many fields of mathematics refer to various kinds of expressions as undefined. Therefore, the following examples of undefined expressions are not exhaustive.