In mathematics, mirror descent is an iterative optimization algorithm for finding a local minimum of a differentiable function.

It generalizes algorithms such as gradient descent and multiplicative weights.

Mirror descent was originally proposed by Nemirovski and Yudin in 1983.

In gradient descent with the sequence of learning rates ${\displaystyle (\eta _{n})_{n\geq 0}}$ applied to a differentiable function ${\displaystyle F}$, one starts with a guess ${\displaystyle \mathbf {x} _{0}}$ for a local minimum of ${\displaystyle F,}$ and considers the sequence ${\displaystyle \mathbf {x} _{0},\mathbf {x} _{1},\mathbf {x} _{2},\ldots }$ such that

This can be reformulated by noting that

In other words, ${\displaystyle \mathbf {x} _{n+1}}$ minimizes the first-order approximation to ${\displaystyle F}$ at ${\displaystyle \mathbf {x} _{n}}$ with added proximity term ${\displaystyle \|\mathbf {x} -\mathbf {x} _{n}\|^{2}}$.

This squared Euclidean distance term is a particular example of a Bregman distance. Using other Bregman distances will yield other algorithms such as Hedge which may be more suited to optimization over particular geometries.

We are given convex function ${\displaystyle f}$ to optimize over a convex set ${\displaystyle K\subset \mathbb {R} ^{n}}$, and given some norm ${\displaystyle \|\cdot \|}$ on ${\displaystyle \mathbb {R} ^{n}}$.