In computer vision, pattern recognition, and robotics, point-set registration, also known as point-cloud registration or scan matching, is the process of finding a spatial transformation (e.g., scaling, rotation and translation) that aligns two point clouds. The purpose of finding such a transformation includes merging multiple data sets into a globally consistent model (or coordinate frame), and mapping a new measurement to a known data set to identify features or to estimate its pose. Raw 3D point cloud data are typically obtained from Lidars and RGB-D cameras. 3D point clouds can also be generated from computer vision algorithms such as triangulation, bundle adjustment, and more recently, monocular image depth estimation using deep learning. For 2D point set registration used in image processing and feature-based image registration, a point set may be 2D pixel coordinates obtained by feature extraction from an image, for example corner detection. Point cloud registration has extensive applications in autonomous driving, motion estimation and 3D reconstruction, object detection and pose estimation, robotic manipulation, simultaneous localization and mapping (SLAM), panorama stitching, virtual and augmented reality, and medical imaging.

As a special case, registration of two point sets that only differ by a 3D rotation (i.e., there is no scaling and translation), is called the Wahba Problem and also related to the orthogonal procrustes problem.

The problem may be summarized as follows: Let ${\displaystyle \lbrace {\mathcal {M}},{\mathcal {S}}\rbrace }$ be two finite size point sets in a finite-dimensional real vector space ${\displaystyle \mathbb {R} ^{d}}$, which contain ${\displaystyle M}$ and ${\displaystyle N}$ points respectively (e.g., ${\displaystyle d=3}$ recovers the typical case of when ${\displaystyle {\mathcal {M}}}$ and ${\displaystyle {\mathcal {S}}}$ are 3D point sets). The problem is to find a transformation to be applied to the moving "model" point set ${\displaystyle {\mathcal {M}}}$ such that the difference (typically defined in the sense of point-wise Euclidean distance) between ${\displaystyle {\mathcal {M}}}$ and the static "scene" set ${\displaystyle {\mathcal {S}}}$ is minimized. In other words, a mapping from ${\displaystyle \mathbb {R} ^{d}}$ to ${\displaystyle \mathbb {R} ^{d}}$ is desired which yields the best alignment between the transformed "model" set and the "scene" set. The mapping may consist of a rigid or non-rigid transformation. The transformation model may be written as ${\displaystyle T}$, using which the transformed, registered model point set is:

The output of a point set registration algorithm is therefore the optimal transformation ${\displaystyle T^{\star }}$ such that ${\displaystyle {\mathcal {M}}}$ is best aligned to ${\displaystyle {\mathcal {S}}}$, according to some defined notion of distance function ${\displaystyle \operatorname {dist} (\cdot ,\cdot )}$:

where ${\displaystyle {\mathcal {T}}}$ is used to denote the set of all possible transformations that the optimization tries to search for. The most popular choice of the distance function is to take the square of the Euclidean distance for every pair of points:

where ${\displaystyle \|\cdot \|_{2}}$ denotes the vector 2-norm, ${\displaystyle s_{m}}$ is the corresponding point in set ${\displaystyle {\mathcal {S}}}$ that attains the shortest distance to a given point ${\displaystyle m}$ in set ${\displaystyle {\mathcal {M}}}$ after transformation. Minimizing such a function in rigid registration is equivalent to solving a least squares problem.

When the correspondences (i.e., ${\displaystyle s_{m}\leftrightarrow m}$) are given before the optimization, for example, using feature matching techniques, then the optimization only needs to estimate the transformation. This type of registration is called correspondence-based registration. On the other hand, if the correspondences are unknown, then the optimization is required to jointly find out the correspondences and transformation together. This type of registration is called simultaneous pose and correspondence registration.

Given two point sets, rigid registration yields a rigid transformation which maps one point set to the other. A rigid transformation is defined as a transformation that does not change the distance between any two points. Typically such a transformation consists of translation and rotation. In rare cases, the point set may also be mirrored. In robotics and computer vision, rigid registration has the most applications.