Hubbry Logo
search
logo
Outlier avatar
Outlier
Outlier
current hub
2259712

Outlier

logo
Community Hub0 Subscribers
Read side by side
Outlier
View on Wikipedia
from Wikipedia

Figure 1. Box plot of data from the Michelson–Morley experiment displaying four outliers in the middle column, as well as one outlier in the first column.

In statistics, an outlier is a data point that differs significantly from other observations[1][2]. An outlier may be due to a variability in the measurement, an indication of novel data, or it may be the result of experimental error; the latter are sometimes excluded from the data set[3][4]. An outlier can be an indication of exciting possibility, but can also cause serious problems in statistical analyses.

Outliers can occur by chance in any distribution, but they can indicate novel behaviour or structures in the data-set, measurement error, or that the population has a heavy-tailed distribution. In the case of measurement error, one wishes to discard them or use statistics that are robust to outliers, while in the case of heavy-tailed distributions, they indicate that the distribution has high skewness and that one should be very cautious in using tools or intuitions that assume a normal distribution. A frequent cause of outliers is a mixture of two distributions, which may be two distinct sub-populations, or may indicate 'correct trial' versus 'measurement error'; this is modeled by a mixture model.

In most larger samplings of data, some data points will be further away from the sample mean than what is deemed reasonable. This can be due to incidental systematic error or flaws in the theory that generated an assumed family of probability distributions, or it may be that some observations are far from the center of the data. Outlier points can therefore indicate faulty data, erroneous procedures, or areas where a certain theory might not be valid. However, in large samples, a small number of outliers is to be expected (and not due to any anomalous condition).

Outliers, being the most extreme observations, may include the sample maximum or sample minimum, or both, depending on whether they are extremely high or low. However, the sample maximum and minimum are not always outliers because they may not be unusually far from other observations.

Naive interpretation of statistics derived from data sets that include outliers may be misleading. For example, if one is calculating the average temperature of 10 objects in a room, and nine of them are between 20 and 25 degrees Celsius, but an oven is at 175 °C, the median of the data will be between 20 and 25 °C but the mean temperature will be between 35.5 and 40 °C. In this case, the median better reflects the temperature of a randomly sampled object (but not the temperature in the room) than the mean; naively interpreting the mean as "a typical sample", equivalent to the median, is incorrect. As illustrated in this case, outliers may indicate data points that belong to a different population than the rest of the sample set.

Estimators capable of coping with outliers are said to be robust: the median is a robust statistic of central tendency, while the mean is not.[5]

Occurrence and causes

[edit]
Relative probabilities in a normal distribution

In the case of normally distributed data, the three sigma rule means that roughly 1 in 22 observations will differ by twice the standard deviation or more from the mean, and 1 in 370 will deviate by three times the standard deviation.[6] In a sample of 1000 observations, the presence of up to five observations deviating from the mean by more than three times the standard deviation is within the range of what can be expected, being less than twice the expected number and hence within 1 standard deviation of the expected number – see Poisson distribution – and not indicate an anomaly. If the sample size is only 100, however, just three such outliers are already reason for concern, being more than 11 times the expected number.

In general, if the nature of the population distribution is known a priori, it is possible to test if the number of outliers deviate significantly from what can be expected: for a given cutoff (so samples fall beyond the cutoff with probability p) of a given distribution, the number of outliers will follow a binomial distribution with parameter p, which can generally be well-approximated by the Poisson distribution with λ = pn. Thus if one takes a normal distribution with cutoff 3 standard deviations from the mean, p is approximately 0.3%, and thus for 1000 trials one can approximate the number of samples whose deviation exceeds 3 sigmas by a Poisson distribution with λ = 3.

Causes

[edit]

Outliers can have many anomalous causes. A physical apparatus for taking measurements may have suffered a transient malfunction. There may have been an error in data transmission or transcription. Outliers arise due to changes in system behaviour, fraudulent behaviour, human error, instrument error or simply through natural deviations in populations. A sample may have been contaminated with elements from outside the population being examined. Alternatively, an outlier could be the result of a flaw in the assumed theory, calling for further investigation by the researcher. Additionally, the pathological appearance of outliers of a certain form appears in a variety of datasets, indicating that the causative mechanism for the data might differ at the extreme end (King effect).

Definitions and detection

[edit]

There is no rigid mathematical definition of what constitutes an outlier; determining whether or not an observation is an outlier is ultimately a subjective exercise.[7] There are various methods of outlier detection, some of which are treated as synonymous with novelty detection.[8][9][10][11][12] Some are graphical such as normal probability plots. Others are model-based. Box plots are a hybrid.

Model-based methods which are commonly used for identification assume that the data are from a normal distribution, and identify observations which are deemed "unlikely" based on mean and standard deviation:

Peirce's criterion

[edit]
Main article: Peirce's criterion

It is proposed to determine in a series of observations the limit of error, beyond which all observations involving so great an error may be rejected, provided there are as many as such observations. The principle upon which it is proposed to solve this problem is, that the proposed observations should be rejected when the probability of the system of errors obtained by retaining them is less than that of the system of errors obtained by their rejection multiplied by the probability of making so many, and no more, abnormal observations. (Quoted in the editorial note on page 516 to Peirce (1982 edition) from A Manual of Astronomy 2:558 by Chauvenet.) [14][15][16][17]

Tukey's fences

[edit]

Other methods flag observations based on measures such as the interquartile range. For example, if and are the lower and upper quartiles respectively, then one could define an outlier to be any observation outside the range:

for some nonnegative constant . John Tukey proposed this test, where indicates an "outlier", and indicates data that is "far out".[18]

In anomaly detection

[edit]
Main article: Anomaly detection

In various domains such as, but not limited to, statistics, signal processing, finance, econometrics, manufacturing, networking and data mining, the task of anomaly detection may take other approaches. Some of these may be distance-based[19][20] and density-based such as Local Outlier Factor (LOF).[21] Some approaches may use the distance to the k-nearest neighbors to label observations as outliers or non-outliers.[22]

Modified Thompson Tau test

[edit]
See also: Studentized residual § Distribution

The modified Thompson Tau test is a method used to determine if an outlier exists in a data set.[23] The strength of this method lies in the fact that it takes into account a data set's standard deviation, average and provides a statistically determined rejection zone; thus providing an objective method to determine if a data point is an outlier.[citation needed][24] How it works: First, a data set's average is determined. Next the absolute deviation between each data point and the average are determined. Thirdly, a rejection region is determined using the formula:

;

where is the critical value from the Student t distribution with n-2 degrees of freedom, n is the sample size, and s is the sample standard deviation. To determine if a value is an outlier: Calculate . If δ > Rejection Region, the data point is an outlier. If δ ≤ Rejection Region, the data point is not an outlier.

The modified Thompson Tau test is used to find one outlier at a time (largest value of δ is removed if it is an outlier). Meaning, if a data point is found to be an outlier, it is removed from the data set and the test is applied again with a new average and rejection region. This process is continued until no outliers remain in a data set.

Some work has also examined outliers for nominal (or categorical) data. In the context of a set of examples (or instances) in a data set, instance hardness measures the probability that an instance will be misclassified ( where y is the assigned class label and x represent the input attribute value for an instance in the training set t).[25] Ideally, instance hardness would be calculated by summing over the set of all possible hypotheses H:

Practically, this formulation is unfeasible as H is potentially infinite and calculating is unknown for many algorithms. Thus, instance hardness can be approximated using a diverse subset :

where is the hypothesis induced by learning algorithm trained on training set t with hyperparameters . Instance hardness provides a continuous value for determining if an instance is an outlier instance.

Working with outliers

[edit]

The choice of how to deal with an outlier should depend on the cause. Some estimators are highly sensitive to outliers, notably estimation of covariance matrices.

Retention

[edit]

Even when a normal distribution model is appropriate to the data being analyzed, outliers are expected for large sample sizes and should not automatically be discarded if that is the case.[26] Instead, one should use a method that is robust to outliers to model or analyze data with naturally occurring outliers.[26]

Exclusion

[edit]

When deciding whether to remove an outlier, the cause has to be considered. As mentioned earlier, if the outlier's origin can be attributed to an experimental error, or if it can be otherwise determined that the outlying data point is erroneous, it is generally recommended to remove it.[26][27] However, it is more desirable to correct the erroneous value, if possible.

Removing a data point solely because it is an outlier, on the other hand, is a controversial practice, often frowned upon by many scientists and science instructors, as it typically invalidates statistical results.[26][27] While mathematical criteria provide an objective and quantitative method for data rejection, they do not make the practice more scientifically or methodologically sound, especially in small sets or where a normal distribution cannot be assumed. Rejection of outliers is more acceptable in areas of practice where the underlying model of the process being measured and the usual distribution of measurement error are confidently known.

The two common approaches to exclude outliers are truncation (or trimming) and Winsorising. Trimming discards the outliers whereas Winsorising replaces the outliers with the nearest "nonsuspect" data.[28] Exclusion can also be a consequence of the measurement process, such as when an experiment is not entirely capable of measuring such extreme values, resulting in censored data.[29]

In regression problems, an alternative approach may be to only exclude points which exhibit a large degree of influence on the estimated coefficients, using a measure such as Cook's distance.[30]

If a data point (or points) is excluded from the data analysis, this should be clearly stated on any subsequent report.

Non-normal distributions

[edit]

The possibility should be considered that the underlying distribution of the data is not approximately normal, having "fat tails". For instance, when sampling from a Cauchy distribution,[31] the sample variance increases with the sample size, the sample mean fails to converge as the sample size increases, and outliers are expected at far larger rates than for a normal distribution. Even a slight difference in the fatness of the tails can make a large difference in the expected number of extreme values.

Set-membership uncertainties

[edit]

A set membership approach considers that the uncertainty corresponding to the ith measurement of an unknown random vector x is represented by a set Xi (instead of a probability density function). If no outliers occur, x should belong to the intersection of all Xi's. When outliers occur, this intersection could be empty, and we should relax a small number of the sets Xi (as small as possible) in order to avoid any inconsistency.[32] This can be done using the notion of q-relaxed intersection. As illustrated by the figure, the q-relaxed intersection corresponds to the set of all x which belong to all sets except q of them. Sets Xi that do not intersect the q-relaxed intersection could be suspected to be outliers.

Figure 5. q-relaxed intersection of 6 sets for q=2 (red), q=3 (green), q= 4 (blue), q= 5 (yellow).

Alternative models

[edit]

In cases where the cause of the outliers is known, it may be possible to incorporate this effect into the model structure, for example by using a hierarchical Bayes model, or a mixture model.[33][34]

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Outlier

View on Grokipedia
from Grokipedia
In statistics, an outlier is an observation that lies an abnormal distance from other values in a dataset, deviating markedly from the expected pattern or distribution of the data.[1][2] These anomalous points can emerge from various sources, including measurement or recording errors, variability in data collection processes, or true rare phenomena that reflect genuine deviations in the underlying population.[3] While outliers may sometimes represent valuable insights—such as indicators of fraud, system failures, or novel discoveries—they often distort key statistical measures like the mean and standard deviation, leading to skewed analyses and unreliable inferences.[4][5] Detecting outliers is a fundamental step in data analysis to ensure robust results, particularly in fields like machine learning, quality control, and scientific research where assumptions of normality or linearity are common.[6] Common univariate methods include the interquartile range (IQR) approach, which identifies outliers as values falling below Q1 - 1.5 × IQR or above Q3 + 1.5 × IQR (where Q1 and Q3 are the first and third quartiles), and the z-score method, flagging points with absolute z-scores exceeding 3 as potential outliers under a normal distribution assumption.[1][7] For multivariate data, techniques such as Mahalanobis distance or density-based methods like local outlier factor (LOF) account for correlations among variables to pinpoint anomalies.[5][6] Once identified, outliers require careful handling—options include removal if erroneous, robust statistical methods that downweight their influence (e.g., median-based estimators), or separate investigation to uncover underlying causes—balancing the risk of discarding informative data against the need for accurate modeling.[8] The choice of detection and treatment strategy depends on the dataset's context, size, and analytical goals, underscoring outliers' dual role as both challenges and opportunities in statistical practice.[4]

Fundamentals

Definition

In statistics, an outlier is defined as an observation that appears to be inconsistent with the remainder of a dataset, differing significantly from other data points. This concept arises in contexts where a value deviates markedly from the expected pattern, potentially indicating variability in measurement, experimental error, or a genuine rare event.[9] Such deviations are often quantified using measures of central tendency and dispersion, such as the mean or median. For instance, in a univariate dataset {x1,…,xn}\{x_1, \dots, x_n\}, a data point xix_i is considered an outlier if it satisfies ∣xi−μ∣>kσ|x_i - \mu| > k \sigma, where μ\mu denotes the sample mean, σ\sigma the sample standard deviation, and kk a predefined threshold, commonly set to 3 in the "3-sigma rule" under assumptions of approximate normality. This formulation provides a formal criterion for identifying extremes relative to the dataset's overall spread.[9][10] The identification of outliers is inherently contextual, depending on the underlying distribution of the data, its scale, and domain-specific knowledge; what constitutes an outlier in one dataset may be typical in another with a different structure or generating process.[11] The recognition of outliers dates back to the 18th century in astronomy and early error theory, where astronomers like Roger Joseph Boscovich (1755) and Daniel Bernoulli (1777) discussed rejecting aberrant observations in measurements of celestial positions and physical constants to improve estimates. Formalization advanced with Karl Pearson's introduction of the standard deviation as a measure of dispersion in 1894, enabling systematic quantification of deviations in statistical analysis.[9][10]

Types

Outliers are categorized into distinct types based on their dimensionality, contextual relevance, and patterns of occurrence, extending the foundational concept of a data point that significantly deviates from expected patterns in a dataset. Univariate outliers occur as deviations in data involving a single variable, where an observation stands out markedly from the distribution of values in that dimension alone; for instance, an extreme height value in a dataset of human measurements. These outliers are typically assessed relative to summary statistics like the mean and standard deviation within the univariate context. Multivariate outliers, in contrast, manifest in multi-dimensional data where a point appears normal when examined variable-by-variable but is anomalous overall due to inter-variable relationships; an example is a combination of features that, while individually typical, collectively stray far from the data cloud.[12] A standard metric for identifying such outliers is the Mahalanobis distance, calculated as
D2=(x−μ)TΣ−1(x−μ), D^2 = (\mathbf{x} - \boldsymbol{\mu})^T \Sigma^{-1} (\mathbf{x} - \boldsymbol{\mu}),
where x\mathbf{x} is the data point, μ\boldsymbol{\mu} is the multivariate mean, and Σ\Sigma is the covariance matrix, with the point deemed an outlier if D2D^2 exceeds a χ2\chi^2 distribution threshold.[12] This approach accounts for correlations, making it suitable for high-dimensional spaces.[12] Contextual outliers are observations that conform to the global data distribution but deviate unusually within a specific subset or condition, such as a temperature reading that is unremarkable across all seasons yet anomalous for summer data alone.[13] These require defining behavioral rules or contexts to delineate normalcy.[13] Collective outliers involve groups of related data points that, when considered together, deviate from the broader dataset, even if individual points within the group do not; a representative case is a cluster of transactions that signal fraud as a set, despite each appearing benign in isolation.[13] This type emphasizes patterns in subsets rather than solitary deviations.[13] Outliers may further be viewed as point outliers, which are isolated instances anomalous on their own, versus global outliers, evaluated against the entire dataset's distribution for broader inconsistency.[13] While the terms overlap, point outliers highlight local isolation, whereas global ones stress dataset-wide relativity.[13]

Origins and Causes

Sources

Outliers in datasets can originate from multiple mechanisms that introduce deviations from the expected statistical distribution. These sources range from procedural mistakes during data acquisition to inherent variabilities in the underlying phenomena being studied. Measurement errors constitute a primary source of outliers, often stemming from instrumental faults or inaccuracies in the recording process. For instance, faulty sensors in scientific instruments may produce extreme readings that do not reflect true values, while human transcription mistakes, such as swapping digits in numerical entries, can lead to implausibly large or small data points.[14][15] Process anomalies represent another key origin, involving rare or irregular events within the data-generating system. In manufacturing contexts, equipment malfunctions like unexpected power failures can yield atypical product measurements that stand out from standard outputs. Similarly, transient environmental disruptions in natural observations may generate isolated extreme values.[14][16] Data entry issues frequently introduce outliers through inadvertent human or systemic lapses, such as inputting values outside permissible ranges or biases arising from non-representative sampling procedures. Examples include clerical errors where a valid score like 9 is mistakenly entered as 99, or skewed samples drawn from incorrect subpopulations that contaminate the dataset.[17][15] True extremes occur as legitimate but infrequent phenomena that align with the data's broader variability, rather than errors. In financial datasets, black swan events—unpredictable occurrences with outsized impacts, such as sudden market crashes—manifest as extreme outliers that, while rare, are genuine reflections of systemic risks. These differ from errors by being valid data points within the population's potential range.[18][14] Data contamination arises when observations from disparate populations are inadvertently mixed, leading to points that deviate due to differing underlying distributions. In robust statistical frameworks, this is modeled as a small proportion of "bad" data infiltrating a primarily clean dataset, such as combining healthy and diseased samples in medical studies, thereby creating apparent outliers relative to the majority.[19][20]

Distinctions from Anomalies

In statistics, an outlier is classically defined as an observation that deviates markedly from other observations in a dataset, raising suspicion that it was produced by a different underlying mechanism. This deviation is typically assessed relative to the expected distribution of the data, often implying a potential error in measurement or recording. In contrast, an anomaly refers to a pattern or instance in the data that does not conform to anticipated normal behavior, but it frequently carries connotations of intrinsic interest or significance rather than mere error, such as in contexts like fraud detection where the deviation signals a valuable or actionable event.[21][22] While the terms are sometimes used interchangeably, outliers emphasize statistical extremity within a known framework, whereas anomalies highlight unexpectedness that may warrant investigation beyond dismissal.[21] Outliers also differ from noise, which represents random fluctuations or measurement errors scattered throughout the dataset without systematic deviation. Noise arises from inherent variability in the data-generating process, such as sensor inaccuracies or environmental factors, and is generally regarded as uninformative variation to be smoothed or filtered out during preprocessing.[6] Unlike noise, which permeates the data uniformly and dilutes signal quality, outliers manifest as isolated, extreme points that can disproportionately influence statistical estimates like means or regressions if not addressed. This distinction underscores the need to remove noise prior to outlier analysis, as random errors can mask or mimic true extremes.[6][23] A further boundary exists between outliers and novelties, where novelties pertain to entirely new or unseen patterns emerging in the data, often from an unknown distribution or class not represented in the training set. Outlier detection focuses on identifying extremes within an established normal distribution, assuming a single dominant mechanism, whereas novelty detection aims to flag instances that do not belong to any known category, enabling the recognition of emerging phenomena.[24] For instance, an outlier might be an unusually high value in a familiar sales dataset, while a novelty could introduce a previously unobserved transaction type indicative of market shifts.[24] In machine learning, these conceptual lines blur, with outliers frequently relabeled as anomalies to emphasize their role in predictive modeling, such as training robust classifiers that treat deviations as potential threats or opportunities. This semantic shift prioritizes the practical utility of extremes in enhancing model generalization over strict statistical purity.[21] Philosophically, a debate persists on whether outliers should be viewed uniformly as errors to be excised or as rare events harboring deeper insights into underlying processes. Traditional approaches often discard them to preserve data integrity, yet reframing outliers as meaningful variations—such as rare biological adaptations—can reveal evolutionary or systemic truths otherwise overlooked. This perspective advocates reintegrating outliers into analysis to foster a more holistic understanding of variability, challenging the error-centric paradigm in scientific inquiry.[25]

Detection Methods

Univariate Techniques

Univariate techniques for outlier detection focus on analyzing a single variable in a dataset, assuming the data approximately follow a normal distribution or using non-parametric approaches to identify deviations from the central tendency and spread. These methods are foundational in statistics, providing simple, computationally efficient ways to flag potential outliers without requiring complex models. They are particularly useful for preliminary data exploration in small to moderate-sized samples, where assumptions about data distribution can be reasonably validated. The Z-score method standardizes data points relative to the sample mean and standard deviation to measure their extremity. For a data point xx, the Z-score is calculated as $ z = \frac{x - \mu}{\sigma} $, where μ\mu is the mean and σ\sigma is the standard deviation. Points with $ |z| > 3 $ are typically flagged as outliers, as this threshold corresponds to deviations exceeding three standard deviations from the mean, which occurs with low probability (less than 0.3%) under a normal distribution.[26] This rule of thumb is widely applied in exploratory data analysis, though it assumes normality and can be sensitive to skewed distributions or small samples where the mean and standard deviation may be influenced by outliers themselves. For robustness, a modified Z-score using the median and median absolute deviation (MAD) is sometimes preferred, flagging points where the absolute modified Z-score exceeds 3.5.[26] The interquartile range (IQR) method, also known as Tukey's fences, is a non-parametric approach that identifies outliers based on the spread of the middle 50% of the data, making it less sensitive to extreme values. The IQR is defined as $ \text{IQR} = Q_3 - Q_1 $, where $ Q_1 $ and $ Q_3 $ are the first and third quartiles, respectively. Data points below $ Q_1 - 1.5 \times \text{IQR} $ or above $ Q_3 + 1.5 \times \text{IQR} $ are considered mild outliers, while those beyond $ Q_1 - 3 \times \text{IQR} $ or $ Q_3 + 3 \times \text{IQR} $ are extreme outliers. This method, introduced in exploratory data analysis, visualizes outliers effectively via boxplots and performs well on non-normal data, as it relies solely on order statistics rather than parametric assumptions.[27] Peirce's criterion provides a probabilistic framework for rejecting outliers in small samples, particularly useful when the number of observations $ n $ is low (typically 3 to 30) and the expected number of outliers is small. It assumes a normal distribution and determines a critical ratio $ R $ from precomputed tables based on $ n $, the probability $ p $ (often 0.05) of falsely rejecting a good observation, and the number of potential outliers. An observation is rejected if its absolute residual exceeds $ R $ times the standard error, with the process applied iteratively to test and remove the most deviant point. Developed in the context of astronomical observations, this method balances Type I and Type II errors in limited data scenarios.[28][29] The modified Thompson Tau test extends the original Thompson test for iterative outlier rejection in normally distributed data, using Student's t-distribution to account for small sample uncertainty. It begins by identifying the most extreme observation (maximum $ |\delta_i| $, where $ \delta_i = x_i - \bar{x} $) and computes the test statistic $ \tau = \frac{ \sqrt{n} |\delta_{\max}| }{ s \sqrt{n-1 + \frac{ |\delta_{\max}|^2 }{ s^2 } } } $, where $ s $ is the sample standard deviation and $ n $ is the sample size. The observation is rejected if $ \tau $ exceeds the critical value from the t-distribution with $ n-2 $ degrees of freedom at significance level $ \alpha / n $ (to control family-wise error). This process repeats until no further rejections occur, making it suitable for datasets with up to a few potential outliers. The modification improves upon the original z-based approach by incorporating t-critical values, enhancing reliability for small $ n $.[30] Grubbs' test is a hypothesis-testing procedure designed to detect a single outlier in a univariate normal sample, focusing on the maximum deviation from the mean. The test statistic is $ G = \frac{ \max_i |x_i - \bar{x}| }{ s } $, where $ \bar{x} $ is the sample mean and $ s $ is the standard deviation. Under the null hypothesis of no outliers, $ G $ follows a known distribution, and rejection occurs if $ G > G_{\text{crit}} $ at a chosen significance level (e.g., 0.05), with critical values tabulated or approximated via $ G_{\text{crit}} = \frac{ t_{1-\alpha/ n, n-2} }{ \sqrt{n} } \sqrt{ \frac{ n-1 + t_{1-\alpha/ n, n-2}^2 }{ n - t_{1-\alpha/ n, n-2}^2 } } $. For multiple potential outliers, the test can be applied sequentially after removal, though power decreases with more iterations. This method is influential for its explicit control of false positives in quality control and experimental data.

Multivariate and Modern Approaches

In multivariate outlier detection, the Mahalanobis distance provides a measure of deviation from the multivariate mean that accounts for correlations between variables by incorporating the covariance matrix.[31] For data assumed to follow a multivariate normal distribution, the squared Mahalanobis distance follows a chi-squared distribution with degrees of freedom equal to the number of variables, allowing outliers to be identified by exceeding a significance threshold, such as the 99th percentile of the chi-squared distribution.[32] This approach is particularly effective for detecting outliers in correlated datasets, as it scales distances inversely with variable variances and covariances, unlike Euclidean distance which treats variables independently. The Local Outlier Factor (LOF) algorithm offers a density-based method for identifying outliers in multivariate settings by computing the local reachability density of each point relative to its k-nearest neighbors.[33] The outlier score for a point is the ratio of the average local density of its neighbors to its own local density; points with low density compared to their surroundings receive high LOF scores, indicating potential outliers.[33] This method excels in datasets with varying densities, where global approaches might misclassify points in sparse regions as outliers, and it has been widely adopted for its ability to provide a continuous outlier degree rather than binary classification. Isolation Forest is an ensemble-based technique designed for efficient outlier detection in high-dimensional data, operating by constructing multiple isolation trees through random partitioning of the feature space.[34] Anomalies are isolated faster than normal points because they require fewer splits to separate due to their distinctiveness, resulting in shorter path lengths in the trees; the anomaly score is derived from the average path length across the forest, with shorter averages signaling outliers.[34] Its linear time complexity and effectiveness on large-scale, high-dimensional datasets make it suitable for real-world applications like fraud detection, outperforming distance-based methods in scalability. In modern anomaly detection, unsupervised methods such as one-class support vector machines (SVM) and autoencoders are employed, particularly for streaming data where labels are scarce. One-class SVM learns a boundary around normal data in a high-dimensional feature space, classifying points outside this hypersphere as anomalies based on a user-defined fraction of outliers.[35] Autoencoders, neural networks trained to reconstruct input data, detect outliers by measuring reconstruction error; high errors indicate anomalies, and variants like variational autoencoders can handle streaming data through online updates.[36] These approaches are adaptable to non-stationary streams by retraining on recent windows, providing robust detection in dynamic environments. Density-based spatial clustering of applications with noise (DBSCAN) identifies outliers as points that do not belong to any cluster, using parameters for neighborhood radius and minimum points to define dense regions.[37] Core points form clusters if they have sufficient neighbors within the radius, while border and noise points are distinguished accordingly; noise points, lacking dense surroundings, are flagged as outliers.[37] This method is valuable for multivariate data with arbitrary cluster shapes and noise, requiring no prior knowledge of cluster count and handling varying densities effectively.

Handling Strategies

Retention and Inclusion

Retaining outliers in statistical analysis is often justified when they represent genuine rare events or true signals within the data, rather than errors, as these points can provide critical insights into underlying processes or variability that would otherwise be overlooked.[26][7] For instance, in fields like finance or epidemiology, outliers may capture extreme but legitimate occurrences, such as market crashes or disease outbreaks, which inform model robustness and predictive accuracy.[38] Robust statistical methods facilitate the retention of outliers by employing estimators less sensitive to extreme values, thereby preserving data integrity while mitigating undue influence. The median, as a location estimator, is particularly resilient, tolerating up to 50% contamination from outliers before breakdown, in contrast to the mean's 0% breakdown point.[39] M-estimators, introduced by Huber, further enhance this approach by minimizing a weighted sum of residuals, where weights downplay the impact of large deviations through a convex loss function, allowing inclusion of all data points while achieving near-maximum efficiency under nominal distributions.[40] Winsorizing offers a targeted retention strategy by replacing outlier values with the nearest non-extreme observations, effectively capping extremes without full exclusion; for example, values beyond the 95th percentile may be set to that threshold, reducing skewness while retaining the dataset's overall structure.[41] This method, attributed to biostatistician Charles P. Winsor, preserves sample size and informational content, making it suitable for preliminary analyses where complete data retention is prioritized.[42] Transformations provide another means to retain outliers by altering the data scale to lessen their leverage, such as applying a logarithmic transformation to compress high values in positively skewed distributions. The Box-Cox transformation generalizes this by estimating an optimal power parameter λ to stabilize variance and approximate normality, enabling the inclusion of extremes in parametric models without removal. Retaining outliers through these methods influences statistical inference by typically widening confidence intervals and reducing test power due to increased variance, though it avoids biases from arbitrary exclusion and supports more reliable hypothesis testing in heterogeneous populations. For example, in t-tests, inclusion can lead to conservative p-values that better reflect true uncertainty, preventing overconfidence in results from cleaned datasets.[43][44]

Exclusion and Adjustment

Exclusion and adjustment methods aim to mitigate the influence of outliers by either removing them or modifying their values, thereby enhancing the reliability of statistical analyses while preserving as much data integrity as possible. Deletion involves outright removal of identified outliers, typically after verification through multiple detection techniques to ensure they are not legitimate extreme values. For instance, the National Institute of Standards and Technology recommends deleting outlying points only if they are confirmed erroneous, such as through data collection errors, to avoid arbitrary exclusion.[26] However, in small samples, such removals can introduce substantial bias, as even a single deletion may disproportionately alter parameter estimates and inflate Type I error rates.[43] Winsorizing represents a systematic approach to adjustment by capping extreme values rather than eliminating them entirely, which helps maintain sample size while reducing outlier impact. Winsorizing, named after biostatistician Charles P. Winsor, replaces the top and bottom percentages of data—commonly 5% each—with the nearest non-extreme values, thereby bounding the dataset and improving robustness for estimators like the mean.[45] This method moderates variance without fully discarding observations, making it suitable for datasets where complete removal might lead to loss of representativeness.[46] Imputation offers an alternative adjustment strategy by replacing outlier values with estimated substitutes derived from the remaining data, preserving the full dataset structure. Simple techniques substitute outliers with the sample mean or median to centralize extremes, while more advanced methods use regression-based predictions or k-nearest neighbors (k-NN) to impute values based on similarity to non-outlying points.[47] For example, k-NN imputation identifies the k closest observations (often k=5 or 10) and averages their values for the outlier position, effectively leveraging local patterns to restore plausibility.[48] These approaches are particularly useful in multivariate settings where outliers may stem from measurement noise rather than irrelevance. Despite their utility, exclusion and adjustment carry inherent risks that can compromise analysis validity. Deleting or modifying outliers often results in information loss, as genuine extremes may contain critical insights into underlying processes, potentially leading to biased distributions and underestimated variability.[43] In regulatory contexts, such as clinical trials, improper adjustments can invalidate compliance with standards like those from the FDA, where transparency in outlier handling is mandatory to ensure reproducible results.[49] Moreover, these methods may distort power and inference, especially when outliers are not uniformly distributed across groups. Ethically, exclusion and adjustment demand caution to prevent the suppression of inconvenient data that challenges hypotheses or reveals systemic issues. Researchers must document all decisions transparently to uphold scientific integrity, as selective removal without justification can mislead interpretations and erode trust in findings.[50] This contrasts with retention strategies, which prioritize inclusion to capture full variability, though purification techniques like adjustment remain essential when data quality demands it.

Applications

In Statistics and Data Analysis

Outliers significantly distort descriptive statistics, particularly measures of central tendency and dispersion. The sample mean is highly sensitive to extreme values, as a single outlier can pull the mean toward itself, leading to a biased representation of the data's center. Similarly, outliers inflate the variance and standard deviation by increasing the sum of squared deviations from the mean, exaggerating the perceived spread of the data. In bivariate analysis, outliers can alter correlation coefficients, either artificially strengthening or weakening the apparent linear relationship between variables depending on their position relative to the data cloud.[51] In regression analysis, outliers manifest as leverage points—observations distant from the center of the predictor space—or influential observations that disproportionately affect model parameters. Leverage points amplify the impact of residuals, potentially leading to biased slope estimates and poor model fit. To quantify influence, Cook's distance is employed, defined as
Di=∑j=1n(y^j−y^(i)j)2p⋅MSE, D_i = \frac{ \sum_{j=1}^n (\hat{y}_j - \hat{y}_{(i)j})^2 }{ p \cdot \mathrm{MSE} },
where y^j\hat{y}_j are the fitted values using all data, y^(i)j\hat{y}_{(i)j} are the fitted values excluding the ii-th observation, pp is the number of parameters, and MSE\mathrm{MSE} is the mean squared error; values of Di>4/nD_i > 4/n (with nn the sample size) indicate substantial influence.[52] Outliers also compromise hypothesis testing by inflating Type I error rates in parametric procedures like the t-test, as they increase variance estimates and distort test statistics, leading to false rejections of the null hypothesis.[53] Robust alternatives, such as the bootstrap method, mitigate this by resampling the data to generate empirical distributions of the test statistic, providing reliable inference even with contaminants.[54] In exploratory data analysis (EDA), visual tools facilitate initial outlier detection without assuming underlying distributions. Box plots, based on the interquartile range (IQR), flag potential outliers as points beyond 1.5 times the IQR from the quartiles, offering a quick assessment of univariate deviations.[55] Scatter plots complement this by revealing multivariate outliers through deviations from linear patterns or clusters in two-dimensional projections.[55] Big data environments exacerbate outlier challenges due to scalability constraints, where even rare extreme values in massive datasets can disproportionately skew aggregate statistics or overwhelm computational resources during analysis.[56] This demands distributed algorithms to assess outlier impact efficiently across high-volume, high-velocity data streams.[57]

In Specific Domains

In finance, outlier detection plays a crucial role in identifying extreme market events such as crashes and potential insider trading through anomalous returns. For instance, analysis of drawdowns in major stock indices and currency markets has revealed 49 outliers across global datasets, with 25 classified as endogenous crashes driven by speculative bubbles and 22 as exogenous ones triggered by external shocks, enabling better risk assessment and regulatory oversight.[58] Similarly, machine learning methods, including unsupervised techniques like random forests and isolation forests, support surveillance by flagging unusual trading patterns indicative of insider activity, such as abnormal volume spikes or timing deviations, improving detection rates in high-frequency data environments.[59] In medicine, outliers facilitate the identification of rare diseases and errors in clinical measurements, enhancing diagnostic accuracy and trial integrity. Transcriptome-wide analysis of splicing outliers has diagnosed individuals with rare genetic disorders by detecting aberrant RNA patterns missed by standard genomic sequencing.[60] In clinical trials, anomaly detection algorithms applied to real-world data identify measurement errors from careless data entry or protocol deviations, such as implausible vital signs like extreme blood pressure readings, thereby reducing bias and ensuring reliable efficacy assessments.[61] Multi-omics outlier workflows, integrating proteomics and RNA sequencing, have resolved 15% of previously unsolved rare disease cases by pinpointing protein expression anomalies linked to variants in genes like MSTO1 and SHMT2.[62] In engineering, outlier detection is essential for fault identification in sensor networks, where anomalous readings signal equipment failures. Ensemble learning approaches, combining isolation forests and local outlier factors in sliding windows, effectively process streaming sensor data to isolate faults like irregular vibrations in machinery, achieving high precision in real-time industrial monitoring.[63] Improved support vector data description methods adaptively model normal sensor behaviors, detecting outliers in multivariate time series from IoT devices, such as temperature or pressure deviations in manufacturing systems, to prevent downtime and safety risks.[64] In environmental science, outliers in climate datasets highlight extreme weather events, informing model refinements and policy responses. Machine learning algorithms, including kernel principal component analysis and local outlier factors, applied to 40 years of meteorological records from regions like Burkina Faso, identify approximately 5% of data points as anomalies in variables such as maximum temperature, correlating with intensified droughts and heatwaves.[65] Climate models often exhibit outlier projections for heatwave frequency, where certain simulations overestimate tail-end extremes by over 0.5°C per decade, underscoring the need for outlier-aware attribution to distinguish natural variability from anthropogenic influences.[66] In social sciences, survey response outliers reveal biases, such as careless or patterned answering that skews results. Person fit statistics based on item response theory detect atypical patterns in questionnaire data, identifying up to 4.6% of respondents with misfitting responses (e.g., extreme inconsistencies) that indicate response bias, thereby improving data quality in studies on health or behavior.[67] Response time outliers, flagged via thresholds like interquartile ranges, uncover temporary disengagement or straight-lining in surveys, reducing estimation errors in fields like psychology and sociology.[68] As of 2025, AI-driven outlier detection has advanced cybersecurity by enabling proactive intrusion alerts through real-time anomaly identification. Comprehensive reviews highlight deep learning models that analyze network traffic for deviations like unusual packet flows, achieving high accuracy in detecting advanced persistent threats and zero-day attacks.[69] In critical infrastructure, hybrid AI frameworks fuse cyber and physical data to spot outliers signaling intrusions, such as anomalous DNP3 protocol commands in power grids, enhancing resilience against sophisticated cyber threats.[70]

References

  1. 3.2 - Identifying Outliers: IQR Method | STAT 200
  2. DS 352 Anomaly/Outlier Detection
  3. [PDF] The power of outliers (and why researchers should ALWAYS check ...
  4. [PDF] A Review and Comparison of Methods for Detecting Outliers in ...
  5. [PDF] Outlier Detection: A Novel Depth Approach
  6. [PDF] Lecture 8: Outlier Detection - UNM CS
  7. [PDF] Lab 5: Testing Our Way to Outliers - Statistics & Data Science
  8. Outliers - University of Notre Dame
  9. [PDF] Outliers in - The World Bank
  10. III. Contributions to the mathematical theory of evolution - Journals
  11. Rigorous definition of an outlier? - Cross Validated - Stack Exchange
  12. Univariate and Multivariate Outliers - Statistics Solutions
  13. [PDF] Outlier Analysis Second Edition - Charu Aggarwal
  14. Guidelines for Removing and Handling Outliers in Data
  15. [PDF] Outlier detection and treatment - The World Bank
  16. Statistical Outliers in the Laboratory Setting
  17. Statistical data preparation: management of missing values and ...
  18. Black Swan in the Stock Market: What Is It, With Examples and History
  19. [PDF] Robust statistics for outlier detection - KU Leuven
  20. Full article: Robust variable selection under cellwise contamination
  21. Anomaly detection: A survey: ACM Computing Surveys: Vol 41, No 3
  22. Outliers and anomalies in training and testing datasets for AI ... - NIH
  23. Noise Versus Outliers - Secondary Analysis of Electronic ... - NCBI
  24. [PDF] arXiv:1911.07116v1 [cs.LG] 16 Nov 2019
  25. The Philosophy of Outliers: Reintegrating Rare Events Into ...
  26. 1.3.5.17. Detection of Outliers - Information Technology Laboratory
  27. [PDF] Exploratory-Data-Analysis-1977-John-Tukey.pdf - Consoleflare
  28. Criterion for the rejection of doubtful observations - Harvard University
  29. On a Criterion for the Rejection of Observations and the Distribution ...
  30. On the Generalised Distance in Statistics
  31. 4.4 - Multivariate Normality and Outliers | STAT 505
  32. [PDF] LOF: Identifying Density-Based Local Outliers
  33. [PDF] Isolation Forest - LAMDA
  34. [PDF] Support Vector Method for Novelty Detection
  35. Anomaly Detection Using Autoencoders with Nonlinear ...
  36. [PDF] A Density-Based Algorithm for Discovering Clusters in Large Spatial ...
  37. How to Find Outliers | 4 Ways with Examples & Explanation - Scribbr
  38. To Improve Customer Experience, Embrace the Outliers in Your Data
  39. [PDF] Robust Statistics
  40. Robust Estimation of a Location Parameter - Project Euclid
  41. "Winsorizing" by Bruce E. Blaine - Fisher Digital Publications
  42. [PDF] Winsor Approach in Regression Analysis with Outlier - m-hikari.com
  43. Effect of removing outliers on statistical inference - PubMed Central
  44. (PDF) Effect of Removing Outliers on Statistical Inference
  45. Winsorization - an overview | ScienceDirect Topics
  46. Trimmed and Winsorized Means
  47. 7.4. Imputation of missing values — scikit-learn 1.7.2 documentation
  48. K-nearest neighbor algorithm for imputing missing longitudinal ... - NIH
  49. [PDF] Sample Size, Outliers, and Exclusion Criteria - NIH Grants & Funding
  50. Outlier Removal and the Relation with Reporting Errors and Quality ...
  51. Pearson Product-Moment Correlation (cont...) - Laerd Statistics
  52. Detection of Influential Observation in Linear Regression
  53. [PDF] Outlier Impact and Accommodation Methods: Multiple Comparisons ...
  54. Bootstrap estimation of the proportion of outliers in robust regression
  55. 7.1.6. What are outliers in the data?
  56. Challenges of Big Data Analysis - PMC - PubMed Central
  57. [PDF] Distributed Local Outlier Detection in Big Data
  58. Endogenous versus Exogenous Crashes in Financial Markets - arXiv
  59. A machine learning approach to support decision in insider trading ...
  60. Transcriptome-wide outlier approach identifies individuals with ...
  61. Anomaly Detection Algorithm for Real-World Data and Evidence in ...
  62. An outlier approach: advancing diagnosis of neurological diseases ...
  63. Outlier Detection in Sensor Data using Ensemble Learning
  64. Outlier Detection Using Improved Support Vector Data Description in ...
  65. Machine Learning-Based Outlier Detection in Long-Term Climate Data
  66. Global emergence of regional heatwave hotspots outpaces climate ...
  67. Using Person Fit Statistics to Detect Outliers in Survey Research
  68. Investigating the Adequacy of Response Time Outlier Definitions in ...
  69. Advancing cybersecurity: a comprehensive review of AI-driven ...
  70. AI-driven cybersecurity framework for anomaly detection in power ...
User Avatar
No comments yet.