Hubbry Logo
All models are wrongAll models are wrongMain
Open search
All models are wrong
Community hub
All models are wrong
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
All models are wrong
All models are wrong
All models are wrong
View on Wikipedia
from Wikipedia

"All models are wrong" is a common aphorism in statistics. It is often expanded as "All models are wrong, but some are useful". The aphorism acknowledges that statistical models always fall short of the complexities of reality but can still be useful nonetheless. The aphorism is generally attributed to George E. P. Box, a British statistician, although the underlying concept predates Box's writings.

History

[edit]
The phrase is attributed to George Box

The phrase "all models are wrong" was attributed[1] to George Box who used the phrase in a 1976 paper to refer to the limitations of models, arguing that while no model is ever completely accurate, simpler models can still provide valuable insights if applied judiciously.[2]: 792 

In their 1983 book on generalized linear models, Peter McCullagh and John Nelder stated that while modeling in science is a creative process, some models are better than others, even though none can claim eternal truth.[3][4] In 1996, an Applied Statistician's Creed was proposed by M.R. Nester, which incorporated the aphorism as a central tenet.[1]

The longer form appears on in a 1987 book by Box and Norman Draper in a section "The Use of Approximating Functions,":

"The fact that the polynomial is an approximation does not necessarily detract from its usefulness because all models are approximations. Essentially, all models are wrong, but some are useful."[5]: 424 

Discussions

[edit]

Box used the aphorism again in 1979, where he expanded on the idea by discussing how models serve as useful approximations, despite failing to perfectly describe empirical phenomena.[6] He reiterated this sentiment in his later works, where he discussed how models should be judged based on their utility rather than their absolute correctness.[7][8]

David Cox, in a 1995 commentary, argued that stating all models are wrong is unhelpful, as models by their nature simplify reality. He emphasized that statistical models, like other scientific models, aim to capture important aspects of systems through idealized representations.[9]

In their 2002 book on statistical model selection, Burnham and Anderson reiterated Box's statement, noting that while models are simplifications of reality, they vary in usefulness, from highly useful to essentially useless.[10]

J. Michael Steele used the analogy of city maps to explain that models, like maps, serve practical purposes despite their limitations, emphasizing that certain models, though simplified, are not necessarily wrong.[11] In response, Andrew Gelman acknowledged Steele's point but defended the usefulness of the aphorism, particularly in drawing attention to the inherent imperfections of models.[12]

Philosopher Peter Truran, in a 2013 essay, discussed how seemingly incompatible models can make accurate predictions by representing different aspects of the same phenomenon, illustrating the point with an example of two observers viewing a cylindrical object from different angles.[13]

In 2014, David Hand reiterated that models are meant to aid in understanding or decision-making about the real world, a point emphasized by Box's famous remark.[14]

See also

[edit]

Notes

[edit]

References

[edit]

Further reading

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

All models are wrong

View on Grokipedia
from Grokipedia
"All models are wrong, but some are useful" is a renowned in and scientific modeling, attributed to the British statistician , which encapsulates the idea that mathematical and statistical models are always simplifications of complex reality and thus inevitably contain errors or omissions, yet their worth lies in their ability to provide practical insights and predictions when appropriately applied. This principle highlights the between model accuracy and , urging practitioners to evaluate models based on their effectiveness in addressing specific problems rather than pursuing unattainable perfection. The origins of the phrase trace back to 's 1976 paper "Science and Statistics", published in the Journal of the , where he wrote, "Since all models are wrong the scientist cannot obtain a 'correct' one by excessive elaboration" and further noted that scientists must remain "alert to what is importantly wrong." expanded on this in the 1987 book Empirical Model-Building and Response Surfaces, co-authored with Norman R. Draper, stating, "Essentially, all models are wrong, but some are useful," in the context of and empirical modeling techniques. These statements emerged from 's extensive work in experimental design and , influenced by his experiences during in developing statistical methods for processes. The aphorism's core meaning revolves around the philosophical and practical limitations of modeling: all models rely on assumptions that abstract away real-world complexities, such as nonlinearity, interactions, or unmeasured variables, making them "wrong" in an absolute sense. However, Box emphasized that usefulness arises from a model's parsimony—favoring simpler structures per —and its validation through empirical testing, ensuring it captures essential patterns without . This perspective counters over-reliance on models as infallible truths, promoting iterative refinement and awareness of biases. In broader applications, the principle has influenced diverse fields beyond statistics, including , and science, and medicine. For instance, in and , it guides the construction of models that, despite simplifications, enable hypothesis testing and simulation of biological processes. The aphorism remains a cornerstone of model evaluation, frequently cited to advocate for transparency in assumptions and robust validation to maximize real-world impact.

Introduction

The Aphorism

The aphorism "All models are wrong, but some are useful" is a foundational statement in statistical philosophy, commonly attributed to the British statistician George E. P. Box. A related phrasing, "Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful," appears on page 424 of the 1987 book Empirical Model-Building and Response Surfaces, co-authored by Box and Norman R. Draper. This succinct expression captures the inherent limitations of mathematical and statistical models while highlighting their potential value in practical applications. George E. P. Box (1919–2013) was a pioneering statistician whose work profoundly influenced experimental design, time-series analysis, and industrial quality control. He is particularly renowned for developing response surface methodology in collaboration with K. B. Wilson in 1951, a technique for optimizing processes through sequential experimentation, and for co-authoring the influential Box-Jenkins methodology for ARIMA models in 1970. Box's career spanned academia and industry, including key roles at the University of Wisconsin-Madison and contributions to wartime chemical engineering efforts during World War II. The core idea behind the traces to Box's 1976 paper "Science and Statistics," published in the Journal of the American Statistical Association, where he stated, "Since all models are wrong the scientist cannot obtain a 'correct' one by excessive elaboration." In this work, Box aimed to caution researchers in empirical sciences against over-literal interpretations of models, urging instead a focus on their approximate nature and the importance of iterative refinement to address key discrepancies with reality. This perspective underscores that models serve as tools for insight rather than perfect representations, a notion that resonates with earlier philosophical distinctions like Alfred Korzybski's 1933 assertion that "the map is not the territory."

Core Principle

The core principle underlying the aphorism "all models are wrong, but some are useful" posits that models serve as simplified abstractions of complex reality, capturing key patterns while inevitably excluding numerous details and nuances. In statistical and scientific contexts, a model is defined as a mathematical or conceptual representation that approximates the relationships within observed data, relying on assumptions to make the system tractable for analysis and prediction. These simplifications—such as assuming specific distributions or functional forms—render every model inherently inaccurate, as no representation can fully encapsulate the infinite intricacies of the real world. Despite this fundamental wrongness, models derive value from their pragmatic utility in facilitating understanding, forecasting outcomes, or informing decisions, provided they align sufficiently with in targeted scenarios. The principle emphasizes that usefulness arises not from pursuing unattainable perfect truth, but from iterative refinement where models are tested, critiqued, and adjusted against real-world data to highlight significant discrepancies over minor ones. For instance, exemplifies this by assuming a straight-line relationship between variables, which is often untrue in nonlinear phenomena, yet it effectively approximates many such relationships when the deviations are not extreme, enabling reliable predictions in fields like or . This distinction between absolute truth—which remains elusive in multifaceted systems—and practical utility underscores the aphorism's philosophical essence: models should be evaluated by their capacity to yield actionable insights rather than their completeness. As articulated by statistician , the focus must lie on models that, though flawed, advance scientific inquiry through robust, falsifiable approximations.

Historical Origins

Precedents Before Box

The intellectual precursors to the aphorism "all models are wrong" can be traced to early 20th-century discussions in , semantics, and statistics that underscored the inherent limitations of abstract representations in capturing complex realities. A foundational influence emerged from Alfred Korzybski's development of , where he introduced the dictum "the map is not the territory" to highlight the epistemological and semantic boundaries between linguistic or symbolic abstractions and the actual world they describe. In Science and Sanity (1933), Korzybski argued that all human knowledge operates through such "maps"—simplified structures that resemble but never fully encompass the "territory" of reality, leading to potential errors in scientific reasoning if the distinction is ignored. This principle emphasized that abstractions, while useful for navigation and prediction, impose structural similarities rather than identical replications, thereby establishing a cautionary framework for scientific modeling that prioritizes awareness of representational incompleteness. Building on semantic critiques, statistical thinkers like Walter Shewhart advanced ideas about approximations in , particularly in . In his 1939 monograph Statistical Method from the Viewpoint of , Shewhart described statistical control charts as practical tools for approximating process variability in , acknowledging that no mathematical framework could exhaustively characterize the dynamic, chance-influenced nature of physical systems. He posited that such methods enable rational prediction and control but are inherently limited by the unpredictability of assignable causes versus common variation, framing statistical models as operational approximations rather than definitive truths. This perspective influenced by promoting models that balance utility with an explicit recognition of their partial coverage of real-world phenomena. In the realm of and simulations, contributed a related insight during the mid-1940s, amid early efforts to model complex systems numerically. In 1947, von Neumann remarked that "truth is much too complicated to allow anything but approximations," reflecting on the challenges of exact modeling in computational contexts like simulations and . This observation arose from his work on electronic at the Institute for Advanced Study, where he recognized that the intricate dynamics of physical and biological processes defied precise mathematical replication, necessitating iterative, approximate methods for practical advancement. His view positioned computational models as indispensable yet flawed instruments for exploring otherwise intractable realities. Complementing these developments, Bertrand Russell's early 20th-century philosophy of logic and provided broader conceptual groundwork through his emphasis on logical constructions as approximations to empirical reality. In works such as Our Knowledge of the External World (1914), Russell advocated for , wherein complex phenomena are analyzed into simpler propositional structures that approximate the underlying order of the world, but he cautioned that such analyses are mediated by incomplete sense data and cannot fully mirror unobservable essences. His approach in The Analysis of Matter (1927) further elaborated how scientific theories construct relational models that capture structural analogies to reality rather than its intrinsic nature, influencing positivist traditions by treating logical approximations as the closest achievable proxies in and . These ideas reinforced the notion that all representational systems, logical or otherwise, entail necessary simplifications.

George Box's Formulation

George Edward Pelham Box, a prominent British , first articulated the core idea behind the in his 1976 presidential address to the , published in the Journal of the American Statistical Association. In the "Science and Statistics," Box stated: "Since all models are wrong the cannot obtain a 'correct' one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena." This formulation emphasized parsimony in modeling, arguing that scientists should prioritize simple, adequate approximations over complex but still flawed representations to uncover new insights about reality. Box's background in experimental design profoundly shaped this perspective. During , he contributed to statistics at , where he developed methods for optimizing industrial processes. His seminal 1951 collaboration with K. B. Wilson introduced (RSM), a technique for modeling and analyzing problems in which a response of interest is influenced by several variables, enabling efficient experimental designs to approximate optimal conditions. Later, Box advanced through works like his 1973 book with George C. Tiao, applying probabilistic approaches to model updating and uncertainty quantification in statistical analysis. Box expanded the aphorism in 1979, in his paper "Robustness in the Strategy of Scientific ", where he famously declared: "All models are wrong but some are useful." In this context, Box illustrated the point using the (PV = nRT) as an example of a model that, despite its inaccuracies in capturing behavior, provides valuable predictions and guides further inquiry into . This work popularized the phrase within statistical literature, highlighting how imperfect models can still drive practical advancements when they align with empirical data. Within , Box employed the aphorism to advocate for iterative model refinement, a process where initial quadratic approximations are fitted to experimental data, evaluated for adequacy, and sequentially improved through additional designs to better capture system responses. This sequential approach, rooted in his 1951 RSM framework, underscores the aphorism's role in promoting adaptive, usefulness-focused modeling over static perfectionism. In his 1987 book with Draper, Empirical Model-Building and Response Surfaces, Box further elaborated on models as parsimonious simplifications of complex realities, akin to caricatures that exaggerate key features while omitting minor details to facilitate understanding and prediction. Box's ideas echoed earlier sentiments, such as John von Neumann's remark that "with four parameters I can fit an elephant, and with five I can make him wiggle his trunk," which similarly critiqued overparameterized models.

Conceptual Foundations

Nature of Models

In statistical and scientific modeling, a fundamental representation of the relationship between observed outcomes and predictors is given by the equation Y=f(X,θ)+ϵ,Y = f(X, \theta) + \epsilon, where YY is the response variable, XX represents the input features, θ\theta denotes the model parameters, ff is the functional form capturing the systematic relationship, and ϵ\epsilon is the error term accounting for unmodeled variability or noise, often assumed to have mean zero. This formulation underscores that all models are inherently approximations, as ϵ\epsilon encapsulates aspects of reality that the model cannot fully capture, such as random fluctuations or omitted complexities. Models are broadly categorized into parametric and non-parametric types based on their structural assumptions. Parametric models assume a fixed functional form with a finite number of parameters θ\theta, such as a Gaussian distribution for error terms or linear relationships in regression, which allows for efficient but risks misspecification if the assumed form does not match the data-generating process. In contrast, non-parametric models avoid rigid assumptions about the form of ff, instead estimating it flexibly from the data, such as through kernel smoothing or splines; however, they require larger sample sizes to achieve reliable performance and can suffer from in high dimensions. These categories highlight the between simplicity and adaptability inherent in modeling. The "wrongness" of models arises from several key sources of misspecification. Omitted variables occur when relevant predictors are excluded, leading to biased estimates of the included parameters, as the model attributes effects of the missing factors to the observed ones—a phenomenon known as . Assumption violations, such as non-linearity in a or non-normal errors where normality is presumed, distort and prediction accuracy by invalidating the theoretical basis for estimation. In complex systems, scaling issues further exacerbate flaws, where models calibrated at one scale (e.g., local interactions) fail to capture emergent behaviors or interactions at larger scales, due to nonlinear dynamics or hierarchical structures that amplify small discrepancies. A classic illustration of these limitations is Newtonian physics, which serves as a local approximation to under conditions of low velocities and weak gravitational fields but breaks down at relativistic scales, such as near the or in strong gravity, where spacetime curvature introduces unmodeled effects like . This example reminds us, as articulated in George Box's aphorism, that models are simplifications destined to be wrong in some respects yet potentially useful within their intended domains.

Criteria for Usefulness

The usefulness of a model lies in its ability to deliver practical value despite inevitable simplifications and errors, primarily through strong , interpretability, generalizability, and cost-effectiveness. refers to the model's capacity to accurately forecast outcomes on unseen data, enabling reliable in real-world scenarios. Interpretability allows users to comprehend the relationships and factors driving predictions, fostering trust and actionable insights. Generalizability measures how well the model performs across diverse datasets or conditions, avoiding degradation in novel contexts. Cost-effectiveness evaluates the between the model's benefits and the resources required for development, training, and deployment, ensuring feasibility in resource-constrained environments. A fundamental principle underlying model usefulness is the bias-variance tradeoff, which highlights the tension between model simplification—introducing through assumptions that deviate from true —and , which increases variance by capturing noise in training data at the expense of broader applicability. This tradeoff is formally captured in the decomposition of (MSE), a common measure of predictive accuracy: MSE=Bias2+Variance+Irreducible Error\text{MSE} = \text{Bias}^2 + \text{Variance} + \text{Irreducible Error} Here, bias² quantifies systematic errors from model assumptions, variance reflects sensitivity to data fluctuations, and irreducible error represents inherent in the that no model can eliminate. Effective models minimize total MSE by balancing these components, prioritizing configurations that enhance overall utility without excessive complexity. For instance, in decisions, simple linear models are frequently favored over intricate alternatives precisely because their straightforward coefficients and assumptions enhance interpretability, allowing policymakers to readily discern causal influences and build stakeholder confidence.

Key Interpretations

In Statistics and Modeling

In statistical practice, the aphorism "all models are wrong" underscores the inherent approximations in modeling real-world phenomena, influencing approaches to and validation by encouraging practitioners to prioritize usefulness over perfection. This perspective, originally formulated by George Box, reminds statisticians that no model fully captures the complexity of data-generating processes, prompting techniques that account for and model inadequacy. A key interpretation appears in the development of generalized linear models (GLMs), where Peter McCullagh and John Nelder frame models explicitly as approximations rather than exact representations of truth. In their foundational text, they emphasize the goal of identifying models that provide adequate fits for practical inference, particularly in handling non-normal response variables through link functions and variance structures. This view promotes rigorous diagnostic checks and iterative refinement to mitigate the inevitable discrepancies between model assumptions and data. McCullagh and Nelder stress that the approximate nature of models must always be acknowledged to avoid overconfidence in predictions or parameter estimates. Building on this, Kenneth P. Burnham and David R. Anderson advocate for model averaging as a strategy to address the flaws in any single model, recognizing that often favors one at the expense of others, leading to biased inferences. In their information-theoretic framework, they recommend using Akaike's Information Criterion (AIC) to weigh multiple candidate models and compute weighted averages of predictions, thereby incorporating model uncertainty and reducing the risks associated with relying on a "best" but inevitably wrong model. This multimodel inference approach has become widely adopted in and beyond, as it aligns with the by treating all models as provisional tools for approximating reality rather than definitive truths. Mark R. Nester integrates the into his " for modelers," a set of guiding principles for applied statisticians that emphasizes in the face of model limitations. Presented as a against overreliance on rigid assumptions, Nester's lists "all models are wrong" as a core tenet, urging modelers to focus on robust , sensitivity analyses, and communication of uncertainties rather than pursuing unattainable exactitude. This promotes validation techniques like cross-validation and residual to reveal model inadequacies, fostering a practice-oriented mindset that values practical utility in decision-making. John Michael Steele extends the discussion with an to city , portraying statistical models as selective representations designed for specific purposes, such as a that depicts a building as a simple square rather than its full architectural detail to serve navigation needs. In this view, models are not wrong in an absolute sense but are intentionally simplified to serve targeted inquiries, such as or testing; their "wrongness" arises only if mismatched to the intended use. Steele's reinforces validation practices that assess model fit relative to the problem at hand, encouraging statisticians to choose and critique models based on their representational fidelity for practical tasks.

In Philosophy of Science

The "all models are wrong" aligns closely with scientific , particularly Bas van Fraassen's constructive , which posits that scientific models function as instruments for empirical adequacy rather than as literal depictions of reality. In constructive , the success of a theory or model is measured by its ability to accurately represent phenomena, without committing to the truth of theoretical entities or structures beyond what is empirically accessible. This view echoes the by emphasizing that models need not capture the "whole truth" to be valuable; instead, their utility lies in and empirical fit, acknowledging inherent approximations and idealizations that prevent perfect representation. The epistemological limits highlighted in the also draw from Alfred Korzybski's , where his dictum "the map is not the territory" underscores that abstractions and representations inevitably distort the complexity of reality. Korzybski argued in his foundational work that linguistic and symbolic models impose structural constraints on human understanding, leading to potential errors if treated as identical to the phenomena they describe. This influence revisits broader philosophical concerns about the boundaries of , reinforcing the 's caution against conflating models with objective truth and promoting awareness of their abstract nature in scientific . In discussions of scientific inference, David J. Hand has elaborated on the aphorism to advocate a shift from mere prediction toward decision-making utility, arguing that models' value extends beyond explanatory accuracy to informing practical choices under uncertainty. Hand contends that while models cannot fully replicate reality, their role in science increasingly involves supporting robust decisions by highlighting probabilistic outcomes and risks, rather than aspiring to unattainable precision in forecasting. This perspective integrates the aphorism into pragmatic philosophy, where model assessment prioritizes actionable insights over idealized truth claims. Philosopher Peter Truran further illustrates the aphorism's implications by observing that even simplistic models, such as approximating non-cylindrical objects as cylinders in physical analyses, can disclose partial truths about underlying mechanisms despite their inaccuracies. Truran emphasizes that such models reveal aspects of by focusing on relevant features while abstracting away irrelevant details, thereby contributing to without pretending to completeness. This approach supports the philosophical acceptance of as a necessary feature of scientific progress, where the "wrongness" of models does not negate their capacity to illuminate truths incrementally.

Applications Across Fields

In Traditional Sciences

In physics, the underscores the role of approximate models in simulations that facilitate practical engineering applications, despite inherent inaccuracies. For instance, (CFD) models approximate complex turbulent flows by simplifying small-scale eddies and using parameterization for unresolved phenomena, enabling designs for and vehicles that reduce the need for physical prototypes. These models, such as those based on the Navier-Stokes equations, are iteratively refined through validation against experimental data to improve predictive accuracy for macroscopic behaviors, even as they fail to capture every molecular interaction. In engineering, George Box applied the principle through his development of (RSM) for optimizing chemical processes at . RSM uses quadratic polynomial approximations to model relationships between input variables and process outputs, allowing engineers to identify optimal operating conditions despite the models' simplifications of nonlinear dynamics. This approach emphasizes iterative experimentation to refine models, balancing computational feasibility with actionable insights for industrial efficiency, as seen in yield maximization for chemical reactors. In , ecological models exemplify the as deliberate simplifications that inform conservation strategies, highlighting the value of iterative updates based on field data. The Lotka-Volterra equations, which describe idealized predator-prey dynamics through coupled differential equations, ignore factors like and environmental stochasticity but provide foundational insights into population oscillations and stability thresholds. Such models are refined over time—incorporating extensions for multiple species or —to guide planning, such as predicting extinction risks in island ecosystems and informing control efforts.

In Machine Learning and AI

In the domain of explainable AI (XAI), the inspires variants like "All models are wrong, but some are dangerous," which stress the perils of deploying opaque models in high-stakes applications without transparency mechanisms. This perspective is exemplified in the development of local interpretable model-agnostic explanations (LIME), where Ribeiro et al. (2016) argue that understanding model predictions is essential to mitigate risks from erroneous or biased outputs, enabling users to and trust black-box classifiers. Such approaches have become foundational in XAI, promoting surrogate models that approximate complex decisions locally to reveal potential flaws. The limitations of large language models (LLMs) further illustrate the , as claims of emergent abilities—sudden performance jumps in tasks like arithmetic or —often prove misleading due to inherent model flaws and evaluation artifacts. Schaeffer et al. (2023) show that these "emergences" vanish when using continuous, non-smooth metrics, attributing the phenomenon to discontinuities in measurement rather than genuine capabilities, thus highlighting how LLMs remain fundamentally approximate and prone to . This underscores the need for cautious interpretation in AI scaling, where model wrongs manifest as unreliable generalizations. Physics-enhanced machine learning leverages the idea to advocate hybrid models that incorporate domain knowledge, acknowledging pure data-driven approaches as wrong in data-sparse regimes but improvable through physical constraints. In a position paper, Cicirello (2024) invokes the aphorism to justify physics-informed neural networks for dynamical systems, where embedding governing equations reduces extrapolation errors and enhances utility over standalone ML, as demonstrated in simulations of fluid flows and structural mechanics. This integration yields more robust predictions, bridging the gap between approximate models and real-world applicability. Assessing variable importance in ML also draws on the principle by shifting focus from single flawed models to entire classes, yielding stable insights despite individual inaccuracies. Fisher et al. (2019) introduce model class reliance, a method that aggregates reliance across prediction models to quantify feature effects robustly, even for black-box or misspecified systems, as validated on datasets like UCI benchmarks showing reduced variance in importance rankings. Recent extensions, such as targeted learning frameworks in 2024, build on this for in AI. Contemporary developments in trustworthy AI and arbitrariness in ML continue to reference the aphorism, emphasizing model-agnostic limits and selection strategies. For instance, works on distribution-free prediction bounds establish fundamental hardness results, confirming that detecting model wrongs is challenging without assumptions, informing safer AI deployment (Müller, 2025). Similarly, explorations of reveal how random seeds or architectures lead to divergent outcomes, advocating ensemble methods for reliability in trustworthy systems (Ganesh, 2025). These preprints from 2023–2025 highlight ongoing efforts to make "wrong" models more predictably useful.

Criticisms and Debates

Primary Critiques

One prominent critique of the aphorism "all models are wrong" comes from David Cox, who in 1995 argued that the statement is unhelpful because the very concept of a model inherently involves simplification and to aid understanding of complex phenomena, rather than aiming for exact replication of reality. Cox emphasized that such declarations overlook the practical roles models play in hypothesis testing and , where their utility derives from targeted approximations rather than absolute accuracy. Critics have also pointed to an overemphasis on the "wrongness" of models, which can inadvertently discourage rigorous validation and improvement efforts. In the context of artificial intelligence, particularly large language models (LLMs), analyses of LLM risks highlight how inherent flaws in training data—such as embedded biases and incomplete representations of language—can propagate discriminatory outputs, underscoring the need for proactive mitigation of ethical issues like bias amplification and societal harms. Philosophical discussions highlight a tension between instrumentalist views, which emphasize models as useful tools for , and , which suggests that refined models can approximate underlying truths about the world.

Responses and Evolutions

In response to the inherent limitations of models, emphasized in 1987 the value of iterative refinement in empirical model-building processes to progressively enhance their practical utility, viewing model development as a cyclical endeavor involving , , experimentation, and verification. This approach counters the aphorism's acknowledgment of imperfection by focusing on incremental improvements that make models increasingly useful for and , even if they never achieve perfect accuracy. Building on this, P. Burnham advanced the discourse in 2002 by promoting multi-model inference as a robust alternative to reliance on a single "best" model, which often amplifies flaws due to and uncertainty. Through information-theoretic criteria like Akaike's Information Criterion (AIC), Burnham advocated averaging predictions across a set of candidate models weighted by their relative support from the data, thereby mitigating the risks of overconfidence in any one imperfect representation and improving overall inference reliability in statistical applications. In the domain of , recent position papers have extended these ideas to explainable AI (XAI), proposing "explanation agreement" as a mechanism to bolster robustness amid model imperfections. A framework, EXAGREE, formalizes methods to quantify and resolve disagreements in feature importance rankings across multiple explainers, enabling the identification of stable insights that persist despite individual model errors and thus enhancing trust in AI decisions for high-stakes scenarios like healthcare diagnostics. Ongoing debates in 2025 continue to explore formalizations of model trustworthiness under imperfections, with preprints surveying trustworthiness dimensions—such as robustness, fairness, and explainability—in large language models (LLMs) and proposing metrics to quantify reliability when perfect fidelity to reality is unattainable. These works underscore the aphorism's relevance by developing protocols for and in ensemble systems, ensuring that even flawed models contribute to verifiable, safe AI deployments.

References

  1. https://www.neurology.org/doi/10.1212/WNL.0000000000001769
  2. https://pmc.ncbi.nlm.nih.gov/articles/PMC7880041/
  3. https://www-sop.inria.fr/members/Ian.Jermyn/philosophy/writings/Boxonmaths.pdf
  4. https://news.wisc.edu/renowned-statistician-george-box-dies-at-93/
  5. https://issues.org/andrea/
  6. https://pmc.ncbi.nlm.nih.gov/articles/PMC6297095/
  7. https://books.google.com/books?id=QO2dDRufJEAC
  8. https://en.wikiquote.org/wiki/George_E._P._Box
  9. https://mathshistory.st-andrews.ac.uk/Biographies/Box/
  10. https://www.tandfonline.com/doi/pdf/10.1080/01621459.1976.10480949
  11. https://graduate.northeastern.edu/knowledge-hub/statistical-modeling-for-data-analysis/
  12. https://people.duke.edu/~rnau/testing.htm
  13. https://www.holybooks.com/wp-content/uploads/Science-and-Sanity.pdf
  14. https://archive.org/details/CAT10502416
  15. https://todayinsci.com/V/VonNeumann_John/VonNeumannJohn-Quotations.htm
  16. https://gwern.net/doc/statistics/decision/1979-box.pdf
  17. https://www.sciencedirect.com/science/article/pii/S2590260121000321
  18. https://pmc.ncbi.nlm.nih.gov/articles/PMC8613103/
  19. https://sesmo.org/article/view/18156
  20. https://philsci-archive.pitt.edu/17236/1/reduction_shpmp.pdf
  21. https://www.dam.brown.edu/people/documents/bias-variance.pdf
  22. http://www-stat.wharton.upenn.edu/~steele/Rants/ModelsMandLE.html
  23. https://www.taylorfrancis.com/books/mono/10.1201/9780203753735/generalized-linear-models-peter-mccullagh-john-nelder
  24. https://link.springer.com/book/10.1007/b97636
  25. https://aeon.co/essays/all-scientific-models-are-wrong-but-some-at-least-are-useful
  26. https://tcg.mae.cornell.edu/pubs/Pope_PF_2011.pdf
  27. https://journals.sagepub.com/doi/10.1177/87560879241240242
  28. https://pmc.ncbi.nlm.nih.gov/articles/PMC5469619/
  29. https://www.siam.org/media/3wrd1pns/4-cats-rabbits-birds-and-viruses-oh-my.pdf
  30. https://arxiv.org/abs/2405.05987
  31. https://arxiv.org/abs/1801.01489
  32. https://arxiv.org/abs/2502.06765
  33. https://arxiv.org/abs/2501.14959
  34. https://www.jstor.org/stable/2983440
  35. https://pmc.ncbi.nlm.nih.gov/articles/PMC7442146/
  36. https://link.springer.com/book/9780387953649
  37. https://arxiv.org/abs/2411.01956
  38. https://arxiv.org/abs/2502.15871
Add your contribution
Related Hubs
User Avatar
No comments yet.