Hubbry Logo
Credible intervalCredible intervalMain
Open search
Credible interval
Community hub
Credible interval
logo
8 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Credible interval
Credible interval
Credible interval
View on Wikipedia
from Wikipedia
The 90%-smallest credible interval of a distribution is the smallest interval that contains 90% of the distribution mass.
Part of a series on
Bayesian statistics
Posterior = Likelihood × Prior ÷ Evidence
Background
Model building
Posterior approximation
Estimators
Evidence approximation
Model evaluation

In Bayesian statistics, a credible interval is an interval used to characterize a probability distribution. It is defined such that an unobserved parameter value has a particular probability to fall within it. For example, in an experiment that determines the distribution of possible values of the parameter , if the probability that lies between 35 and 45 is , then is a 95% credible interval.

Credible intervals are typically used to characterize posterior probability distributions or predictive probability distributions.[1] Their generalization to disconnected or multivariate sets is called credible set or credible region.

Credible intervals are a Bayesian analog to confidence intervals in frequentist statistics.[2] The two concepts arise from different philosophies:[3] Bayesian intervals treat their bounds as fixed and the estimated parameter as a random variable, whereas frequentist confidence intervals treat their bounds as random variables and the parameter as a fixed value. Also, Bayesian credible intervals use (and indeed, require) knowledge of the situation-specific prior distribution, while the frequentist confidence intervals do not.

Definitions

[edit]

Credible sets are not unique, as any given probability distribution has an infinite number of -credible sets, i.e. sets of probability . For example, in the univariate case, there are multiple definitions for a suitable interval or set:

  • The smallest credible interval (SCI), sometimes also called the highest density interval. This interval necessarily contains the median whenever . When the distribution is unimodal, this interval also contains the mode.
  • The smallest credible set (SCS), sometimes also called the highest density region. For a multimodal distribution, this is not necessarily an interval as it can be disconnected. This set always contains the mode.
  • A quantile-based credible interval, which is computed by taking the inter-quantile interval for some predefined . For instance, the median credible interval (MCI) of probability is the interval where the probability of being below the interval is as likely as being above it, that is to say the interval . It is sometimes also called the equal-tailed interval, and it always contains the median. Other quantile-based credible intervals can be defined, such as the lowest credible interval (LCI) which is , or the highest credible interval (HCI) which is . These intervals may be more suited for bounded variables.

One may also define an interval for which the mean is the central point, assuming that the mean exists.

-Smallest Credible Sets (-SCS) can easily be generalized to the multivariate case, and are bounded by probability density contour lines.[4] They always contain the mode, but not necessarily the mean, the coordinate-wise median, nor the geometric median.

Credible intervals can also be estimated through the use of simulation techniques such as Markov chain Monte Carlo.[5]

Contrasts with confidence interval

[edit]

A frequentist 95% confidence interval means that with a large number of repeated samples, 95% of such calculated confidence intervals would include the true value of the parameter. In frequentist terms, the parameter is fixed (cannot be considered to have a distribution of possible values) and the confidence interval is random (as it depends on the random sample).

Bayesian credible intervals differ from frequentist confidence intervals by two major aspects:

  • Credible intervals are intervals whose values have a (posterior) probability density, representing the plausibility that the parameter has those values, whereas confidence intervals regard the population parameter as fixed and therefore not the object of probability. Within confidence intervals, confidence refers to the randomness of the very confidence interval under repeated trials, whereas credible intervals analyze the uncertainty of the target parameter given the data at hand.
  • Credible intervals and confidence intervals treat nuisance parameters in radically different ways.

For the case of a single parameter and data that can be summarised in a single sufficient statistic, it can be shown that the credible interval and the confidence interval coincide if the unknown parameter is a location parameter (i.e. the forward probability function has the form ), with a prior that is a uniform flat distribution;[6] and also if the unknown parameter is a scale parameter (i.e. the forward probability function has the form ), with a Jeffreys' prior [6] — the latter following because taking the logarithm of such a scale parameter turns it into a location parameter with a uniform distribution. But these are distinctly special (albeit important) cases; in general no such equivalence can be made.

References

[edit]

Further reading

[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Credible interval

View on Grokipedia
from Grokipedia
A credible interval, also known as a Bayesian credible interval, is an interval estimate for a in that contains the true value with a specified , such as 95%, given the observed and prior beliefs. Unlike frequentist confidence intervals, which provide a long-run frequency interpretation over repeated samples, credible intervals treat the parameter as a and directly quantify the probability that it falls within the interval based on the posterior distribution. In , credible intervals are derived from the posterior distribution, which combines the likelihood of the with the prior distribution on the . For a θ\theta, a 100(1−α)%100(1 - \alpha)\% credible interval [a,b][a, b] satisfies P(a≤θ≤b∣[data](/page/Data))=1−αP(a \leq \theta \leq b \mid \text{[data](/page/Data)}) = 1 - \alpha, where the probability is computed over the posterior. This probabilistic interpretation allows for intuitive statements about parameter uncertainty, such as "there is a 95% probability that the true lies within this interval," which is not possible with confidence intervals. There are several types of credible intervals, with the two most common being the equal-tailed interval and the highest posterior density (HPD) interval. The equal-tailed interval is constructed by taking the central 1−α1 - \alpha portion of the posterior, specifically the quantiles from α/2\alpha/2 to 1−α/21 - \alpha/2, which is symmetric in probability mass but may not be the shortest possible interval. In contrast, the HPD interval consists of the set of values where the posterior density exceeds a certain threshold, ensuring it is the smallest interval (or region in multiple dimensions) with the desired ; this makes it particularly useful for skewed distributions. Both types can be computed analytically for conjugate priors or numerically via methods like for complex models. Credible intervals offer advantages in incorporating prior information and providing direct uncertainty quantification, which is especially valuable in fields like , physics, and where subjective priors may reflect expert knowledge. However, their properties depend on the choice of prior, and in some cases, they may differ substantially from intervals, particularly with informative priors or small sample sizes. Overall, credible intervals exemplify the Bayesian paradigm's emphasis on updating beliefs with data, enabling more flexible and interpretable inference compared to purely frequentist approaches.

Bayesian Foundations

Definition and Interpretation

In , a credible interval for an unknown θ\theta is a range of values derived from the posterior distribution p(θ∣X)p(\theta \mid X), where XX denotes the observed , such that the probability that θ\theta lies within the interval equals a specified level, typically 1−α1 - \alpha. Formally, a 100(1−α)%100(1 - \alpha)\% credible interval is a random interval C(X)C(X) satisfying P(θ∈C(X)∣X)=1−αP(\theta \in C(X) \mid X) = 1 - \alpha, with the probability computed with respect to the posterior distribution updated by the and the prior p(θ)p(\theta). This interval directly quantifies the uncertainty about the parameter θ\theta given the data and prior beliefs, providing a probabilistic statement about the location of θ\theta in the parameter space after incorporating evidence from the likelihood p(X∣θ)p(X \mid \theta). Unlike approaches based on sampling distributions of estimators, credible intervals treat θ\theta as a random variable under the posterior, enabling interpretations such as "there is a 95%95\% posterior probability that θ\theta falls within this range" for an α=0.05\alpha = 0.05 interval. One common type is the equal-tailed credible interval, which divides the posterior probability mass equally between the lower and upper tails. For a 100(1−α)%100(1 - \alpha)\% equal-tailed interval [L,U][L, U], it satisfies ∫−∞Lp(θ∣X) dθ=α2,∫U∞p(θ∣X) dθ=α2,\int_{-\infty}^{L} p(\theta \mid X) \, d\theta = \frac{\alpha}{2}, \quad \int_{U}^{\infty} p(\theta \mid X) \, d\theta = \frac{\alpha}{2}, corresponding to the α/2\alpha/2 and 1−α/21 - \alpha/2 quantiles of the posterior distribution; this construction is straightforward for symmetric posteriors and is often the default choice due to its simplicity. The term "credible interval" was coined by Edwards, Lindman, and Savage in 1963 to emphasize its basis in subjective Bayesian probabilities, distinguishing it from frequentist concepts while highlighting the interval's believability given the posterior evidence.

Role in Bayesian Inference

In Bayesian inference, the process begins with specifying a prior distribution p(θ)p(\theta) that encodes beliefs about the unknown parameter θ\theta before observing data XX. This prior is then updated using the likelihood p(X∣θ)p(X | \theta), which describes the probability of the data given the parameter, to obtain the posterior distribution p(θ∣X)p(\theta | X). Credible intervals are derived directly from this posterior, providing a range of plausible values for θ\theta that contains a specified probability mass, such as 95%, under the updated beliefs. The posterior is formally derived via as p(θ∣X)=p(X∣θ)p(θ)∫p(X∣θ)p(θ) dθ,p(\theta | X) = \frac{p(X | \theta) p(\theta)}{\int p(X | \theta) p(\theta) \, d\theta}, where the denominator serves as the , or , ensuring the posterior integrates to 1. This normalization step integrates over all possible values of θ\theta, making the posterior a proper . The choice of prior profoundly influences the location and width of the resulting credible intervals; for instance, informative priors can shift the interval toward expected values or narrow it by incorporating external knowledge, while non-informative priors yield intervals more driven by the data alone. A classic example of prior dependence occurs with conjugate priors, where the posterior retains the same distributional form as the prior, facilitating analytical computation. For a binomial likelihood modeling success probability θ\theta with nn trials and kk successes, a beta prior Beta(α,β)\text{Beta}(\alpha, \beta) yields a beta posterior Beta(α+k,β+n−k)\text{Beta}(\alpha + k, \beta + n - k), whose credible intervals can then be computed from the updated parameters, demonstrating how the prior hyperparameters α\alpha and β\beta adjust the interval's position and spread based on prior "pseudo-observations." Credible intervals also play a key role in Bayesian testing and , particularly for evaluating point null hypotheses like H0:θ=θ0H_0: \theta = \theta_0, where the interval's inclusion or exclusion of θ0\theta_0 informs the of the null. They serve as building blocks for Bayes factors, which quantify evidence for competing models by comparing posterior odds to prior odds, aiding decisions under uncertainty without relying on p-values.

Comparison to Frequentist Intervals

Confidence Intervals Overview

In frequentist statistics, a provides a range of plausible values for an unknown based on sample . Specifically, a (1−α)100%(1-\alpha)100\% for a θ\theta is defined as a random interval [L(X),U(X)][L(X), U(X)], where XX represents the observed , such that the probability P(L(X)≤θ≤U(X))=1−αP(L(X) \leq \theta \leq U(X)) = 1-\alpha, with this probability taken over the of XX conditional on the fixed true value of θ\theta. The frequentist interpretation of a confidence interval emphasizes long-run frequency coverage rather than a direct probability statement about the for a given sample. In repeated sampling from the same , approximately 1−α1-\alpha proportion of the constructed intervals will contain the of θ\theta, reflecting the reliability of the procedure over many hypothetical experiments. This perspective treats θ\theta as fixed but unknown, and the interval as random due to variability in the sample. Confidence intervals are commonly constructed using pivot-based methods, which exploit known properties of the to achieve the desired . For instance, when estimating the μ\mu of a with known standard deviation σ\sigma based on a sample of size nn, the (1−α)100%(1-\alpha)100\% is given by [Xˉ−zα/2σn,Xˉ+zα/2σn],\left[ \bar{X} - z_{\alpha/2} \frac{\sigma}{\sqrt{n}}, \bar{X} + z_{\alpha/2} \frac{\sigma}{\sqrt{n}} \right],
Add your contribution
Related Hubs
User Avatar
No comments yet.