Hubbry Logo
Maximum and minimumMaximum and minimumMain
Open search
Maximum and minimum
Community hub
Maximum and minimum
logo
8 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Maximum and minimum
Maximum and minimum
from Wikipedia
Local and global maxima and minima for cos(3πx)/x, 0.1≤ x ≤1.1

In mathematical analysis, the maximum and minimum[a] of a function are, respectively, the greatest and least value taken by the function. Known generically as extrema,[b] they may be defined either within a given range (the local or relative extrema) or on the entire domain (the global or absolute extrema) of a function.[1][2][3] Pierre de Fermat was one of the first mathematicians to propose a general technique, adequality, for finding the maxima and minima of functions.

As defined in set theory, the maximum and minimum of a set are the greatest and least elements in the set, respectively. Unbounded infinite sets, such as the set of real numbers, have no minimum or maximum.

In statistics, the corresponding concept is the sample maximum and minimum.

Definition

[edit]

A real-valued function f defined on a domain X has a global (or absolute) maximum point at x, if f(x) ≥ f(x) for all x in X. Similarly, the function has a global (or absolute) minimum point at x, if f(x) ≤ f(x) for all x in X. The value of the function at a maximum point is called the maximum value of the function, denoted , and the value of the function at a minimum point is called the minimum value of the function, (denoted for clarity). Symbolically, this can be written as follows:

is a global maximum point of function if

The definition of global minimum point also proceeds similarly.

If the domain X is a metric space, then f is said to have a local (or relative) maximum point at the point x, if there exists some ε > 0 such that f(x) ≥ f(x) for all x in X within distance ε of x. Similarly, the function has a local minimum point at x, if f(x) ≤ f(x) for all x in X within distance ε of x. A similar definition can be used when X is a topological space, since the definition just given can be rephrased in terms of neighbourhoods. Mathematically, the given definition is written as follows:

Let be a metric space and function . Then is a local maximum point of function if such that

The definition of local minimum point can also proceed similarly.

In both the global and local cases, the concept of a strict extremum can be defined. For example, x is a strict global maximum point if for all x in X with xx, we have f(x) > f(x), and x is a strict local maximum point if there exists some ε > 0 such that, for all x in X within distance ε of x with xx, we have f(x) > f(x). Note that a point is a strict global maximum point if and only if it is the unique global maximum point, and similarly for minimum points.

A continuous real-valued function with a compact domain always has a maximum point and a minimum point. An important example is a function whose domain is a closed and bounded interval of real numbers (see the graph above).

[edit]

Finding global maxima and minima is the goal of mathematical optimization. If a function is continuous on a closed interval, then by the extreme value theorem, global maxima and minima exist. Furthermore, a global maximum (or minimum) either must be a local maximum (or minimum) in the interior of the domain, or must lie on the boundary of the domain. So a method of finding a global maximum (or minimum) is to look at all the local maxima (or minima) in the interior, and also look at the maxima (or minima) of the points on the boundary, and take the greatest (or least) one.

For differentiable functions, Fermat's theorem states that local extrema in the interior of a domain must occur at critical points (or points where the derivative equals zero).[4] However, not all critical points are extrema. One can often distinguish whether a critical point is a local maximum, a local minimum, or neither by using the first derivative test, second derivative test, or higher-order derivative test, given sufficient differentiability.[5]

For any function that is defined piecewise, one finds a maximum (or minimum) by finding the maximum (or minimum) of each piece separately, and then seeing which one is greatest (or least).

Examples

[edit]
The global maximum of xx occurs at x = e.
Function Maxima and minima
x2 Unique global minimum at x = 0.
x3 No global minima or maxima. Although the first derivative (3x2) is 0 at x = 0, this is an inflection point. (2nd derivative is 0 at that point.)
Unique global maximum at x = e. (See figure at right)
xx Unique global maximum over the positive real numbers at x = 1/e.
x3/3 − x First derivative x2 − 1 and second derivative 2x. Setting the first derivative to 0 and solving for x gives stationary points at −1 and +1. From the sign of the second derivative, we can see that −1 is a local maximum and +1 is a local minimum. This function has no global maximum or minimum.
|x| Global minimum at x = 0 that cannot be found by taking derivatives, because the derivative does not exist at x = 0.
cos(x) Infinitely many global maxima at 0, ±2π, ±4π, ..., and infinitely many global minima at ±π, ±3π, ±5π, ....
2 cos(x) − x Infinitely many local maxima and minima, but no global maximum or minimum.
cos(3πx)/x with 0.1 ≤ x ≤ 1.1 Global maximum at x = 0.1 (a boundary), a global minimum near x = 0.3, a local maximum near x = 0.6, and a local minimum near x = 1.0. (See figure at top of page.)
x3 + 3x2 − 2x + 1 defined over the closed interval (segment) [−4,2] Local maximum at x = −1−15/3, local minimum at x = −1+15/3, global maximum at x = 2 and global minimum at x = −4.

For a practical example,[6] assume a situation where someone has feet of fencing and is trying to maximize the square footage of a rectangular enclosure, where is the length, is the width, and is the area:

The derivative with respect to is:

Setting this equal to

reveals that is our only critical point. Now retrieve the endpoints by determining the interval to which is restricted. Since width is positive, then , and since , that implies that . Plug in critical point , as well as endpoints and , into , and the results are and respectively.

Therefore, the greatest area attainable with a rectangle of feet of fencing is .[6]

Functions of more than one variable

[edit]
Peano surface, a counterexample to some criteria of local maxima of the 19th century
The global maximum is the point at the top
Counterexample: The red dot shows a local minimum that is not a global minimum

For functions of more than one variable, similar conditions apply. For example, in the (enlargeable) figure on the right, the necessary conditions for a local maximum are similar to those of a function with only one variable. The first partial derivatives as to z (the variable to be maximized) are zero at the maximum (the glowing dot on top in the figure). The second partial derivatives are negative. These are only necessary, not sufficient, conditions for a local maximum, because of the possibility of a saddle point. For use of these conditions to solve for a maximum, the function z must also be differentiable throughout. The second partial derivative test can help classify the point as a relative maximum or relative minimum. In contrast, there are substantial differences between functions of one variable and functions of more than one variable in the identification of global extrema. For example, if a bounded differentiable function f defined on a closed interval in the real line has a single critical point, which is a local minimum, then it is also a global minimum (use the intermediate value theorem and Rolle's theorem to prove this by contradiction). In two and more dimensions, this argument fails. This is illustrated by the function

whose only critical point is at (0,0), which is a local minimum with f(0,0) = 0. However, it cannot be a global one, because f(2,3) = −5.

Maxima or minima of a functional

[edit]

If the domain of a function for which an extremum is to be found consists itself of functions (i.e. if an extremum is to be found of a functional), then the extremum is found using the calculus of variations.

In relation to sets

[edit]

Maxima and minima can also be defined for sets. In general, if an ordered set S has a greatest element m, then m is a maximal element of the set, also denoted as . Furthermore, if S is a subset of an ordered set T and m is the greatest element of S with (respect to order induced by T), then m is a least upper bound of S in T. Similar results hold for least element, minimal element and greatest lower bound. The maximum and minimum function for sets are used in databases, and can be computed rapidly, since the maximum (or minimum) of a set can be computed from the maxima of a partition; formally, they are self-decomposable aggregation functions.

In the case of a general partial order, a least element (i.e., one that is less than all others) should not be confused with the minimal element (nothing is lesser). Likewise, a greatest element of a partially ordered set (poset) is an upper bound of the set which is contained within the set, whereas the maximal element m of a poset A is an element of A such that if mb (for any b in A), then m = b. Any least element or greatest element of a poset is unique, but a poset can have several minimal or maximal elements. If a poset has more than one maximal element, then these elements will not be mutually comparable.

In a totally ordered set, or chain, all elements are mutually comparable, so such a set can have at most one minimal element and at most one maximal element. Then, due to mutual comparability, the minimal element will also be the least element, and the maximal element will also be the greatest element. Thus in a totally ordered set, we can simply use the terms minimum and maximum.

If a chain is finite, then it will always have a maximum and a minimum. If a chain is infinite, then it need not have a maximum or a minimum. For example, the set of natural numbers has no maximum, though it has a minimum. If an infinite chain S is bounded, then the closure Cl(S) of the set occasionally has a minimum and a maximum, in which case they are called the greatest lower bound and the least upper bound of the set S, respectively.

Argument of the maximum

[edit]
As an example, both unnormalised and normalised sinc functions above have of {0} because both attain their global maximum value of 1 at x = 0.

The unnormalised sinc function (red) has arg min of {−4.49, 4.49}, approximately, because it has 2 global minimum values of approximately −0.217 at x = ±4.49. However, the normalised sinc function (blue) has arg min of {−1.43, 1.43}, approximately, because their global minima occur at x = ±1.43, even though the minimum value is the same.[7]
In mathematics, the arguments of the maxima (abbreviated arg max or argmax) and arguments of the minima (abbreviated arg min or argmin) are the input points at which a function output value is maximized and minimized, respectively.[8] While the arguments are defined over the domain of a function, the output is part of its codomain.

See also

[edit]

Notes

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia
from Grokipedia
In , the maximum and minimum of a set are defined as the greatest and least elements in the set, respectively, provided such elements exist. For functions, maxima and minima, collectively known as extrema, refer to the largest and smallest values that the function attains over its domain, occurring at points where the function reaches these peak or valley values. These concepts are fundamental in various mathematical fields, including calculus, optimization, and order theory. In the context of real-valued functions of one or more variables, extrema are classified as absolute (or global), which represent the overall highest or lowest values across the entire domain, or local (or relative), which are the highest or lowest values in a neighborhood around a specific point. Local maxima and minima often occur at critical points, where the derivative of the function is zero or undefined, allowing for the identification of potential peaks and valleys through techniques like the first and second derivative tests. The study of maxima and minima plays a crucial role in , particularly in optimization problems, where one seeks to maximize profit, minimize , or find equilibrium states in physical systems. For instance, in and engineering, these principles underpin and nonlinear optimization algorithms. In more advanced settings, such as , partial derivatives help locate extrema on surfaces, with the providing insight into their nature (maximum, minimum, or ). Not all functions possess maxima or minima; unbounded functions, like linear ones extending to , may lack them, highlighting the importance of domain considerations.

Fundamental Definitions

Maxima and Minima for Functions

In mathematics, a real-valued function ff defined on a domain DD attains a maximum value at a point x0Dx_0 \in D if f(x0)f(x)f(x_0) \geq f(x) for all xDx \in D, making f(x0)f(x_0) the largest value of the function over its entire domain; similarly, it attains a minimum value at x1Dx_1 \in D if f(x1)f(x)f(x_1) \leq f(x) for all xDx \in D, making f(x1)f(x_1) the smallest value. These are known as absolute or global maxima and minima, as they represent the extremal values across the full domain. In contrast, a local maximum occurs at x0Dx_0 \in D if there exists an open interval II containing x0x_0 such that f(x0)f(x)f(x_0) \geq f(x) for all xIDx \in I \cap D, and a local minimum satisfies f(x1)f(x)f(x_1) \leq f(x) for all xIDx \in I \cap D; these relative extrema hold only in a neighborhood, not necessarily globally. The existence of global maxima and minima depends on the domain and properties of the function. For a ff on a closed and bounded interval [a,b][a, b], the guarantees that ff attains both an absolute maximum and an absolute minimum on [a,b][a, b]. This theorem requires continuity on the compact set [a,b][a, b], ensuring the function's is also compact and thus achieves its supremum and infimum. On unbounded domains, such as the real line, continuous functions may not attain extrema; for instance, f(x)=xf(x) = x has no global maximum or minimum./03:_Functions_from_R_to_R/3.05:_Extreme_Values) Global extrema are denoted by maxxDf(x)\max_{x \in D} f(x) for the maximum value and minxDf(x)\min_{x \in D} f(x) for the minimum value, with the points of attainment specified as argmax\arg\max or argmin\arg\min when needed. For a f(x)=cf(x) = c on any domain DD, every point in DD is both a global maximum and minimum, as f(x)=cf(x) = c for all xDx \in D. Local extrema detection often involves derivatives, though full methods are covered elsewhere. In unbounded cases where extrema are not attained, the supremum and infimum provide least upper and greatest lower bounds, respectively, as discussed in ordered sets. The study of maxima and minima originated in early calculus, with Pierre de Fermat developing foundational methods around the 1630s through correspondence, using adequacy to identify points where function differences vanish near extrema.

Local and Global Extrema

In the context of a function ff defined on a domain DRD \subseteq \mathbb{R}, a point cDc \in D is a local maximum if there exists an open neighborhood NN around cc such that f(c)f(x)f(c) \geq f(x) for all xNDx \in N \cap D. Similarly, cc is a local minimum if f(c)f(x)f(c) \leq f(x) for all xNDx \in N \cap D. These local extrema represent relative peaks or valleys in the function's graph within a restricted vicinity, without regard to the behavior elsewhere in the domain. A global maximum (or absolute maximum) occurs at a point cDc \in D where f(c)f(x)f(c) \geq f(x) for every xDx \in D, making it the highest value attained over the entire domain; a global minimum is defined analogously with the inequality reversed. A global extremum may coincide with one or more local extrema, but uniqueness is not guaranteed—a function can have multiple global maxima if it is constant on some interval, for instance. On domains with boundaries, such as closed intervals [a,b][a, b], global extrema can occur at the endpoints (boundary points) even if they are not local extrema in the interior. For example, the function f(x)=xf(x) = x on [0,1][0, 1] has its global maximum at the boundary point x=1x = 1 and global minimum at x=0x = 0. The existence of global extrema is guaranteed under certain conditions by the : if ff is continuous on a compact set KRK \subseteq \mathbb{R} (i.e., closed and bounded), then ff attains both a global maximum and a global minimum on KK. These extrema occur either at critical points in the interior or at boundary points of KK. A proof sketch proceeds as follows: since KK is compact and ff is continuous, the image f(K)f(K) is also compact; by the Heine-Borel theorem, f(K)f(K) is closed and bounded in R\mathbb{R}. Boundedness implies f(K)f(K) has a supremum MM and infimum mm; closedness ensures M,mf(K)M, m \in f(K), so there exist points in KK where these values are attained. The further supports attainment by ensuring the continuous image connects all intermediate values between mm and MM. Critical points, where the is zero or undefined, are relevant for locating interior extrema, as detailed in analytical techniques for single-variable functions. Global extrema may fail to exist if the domain is not compact or if ff is discontinuous. For instance, on the open interval (0,1](0, 1], the continuous function f(x)=xf(x) = x has no global minimum, as values approach 0 (the infimum) near x=0x = 0 but never attain it within the domain, despite having a global maximum of 1 at x=1x = 1. Discontinuous functions on compact sets also lack guaranteed extrema; for example, consider the function defined by f(x)=xf(x) = x for 0x<10 \leq x < 1 and f(1)=0f(1) = 0 on [0,1][0, 1], which has no maximum since its supremum of 1 is not attained (though the infimum of 0 is attained at x=0x = 0 and x=1x = 1). In cases without a maximum, the supremum serves as the least upper bound, a concept explored further in ordered sets.

Methods for Finding Extrema

Analytical Techniques for Single-Variable Functions

Analytical techniques for identifying maxima and minima in single-variable functions rely on the properties of derivatives, assuming the function is differentiable. These methods locate critical points where the derivative is zero or undefined and classify them as local extrema based on the behavior of the function nearby. For functions defined on closed intervals, additional evaluation at boundaries determines global extrema. Fermat's theorem states that if a function ff has a local extremum at an interior point cc where ff is differentiable, then the first derivative f(c)=0f'(c) = 0. This theorem identifies potential locations for local maxima or minima, known as critical points, but does not distinguish between them. The first derivative test classifies critical points by examining the sign of ff' in intervals around cc. If ff' changes from negative to positive at cc, then f(c)f(c) is a local minimum; if from positive to negative, then f(c)f(c) is a local maximum. No sign change indicates neither. The second derivative test provides a quicker classification using the concavity at cc. Compute the second derivative, defined as f(x)=d2fdx2.f''(x) = \frac{d^2 f}{dx^2}. If f(c)=0f'(c) = 0 and f(c)>0f''(c) > 0, then f(c)f(c) is a local minimum; if f(c)<0f''(c) < 0, then a local maximum. If f(c)=0f''(c) = 0, the test is inconclusive. For inconclusive cases where f(c)=0f''(c) = 0, higher-order derivative tests extend the analysis. Consider the first non-zero higher derivative at cc: if it is the nnth derivative with f(n)(c)>0f^{(n)}(c) > 0 and nn even, then a local minimum; if f(n)(c)<0f^{(n)}(c) < 0 and nn even, a local maximum; odd nn indicates neither. For example, the third derivative test applies when the second is zero. To find global extrema on a closed interval [a,b][a, b], evaluate ff at critical points in the interior and at the endpoints aa and bb, then compare values; the largest is the global maximum, the smallest the global minimum. This follows from the extreme value theorem for continuous functions on compact sets. Consider the example f(x)=x33xf(x) = x^3 - 3x on [2,2][-2, 2]. First, find critical points: f(x)=3x23=0f'(x) = 3x^2 - 3 = 0 implies x2=1x^2 = 1, so x=±1x = \pm 1. These are interior points. Apply the first derivative test: f(x)<0f'(x) < 0 for x<1|x| < 1, and f(x)>0f'(x) > 0 for x>1|x| > 1, so x=1x = -1 is a local maximum and x=1x = 1 is a local minimum. Confirm with the second derivative: f(x)=6xf''(x) = 6x, so f(1)=6<0f''(-1) = -6 < 0 (maximum) and f(1)=6>0f''(1) = 6 > 0 (minimum). Evaluate at endpoints and critical points: f(2)=2f(-2) = -2, f(1)=2f(-1) = 2, f(1)=2f(1) = -2, f(2)=2f(2) = 2. Thus, global maxima are 2 at x=1x = -1 and x=2x = 2, global minima are -2 at x=1x = 1 and x=2x = -2.

Numerical and Search Methods

Numerical and search methods provide iterative approaches to approximate maxima and minima of functions, particularly when analytical solutions are unavailable or computationally infeasible due to the function's complexity or lack of closed-form . These techniques rely on repeated evaluations of the function (and possibly its ) to refine estimates within a search interval, often assuming properties like to ensure progress toward an extremum. They are essential for practical optimization in fields such as and , where exact methods may fail for non-polynomial or high-dimensional problems. The , adapted for finding extrema in functions, operates by successively halving an initial interval [a,b][a, b] containing the extremum based on function evaluations at the and comparisons to endpoints. For a function ff with a maximum, evaluate ff at the m=(a+b)/2m = (a + b)/2; if f(m)>f(b)f(m) > f(b), the maximum lies in [a,m][a, m] (set b=mb = m); otherwise, it lies in [m,b][m, b] (set a=ma = m). This process reduces the interval length by half each iteration until a tolerance is met, guaranteeing convergence to the global extremum in the interval under unimodality. Ternary search extends this interval reduction for unimodal functions without requiring derivatives, dividing the interval into thirds rather than halves to more efficiently discard unpromising regions. Given [l,r][l, r], compute points m1=l+(rl)/3m_1 = l + (r - l)/3 and m2=r(rl)/3m_2 = r - (r - l)/3; evaluate f(m1)f(m_1) and f(m2)f(m_2). For a maximum, if f(m1)<f(m2)f(m_1) < f(m_2), discard [l,m1][l, m_1] (set l=m1l = m_1); otherwise, discard [m2,r][m_2, r] (set r=m2)r = m_2). Repeat until the interval is sufficiently small, achieving a convergence rate where the interval shrinks by a factor of approximately 0.666 per iteration. This method is particularly useful for black-box functions where derivative computation is expensive or impossible. Newton's method for optimization iteratively refines an estimate xnx_n of a local extremum using second-order information, updating via the formula xn+1=xnf(xn)f(xn),x_{n+1} = x_n - \frac{f'(x_n)}{f''(x_n)}, which approximates the function quadratically near the root of f(x)=0f'(x) = 0. It requires the function to be twice continuously differentiable, with f(x)f''(x) nonzero and of consistent sign at the extremum (positive for minima, negative for maxima) to ensure the update moves toward the stationary point. Local quadratic convergence—where the error en+1Cen2e_{n+1} \approx C e_n^2 for some constant CC—holds if the initial guess is sufficiently close and the Hessian (second derivative) is Lipschitz continuous, making it highly efficient for smooth problems once near the solution. However, it may diverge if started far from the extremum or if f(xn)f''(x_n) changes sign. In one dimension, gradient ascent (for maxima) or descent (for minima) updates the estimate in the direction of the steepest increase or decrease, given by xn+1=xn+αf(xn)x_{n+1} = x_n + \alpha f'(x_n) for ascent, where α>0\alpha > 0 is a step size. This method assumes differentiability and moves along the until f(xn)<ϵ|f'(x_n)| < \epsilon, converging linearly under Lipschitz continuity of ff' but potentially slowly if the function is ill-conditioned. While foundational, it is less common in one dimension than in multivariable settings, where it generalizes to higher-dimensional gradients. Global optimization poses challenges when functions exhibit multiple local extrema, trapping local methods in suboptimal points; stochastic approaches like simulated annealing address this by mimicking the physical annealing process to explore the search space probabilistically. Starting from an initial solution, it iteratively perturbs the current state to a neighbor, accepting the move if it improves the objective or, with probability exp(ΔE/T)\exp(-\Delta E / T) (where ΔE\Delta E is the change in function value and TT is a decreasing "temperature" parameter), if it worsens it—allowing escape from local minima early on while favoring improvements as TT cools. This seminal algorithm converges to the global optimum in probability under suitable cooling schedules, though it requires tuning and may be computationally intensive. Error analysis in these methods focuses on convergence rates and stopping criteria to balance accuracy and efficiency. Linear convergence, as in bisection or ternary search, reduces error by a constant factor r<1r < 1 per iteration (en+1rene_{n+1} \leq r e_n), while quadratic rates in Newton's method square the error near the solution. Common stopping criteria include a tolerance ϵ\epsilon on the interval length (ba<ϵ|b - a| < \epsilon) for bracketing methods, on the gradient magnitude (f(xn)<ϵ|f'(x_n)| < \epsilon) for derivative-based approaches, or on the change in iterates (xn+1xn<ϵ|x_{n+1} - x_n| < \epsilon), ensuring the approximation meets a predefined precision without excessive computation. These criteria must account for function scaling and noise to avoid premature or infinite termination.

Extrema in Multivariable Settings

Functions of Several Variables

In the context of functions of several variables, a function f:RnRf: \mathbb{R}^n \to \mathbb{R} has a local maximum at a point cRnc \in \mathbb{R}^n if there exists a neighborhood UU around cc such that f(c)f(x)f(c) \geq f(x) for all xUx \in U; similarly, cc is a local minimum if f(c)f(x)f(c) \leq f(x) for all xUx \in U . A global maximum (or absolute maximum) occurs if f(c)f(x)f(c) \geq f(x) for all xx in the domain of ff, and analogously for a global minimum . These definitions extend the one-variable case by considering neighborhoods in the higher-dimensional Euclidean space Rn\mathbb{R}^n, where the neighborhood is typically an open ball centered at cc . To identify potential local extrema, critical points are defined as points cc where the gradient f(c)=0\nabla f(c) = 0, with f\nabla f being the vector of partial derivatives (fx1,,fxn)\left( \frac{\partial f}{\partial x_1}, \dots, \frac{\partial f}{\partial x_n} \right) . At such points, the tangent hyperplane to the graph of ff is horizontal, analogous to the first derivative test in one variable, though critical points may also include locations where the gradient is undefined . Solving f(c)=0\nabla f(c) = 0 typically involves setting each partial derivative to zero and solving the resulting system of equations . To classify these critical points, the second derivative test employs the Hessian matrix HH, whose entries are the second partial derivatives: Hij=2fxixjH_{ij} = \frac{\partial^2 f}{\partial x_i \partial x_j} for i,j=1,,ni, j = 1, \dots, n . The definiteness of HH at a critical point cc is determined by its eigenvalues: if all eigenvalues are positive (positive definite), then cc is a local minimum; if all are negative (negative definite), a local maximum; and if eigenvalues have mixed signs (indefinite), a saddle point . For functions of two variables, definiteness can also be checked via the determinant of HH and the trace, but eigenvalues provide a general approach for n>2n > 2 . The for multivariable functions states that if f:KRf: K \to \mathbb{R} is continuous and KRnK \subset \mathbb{R}^n is compact (closed and bounded), then ff attains its global maximum and minimum on KK . This guarantees the existence of extrema on such sets, with candidates found among critical points in the interior and boundary values . Consider the function f(x,y)=x2+y2f(x,y) = x^2 + y^2. The is f=(2x,2y)\nabla f = (2x, 2y), so the critical point is at (0,0)(0,0) where f=0\nabla f = 0 . The Hessian is H=(2002),H = \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix}, with eigenvalues 2 and 2, both positive, confirming a local (and global) minimum at (0,0)(0,0) where f(0,0)=0f(0,0) = 0 .

Constrained Optimization Problems

involves finding the maximum or minimum of an objective function subject to one or more constraints that define a , often visualized geometrically as extrema occurring on manifolds defined by equality constraints or within bounded s for inequalities. This setup contrasts with unconstrained problems by requiring the of the objective to align with the constraint s at the optimum, ensuring the search direction respects the boundary. For equality constraints of the form g(x)=0g(\mathbf{x}) = 0, where f(x)f(\mathbf{x}) is the objective function to extremize, the method of Lagrange multipliers introduces scalar multipliers λ\lambda such that the gradients satisfy f(x)=λg(x)\nabla f(\mathbf{x}) = \lambda \nabla g(\mathbf{x}). This condition arises from forming the Lagrangian L(x,λ)=f(x)+λg(x)\mathcal{L}(\mathbf{x}, \lambda) = f(\mathbf{x}) + \lambda g(\mathbf{x}) and setting its partial derivatives to zero, yielding the system: f(x)+λg(x)=0,g(x)=0.\nabla f(\mathbf{x}) + \lambda \nabla g(\mathbf{x}) = 0, \quad g(\mathbf{x}) = 0. The multiplier λ\lambda interprets the sensitivity of the objective to perturbations in the constraint, representing the rate of change of the extremum value with respect to the constraint constant. This approach, pioneered by Joseph-Louis Lagrange in his 1788 work Mécanique Analytique, transforms the constrained problem into solving an unconstrained system in extended variables. To classify these critical points, second-order conditions employ the bordered Hessian matrix of the Lagrangian, which augments the standard Hessian with rows and columns from the constraint . For a single constraint in two variables, the bordered Hessian is Hb=0gxgygx2Lx22Lxygy2Lyx2Ly2.H_b = \begin{vmatrix} 0 & \frac{\partial g}{\partial x} & \frac{\partial g}{\partial y} \\ \frac{\partial g}{\partial x} & \frac{\partial^2 \mathcal{L}}{\partial x^2} & \frac{\partial^2 \mathcal{L}}{\partial x \partial y} \\ \frac{\partial g}{\partial y} & \frac{\partial^2 \mathcal{L}}{\partial y \partial x} & \frac{\partial^2 \mathcal{L}}{\partial y^2} \end{vmatrix}.
Add your contribution
Related Hubs
Contribute something
User Avatar
No comments yet.