Questions tagged [lagrange-multiplier]

This tag is for the questions on Lagrange multipliers. The method of Lagrange multipliers (named after Joseph Louis Lagrange) provides a strategy for finding the local maxima and minima of a function subject to equality constraints.

When are Lagrange multipliers useful?

One of the most common problems in calculus is that of finding maxima or minima (in general, "extrema") of a function, but it is often difficult to find a closed form for the function being extremized. Such difficulties often arise when one wishes to maximize or minimize a function subject to fixed outside conditions or constraints. The method of Lagrange multipliers is a powerful tool for solving this class of problems without the need to explicitly solve the conditions and use them to eliminate extra variables.

Put more simply, it's usually not enough to ask, "How do I minimize the aluminum needed to make this can?" (The answer to that is clearly "Make a really, really small can!") You need to ask, "How do I minimize the aluminum while making sure the can will hold $10$ ounces of soup?" Or similarly, "How do I maximize my factory's profit given that I only have $\$15,000$ to invest?" Or to take a more sophisticated example, "How quickly will the roller coaster reach the ground assuming it stays on the track?" In general, Lagrange multipliers are useful when some of the variables in the simplest description of a problem are made redundant by the constraints.

The mathematics of Lagrange multipliers:

To find critical points of a function $f(x, y, z)$ on a level surface $g(x, y, z) = C$ (or subject to the constraint $g(x, y, z) = C$), we must solve the following system of simultaneous equations: $$∇f(x, y, z) = λ∇g(x, y, z)$$ $$g(x, y, z) = C$$ Remembering that $∇f$ and $∇g$ are vectors, we can write this as a collection of four equations in the four unknowns $x, y, z,$ and $λ$ : $$f_x(x, y, z) = λ~g_x(x, y, z)$$ $$f_y(x, y, z) = λ~g_y(x, y, z)$$ $$fz(x, y, z) = λ~g_z(x, y, z)$$ $$g(x, y, z) = C$$ The variable $~λ~$ is a dummy variable called a Lagrange multiplier; we only really care about the values of $x, y,$ and $z$ .

Once we have found all the critical points, we plug them into $~f~$ to see where the maxima and minima. The critical points where $~f~$ is greatest are maxima and the critical points where $~f~$ is smallest are minima.

Note:

Solving the system of equations can be hard! Here are some tricks that may help:

$1.\quad$ Since we don’t actually care what $~λ~$ is, you can first solve for $~λ~$ in terms of $~x,~ y,~$ and $~z~$ to remove $~λ~$ from the equations.

$2.\quad$ Try first solving for one variable in terms of the others.

$3.\quad$ Remember that whenever you take a square root, you must consider both the positive and the negative square roots.

$4.\quad$ Remember that whenever you divide an equation by an expression, you must be sure that the expression is not $~0~$. It may help to split the problem into two cases: first solve the equations assuming that a variable is $~0~$, and then solve the equations assuming that it is not $~0~$.

Reference:

https://en.wikipedia.org/wiki/Lagrange_multiplier

2512 questions
272
votes
8 answers

Please explain the intuition behind the dual problem in optimization.

I've studied convex optimization pretty carefully, but don't feel that I have yet "grokked" the dual problem. Here are some questions I would like to understand more deeply/clearly/simply: How would somebody think of the dual problem? What…
48
votes
4 answers

How do Lagrange multipliers work to find the lowest value of a function subject to a constraint?

I have been using Lagrange multipliers in constrained optimization problems, but I don't see how they actually work to simultaneously satisfy the constraint and find the lowest possible value of an objective function.
Tom Stephens
  • 523
  • 2
  • 15
  • 25
36
votes
5 answers

How to know whether Lagrange multipliers gives maximum or minimum?

My book tells me that of the solutions to the Lagrange system, the smallest is the minimum of the function given the constraint and the largest is the maximum given that one actually exists. But what if we only have one point as a solution? How to…
35
votes
3 answers

How to prove Lagrange multiplier theorem in a rigorous but intuitive way?

Following some text books, the Lagrange multiplier theorem can be described as follows. Let $U \subset \mathbb{R}^n$ be an open set and let $f:U\rightarrow \mathbb{R}, g:U\rightarrow \mathbb{R}$ be $C^1$ functions. Let $\mathbf{x}_{0} \in U,…
26
votes
2 answers

Derivation of the method of Lagrange multipliers?

I've always used the method of Lagrange multipliers with blind confidence that it will give the correct results when optimizing problems with constraints. But I would like to know if anyone can provide or recommend a derivation of the method at…
24
votes
3 answers

Lagrange multipliers with inequality constraints: minimize $f$ on the region $0 \leq x,y \leq 1$

I do not have much experience with constrained optimization, but I am hoping that you can help. My current problem involves a more complex function, but the constraints are similar to the ones below. Just so that I can see how to apply Lagrange…
23
votes
1 answer

Constrained variational problems intuition

Problem: minimise $F(x,y,y')$ over $x$, constrained by $G(x,y,y')=0$. $$J_1(x,y,y')=\large \int_{x_0}^{x_1}F(x,y,y')+ \lambda (x) G(x,y,y')dx$$ I understand the Euler-Lagrange equation and Lagrange multipliers in multivariable (i.e. not variational)…
Meow
  • 5,853
  • 6
  • 38
  • 59
20
votes
4 answers

Is it possible for the Lagrange multiplier to be equal to zero?

I would like to find the extrema of the function $f(x,y)=x^2+4xy+4y^2$ subject to $x^2+2y^2=4$ using Lagrange Multipliers. Is it possible to get for the Lagrange multipliers the value zero? I don't think so because the gradient vectors for $f$ and…
19
votes
3 answers

Pointwise infimum of affine functions is concave

So I was just starting on convex optimization and was having a slightly hard time visualizing the lagrangian being always concave because it is the pointwise infimum of a family of affine functions. Can anyone help explain this? I've googled…
Achint
  • 321
  • 1
  • 2
  • 6
17
votes
0 answers

Lagrange multipliers in Calculus of Variations

I am trying to learn about Calculus of Variations and I am beginning to see some constrained optimization problems in the domain of functionals, by using Lagrange multipliers. It seems that things work like in the finite-dimensional calculus but I…
15
votes
1 answer

Calculus of variations: Lagrange multipliers

Given a functional $$J(y)=\int_a^b F(x,y,y')dx, \tag{1}$$ where $y$ is a function of $x$, and a constraint $$\int_a^b K(x,y,y')dx=l, \tag{2}$$ if $y=y(x)$ is an extreme of (1) under the constraint (2), then there exists a constant $\lambda$…
MIMIGA
  • 911
  • 2
  • 8
  • 16
15
votes
3 answers

Intuition behind Lagrange multiplier

I noticed that all attempts of showcasing the intuition behind Lagrange's multipliers basically resort to the following example (taken from Wikipedia): The reason why such examples make sense is that the level curves of the f function are either…
13
votes
2 answers

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with penalty definition: $$\min\limits_{\beta}…
Timespace
  • 369
  • 1
  • 2
  • 11
13
votes
4 answers

Lagrange multipliers and KKT conditions - what do we gain?

I'm working through an optimization problem that reformulates the problem in terms of KKT conditions. Can someone please have a go at explaining the following in simple terms? What do we gain by rewriting an optimization problem in terms of KKT…
Flash
  • 639
  • 1
  • 6
  • 13
13
votes
7 answers

How to prove the inequality $ \frac{a}{\sqrt{1+a}}+\frac{b}{\sqrt{1+b}}+\frac{c}{\sqrt{1+c}} \ge \frac{3\sqrt{2}}{2}$

How to prove the inequality $$ \frac{a}{\sqrt{1+a}}+\frac{b}{\sqrt{1+b}}+\frac{c}{\sqrt{1+c}} \ge \frac{3\sqrt{2}}{2}$$ for $a,b,c>0$ and $abc=1$? I have tried prove $\frac{a}{\sqrt{1+a}}\ge \frac{3a+1}{4\sqrt{2}}$ Indeed,$\frac{{{a}^{2}}}{1+a}\ge…
1
2 3
99 100