Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [convex-optimization]

A convex optimization problem consists of either minimizing a convex objective or maximizing a concave objective over a convex feasible region.

Convex Optimization is a special case of mathematical optimization where the feasible region is convex and the objective is to either minimize a convex function or maximize a concave function. Linear Programming is a special case. Convex Optimization problems as a class are easier to solve numerically than general mathematical optimization problems.

The following problems are all convex minimization problems, or can be transformed into convex minimizations problems via a change of variables:

  • Least squares
  • Linear programming
  • Convex quadratic minimization with linear constraints
  • Quadratic minimization with convex quadratic constraints
  • Conic optimization
  • Geometric programming
  • Second order cone programming
  • Semidefinite programming
  • Entropy maximization with appropriate constraints
6604 questions
272
votes
8 answers

Please explain the intuition behind the dual problem in optimization.

I've studied convex optimization pretty carefully, but don't feel that I have yet "grokked" the dual problem. Here are some questions I would like to understand more deeply/clearly/simply: How would somebody think of the dual problem? What…
littleO
  • 48,104
  • 8
  • 84
  • 154
146
votes
10 answers

The median minimizes the sum of absolute deviations (the $ {\ell}_{1} $ norm)

Suppose we have a set $S$ of real numbers. Show that $$\sum_{s\in S}|s-x| $$ is minimal if $x$ is equal to the median. This is a sample exam question of one of the exams that I need to take and I don't know how to proceed.
hattenn
  • 1,847
  • 3
  • 13
  • 11
60
votes
4 answers

Difference between supremum and maximum

Referring to this lecture , I want to know what is the difference between supremum and maximum. It looks same as far as the lecture is concerned when it explains pointwise supremum and pointwise maximum
user31820
  • 893
  • 2
  • 8
  • 12
53
votes
2 answers

What is the difference between minimum and infimum?

What is the difference between minimum and infimum? I have a great confusion about this.
Manoj
  • 1,617
  • 1
  • 14
  • 28
46
votes
7 answers

How do you prove that $\{ Ax \mid x \geq 0 \}$ is closed?

Let $A$ be a real $m \times n$ matrix. How do you prove that $\{ Ax \mid x \geq 0, x \in \mathbb R^n \}$ is closed (as in, contains all its limit points)? The inequality $x \geq 0$ is interpreted component-wise. This fact is used in some proofs…
littleO
  • 48,104
  • 8
  • 84
  • 154
45
votes
4 answers

Geometric intuition of conjugate function

I am looking for a geometric and intuitive explanation of the conjugate function and how it maps to the below analytical formula. $$ f^*(y)= \sup_{x \in \operatorname{dom} f } (y^Tx-f(x))$$
Abhishek Bhatia
  • 1,378
  • 7
  • 19
  • 35
32
votes
1 answer

Convexity of the product of two functions in higher dimensions

Exercise 3.32 page 119 of Convex Optimization is concerned with the proof that if $f:\mathbb{R}\rightarrow\mathbb{R}:x\mapsto f(x)$ and $g:\mathbb{R}\rightarrow\mathbb{R}:x\mapsto g(x)$ are both convex, nondecreasing (or nonincreasing) and positive,…
Anthony Labarre
  • 838
  • 1
  • 9
  • 14
26
votes
4 answers

Convexity and Affineness

In reading about convex optimization, the author states that all convex sets are affine. Are affinity and convexity equivalent? If I understand, both definitions incorporate the notion that a set is affine/convex iff for every two points in the…
analysisj
  • 2,600
  • 3
  • 24
  • 36
25
votes
4 answers

KKT and Slater's condition

I was studying Stephen Boyd's text book and got confused in the KKT part. The book says the following: "For any convex optimization problem with differentiable objective and constraint function, any points that satisfy the KKT conditions are primal…
DSKim
  • 987
  • 3
  • 12
  • 18
22
votes
4 answers

How the dual LP solves the primal LP

When I heard someone discussing LP the other day, I heard him say, "Well, we could just solve the dual." I know that both the primal LP and its dual must have the same optimal objective value (assuming both are feasible and bounded). I also…
user
  • 353
  • 1
  • 2
  • 14
20
votes
4 answers

Why is the affine hull of the unit circle $\mathbb R^2$?

In Boyd's "Convex Optimization" it defines the affine hull of a subset $C$ of $\mathbb R^n$ as $$\text{aff} C = \left\{\theta_1 x_1 + \ldots +\theta_k x_k \mid x_1, \ldots x_k \in C, \theta_1 + \ldots \theta_k = 1 \right\}.$$ Then, it claims…
Palace Chan
  • 1,215
  • 1
  • 11
  • 18
20
votes
5 answers

Why is convexity more important than quasi-convexity in optimization?

In the mathematical optimization literature it is common to distinguish problems according to whether or not they are convex. The reason seems to be that convex problems are guaranteed to have globally optimal solutions, so one can use methods such…
Amelio Vazquez-Reina
  • 819
  • 6
  • 16
19
votes
3 answers

Pointwise infimum of affine functions is concave

So I was just starting on convex optimization and was having a slightly hard time visualizing the lagrangian being always concave because it is the pointwise infimum of a family of affine functions. Can anyone help explain this? I've googled…
Achint
  • 321
  • 1
  • 2
  • 6
19
votes
1 answer

How many convex functions are there in $[0,1]^2$?

Consider the square $[0,1]^2$. Assume that this region is divided into $N=K^2$ equispaced grid points. How many convex curves can be drawn in terms of $K$? The points $(0,1)$ and $(1,0)$ are known to be on the convex curve. I am interested in the…
Seyhmus Güngören
  • 7,666
  • 3
  • 24
  • 43
19
votes
2 answers

Convex analysis books and self study.

I have taken some courses in Convex optimization. Now I would like to know a little bit more about the pure mathematical side. Is there any good books in convex analysis? I have read and worked with Boyds Convex Optimization book. Is there any video…
Johan
  • 3,682
  • 1
  • 22
  • 53
1
2 3
…
99 100