A non-convex optimization problem is one where either the objective function is non-convex in a minimization problem (or non-concave in a maximization problem) or where the feasible region is not convex.

# Questions tagged [non-convex-optimization]

569 questions

**29**

votes

**3**answers

### What is the definition of a first order method?

The term "first order method" is often used to categorize a numerical optimization method for say constrained minimization, and similarly for a "second order method".
I wonder what is the exact definition of a first (or second) order method.
Does…

Wazowski

- 457
- 1
- 4
- 10

**22**

votes

**1**answer

### Gradient descent on non-convex function works. How?

For Netflix Prize competition on recommendations one method used a stochastic gradient descent, popularized by Simon Funk who used it to solve an SVD approximately. The math is better explained here on pg 152. A rating is predicted by…

BBSysDyn

- 15,011
- 7
- 35
- 54

**15**

votes

**3**answers

### How do I solve $\min_x \max(c_1^Tx, c_2^Tx, \dots, c_k^Tx)$ for $\lVert x \rVert_2 = 1$.

Let $f(x) = \max(c_1^Tx, c_2^Tx, \dots, c_k^Tx)$.
where $x, c_1, c_2, \dots, c_k \in \mathbb R^n$.
What fast iterative methods are available for finding the (approximate) min of $f$ with the constraint $\lVert x \rVert_2 = 1$?
Notes:
$f$ is convex…

douglasquaid84

- 173
- 2
- 12

**14**

votes

**7**answers

### Maximize the value of $v^{T}Av$

Let $A$ be a symmetric, real matrix. The goal is to find a unit vector $v$ such that the value $v^{T}Av$ is
maximized, and
minimized.
The answer is that $v$ should be the eigenvector of $A$ with
largest eigenvalue, and
smallest eigenvalue.…

user4205580

- 1,883
- 1
- 31
- 47

**13**

votes

**1**answer

### Why is the non-negative matrix factorization problem non-convex?

Supposing $\mathbf{X}\in\mathbb{R}_+^{m\times n}$, $\mathbf{Y}\in\mathbb{R}_+^{m\times r}$, $\mathbf{W}\in\mathbb{R}_+^{r\times n}$, the non-negative matrix factorization problem is defined…

no_name

- 405
- 1
- 5
- 9

**13**

votes

**3**answers

### Solve least-squares minimization from overdetermined system with orthonormal constraint

I would like to find the rectangular matrix $X \in \mathbb{R}^{n \times k}$ that solves the following minimization problem:
$$
\mathop{\text{minimize }}_{X \in \mathbb{R}^{n \times k}} \left\| A X - B \right\|_F^2 \quad \text{ subject to } X^T X =…

Alec Jacobson

- 424
- 2
- 13

**11**

votes

**2**answers

### Linear programming with one quadratic equality constraint

I have a problem that can be formulated as a linear program with one quadratic equality constraint:
where variable $x$ is an $n$-dimensional vector and $H$ is a positive semidefinite $n \times n$ matrix.
I know this optimization problem can always…

user123346

- 111
- 1
- 3

**11**

votes

**0**answers

### Balanced linear partitioning of a set of points in $R^d$

Suppose we have a set of points in $R^d$ and for a given constant $\epsilon>0$ we want to find a hyperplane such that it divides the dataset into two balanced partitions, and that the number of points that are $\epsilon$-close the hyperplane is…

kvphxga

- 533
- 3
- 15

**11**

votes

**3**answers

### Stable strict local minimum implies local convexity

Let $\bar{x}\in\mathbb{R}^n$ and $f:\mathbb{R}^n\rightarrow\mathbb{R}$ be a $C^2$ function.
We have known that if $\nabla f(\bar{x})=0$ and $\nabla^2f(\bar{x})>0$, i.e. $\nabla^2f(\bar{x})$ is positive definite, then $\bar{x}$ is a strict local…

Blind

- 982
- 4
- 22

**10**

votes

**3**answers

### Box-constrained orthogonal matrix

Given constants $\ell, u \in \mathbb{R}^{3 \times 3}$ and the following system of constraints in $P \in \mathbb{R}^{3 \times 3}$
$$
P^T P = I_{3 \times 3},\quad \ell_{ij} \leq P_{ij} \leq u_{ij},
$$
I would like to find a matrix $P$ which satisfies…

Alex Shtof

- 267
- 6
- 20

**9**

votes

**0**answers

### Does there exist a strongly convex function that is strongly convex with respect to norm $\|\cdot\|_p$ for any $p > 2$?

A function $f$ is said to be strongly convex with respect to a norm $\|\cdot\|_p$ if for all $x,y$, $$f(x) \geq f(y) + \nabla f(y)^T(x-y) + \frac{1}{2}\|x-y\|^2_p.$$
There are a bunch of functions used in machine learning, statistics, etc. that are…

Norman

- 966
- 10
- 24

**7**

votes

**1**answer

### Maximization of complex quadratic function with quadratic and cubic equality constraints

I am trying to solve
$$\max_{g_0,...,g_{M-1}} \left|\sum_{m=0}^{M-1} g_m\right|^2\ \text{s.t.}\ \sum_{m=0}^{M-1} |g_m|^2=M,\ \sum_{m=0}^{M-1} g_m |g_m|^2=0.$$
where $g_m$ can be complex. The problem does not seem easy to solve for me.
Intuitively, I…

François2021

- 71
- 3

**7**

votes

**1**answer

### How to find the maximum of $\boldsymbol{x}^T \boldsymbol{A} \boldsymbol{x}$ subject to $\boldsymbol{q}^T \boldsymbol{x}=1$?

I want to solve the following problem in $\boldsymbol{x} \in \mathbb R^{n}$
$$\begin{array}{ll} \text{maximize} & \boldsymbol{x}^T \boldsymbol{A} \boldsymbol{x}\\ \text{subject to} & \boldsymbol{q}^T \boldsymbol{x} = 1\\ & x_i \geq…

Kris Prokins

- 321
- 1
- 6

**6**

votes

**2**answers

### Minimize the maximum inner product with vectors in a given set

Given a finite set $S$ of non-negative unit vectors in $\mathbb R_+^n$, find a non-negative unit vector $x$ such that the largest inner product of $x$ and a vector $v \in S$ is minimized. That is,
$$
\min_{x\in \mathbb R_+^n,\|x\|_2=1}\max_{v\in S}…

cangrejo

- 1,169
- 1
- 6
- 12

**6**

votes

**2**answers

### Question about KKT conditions and strong duality

I am confused about the KKT conditions. I have seen similar questions asked here, but I think none of the questions/answers cleared up my confusion.
In Boyd and Vandenberghe's Convex Optimization [Sec 5.5.3] , KKT is explained in the following…

user2020

- 63
- 5