Questions tagged [non-convex-optimization]

A non-convex optimization problem is one where either the objective function is non-convex in a minimization problem (or non-concave in a maximization problem) or where the feasible region is not convex.

569 questions
29
votes
3 answers

What is the definition of a first order method?

The term "first order method" is often used to categorize a numerical optimization method for say constrained minimization, and similarly for a "second order method". I wonder what is the exact definition of a first (or second) order method. Does…
22
votes
1 answer

Gradient descent on non-convex function works. How?

For Netflix Prize competition on recommendations one method used a stochastic gradient descent, popularized by Simon Funk who used it to solve an SVD approximately. The math is better explained here on pg 152. A rating is predicted by…
15
votes
3 answers

How do I solve $\min_x \max(c_1^Tx, c_2^Tx, \dots, c_k^Tx)$ for $\lVert x \rVert_2 = 1$.

Let $f(x) = \max(c_1^Tx, c_2^Tx, \dots, c_k^Tx)$. where $x, c_1, c_2, \dots, c_k \in \mathbb R^n$. What fast iterative methods are available for finding the (approximate) min of $f$ with the constraint $\lVert x \rVert_2 = 1$? Notes: $f$ is convex…
14
votes
7 answers

Maximize the value of $v^{T}Av$

Let $A$ be a symmetric, real matrix. The goal is to find a unit vector $v$ such that the value $v^{T}Av$ is maximized, and minimized. The answer is that $v$ should be the eigenvector of $A$ with largest eigenvalue, and smallest eigenvalue.…
13
votes
1 answer

Why is the non-negative matrix factorization problem non-convex?

Supposing $\mathbf{X}\in\mathbb{R}_+^{m\times n}$, $\mathbf{Y}\in\mathbb{R}_+^{m\times r}$, $\mathbf{W}\in\mathbb{R}_+^{r\times n}$, the non-negative matrix factorization problem is defined…
13
votes
3 answers

Solve least-squares minimization from overdetermined system with orthonormal constraint

I would like to find the rectangular matrix $X \in \mathbb{R}^{n \times k}$ that solves the following minimization problem: $$ \mathop{\text{minimize }}_{X \in \mathbb{R}^{n \times k}} \left\| A X - B \right\|_F^2 \quad \text{ subject to } X^T X =…
11
votes
2 answers

Linear programming with one quadratic equality constraint

I have a problem that can be formulated as a linear program with one quadratic equality constraint: where variable $x$ is an $n$-dimensional vector and $H$ is a positive semidefinite $n \times n$ matrix. I know this optimization problem can always…
11
votes
0 answers

Balanced linear partitioning of a set of points in $R^d$

Suppose we have a set of points in $R^d$ and for a given constant $\epsilon>0$ we want to find a hyperplane such that it divides the dataset into two balanced partitions, and that the number of points that are $\epsilon$-close the hyperplane is…
11
votes
3 answers

Stable strict local minimum implies local convexity

Let $\bar{x}\in\mathbb{R}^n$ and $f:\mathbb{R}^n\rightarrow\mathbb{R}$ be a $C^2$ function. We have known that if $\nabla f(\bar{x})=0$ and $\nabla^2f(\bar{x})>0$, i.e. $\nabla^2f(\bar{x})$ is positive definite, then $\bar{x}$ is a strict local…
10
votes
3 answers

Box-constrained orthogonal matrix

Given constants $\ell, u \in \mathbb{R}^{3 \times 3}$ and the following system of constraints in $P \in \mathbb{R}^{3 \times 3}$ $$ P^T P = I_{3 \times 3},\quad \ell_{ij} \leq P_{ij} \leq u_{ij}, $$ I would like to find a matrix $P$ which satisfies…
9
votes
0 answers

Does there exist a strongly convex function that is strongly convex with respect to norm $\|\cdot\|_p$ for any $p > 2$?

A function $f$ is said to be strongly convex with respect to a norm $\|\cdot\|_p$ if for all $x,y$, $$f(x) \geq f(y) + \nabla f(y)^T(x-y) + \frac{1}{2}\|x-y\|^2_p.$$ There are a bunch of functions used in machine learning, statistics, etc. that are…
7
votes
1 answer

Maximization of complex quadratic function with quadratic and cubic equality constraints

I am trying to solve $$\max_{g_0,...,g_{M-1}} \left|\sum_{m=0}^{M-1} g_m\right|^2\ \text{s.t.}\ \sum_{m=0}^{M-1} |g_m|^2=M,\ \sum_{m=0}^{M-1} g_m |g_m|^2=0.$$ where $g_m$ can be complex. The problem does not seem easy to solve for me. Intuitively, I…
7
votes
1 answer

How to find the maximum of $\boldsymbol{x}^T \boldsymbol{A} \boldsymbol{x}$ subject to $\boldsymbol{q}^T \boldsymbol{x}=1$?

I want to solve the following problem in $\boldsymbol{x} \in \mathbb R^{n}$ $$\begin{array}{ll} \text{maximize} & \boldsymbol{x}^T \boldsymbol{A} \boldsymbol{x}\\ \text{subject to} & \boldsymbol{q}^T \boldsymbol{x} = 1\\ & x_i \geq…
6
votes
2 answers

Minimize the maximum inner product with vectors in a given set

Given a finite set $S$ of non-negative unit vectors in $\mathbb R_+^n$, find a non-negative unit vector $x$ such that the largest inner product of $x$ and a vector $v \in S$ is minimized. That is, $$ \min_{x\in \mathbb R_+^n,\|x\|_2=1}\max_{v\in S}…
6
votes
2 answers

Question about KKT conditions and strong duality

I am confused about the KKT conditions. I have seen similar questions asked here, but I think none of the questions/answers cleared up my confusion. In Boyd and Vandenberghe's Convex Optimization [Sec 5.5.3] , KKT is explained in the following…
1
2 3
37 38