# Questions tagged [sparsity]

53 questions

**14**

votes

**2**answers

### Are greedy methods such as orthogonal matching pursuit considered obsolete for finding sparse solutions?

When researchers first began seeking sparse solutions to $Ax = b$, they used greedy methods such as orthogonal matching pursuit (OMP). In OMP, we activate components of $x$ one by one, and at each stage we select the component $i$ such that the…

littleO

- 48,104
- 8
- 84
- 154

**9**

votes

**2**answers

### How Can $ {L}_{1} $ Norm Minimization with Linear Equality Constraints (Basis Pursuit / Sparse Representation) Be Formulated as Linear Programming?

Problem Statement
Show how the $L_1$-sparse reconstruction problem:
$$\min_{x}{\left\lVert x\right\rVert}_1 \quad \text{subject to} \; y=Ax$$
can be reduced to a linear programming problem of form similar to:
$$\min_{u}{b^Tu} \quad \text{subject to}…

p.koch

- 526
- 1
- 5
- 9

**7**

votes

**1**answer

### Sparse PCA vs Orthogonal Matching Pursuit

Can't wrap my head around the difference between Sparse PCA and OMP.
Both try to find a sparse linear combination.
Of course, the optimization criteria is different.
In Sparse PCA we have:
\begin{aligned} \max & x^{T} \Sigma x \\ \text { subject to…

Natan ZB

- 83
- 5

**5**

votes

**1**answer

### Sparse Approximation in the Mahalanobis Distance

Given a vector $z \in \mathbb{R}^n$ and $k < n$, finding the best $k$-sparse approximation to $z$ in terms of the Euclidean distance means solving
$$\min_{\{x \in \mathbb{R}^n : ||x||_0 \le k\}} ||z - x||_2$$ This can easily be done by choosing $x$…

Claudio Moneo

- 1,856
- 1
- 5
- 16

**5**

votes

**0**answers

### Controlling the number of nonzero components in the LASSO solution

Let $A$ be a real $m \times n$ matrix. The Lasso optimization problem is
$$
\text{minimize} \quad \frac12 \| Ax - b \|_2^2 + \lambda \| x \|_1
$$
The optimization variable is $x \in \mathbb R^n$.
The $\ell_1$-norm regularization term encourages…

littleO

- 48,104
- 8
- 84
- 154

**4**

votes

**1**answer

### Finding a sparse solution to $A x = b$ via linear programming

I'm trying to solve a system $Ax = b$ where all entries of $x$ are nonnegative, and most are zero. So if $x$ has $N$ entries, then $\epsilon N$ of them are nonzero, where $\epsilon > 0$ is some small constant. Is it possible to use linear…

Alex

- 239
- 2
- 11

**3**

votes

**2**answers

### Finding the unit vector minimizing the sum of the absolute values of the projections of a set of points

Consider
$$
\min_{\mathbf{w} \in \mathbb{R}^d} \|\mathbf{X}^T\mathbf{w}\|_1 \qquad\text{subject to } \quad \|\mathbf{w}\|_2^2=1,
$$
where $\mathbf{X}\in\mathbb{R}^{d\times m}$ is a set of $d$-dimensional points and $m > d$.
How can I solve this…

Bernard Ghanem

- 61
- 4

**3**

votes

**2**answers

### Stability of the Solution of $ {L}_{1} $ Regularized Least Squares (LASSO) Against Inclusion of Redundant Elements

The problem of finding
$$ \substack{{\rm min}\\x}\left( \|Ax-b\|^2_2+\lambda \|x\|_1\right),$$
where $\|\cdot\|_2$ and $\|\cdot\|_1$ are the $L_2$ and $L_1$ norms, respectively, is usually called the LASSO. $A$ is a matrix, $x$ and $b$ are…

thedude

- 1,647
- 7
- 15

**2**

votes

**0**answers

### Newton Polytope of a symmetric polynomial with few vertices

For an $n$-variate polynomial $f = \sum_{a_1,\dotsc,a_n} x_1^{a_1}x_2^{a_2} \cdots x_n^{a_n}$, its Newton polytope $P_f$ is defined as the convex hull of all exponent vectors in the support of $f$. There are known examples where number of vertices…

Pranav Bisht

- 325
- 1
- 12

**2**

votes

**0**answers

### Orthogonal projection into a sparse subspace with $s$ dimension

Traditional orthogonal projection of a given point $y \in \mathbb{R}^n$ into a closed and convex set $D\in \mathbb{R}^n$ is defined as the follwing:
$$
P_D(y)=\arg\min_{x \in D}||x-y||_2^2
$$
Now suppose one wants to find the orthogonal projection…

Sepide

- 881
- 5
- 11

**2**

votes

**1**answer

### Efficiently enumerate the boolean matrix of sparsity k such that no rows and columns are all 0s.

Is there a way to enumerate the boolean matrix of $k$ entries of $1$ with no rows and columns all being $0$s?
e.g.
$k=1$, $\begin{bmatrix}1\end{bmatrix}$
$k=2$, $\begin{bmatrix}1 & 1\end{bmatrix}$
$\begin{bmatrix}1 \\…

Chen Xu

- 25
- 4

**2**

votes

**1**answer

### References for finding sparse solutions of an unconstrained non-convex optimization problem.

Given a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ where $n$ is large and $f$ is non-convex. The following characterises the sparse minimizer of $f$.
$$
x^* = \arg \min_{x} f + ||x||_0
$$
where $||x||_0$ is the number of nonzero elements of…

Saeed

- 4,031
- 1
- 8
- 23

**2**

votes

**1**answer

### If $ {L}_{0} $ Regularization Can be Done via the Proximal Operator, Why Are People Still Using LASSO?

I have just learned that a general framework in constrained optimization is called "proximal gradient optimization". It is interesting that the $\ell_0$ "norm" is also associated with a proximal operator. Hence, one can apply iterative hard…

ArtificiallyIntelligence

- 641
- 3
- 14

**2**

votes

**1**answer

### Sparsity of the inverse of a non-negative matrix

Let $A \in \mathbb{R}^{b \times b}$ be a matrix with non-negative entries. Suppose that $A$ is invertible. Further suppose that all the diagonal entries of $A$ are non-null. My claim is the following:…

pulosky

- 595
- 2
- 15

**2**

votes

**4**answers

### Non-sparse solution for a linear programming problem

I have formulated a linear program with equality, inequality and non-negativity constraints. My objective function is the minimization of a linear combination of decision variables (with different coefficients for different variables). I have 7803…

Fred

- 23
- 3