Questions tagged [numerical-linear-algebra]

Questions on the various algorithms used in linear algebra computations (matrix computations).

Questions tagged with this tag can be about, but not limited to:

  1. Matrix decompositions like SVD, QR, Cholesky, etc.
  2. The solution of linear systems and least squares problems.
  3. Analysis of numerical linear algebra algorithms like condition numbers and stability analysis.
  4. Eigenvalue problems.
  5. The designs of direct or iterative methods to solve linear systems.
3203 questions
61
votes
2 answers

Distance/Similarity between two matrices

I'm in the process of writing an application which identifies the closest matrix from a set of square matrices $M$ to a given square matrix $A$. The closest can be defined as the most similar. I think finding the distance between two given matrices…
Synex
  • 855
  • 2
  • 11
  • 15
40
votes
3 answers

block matrix multiplication

If $A,B$ are $2 \times 2$ matrices of real or complex numbers, then $$AB = \left[ \begin{array}{cc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array} \right]\cdot \left[ \begin{array}{cc} b_{11} & b_{12} \\ b_{21} & b_{22} \end{array} \right] =…
cactus314
  • 23,583
  • 4
  • 35
  • 88
25
votes
6 answers

Determinant of large matrices: there's GOTTA be a faster way

WARNING this is a very long report and is likely going to cause boredom. Be warned!! I've heard of the determinant of small matrices, such as: $$\det \begin{pmatrix} a&b\\ c&d\\ \end{pmatrix} = ad-bc $$ case in point: $$\det \begin{pmatrix}…
23
votes
4 answers

Fast algorithm for solving system of linear equations

I have a system of $N$ linear equations, $Ax=b$, in $N$ unknowns (where $N$ is large). If I am interested in the solution for only one of the unknowns, what are the best approaches? For example, assume $N=50,000$. We want the solution for $x_1$…
mghandi
  • 1,732
  • 1
  • 11
  • 26
20
votes
4 answers

Determining whether a symmetric matrix is positive-definite (algorithm)

I'm trying to create a program, that will decompose a matrix using the Cholesky decomposition. The decomposition itself isn't a difficult algorithm, but a matrix, to be eligible for Cholesky decomposition, must be symmetric and positive-definite.…
19
votes
4 answers

How to compute the smallest eigenvalue using the power iteration algorithm?

I need to write a program which computes the largest and the smallest (in terms of absolute value) eigenvalues using both power iteration and inverse iteration. I can find them using the inverse iteration, and I can also find the largest one using…
18
votes
3 answers

Fast computation/estimation of the nuclear norm of a matrix

The nuclear norm of a matrix is defined as the sum of its singular values, as given by the singular value decomposition (SVD) of the matrix itself. It is of central importance in Signal Processing and Statistics, where it is used for matrix…
18
votes
2 answers

The benefit of LU decomposition over explicitly computing the inverse

I'm going to teach a linear algebra course in the fall, and I want to motivate the topic of matrix factorizations such as the LU decomposition. A natural question one can ask is, why care about this when one already knows how to compute $A^{-1}$ and…
15
votes
2 answers

Uniform sampling of points on a simplex

I have this problem: I'm trying to sample the relation $$ \sum_{i=1}^N x_i = 1 $$ in the domain where $x_i>0\ \forall i$. Right now I'm just extracting $N$ random numbers $u_i$ from a uniform distribution $[0,1]$ and then I transform them into $x_i$…
14
votes
1 answer

Using the Arnoldi Iteration to find the k largest eigenvalues of a matrix

I'm trying to obtain a general understanding of this algorithm which determines the k-largest eigenvalues of a matrix $A\in \mathbb{R}^{n\times n}$. How I see it: power iteration: take random starting vector $b \in \mathbb{R}^{1\times n}$ find…
14
votes
3 answers

What is the most efficient way to find the inverse of large matrix?

Let $A$ be a large square $(n+1) \times (n+1)$ invertible matrix, where $n \approx 1000$. $$A = \begin{bmatrix} -1 & 0 & 0 &\cdots & 0 & a_0\\ 1 & -1 & 0 &\cdots & 0 & a_1\\ 0 & 1 & -1 &\cdots & 0 & a_2\\ \vdots & \vdots & \vdots &\ddots & \vdots &…
13
votes
1 answer

Computing the SVD factorization on C++ (using the proof of the existence of the SVD factorization)

I am doing a C++ program that computes the SVD factorization of a real matrix A without using any known library of algebra that contains the implementation. In addition, QR descomposition is not allowed in any step (because one of my goals is to…
13
votes
2 answers

How to get the SVD of $2AA^T-\operatorname{diag}(AA^T)$ given $A$ and its SVD $A=USV^T$?

Given a matrix $A\in R^{n\times d}$ with $n>d$, and we can have some fast ways to (approximately) calculate the SVD (Singular Value Decomposition) of $A$, saying $A=USV^T$ and $V\in R^{d\times d}$. It is straightforward to know that the SVD of…
olivia
  • 1,438
  • 1
  • 11
  • 22
13
votes
3 answers

What is the minimum and maximum number of eigenvectors?

I am given the eigenvalues of a square, 8x8, matrix. They are all non-zero. I have determined that the matrix is diagonalizable and has an inverse. In one part of the problem, I am asked to find the maximum and minimum number of eigenvectors that…
Ayoshna
  • 1,313
  • 3
  • 24
  • 51
13
votes
1 answer

Numerically stable method for angle between 3D vectors

I'm looking for a numerically stable method for computing the angle between two 3D vectors. Which of the following methods ought to be preferred? Method 1: $$ u\times v = ||u||~||v|| \sin(\theta) \textbf{n}\\ u\cdot v = ||u||~||v||…
Murphy
  • 131
  • 1
  • 3
1
2 3
99 100