Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3392 questions
19
votes
2 answers

Derivatives of eigenvalues

Say I have a hermitian matrix A with elements $A_{ij}$, given the eigenvalues $e_p$, (and optional eigenvectors $v_p$) Is there an easy(ish) way to calculate $\frac{\partial e_p}{\partial A_{ij}}$? The elements represent potentials between…
byo
  • 193
  • 1
  • 1
  • 6
19
votes
3 answers

Derivative of a vector with respect to a matrix

let $W$ be a $n\times m$ matrix and $\textbf{x}$ be a $m\times1$ vector. How do we calculate the following then? $$\frac{dW\textbf{x}}{dW}$$ Thanks in advance.
arindam mitra
  • 1,361
  • 1
  • 10
  • 17
18
votes
3 answers

Matrix exponential of a skew-symmetric matrix without series expansion

I have the following skew-symmetric matrix $$C = \begin{bmatrix} 0 & -a_3 & a_2 \\ a_3 & 0 & -a_1 \\ -a_2 & a_1 & 0 \end{bmatrix}$$ How do I compute $e^{C}$ without resorting to the series expansion…
17
votes
2 answers

Convexity of matrix exponential

Consider the matrix-valued function $A: \mathbb{R}^n \rightarrow \mathbb{R}^{n \times n}$ defined as $$ A( x ) := \left[ \begin{matrix} x_1 & a_{1,2} & \cdots & a_{1,n} \\ a_{2,1} & x_2 & \cdots & a_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\…
17
votes
2 answers

$\operatorname{rank} A = \operatorname{rank} A^2$ if and only if $\lim_{\lambda \to 0} (A+\lambda I)^{-1}A$ exists

Let $A \in \mathbb C^{n \times n}$. Prove that $\operatorname{rank} A = \operatorname{rank} A^2$ if and only if $\displaystyle\lim_{\lambda \to 0} (A+\lambda I)^{-1}A$ exists. I am stuck on this problem, I don't understand what the limit is…
mysatellite
  • 3,802
  • 3
  • 20
  • 35
17
votes
5 answers

Derivative of matrix exponential w.r.t. to each element of the matrix

I have $x= \exp(At)$ where $A$ is a matrix. I would like to find derivative of $x$ with respect to each element of $A$. Could anyone help with this problem?
16
votes
4 answers

Matrix Calculus in Least-Square method

In the proof of matrix solution of Least Square Method, I see some matrix calculus, which I have no clue. Can anyone explain to me or recommend me a good link to study this sort of matrix calculus? In Least-Square method, we want to find such a…
16
votes
4 answers

Second derivative of $\det(\mathbb{1}+tA)$?

I am given a function $F : \mathbb{R} \to \mathbb{R}$ defined by $$F(t)=\det(\mathbb{1}+tA)$$ where $A \in \mathbb{R}^{n \times n}$. As far as I know, the following is true. $$\frac{d}{dt}\bigg|_{t=0} F(t) = \text{tr}~ A$$ However, how to find the…
16
votes
3 answers

Must a matrix of which all conjugates have zero diagonal be zero?

Let $A$ be an $n \times n$ real matrix with the following property: All the conjugates of $A$ have only zeros on the diagonal. Does $A=0$? (By conjugates, I mean all the matrices similar to it, over $\mathbb{R}$, that is I require the conjugating…
Asaf Shachar
  • 23,159
  • 5
  • 19
  • 106
15
votes
2 answers

Given a unitary matrix $U$, how do I find $A$ such that $U=e^{iA}$?

A unitary matrix $U \in \mathbb C^{n \times n}$ can always be written in exponential form $$U = e^{iA} \tag{1}$$ where $A$ is Hermitian. My goal is to find the Hermitian matrix $A$, given the unitary matrix $U$. I figured out a way by diagonalizing…
14
votes
4 answers

Is the determinant differentiable?

I was wondering, given an $n \times n$ square matrix, let function $\det : \left(a_1,a_2,\ldots,a_{n^2}\right) \to \textbf{R}$ give the determinant, where $a_{k}$'s are the entries of the $n \times n$ matrix. Is this function (determinant) a…
14
votes
4 answers

Matrix derivative $(Ax-b)^T(Ax-b)$

I am trying to find the minimum of $(Ax-b)^T(Ax-b)$ but I am not sure whether I am taking the derivative of this expression properly. What I did is the following: \begin{align*} \frac{\delta}{\delta x_i}\left(\sum_i \sum_j…
dreamer
  • 3,099
  • 6
  • 42
  • 73
13
votes
2 answers

A matrix inequality involving trace norm of a matrix and its inverse

Let $A,B \succeq 0$ be two positive semidefinite matrices. Can we get a closed form expression for the following quantity? $$ \inf_{X \succ 0} \mathrm{tr}(XA) + \mathrm{tr}(X^{-1}B) $$ We assume all matrices involved are symmetric.
13
votes
0 answers

Non-trivial isometries of the left invariant metric on $GL_n$

Let $GL_n^+$ be the group of $n \times n$ real invertible matrices with positive determinant. Let $g$ be the left-invariant Riemannian metric on $GL_n^+$ obtained by left translating the standard Euclidean inner product (Frobenius) on $T_IGL_n^+…
12
votes
4 answers

Prove that $\displaystyle\lim_{k \to \infty} \left( I + \frac{1}{k}A \right)^{k} = e^A$

I'm having a little trouble here to prove the following statement: Let $A$ be an $n \times n$ matrix (real or complex). Prove that $$\lim_{k \to \infty} \left( I + \frac{1}{k} A \right)^{k} = e^{A}$$ Now I'm using matrix and possible…