Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3392 questions
216
votes
6 answers

Why does this matrix give the derivative of a function?

I happened to stumble upon the following matrix: $$ A = \begin{bmatrix} a & 1 \\ 0 & a \end{bmatrix} $$ And after trying a bunch of different examples, I noticed the following remarkable pattern. If $P$ is a polynomial,…
ASKASK
  • 8,711
  • 4
  • 24
  • 44
201
votes
6 answers

How could we define the factorial of a matrix?

Suppose I have a square matrix $\mathsf{A}$ with $\det \mathsf{A}\neq 0$. How could we define the following operation? $$\mathsf{A}!$$ Maybe we could make some simple example, admitted it makes any sense, with $$\mathsf{A} = \left(\begin{matrix} 1…
Laplacian
  • 23,646
  • 9
  • 34
  • 53
81
votes
2 answers

Factorial of a matrix: what could be the use of it?

Recently on this site, the question was raised how we might define the factorial operation $\mathsf{A}!$ on a square matrix $\mathsf{A}$. The answer, perhaps unsurprisingly, involves the Gamma function. What use might it be to take the factorial of…
Oliphaunt
  • 943
  • 6
  • 9
64
votes
5 answers

How to take the gradient of the quadratic form?

It's stated that the gradient of: $$\frac{1}{2}x^TAx - b^Tx +c$$ is $$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$ How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?
victor
  • 753
  • 1
  • 6
  • 6
59
votes
4 answers

Derivative of the inverse of a matrix

In a scientific paper, I've seen the following $$\frac{\delta K^{-1}}{\delta p} = -K^{-1}\frac{\delta K}{\delta p}K^{-1}$$ where $K$ is a $n \times n$ matrix that depends on $p$. In my calculations I would have done the following $$\frac{\delta…
Sara
  • 797
  • 1
  • 7
  • 8
46
votes
4 answers

Vector derivative w.r.t its transpose $\frac{d(Ax)}{d(x^T)}$

Given a matrix $A$ and column vector $x$, what is the derivative of $Ax$ with respect to $x^T$ i.e. $\frac{d(Ax)}{d(x^T)}$, where $x^T$ is the transpose of $x$? Side note - my goal is to get the known derivative formula $\frac{d(x^TAx)}{dx} =…
Asaf R
  • 571
  • 1
  • 5
  • 5
41
votes
5 answers

Integral of matrix exponential

Let $A$ be an $n \times n$ matrix. Then the solution of the initial value problem \begin{align*} \dot{x}(t) = A x(t), \quad x(0) = x_0 \end{align*} is given by $x(t) = \mathrm{e}^{At} x_0$. I am interested in the following matrix \begin{align*} …
samsa44
  • 431
  • 1
  • 4
  • 4
38
votes
2 answers

Taking a derivative with respect to a matrix

I'm studying about EM-algorithm and on one point in my reference the author is taking a derivative of a function with respect to a matrix. Could someone explain how does one take the derivative of a function with respect to a matrix...I don't…
jjepsuomi
  • 8,135
  • 12
  • 49
  • 89
36
votes
6 answers

Not understanding derivative of a matrix-matrix product.

I am trying to figure out a the derivative of a matrix-matrix multiplication, but to no avail. This document seems to show me the answer, but I am having a hard time parsing it and understanding it. Here is my problem: We have $\mathbf{D} \in \Re^{m…
Spacey
  • 927
  • 1
  • 12
  • 17
31
votes
4 answers

How to calculate the gradient of log det matrix inverse?

How to calculate the gradient with respect to $X$ of: $$ \log \mathrm{det}\, X^{-1} $$ here $X$ is a positive definite matrix, and det is the determinant of a matrix. How to calculate this? Or what's the result? Thanks!
pluskid
  • 1,059
  • 2
  • 9
  • 16
31
votes
5 answers

Derivative of Quadratic Form

For the Quadratic Form $X^TAX; X\in\mathbb{R}^n, A\in\mathbb{R}^{n \times n}$ (which simplifies to $\Sigma_{i=0}^n\Sigma_{j=0}^nA_{ij}x_ix_j$), I tried to take the derivative wrt. X ($\Delta_X X^TAX$) and ended up with the following: The $k^{th}$…
Praveen
  • 411
  • 1
  • 4
  • 3
27
votes
4 answers

Derivative of squared Frobenius norm of a matrix

In linear regression, the loss function is expressed as $$\frac1N \left\|XW-Y\right\|_{\text{F}}^2$$ where $X, W, Y$ are matrices. Taking derivative w.r.t $W$ yields $$\frac 2N \, X^T(XW-Y)$$ Why is this so?
wrek
  • 447
  • 1
  • 5
  • 24
26
votes
3 answers

If $\,A^k=0$ and $AB=BA$, then $\,\det(A+B)=\det B$

Assume that the matrices $A,\: B\in \mathbb{R}^{n\times n}$ satisfy $$ A^k=0,\,\, \text{for some $\,k\in \mathbb{Z^+}$}\quad\text{and}\quad AB=BA. $$ Prove that $$\det(A+B)=\det B.$$
Iloveyou
  • 2,443
  • 1
  • 13
  • 17
24
votes
3 answers

$AB=BA$ implies $AB^T=B^TA$ when $A$ is normal

I am looking for an elementary proof (if such exists) of the following: $$ AB=BA \quad\Longrightarrow\quad AB^T=B^TA, $$ where $A$ and $B$ are $n\times n$ real matrices, and $A$ is a normal matrix, i.e., $AA^T=A^TA$ - it is true for complex…
Yiorgos S. Smyrlis
  • 78,494
  • 15
  • 113
  • 210
21
votes
8 answers

Derivative of the nuclear norm

The nuclear norm is defined in the following way $$\|X\|_*=\mathrm{tr} \left(\sqrt{X^T X} \right)$$ I'm trying to take the derivative of the nuclear norm with respect to its argument $$\frac{\partial \|X\|_*}{\partial X}$$ Note that $\|X\|_*$ is a…
Alt
  • 2,375
  • 3
  • 19
  • 39
1
2 3
…
99 100