Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3392 questions
1
vote
2 answers

Gradient of scalar field $a^T X^{-1} b$

During the derivation of GDA as generative algorithm, I am stuck at how to take the gradient $$\nabla_X \left( a^TX^{-1}b \right)$$ where $a, b$ are column vectors independent of $X$. I have tried using trace operator and chain rule, but could not…
1
vote
2 answers

Limit of matrix to a power and diagonalized matrix to a power give different results

I have the following problem. I wish to find $$ \lim_{n \to \infty} \psi M^n 1 $$ where $$ \psi = \begin{bmatrix} 1/4 & 1/2 \end{bmatrix}, \quad 1=\begin{bmatrix} 1 & 1 \end{bmatrix}^T $$ and $$ M= \begin{bmatrix} \left(…
hawkjo
  • 433
  • 3
  • 7
1
vote
0 answers

Computing proximal mapping for a quadratic function

I am trying to calculate the proximal mapping of the quadratic function: $f(x)=\frac{1}{2}x^TAx+b^Tx+c$ , where $T$ is transpose. The solution is: $(I+tA)^{-1}(x-tb)$. taken from pg23 of:…
1
vote
1 answer

Matrix trace gradient

Given $A, B$ positive definite matrices and $f(M) = \textrm{tr}(A^{-1}M)+\textrm{tr}(M^{-1}B)$, what is $\nabla f(N)$ ? According to my source, $\nabla f(N)=A-N^{-1}BN^{-1}$. However, I would expect a vector. How can I compute the trace gradient and…
1
vote
0 answers

Matrix Calculus Reference Request

Looking for a good book on matrix calculus. I have seen the matrix cook book but it is mostly identities/equations without any proofs or work. I want a thorough treatment of the topic.
user3417
1
vote
1 answer

Second derivative of Frobenius norm

Let $A$ be a real-valued matrix and $$f(A) := \lVert A \rVert_F = \sqrt{A:A} = \sqrt{\mbox{tr}(A^TA)}$$ be its Frobenius norm. What is $\frac{\partial^2f}{\partial A^2}$? For the squared Frobenius norm, $f^2$, I found that…
blst
  • 1,291
  • 8
  • 15
1
vote
2 answers

If $A\in\mathbb C^{n\times n}\!$, $\,f\in\mathbb C[t]$ and $f(A)$ diagonalizable, and $f'(A)$ is invertible, then $A$ is diagonalizble.

Suppose that $A$ is a square complex matrix and $f$ is a polynomial in $\mathbb C[t]$ such that $f(A)$ is diagonalizable. If $f'(A)$ is invertible, where $f'$ is the derivative of $f$, prove that $A$ is diagonalizable in $\mathbb C$. My…
1
vote
1 answer

How to solve $F''_{w_{ab}}$? I want to know the detailed calculation process of solving $F''_{w_{ab}}$.

There is a objective function: $f(W)$ = $||W||_{2,1}$ For any element $w_{ab}$ in $W$, we apply $F_{w_{ab}}$ to denote the part of $f(W)$ which is only related to $w_{ab}$. $F'_{w_{ab}}=(DW)_{ab}$ $F''_{w_{ab}}=(D-D^{3}(W\odot W))_{aa}$, (the main…
1
vote
1 answer

Why a matrix-by-matrix derivative is actually a tensor?

For matrices $A$ and $B$, I thought $\frac{\partial A}{\partial B}$ is a matrix $C$ where $C_{ij} = \frac{\partial A_{ij}}{\partial B_{ij}}$. However, when I use this matrix calculus website, it says $$\frac{\partial{A}}{\partial{A}} =…
1
vote
0 answers

Gradient of $2$-norm inside absolute value

Compute the gradient of $$ z = | \lVert L x \lVert_2 - P | $$ where $ x \in \mathbb{C}^N $ and $ P \in \mathbb{R}_{+} $. First, I make $ a = \lVert L x \lVert_2 - P $. Also, since $ | a | $ is non-differentiable, its subdifferential is defined as…
Duns
  • 685
  • 5
  • 15
1
vote
2 answers

How to compute the derivative of $\frac{\partial AB^T }{\partial{A}}$ and $\frac{\partial AB^T }{\partial{B}}$

How to compute the derivative of $$\frac{\partial AB^T }{\partial{A}}$$ and $$\frac{\partial AB^T }{\partial{B}}$$ where $A \in R^{m \times n}$ and $B \in R^{r \times n}$. Also, how can we analysis the dimension of final result? such as…
jason
  • 753
  • 5
  • 14
1
vote
1 answer

Hessian of a $\mathbb{R}^{n \times m} \to \mathbb{R}$ function

I have a function $f : \mathbb{R}^{n \times m} \to \mathbb{R}$ whose input is denoted by $\bf U$. I know that $\nabla_{\mathbf{U}}f \in \mathbb{R}^{n \times m}$. I would like to compute $\nabla_{\mathbf{U}}^2 f$, but am currently stuck. For example,…
1
vote
2 answers

Question about Matrix Calculus and Baker–Campbell–Hausdorff Formula

In the book Ideas and Methods of Supersymmetry and Supergravity, the authors used the following formula $$e^{A+\epsilon B}=e^{A}\left(1+\int_{0}^{1}d\tau e^{-\tau A}\epsilon Be^{\tau A}\right), \tag{1}$$ where $\epsilon$ is an infinitesimal…
1
vote
1 answer

Compute integral about cosh

How to simplify following form? $$(2\pi)^{-d/2} \det{(\Sigma)}^{-1/2} e^{-\mu^T \Sigma^{-1} \mu} \int_{x \in R^d} e^{-\frac{1}{2} x^T \Sigma^{-1}x } \cosh(\mu^T \Sigma^{-1} x) \ln(\cosh(\mu^T \Sigma^{-1} x)) dx $$ where $x \in R^d$, $\Sigma \in…
1
vote
1 answer

Gradient and Hessian of $f(x,y) := a^T \left( x \odot \left[ \exp\left( \mu \ (y \ \oslash \ x) \right) - 1 \right] \right)$, wr.t. $x$ and $y$

How to find the Gradient and Hessian of \begin{align} f(x,y) := a^T \left( x \odot \left[ \exp\left( \mu \ (y \ \oslash \ x) \right) - 1 \right] \right) \ , \end{align} where $a, x, y \in \mathbb{R}^n$, all-ones vector $1 \in \mathbb{R}^n$, and…
learning
  • 557
  • 3
  • 12
1 2 3
99
100