How to calculate the gradient with respect to $X$ of: $$ \log \mathrm{det}\, X^{-1} $$ here $X$ is a positive definite matrix, and det is the determinant of a matrix.
How to calculate this? Or what's the result? Thanks!
How to calculate the gradient with respect to $X$ of: $$ \log \mathrm{det}\, X^{-1} $$ here $X$ is a positive definite matrix, and det is the determinant of a matrix.
How to calculate this? Or what's the result? Thanks!
I assume that you are asking for the derivative with respect to the elements of the matrix. In this cases first notice that
$$\log \det X^{-1} = \log (\det X)^{-1} = -\log \det X$$
and thus
$$\frac{\partial}{\partial X_{ij}} \log \det X^{-1} = -\frac{\partial}{\partial X_{ij}} \log \det X = - \frac{1}{\det X} \frac{\partial \det X}{\partial X_{ij}} = - \frac{1}{\det X} \mathrm{adj}(X)_{ji} = - (X^{-1})_{ji}$$
since $\mathrm{adj}(X) = \det(X) X^{-1}$ for invertible matrices (where $\mathrm{adj}(X)$ is the adjugate of $X$, see http://en.wikipedia.org/wiki/Adjugate).
Or you can check section A.4.1 of the book Stephen Boyd, Lieven Vandenberghe, Convex Optimization for an alternative solution, where they compute the gradient without using the adjugate.
The simplest is probably to observe that $$-\log\det (X+tH) = -\log\det X -\log\det(I+tX^{-1}H) \\= -\log\det X - t \textrm{Tr}(X^{-1}H) + o(t),$$
where is used the "obvious" fact that $\det(I+A) = 1+\textrm{Tr}(A)+o(|A|)$ (all the other terms are quadratic expressions of the coefficients of $A$).
Notice that $\textrm{Tr}(X^{-1}H)=(X^{-T},H)$ in the Frobenius scalar product, hence $\nabla [-\log\det(X)] = -X^{-T}$ in this scalar product. (This gives another proof that $\nabla\det (X) = cof(X)$.)
Of course if $X$ is symmetric positive definite then $-X^{-1}$ is also a valid expression. Moreover, one has in this case, for $X,Y$ positive definite, $(-X^{-1}+Y^{-1},X-Y)\ge 0$.
The answers given so far work only if $X \in \mathbb{R}^{n\times n}$ is not symmetric and has $n^2$ independent variables! If $X$ is symmetric, then it has only $n(n+1)/2$ independent variables and the correct formula is
$$\frac{\partial \log\det X^{-1}}{\partial X} = -\frac{\partial \log\det X}{\partial X} = -(2X^{-1}-\text{diag}(y_{11}, \dots, y_{nn})),$$
where $y_{ii}$ is the $i$ the entry on the diagonal of $X^{-1}$. This question explains why this is the case.