A scalar field is a function of the type $X\to \Bbb R$, where $X$ may be an open set in $\mathbb R^n$ or more generally a smooth manifold.

# Questions tagged [scalar-fields]

286 questions

**139**

votes

**13**answers

### Why is gradient the direction of steepest ascent?

$$f(x_1,x_2,\dots, x_n):\mathbb{R}^n \to \mathbb{R}$$
The definition of the gradient is
$$ \frac{\partial f}{\partial x_1}\hat{e}_1 +\ \cdots +\frac{\partial f}{\partial x_n}\hat{e}_n$$
which is a vector.
Reading this definition makes me consider…

Jing

- 1,999
- 4
- 17
- 27

**93**

votes

**5**answers

### Difference between gradient and Jacobian

Could anyone explain in simple words (and maybe with an example) what the difference between the gradient and the Jacobian is?
The gradient is a vector with the partial derivatives, right?

Math_reald

- 1,157
- 1
- 9
- 12

**67**

votes

**3**answers

### Gradient of 2-norm squared

Could someone please provide a proof for why the gradient of the squared $2$-norm of $x$ is equal to $2x$?
$$\nabla\|x\|_2^2 = 2x$$

user167133

- 793
- 1
- 6
- 4

**64**

votes

**5**answers

### How to take the gradient of the quadratic form?

It's stated that the gradient of:
$$\frac{1}{2}x^TAx - b^Tx +c$$
is
$$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$
How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?

victor

- 753
- 1
- 6
- 6

**38**

votes

**2**answers

### Taking a derivative with respect to a matrix

I'm studying about EM-algorithm and on one point in my reference the author is taking a derivative of a function with respect to a matrix. Could someone explain how does one take the derivative of a function with respect to a matrix...I don't…

jjepsuomi

- 8,135
- 12
- 49
- 89

**31**

votes

**4**answers

### How to calculate the gradient of log det matrix inverse?

How to calculate the gradient with respect to $X$ of:
$$
\log \mathrm{det}\, X^{-1}
$$
here $X$ is a positive definite matrix, and det is the determinant of a matrix.
How to calculate this? Or what's the result? Thanks!

pluskid

- 1,059
- 2
- 9
- 16

**27**

votes

**4**answers

### Derivative of squared Frobenius norm of a matrix

In linear regression, the loss function is expressed as
$$\frac1N \left\|XW-Y\right\|_{\text{F}}^2$$
where $X, W, Y$ are matrices. Taking derivative w.r.t $W$ yields
$$\frac 2N \, X^T(XW-Y)$$
Why is this so?

wrek

- 447
- 1
- 5
- 24

**25**

votes

**2**answers

### The gradient as a row versus column vector

Kaplan's Advanced Calculus defines the gradient of a function $f : \mathbb{R^n} \to \mathbb{R}$ as the $1 \times n$ row vector whose entries respectively contain the $n$ partial derivatives
of $f$. By this definition then, the gradient is just the…

ItsNotObvious

- 10,045
- 11
- 64
- 96

**19**

votes

**5**answers

### Differentiate $f(x)=x^TAx$

Calculate the differential of the function $f: \Bbb R^n \to \Bbb R$ given by $$f(x) = x^T A x$$ with $A$ symmetric. Also, differentiate this function with respect to $x^T$.
How exactly does this work in the case of vectors and matrices? Could…

dreamer

- 3,099
- 6
- 42
- 73

**12**

votes

**5**answers

### Proof for the funky trace derivative : $d (\operatorname{trace} (ABA'C))$?

Given three matrices $A$, $B$ and $C$ such that $ABA^T C$ is a square matrix, the derivative of the trace with respect to $A$ is:
$$
\nabla_A \operatorname{trace}( ABA^{T}C ) = CAB + C^T AB^T
$$
There is a proof here, page 4 (PDF file). However, I…

Harold

- 187
- 1
- 6

**11**

votes

**3**answers

### What is the derivative of $\log \det X$ when $X$ is symmetric?

According to Appendix A.4.1 of Boyd & Vandenberghe's Convex Optimization, the gradient of $f(X):=\log \det X$ is
$$\nabla f(X) = X^{-1}$$
The domain of the $f$ here is the set of symmetric matrices $\mathbf S^n$. However, according to the book…

evangelos

- 372
- 2
- 9

**11**

votes

**2**answers

### Derivative of quadratic matrix form with respect to the matrix

Suppose we have the following quadratic form
$$
f(M)=x^TMx
$$
where $f: \mathbb{R}^{n \times m} \rightarrow \mathbb{R}$, and $x \in \mathbb{R}^n$.
As it is obvious, this function is linear in $M$. What is the derivative of $f(M)$ with respect to…

Saeed

- 4,031
- 1
- 8
- 23

**11**

votes

**1**answer

### Prove that $\nabla_{\mathrm X} \mbox{tr} (\mathrm A \mathrm X^{-1} \mathrm B) = - \mathrm X^{-\top} \mathrm A^\top \mathrm B^\top \mathrm X^{-\top}$

Prove that $$\nabla_{\mathrm X} \mbox{tr} (\mathrm A \mathrm X^{-1} \mathrm B) = - \mathrm X^{-\top} \mathrm A^\top \mathrm B^\top \mathrm X^{-\top}$$
My proof is below. I am interested in other proofs.
My proof
Let
$$f (\mathrm X) := \mbox{tr}…

Rodrigo de Azevedo

- 18,977
- 5
- 36
- 95

**10**

votes

**2**answers

### Gradient of squared Frobenius norm

I would like to find the gradient of $\frac{1}{2} \big \| X A^T \big \|_F^2$ with respect to $X_{ij}$. Going by the chain rule in the Matrix Cookbook (eqn 126), it's something like
$$\frac{\partial}{\partial X_{ij}} \Big[\frac{1}{2} \big\| X A…

purple51

- 223
- 1
- 2
- 6

**8**

votes

**4**answers

### How to differentiate product of vectors (that gives scalar) by vector?

I'm trying to understand derivation of the least squares method in matrices terms:
$$S(\beta) = y^Ty - 2 \beta X^Ty + \beta ^ T X^TX \beta$$
Where $\beta$ is $m \times 1$ vertical vector, $X$ is $n \times m$ matrix and $y$ is $n \times 1$…

Khasan Khafizov

- 83
- 5