70

What is an intuitive explanation of a positive-semidefinite matrix? Or a simple example which gives more intuition for it rather than the bare definition. Say $x$ is some vector in space and $M$ is some operation on vectors.

The definition is:

A $n$ × $n$ Hermitian matrix M is called positive-semidefinite if

$$x^{*} M x \geq 0$$

for all $x \in \mathbb{C}^n$ (or, all $x \in \mathbb{R}^n$ for the real matrix), where $x^*$ is the conjugate transpose of $x$.

Rodrigo de Azevedo
  • 18,977
  • 5
  • 36
  • 95

6 Answers6

143

One intuitive definition is as follows. Multiply any vector with a positive semi-definite matrix. The angle between the original vector and the resultant vector will always be less than or equal $\frac{\pi}{2}$. The positive definite matrix tries to keep the vector within a certain half space containing the vector. This is analogous to what a positive number does to a real variable. Multiply it and it only stretches or contracts the number but never reflects it about the origin.

Andreas G.
  • 183
  • 8
  • 3
    Why less than $\pi/2$? – user10024395 Nov 08 '17 at 06:55
  • 1
    @user10024395 That's 90 degrees in radians. Any angle beyond that, limiting ourselves to up to 180 degrees, would result in a negative inner product with the original vector, which violates the conditions of the definition. – cangrejo Jun 18 '19 at 13:28
  • 1
    $user10024395, Let us say we have original vector [a b] and then we get a new vector [w1*a w2*b] where both w1 and w2 >0. Due to this the new vector will stay in its original quadrant. I think what user17762 meant was the original quadrant – Anant Gupta Jun 21 '19 at 11:12
  • @AnantGupta still why quadrant why not bi-quadrant i.e. $\pi$ – GENIVI-LEARNER Mar 26 '20 at 21:32
19

First I'll tell you how I think about Hermitian positive-definite matrices. A Hermitian positive-definite matrix $M$ defines a sesquilinear inner product $\langle Mv, w \rangle = \langle v, Mw \rangle$, and in fact every inner product on a finite-dimensional inner product space $V$ has this form. In other words it is a way of computing angles between vectors, or a way of projecting vectors onto other vectors; over the real numbers it is the key ingredient to doing Euclidean geometry. An inner product can be recovered from the norm $\langle Mv, v \rangle = \langle v, Mv \rangle$ it induces, and a norm in turn can be recovered from its unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$. This unit sphere is a distorted version of the usual unit sphere; the distortions will occur along axes corresponding to the eigenvectors of $M$, and the amount of distortion corresponds to the (inverses of the) corresponding eigenvalues. For example when $\dim V = 2$ it is an ellipse and when $\dim V = 3$ it is an ellipsoid.

A Hermitian positive-semidefinite matrix $M$ no longer describes an inner product because it is not necessarily positive-definite, but it still defines a sesquilinear form. It also defines a function $\langle Mv, v \rangle$ which is no longer a norm because it is not necessarily positive-definite; some people call these "pseudonorms," I think. The corresponding unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$ might now be lower-dimensional than the usual unit sphere, depending on how many eigenvalues are equal to zero; for example if $\dim V = 3$ it might be an ellipsoid, or an ellipse, or two points.

Qiaochu Yuan
  • 359,788
  • 42
  • 777
  • 1,145
  • 4
    A bit more verbose, but great information! Like the word 'sesquilinear'. :) –  Nov 10 '10 at 21:00
  • 6
    This is a great explanation, because it answers not merely *what* a positive definite matrix is but why we *care*. A minor correction: in 3 dimensions, a positive semidefinite matrix $M$ *maps* the unit sphere to the set $\{Mv:v^Tv=1\}$ which may be an ellipsoid, a filled ellipse, or a line segment, but the set $\{v:v^TMv=1\}$ is either an ellipsoid, an elliptic cylinder, or a pair of parallel planes. –  Nov 11 '10 at 06:18
  • 2
    @Rahul: right. Thanks! – Qiaochu Yuan Nov 11 '10 at 09:49
6

Let's consider the set $E$ of all vectors $y=Mx$, where $x\in\mathbb R^n$ belongs to the unit sphere (i.e. $\|x\|=1$). In other words, $E$ is the image of the unit sphere under the linear transformation. If the matrix is non-degenerate, $E$ is an $n$-dimensional ellipsoid. But if we assume additionally that $M$ is symmetric then we can say much more about the structure of $E$. Namely, the directions of the axes of the ellipsoid are pairwise orthogonal and represented by the eigenvectors of the matrix. Moreover, the lengths of the semiaxes are given by the corresponding eigenvalues.

Andrey Rekalo
  • 7,524
  • 3
  • 45
  • 43
2

Intuitively, it's a matrix that's "like" a single number $\geq 0$.

(Relatedly you can have a positive semidefinite function that's also "like" a number.)

Both matrices and functions in general have many (, many, many) more degrees of freedom than members of $\mathbb R$. But the semidefinite classes of matrices and functions "boil down to" something much simpler.

isomorphismes
  • 4,257
  • 1
  • 39
  • 48
  • Here's my [longer explanation](http://blog.hiremebecauseimsmart.com/post/7308833574/positive-semi-definite). – isomorphismes Aug 17 '11 at 20:54
  • 2
    The link is dead as of Sep 2017. Might be [here](https://hiremebecauseimsmart.wordpress.com/2011/07/06/positive-semi-definite/). – akraf Sep 27 '17 at 14:22
1

Positive-definite matrices are matrices that are congruent to the identity matrix, i.e. that can be written as $P^HP$ for invertible $P$ (for some reason, a lot of authors define congruence as $N=P^TMP$, but here we go by the Hermitian definition $N=P^HMP$).

One reason this is useful is that if two forms $M$ and $N$ are congruent, their corresponding "generalised unitary groups" $\{A^HMA=M\}$ and $\{B^HNB=N\}$ are isomorphic (via conjugation by $P$). So positive-definite matrices (as well as negative-definite matrices, because $-I$ is preserved by the unitary group as well) define a dot product whose geometry is isomorphic to Euclidean geometry.

Similarly, a positive semidefinite matrix defines a geometry that Euclidean geometry is homeomorphic to -- to put it in slightly imprecisely, such a geometry has all the symmetries of Euclidean geometry, and perhaps then some.

See a fuller treatment here.

-2

Let us take an example of a vector V = i + 2j which in matrix notation is

enter image description here

It will be represented as enter image description here

  • POSITIVE DEFINITE MATRIX

A positive definite matrix is

enter image description here

How do we know it is positive definite matrix

enter image description here

The altered vector is

enter image description here

enter image description here

  • POSITIVE SEMI-DEFINITE MATRIX

enter image description here

The altered vector is [0 , 0]

  • NEGATIVE MATRIX

enter image description here

The altered vector in this case is

enter image description here

We can see the quadrant has changed

  • why do we care if the quadrant has changed or not? Also how is the graph for semi-definite looks like? Can you please elaborate more on what do you mean by altered vector – GENIVI-LEARNER Feb 09 '20 at 14:53