Questions tagged [projection-matrices]

This tag is for questions relating to projection matrix, which is an square matrix that gives a vector space projection from to a subspace.

Let $~x ∈ E^n = V ⊕ W~$. Then $~x~$ can be uniquely decomposed into $$~x = x_1 + x_2~ \qquad(\text{where $~x_1 ∈ V~$ and $~x_2 ∈ W~$})~.$$ The transformation that maps $~x~$ into $~x_1~$ is called the projection matrix (or simply projector) onto $~V~$ along $~W~$ and is denoted as $~φ~$. This is a linear transformation; that is, $$φ(a_1~y_1 + a_2~y_2) = a_1~φ(y_1) + a_2~φ(y_2)$$ for any $~y_1,~ y_2 ∈ E^n~$. This implies that it can be represented by a matrix. This matrix is called a projection matrix and is denoted by $~P_{~V \cdot W}~$ .

The vector transformed by $~P_{~V \cdot W}~$ (that is, $~x_1 = ~P_{~V \cdot W}~ x$) is called the projection (or the projection vector) of $~x~$ onto $~V~$ along $~W~$.

${}$

Theorem: The necessary and sufficient condition for a square matrix $~P~$ of order $~n~$ to be the projection matrix onto $~V = \text{Sp}(P )~$ along $~W = \text{Ker}(P )~$ is given by $$P^2 = P$$

  • A square matrix $~{\displaystyle P}~$ is called an orthogonal projection matrix if $~{\displaystyle P^{2}=P=P^{\mathrm {T} }}~$ for a real matrix, and respectively $~{\displaystyle P^{2}=P=P^{\mathrm {H} }}~$ for a complex matrix, where $~{\displaystyle P^{\mathrm {T} }}~$ denotes the transpose of $~{\displaystyle P}~$ and $~{\displaystyle P^{\mathrm {H} }}~$ denotes the Hermitian transpose of $~{\displaystyle P}~$.

  • A projection matrix that is not an orthogonal projection matrix is called an oblique projection matrix.

  • In statistics, the projection matrix $~( {\displaystyle \mathbf {P} })~$, sometimes also called the influence matrix or hat matrix $~( {\displaystyle \mathbf {H} })~$, maps the vector of response values (dependent variable values) to the vector of fitted values (or predicted values). It describes the influence each response value has on each fitted value. The diagonal elements of the projection matrix are the leverages, which describe the influence each response value has on the fitted value for that same observation.

References:

https://en.wikipedia.org/wiki/Projection_(linear_algebra)

https://en.wikipedia.org/wiki/Projection_matrix

http://optics.szfki.kfki.hu/~psinko/alj/menu/04/nummod/Projection_Matrices.pdf

502 questions
31
votes
4 answers

Why is a projection matrix symmetric?

I am looking for an intuitive reason for a projection matrix of an orthogonal projection to be symmetric. The algebraic proof is straightforward yet somewhat unsatisfactory. Take for example another property: $P=P^2$. It's clear that applying the…
Leo
  • 7,400
  • 5
  • 27
  • 56
20
votes
2 answers

Product of a vector and its transpose (Projections)

I am doing a basic course on linear algebra, where the guy says $a^Ta$ is a number and $aa^T$ is a matrix not.m Why? Background: Say we are projecting a vector $b$ onto a vector $a$. By the condition of orthogonality, the dot product is…
Abhishek Bhatia
  • 1,378
  • 7
  • 19
  • 35
16
votes
3 answers

Diagonalization of a projection

If I have a projection $T$ on a finite dimensional vector space $V$, how do I show that $T$ is diagonalizable?
smanoos
  • 2,886
  • 1
  • 18
  • 29
16
votes
1 answer

Show that $P_i$ and $\sum_i P_i$ being idempotent implies $P_i P_j=\delta_{ij}$

Let $X$ be a finite dimensional real linear space, or more generally a finite dimensional vector space over a field of characteristic $0$. Let $(P_i)_{i=1}^n$ be a finite sequence of linear mappings $P_i :X\rightarrow X$ such that $P_i^2=P_i$ for…
Alex
  • 2,111
  • 16
  • 25
16
votes
2 answers

Uniqueness of trace as linearization of the rank

It is not difficult to show that if $A \in M_n(k)$ for some field $k$, and $A^2=A$ then $\operatorname{tr}(A) = \dim(\operatorname{Im}(A))$ In this comment, it is written: This property, together with linearity, determines the trace uniquely, and…
Alphonse
  • 5,627
  • 1
  • 14
  • 40
15
votes
6 answers

Relation between trace and rank for projection matrices

If $A $ is an $n \times n$ matrix over $\mathbb C$ such that $A^2=A$ then is it true that $\operatorname{trace} A = \operatorname{rank} A$?
Learnmore
  • 29,367
  • 8
  • 69
  • 200
9
votes
4 answers

Finding the projection matrix of $\mathbb R^3$ onto the plane $x-y-z=0$

Find the matrix of the projection of $\mathbb{R}^3$ onto the plane $x-y-z = 0.$ I can find a normal unit vector of the plane, which is $\vec{n}=(\frac{1}{\sqrt{3}},-\frac{1}{\sqrt{3}},-\frac{1}{\sqrt{3}})^T$ And then the vectors…
Robben
  • 1,009
  • 1
  • 9
  • 22
8
votes
3 answers

Proof of the CS (cosine-sine) matrix decomposition

The CS decomposition is a way to write the singular value decomposition of a matrix with orthonormal columns. More specifically, taking the notation from these notes (pdf alert), consider a $(n_1+n_2)\times p$ matrix $Q$, with $$Q=\begin{bmatrix}Q_1…
glS
  • 5,543
  • 2
  • 24
  • 46
7
votes
2 answers

Roles of $\bf A^TA$ ($\text {A transpose A}$) matrices in orthogonal projection

$\bf A^TA$ forms (or equivalently (?) positive semidefinite matrices, or more particularly, covariance matrices($\bf \Sigma$)) are linked in practice to many operations in which data points are orthogonally projected: In ordinary linear regression…
Antoni Parellada
  • 7,767
  • 5
  • 30
  • 100
6
votes
1 answer

Computing the distance between vector and its projection on a random subspace

Let $V\in\mathbb{R}^{n\times m}$, where $n>m$, be a random matrix of standard normal gaussians. Given an arbitrary vector $y\in\mathbb{R}^n$, I need to understand the distance $||y-p_V(y)||^2$, where $p_V$ is the projection onto the range of $V$,…
6
votes
2 answers

Do orthogonal projections play a role in diagonalizability?

I'm studying Linear Algebra by myself, and the textbook I use is the fourth edition written by Friedberg, Insel, and Spence. For now, I'm trying to get through Section 6.6 that concerns orthogonal projections and the spectral theorem. The following…
Steve
  • 1,809
  • 8
  • 21
6
votes
1 answer

Show that $(I − P)^2 = I − P$ if $P=P^2$

Let $P $ be an $n \times n$ matrix and $I$ be the $n \times n$ identity matrix. Show that $$ (I − P)^2 = I − P $$ is valid if $P = P^2$. I did the following. $$(I - P)^2 = I^2 - IP - PI + P^2 = I - P$$ where $I^2 = I $ because it is the identity…
6
votes
2 answers

When is the sum and difference of two projection matrices $P_1$ and $P_2$ a projection matrix?

Let $P_1$ and $P_2$ be two projection matrices for orthogonal projections onto $S_1 \in \mathcal{R}^m$ and $S_2 \in \mathcal{R}^m$, respectively. When does $P_1+P_2$ and $P_1-P_2$ result in a projection matrix? Prove it. I am confident that…
Matthew
  • 163
  • 10
6
votes
1 answer

The relationship between spectral decomposition / eigendecomposition and projection operators

I am trying to clarify the relationship between the spectral decomposition / eigendecomposition of a matrix and projection operators. I understand that there is a connection between diagonalizability of a linear operator / matrix and projection…
6
votes
3 answers

The matrix of a projection can never be invertible

I am currently studying linear transformations in order to refresh my knowledge of linear algebra. One statement in my textbook (by David Poole) is: When considering linear transformations from $\mathbb{R}^2$ to $\mathbb{R}^2$, the matrix of a…
1
2 3
33 34