2

Consider random vectors $X,Y\in \mathbb{R}^d$ on a unit-euclidean-ball $\big($i.e, $||X||_2=||Y||_2=1\big)$, and a matrix $M \in \mathbb{R}^{d\times d}$. How to calculate the following expectation: $$\mathbb{E}\big[\big|X^TMY\big|\big]$$

Edit: Added the modulus sign

Thanks in advance!

Vivek Bagaria
  • 378
  • 1
  • 13

2 Answers2

5

I have a proof for a lower bound on that value, but I'm not sure at the moment if it's possible to show a tight upper bound too.

Let me start with a few facts:

  • We can write the random vectors $x, y \in \mathbb{R}^d$ as $x=u/\lVert u \rVert, y=v/\lVert v \rVert$, with $u, v \sim \mathcal{N}(0, \textrm{Id}_d)$. See, e.g., Proof that normalized vector of Gaussian variables is uniformly distributed on the sphere.

  • $u \sim \mathcal{N}(0, \textrm{Id}_d) \implies Au \sim \mathcal{N}(0, AA^T)$ for any matrix $A$; in particular, for $A$ orthogonal, $Au \sim \mathcal{N}(0, \textrm{Id}_d)$.

  • Orthogonal transforms preserve norms induced by inner-products (by definition): $\lVert u \rVert = \langle u, u \rangle^{1/2} = \langle Au, Au \rangle^{1/2} = \lVert Au \rVert$.

  • $M = UDV^T$, a singular value decomposition of $M$.

Thus, from the first three facts we have that $w = U^T x$ and $z = V^T y$ are uniformly distributed on the unit sphere as well. Then, $w^T D z = x^T U D V^T y$ = $x^T M y$, and so we can study the simplified expression $\mathbb{E}\ \lvert x^T D y \rvert$ where $D$ is a diagonal matrix. To obtain a lower bound, proceed as follows

\begin{align} \mathbb{E}\ \lvert x^T D y \rvert &= \mathbb{E}\ \lvert \textrm{Tr}\ x^T D y \rvert &&\text{trace trick} \\ &= \mathbb{E}\ \lvert \textrm{Tr}\ y x^T D \rvert &&\text{trace cyclicity} \\ &\ge \lvert \mathbb{E}\ \textrm{Tr}\ y x^T D \rvert &&\text{Jensen's inequality} \\ &= \lvert \textrm{Tr}\ \mathbb{E} \big[ y x^T \big] D \rvert &&\text{trace linearity} \\ &= \lvert \textrm{Tr}\ \textrm{Id}_d D \rvert &&\text{$x, y \stackrel{\textrm{iid}}{\sim} \mathcal{N}(0, \textrm{Id}_d)$} \\ &= \sum_{i=1}^d \sigma_i(M), &&\text{non-negativity of singular values} \end{align} where $\sigma_i(M)$ is the $i$th largest singular value of $M$.


I think it's also possible to show an upper bound in terms of the absolute value of the sum of the singular values of $M$, as follows. Let $y x^T = U \Sigma V^T$ be a SVD of the random matrix $y x^T$ (note that $U$ and $V$ are different than before, but since I didn't use those ones explicitly, let me re-use their names). Then,

\begin{align} \mathbb{E}\ \lvert x^T D y \rvert &= \mathbb{E}\ \lvert \textrm{Tr}\ U \Sigma V^T D \rvert \\ &= \mathbb{E}\ \Big\lvert \sum_{i=1}^d \sigma_i(M) u_i^T \Sigma v_i \Big\rvert &&\text{trace definition} \\ &\le \mathbb{E}\ \Big\lvert \max\limits_{\lVert u \rVert = \lVert v \rVert = 1} \sum_{i=1}^d \sigma_i(M) u^T \Sigma v \Big\rvert &&\text{upper bound} \\ &= \mathbb{E}\ \Big\lvert \sum_{i=1}^d \sigma_i(M) \sigma_1(y x^T) \Big\rvert &&\text{maximum singular value} \\ &= \Big( \sum_{i=1}^d \sigma_i(M) \Big) \cdot \mathbb{E}\ \big\lvert \sigma_1(y x^T) \big\rvert, \end{align}

but there is this additional factor, the expected value of the (absolute value of the) maximum singular value of the outer product of two standard Gaussian vectors, which I'm not sure how to bound. I am familiar with a result which says that $$\lvert 2 - \mathbb{E}\ \lVert W \rVert \rvert \le o(1),\quad \textrm{for}\ W \sim \mathcal{N}(0, 1)^{n \times n},$$ where $\lVert \cdot \rVert$ denotes the spectral norm (equal to $\sigma_1$), but notice that $y x^T$ does not (necessarily) have normally-distributed entries (see, e.g., Is the product of two Gaussian random variables also a Gaussian?).

Călin
  • 249
  • 2
  • 13
1

Assume $X,Y$ are independent random vectors, chosen uniformly from the standard unit $d$-sphere.

Fix $Y$, and let $Z=MY$.

Then $Z$ is just some fixed vector in $\mathbb{R}^d$.

By symmetry, for $X$ chosen uniformly at random from the standard unit $d$-sphere, $$E[X\cdot Z] = E[(-X)\cdot Z] = E[-(X\cdot Z)]= -E[X\cdot Z]$$ hence $E[X\cdot Z]=0$.

Therefore $E[X^TMY]=0$.

quasi
  • 55,313
  • 3
  • 38
  • 89