42

When it comes to inner product I have thus far only dealt with vectors, and so the concept is very intuitive because one can easily visualize two vectors and how they get multiplied, and it is clear why the dot product of two vectors is defined the way it is. For $v∗u$ it would basically be the length projection of $v$ onto $u$ (the part of $v$ in direction of $u$) multiplied by the length of $u$. So you basically have a measure of how much the vectors move in same direction.

But when it comes to functions, what becomes the meaning of the inner product, why is the formula the way it is and what is the intution behind it. Basically for $f(x),g(x)$ you have....

$\int_{a}^{b} f(x)g(x) dx$

as the inner product for two function $f(x),g(x)$ on the interval [a,b].

Arnold Doveman
  • 977
  • 2
  • 9
  • 16

1 Answers1

73

"Only dealt with vectors", you're dealing with vectors here too! Functions are vectors, and this is an inner product on a vector space!

Really, the integral is exactly the same thing as with the dot product. For two vectors in $\Bbb{R}^n$, the dot product is $(x_1,...,x_n)\cdot (y_1,...,y_n)=x_1y_1+\cdots+x_ny_n$.

For functions, you can think of the dot product being the same thing! You multiply the two components and add them up! The only difference is the measure or "weight" of each point. Since $\Bbb{R}^n$ is discrete, each component has weight $1$, where in these function spaces each component has weight "$dx$".

So loosely, on $\Bbb{R}^n$:

$$(x_1,...,x_n)\cdot (y_1,...,y_n)=x_1y_1 \times1+\cdots+x_ny_n\times 1$$

On a sequence space:

$$(x_1,...)\cdot (y_1,...)=x_1y_1 \times1+\cdots$$

On these function spaces (I write $f\cdot g$ to emphasise that I mean the dot product between the functions $f$ and $g$ and not the product $f(x)g(x)$ between the numbers $f(x)$ and $g(x)$):

$$f \cdot g=f(a)g(a)\times dx+f(a+dx)g(a+dx)\times dx+\cdots=\int_a^b f(x)g(x) dx$$

The inner product on function spaces is exactly the regular dot product, just in infinite dimensions and with a different "weight".

Edit, for the issue of orthogonality. The pressing issue here is that inner products define what it means to be orthogonal. So I issue a challenge to you here. Without referencing an inner product (this includes angles, as the inner product DEFINES angles), what does it mean to be orthogonal? You can't answer this question. The entire notion of angle, orthogonality, etc. are summarized in:

$$(x_1,...,x_n)\cdot (y_1,...,y_n)=x_1y_1 \times1+\cdots+x_ny_n\times 1$$

Even visualizing this is difficult. What do orthogonal vectors look like in 4 dimensions for example? I certainly don't know. Could you try picturing this in infinite dimensions? I can't.

So what does it mean to say that the inner product of two functions is $0$? The same thing it always does, that those two functions are orthogonal! Is there a nice visualization of this? Not really.

For another example, with this inner product comes the norm $\|f(x)\|_2=\sqrt{\int_a^b f^2(x) dx}$. Can you draw a unit circle? That is, all functions $f:[a,b]\to\mathbb R$ such that $\|f(x)\|_2=\sqrt{\int_a^b f^2(x) dx}=1$?

Certainly you can't. Things become less geometric in infinite dimensions.

Stephan Kulla
  • 6,671
  • 26
  • 61
  • 4
    Fanatastic answer, now I see where the integral comes in. But I'm wondering is the value of the integral analogous to the value of a regular dot product. A regular dot product of two vectors would basically tell you how much the two vectors move together, if its 0 they are othogonal. Does the value of this integral then tell you how much the functions move together, I dont quite get that point? – Arnold Doveman Aug 30 '15 at 07:08
  • 2
    @ArnoldDoveman Edited with more information. –  Aug 30 '15 at 07:18
  • 1
    So when we compute the "angle" between two functions, $\cos\theta=\frac{\langle f,g\rangle}{\|f\|\|g\|}$, we don't really know what "angle" means any more ? – gone Nov 03 '15 at 17:33
  • 3
    @pbs We know *exactly* what angle means, it's just difficult to visualize. –  Nov 03 '15 at 19:09
  • Do not know the geometric interpretation, but it is used to test orthogonality of functions, within some given interval. If the integral is zero the functions are orthogonal otherwise not. Currently do not know what any other value for the integral may signify – Arup Hore Nov 28 '19 at 22:09