It's a standard exercise to find the Fourier transform of the Gaussian $e^{-x^2}$ and show that it is equal to itself. Although it is computationally straightforward, this has always somewhat surprised me. My intuition for the Gaussian is as the integrand of normal distributions, and my intuition for Fourier transforms is as a means to extract frequencies from a function. They seem unrelated, save for their use of the exponential function.

How should I understand this property of the Gaussian, or in general, eigenfunctions of the Fourier transform? The Hermite polynomials are eigenfunctions of the Fourier transform and play a central role in probability. Is this an instance of a deeper connection between probability and harmonic analysis?