MT Page

What is the meaning of an undefined mean?

—

by

Mars Tonry
in

The formula for the mean of a probability density function (pdf) f(x) defined on the interval (a,b) is the following:

μ=∫abxf(x)dx\mu=\int_{a}^{b}xf(x)dx

If this seems confusing, here is an interpretation of the formula. The formula multiplies the value (x) with the probability of finding that value in an infinitesimal sliver of the distribution (f(x)dx) and integrates it over the entire distribution. If we repeatedly sampled values from the distribution and kept a running average (updating the current average with each new value), the average value would converge to the mean. Let’s use this formula to calculate the mean of an exponential distribution.

f(x)=e−x, 0≤x<∞f(x)=e^{-x},\ 0\le x<\infin
μ=∫0∞xe−xdx\mu=\int_{0}^{\infin}xe^{-x}dx
=xe−x|0∞+∫0∞e−xdx=\left. xe^{-x}\right|_0^{\infin}+\int_{0}^{\infin}e^{-x}dx
=lim⁡x→∞xe−x+1=\lim{x \to \infin}xe^{-x}+1
=0+1=0+1
=1=1

Let’s sample 1000 times from an exponential distribution, and record the running average over the 1000 samples.

As we can see, the running average has some small spikes at the beginning, but it quickly converges to 1, which is the mean of the standard exponential distribution.

However, let’s use this formula to find the mean of a standard Pareto distribution:

f(x)=1x2, 1≤x<∞f(x)=\frac{1}{x^2},\ 1\le x<\infin
μ=∫1∞x1x2dx\mu=\int_{1}^{\infin}x\frac{1}{x^2}dx
=∫1∞1xdx=\int_{1}^{\infin}\frac{1}{x}dx
=ln⁡x|1∞=\left. \ln x \right|_1^{\infin}
=lim⁡x→∞ln⁡x=\lim{x \to \infin}\ln x

This quantity diverges, which means that the standard Pareto distribution has an undefined mean. This may seem counter-intuitive at first, but it means that if we calculate a running average, it will never converge on value, but rather keep increasing the more we sample.

The running average suffers from spikes which are so frequent, the average never converges on a value no matter how much we sample. However, this is not because the outputs of the distribution are always positive.

I briefly discussed how a standard Pareto distribution is heavy-tailed in my August 13th post. Any sufficiently heavy-tailed distribution has an infinite mean, even ones that produce negative outputs.

One such example is the standard Cauchy distribution. If we try to find the mean, the integral formula will diverge again. If we take the running average, we see the same frequent unpredictable spikes, but extending in both the positive and negative directions.

Comments

Leave a Reply