Intro to Statistics: Part 2: A Random Variable's Distribution and Expected Value

The complete set of all possible outcomes for a random variable is known as its distribution. When we talk about the characteristics of a random variable, we're talking about the characteristics of its distribution.

A random variable's distribution has a few key characteristics. First and foremost is its expected value, sometimes referred to as the mean. The expected value of a random variable is the weighted average of the outcomes in its distribution.

Each outcome is "weighted" by its probability of occurring. Outcomes with higher probability have a higher weight; therefore they affect the average more than outcomes with lower probability (lower weight).

If all outcomes have equal probability, then they all have equal weight, in which case the weighted average is equal to the arithmetic mean (or "non-weighted" average).

The expected value of a random variable X is denoted symbolically as E[X]. It's calculated by taking each outcome, multiplying it by its probability (its weight), then summing across all outcomes:

\begin{align*} \operatorname{E}[X] & = \sum_{i=1}^{k}x_i\cdot p_i\\ \\ & = x_1p_1 + x_2p_2 + \dotsb + x_kp_k\end{align*}

... where...

x₁, x₂, ... x_k are the complete set of possible outcomes
p₁, p₂, ... p_k are the probabilities associated with each outcome

When all outcomes are equally likely (p=1/k, where k is the number of outcomes), then the expected value is equivalent to taking the arithmetic mean:

\begin{align*} \operatorname{E}[X] & = x_1\cdot \frac{1}{k} + x_2\cdot \frac{1}{k} + \dotsb + x_k\cdot \frac{1}{k}\\ \\& = \frac{x_1 + x_2 + \dotsb + x_k}{k}\end{align*}

Expected value examples: coin flip and die roll

For example, suppose the random variable X represents flipping a coin, where heads and tails are assigned numerical values 0 and 1. What's its expected value?

Well, there are only two outcomes: 0 and 1, and each has the same probability of occurring, p=0.5. So we can calculate the expected value by multiplying each outcome by its probability and adding up the results:

\operatorname{E}[X]= 0\cdot 0.5 + 1\cdot 0.5 = 0.5

Or, since we know all outcomes have equal probability, the expected value is the same as arithmetic mean:

\operatorname{E}[X]= \frac{0+1}{2} = \frac{1}{2}

Either way, you get the same expected value: E[X] = 0.5

For another example, suppose the random variable Y represents a die roll. Since all outcomes are equally likely, the expected value is just the arithmetic mean:

Note that the expected value doesn't necessarily have to exist in the set of possible outcomes (which it doesn't in both examples we've looked at so far).

Recap

So to recap what we've covered so far (including from previous posts in the series):

A random variable is described by the characteristics of its complete set of possible outcomes, also known as its distribution.
Each outcome has an associated probability of occurring.
The sum of the probabilities for all possible outcomes in the distribution equals 1.
A random variable's expected value, typically denoted E[X], is the weighted average of all outcomes in the distribution, where each outcome is weighted by its probability.

Expected value examples: coin flip and die roll

Recap

Intro to Statistics: 1 . 2 . 3 . 4 . 5 ...