Theorems

Theorem 3.1

For a discrete random variable $X$ with PMF $P_{X} (x)$ and range $S_{X}$ :

For any $x$ , $P_{X} (x) \geq 0$
$\sum_{x \in S_{X}} P_{X} (x) = 1$
For any event $B \subset S_{X}$ , the probability that $X$ is in the set $B$ is

P [B] = \sum_{x \in B} P_{X} (x)

Theorem 3.2

For any discrete random variable $X$ with range $S_{X} = {x_{1}, x_{2}, \dots}$ satisfying $x_{1} \leq x_{2} \leq \dots$ ,

$F_{X} (- \infty) = 0$ and $F_{X} (\infty) = 1$ .
For all $x^{'} \geq x$ , $F_{X} (x^{'}) \geq F_{X} (x)$ .
For $x_{i} \in S_{X}$ and $ϵ$ , an arbitrarily small positive number,
$F_{X} (x_{i}) - F_{X} (x_{i} - ϵ) = P_{X} (x_{i}) .$
$F_{X} (x) = F_{X} (x_{i})$ for all $x$ such that $x_{i} \leq x < x_{i + 1}$ .

Each property of Theorem 3.2 has an equivalent statement in words:

Going from left to right on the x-axis, $F_{X} (x)$ starts at zero and ends at one.
The CDF never decreases as it goes from left to right.
For a discrete random variable $X$ , there is a jump (discontinuity) at each value of $x_{i} \in S_{X}$ . The height of the jump at $x_{i}$ is $P_{X} (x_{i})$ .
Between jumps, the graph of the CDF of the discrete random variable $X$ is a horizontal line.

Theorem 3.3

For all $b \geq a$ ,

P [a < X \leq b] = F_{X} (b) - F_{X} (a)

Warning

The definition of the CDF contains a loose inequality on the left ( $a < X$ ) and a strict inequality on the right ( $X \leq b$ ) which means the function is continuous from the right and not the left.

Theorem 3.4

The Bernoulli random variable $X$ has expected value $E [X] = p$

Proof

$E [X] = 0 \cdot P_{X} (0) + 1 \cdot P_{X} (1) = 0 (1 - p) + 1 (p) = p$

Theorem 3.5

The geometric random variable $X$ has expected value $E [X] = 1 / p$

Theorem 3.6

The Poisson random variable $X$ has expected value $E [X] = α$

Theorem 3.7

The binomial random variable $X$ has expected value

E [X] = n p

The Pascal random variable $X$ has expected value

E [X] = k / p

The discrete uniform random variable $X$ has expected value

E [X] = (k + l) / 2

Definitions

Random Variable

A random variable assigns a number to an outcome in the sample space.

A random variable is denoted by a capital letter $X$ . The range of a random variable is denoted by $S_{X}$ , which refers to the set of possible values of $X$ , using the sample space as notation.

Note

$X$ becomes a discrete random variable if the set of all possible outcomes is countable.

There are three types of random variables:

The random variable is an observation

Abstract

This is fairly straightforward, given the sample space $S = s_{1}, s_{2}, . . .$ , $X$ maps a number to each outcome or observation. Additionally, in this case, the sample space $S$ is the same as the range of $X$ , $S = S_{X}$ .

The random variable is a function of the observation

Abstract

This essentially means that the random variable refers to a subset of all the outcomes. For example if we are looking at the binary numbers of length 5, we can define the random variable $N$ as the number of "1"s present in each binary number. The range of $N$ in this case is $S_{N} = 0, 1, . . ., 5$ meaning an outcome can have anywhere from 0 to 5 total ones.

The random variable is a function of another random variable

Abstract

This is essentially the same as the previous type of random variable, except that it depends on the outcomes of some other random variable. For example, we could take the previous example and create a random variable $R$ which is defined by the net total of the number if each "1" is +$4 and each "0" is -$2. So $R = r (N) = 4 N - 2 (5 - N) = 6 N - 10$ , so $S_{R} = - 10, - 4, . . ., 20$

Probability Mass Function

The probability mass function (PMF) of a discrete random variable $X$ is

P_{X} (x) = P [X = x]

Abstract

$X = x$ refers to the event consisting of all outcomes s of the underlying experiment such that $X (s) = x$ . $P_{X} (x)$ refers to the function ranging over all real numbers $x$ . That is, for any value $x$ , $P_{X} (x)$ is the probability of $X = x$ .

Bernoulli Random Variable

$X$ is a Bernoulli random variable if the PMF if $X$ has the form

P_{X} (x) = {\begin{cases} 1 - p & x = 0 \\ p & x = 1 \\ 0 & otherwise \end{cases}

where $0 \leq p \leq 1$

Abstract

This PMF applies to sequential experiments with independent trials two possible outcomes. These types of experiments are called Bernoulli trials.

Geometric Random Variable

$X$ is a geometric random variable if the PMF of $X$ has the form

P_{X} (x) = {\begin{cases} p (1 - p)^{x - 1} & x = 1, 2, . . . \\ 0 & otherwise \end{cases}

where $0 \leq p \leq 1$

Abstract

This is useful for sequential experiments with sequential trials with $x > 2$ outcomes. In general, the number of Bernoulli trials that take place until the first observation of the two outcomes is a geometric random variable.

Binomial Random Variable

$X$ is a binomial random variable if the PMF of $X$ has the form

P_{X} (x) = (\binom{n}{x}) p^{x} (1 - p)^{n - x}

where $0 \leq p \leq 1$ and ${n \in Z | n \geq 1}$

Abstract

This is useful when we have a sequence of $n$ independent Bernoulli trials each with a probability of success $p$ , the number of successes is the binomial random variable.

Note

This is also equivalent to a Bernoulli random variable when $n = 1$

Discrete Uniform Random Variable

$X$ is a discrete uniform random variable if the PMF of $X$ has the form

P_{X} (x) = {\begin{cases} \frac{1}{l - k + 1} & x = k, k + 1, . . ., l \\ 0 & otherwise \end{cases}

where $k$ and $l$ are integers such that $k < l$

Abstract

This is used for a constant probability across the range of $X$ . It can be read as " $X$ is uniformly distributed between $k$ and $l$ ".

Todo

Add section for Pascal Random Variable

Poisson Random Variable

$X$ is a Poisson random variable if the PMF of $X$ has the form

P_{X} (x) = {\begin{cases} α^{x} e^{- α} / x! & x = 0, 1, 2, . . . \\ 0 & otherwise \end{cases}

where $α > 0$

Abstract

A Poisson random variable describes occurrence of the phenomenon of interest, or arrival. This model often has an average rate, $λ$ per unit of time, and some time interval $T$ . In this time interval the number of arrivals $X$ has a Poisson PMF with $α = λ T$

Cumulative Distributive Function

The cumulative distributive function (CDF) of random variable $X$ is

F_{X} (x) = P [X \leq x]

Expected Value

The expected value of $X$ is

E [X] = μ_{X} = \sum_{x \in S_{X}} x P_{X} (x)

Abstract

This can also be interpreted as the mean value.

Variance

Variance is defined as

\begin{aligned} V a r [X] & = E [(X - μ_{X})^{2}] \\ = E [X^{2}] - μ_{X}^{2} \\ = E [X^{2}] - (E [X])^{2} \end{aligned}

Info

$V a r [a X + b] = a^{2} V a r [X]$

Todo

Add sections for variance of different families of discrete random variables.

Standard Deviation

Standard deviation is defined as

σ_{X} = \sqrt{V a r [X]}

Sets & Probability

Theorems

Definitions

Related