Todo

Add sections for the intersection of multiple independent events
Add sections for

Axioms

Probability

Axiom 1: For any event $A$ , $P [A] \geq 0$
Axiom 2: $P [S] = 1$
Axiom 3: For any countable collection $A = A_{1}, A_{2}, . . ., A_{m}$ of mutually exclusive events, $A_{i} \cap A_{j} = \emptyset$ for $i \neq j$

P [A_{1} \cup A_{2} \cup . . . \cup A_{m}] = P [A_{1}] + P [A_{2}] + . . . + P [A_{m}]

Conditional Probability

Axiom 1: $P [A | B] \geq 0$
Axiom 2: $P [B | B] = 1$
Axiom 3: For any countable collection $A = A_{1}, A_{2}, . . ., A_{m}$ of mutually exclusive events, $A_{i} \cap A_{j} = \emptyset$ for $i \neq j$

P [A | B] = P [A_{1} | B] + P [A_{2} | B] + . . . + P [A_{m} | B]

Theorems

Theorem 1.2

Given mutually exclusive $A_{1}$ and $A_{2}$ ,

P [A_{1} \cup A_{2}] = P [A_{1}] + [A_{2}]

Theorem 1.3

Given $A = A_{1} \cup A_{2} . . . A_{m}$ and $A_{1} . . A_{m}$ are mutually exclusive.

P [A] = \sum_{i = 1}^{m} P [A_{i}]

Theorem 1.4

Probability measure $P [\cdot]$ satisfies

$P [\emptyset] = 0$
$P [A^{c}] = 1 - P [A]$
For any $A$ and $B$ not necessarily mutually exclusive

P [A \cup B] = P [A] + P [B] - P [A \cap B]

If $A \subset B$ then $P [A] \leq P [B]$

Note

If $A \subset B$ then $A B = A$

Theorem 1.5

Given a probability set (event) $B = s_{1}, s_{2}, . . ., s_{m}$ the sum of the probabilities of each outcome is equal to the probability of $B$ .

P [B] = \sum_{i = 1}^{m} P [s_{i}]

Theorem 1.6

If a sample space $S = s_{1}, . . ., s_{n}$ has equally likely outcomes for each $s_{i}$ , the probability of each outcome is $1 / n$ . Where $n$ is the number of outcomes.

P [s_{i}] = \frac{1}{n} 1 \leq i \leq n

This is because:

P [S] = P [s_{1}] + . . . + P [s_{n}] = n p

we know that $P [S] = 1$ so $p = 1 / n$

Theorem 1.8

Given a partition $B = B_{1}, B_{2}, . . .$ and some event in the sample space $A$ , let $C_{i} = A \cap B_{i}$ . For $i \neq j$ the events $C_{i}$ and $C_{j}$ are mutually exclusive.

A = C_{1} \cup C_{2} \cup . . .

Abstract

This essentially states that given any event $A$ in our sample space, we can divide $A$ among the partitions of $B$ by intersecting them. This will give us subsets of $A$ which are mutually exclusive or $C_{1}, C_{2}, . . .$ . The union of these subsets can reconstruct $A$ .

This concept is vital to understanding probability theory.

Theorem 1.9

For any event $A$ and partition $B_{1}, B_{2}, . . ., B_{m}$

P [A] = \sum_{i = 1}^{m} P [A \cap B_{i}]

Abstract

$A \cap B_{i} = C_{i}$ so this is essentially saying each mutually exclusive subset of $P [A]$ that was divided over partition $B$ can be summed to get $P [A]$

This is mostly used when the sample space can be put in the form of a table. Where each row and column represents a partition.

Pasted image 20241004102927.png

Theorem 1.10 - Total Law of Probability

Given partition $B_{1}, B_{2}, . . . B_{m}$ with $P [B_{i}] > 0$ for all $i$

P [A] = \sum_{i = 1}^{m} P [A | B_{i}] P [B_{i}]

Abstract

This is useful when trying to find the total probability of some event $A$ where $A$ has been subdivided over partition $B$ . It takes the likelihood that $A$ occurs over each subsection of partition $B$ ( $P [A | B_{i}]$ ) and multiplies it the probability that subsection of $B$ occurs as well. Each result is summed to get the total probability of $A$ .

Theorem 1.11

If given $P [A | B]$ we can calculate $P [B | A]$ with

P [B | A] = \frac{P [A | B] P [B]}{P [A]}

Note

Also known as Bayes' Theorem

Notation

Probability of a singleton set:

$P [s_{1}] = P [{s_{1}}]$

Probability of the intersection of two events:

P [A \cap B] = P [A B] = P [A, B]

Definitions

Set Theory	Probability
Element	Outcome
Set	Event
Universal Set	Sample Space

Outcome: One result of an experiment
Event: A set of outcomes of an experiment
Sample Space: The collectively exhaustive set of all outcomes

Conditional Probability

The probability of event $A$ given event $B$ is the probability of $A$ and $B$ occurring divided by the probability of $B$ occurring.

P [A | B] = \frac{P [A B]}{P [B]}

Note

The condition $P [B] > 0$ must be true for this definition to apply. It does not make sense to evaluate the $P [A]$ if $P [B]$ is already 0.

Partition

A partitions divides a sample space $S$ into mutually exclusive events or sets.

Pasted image 20241004094931.png

Independent Events

Two events are independent if and only if

P [A B] = P [A] P [B]

If $A$ and $B$ are non-zero, the following can be used to define independence.

P [A | B] = P [A] P [B | A] = P [B]

Warning

Independent and mutually exclusive are not synonyms.

Mutually exclusive events have no outcomes in common and therefore $P [A B] = 0$ for those events. Independent events on the other hand are not always mutually exclusive and exceptions only occur when $A = 0$ or $B = 0$

Abstract

This definition basically states that given some non-zero probabilities of $A$ and $B$ the probability of some event $A$ or $B$ occurring does not rely on the given event ( $B$ or $A$ respectively) occurring.

Knowing two events are independent allows us to calculate its intersection $P [A \cup B] = P [A B]$