Probability - Bayesian Inference
Vuong Huynh
Different notations in Set and Set Symbols
ALT codes list and for Math
Sets and Events - Introduction
Every event has a set of outcomes that satisfy it. These are favorable outcomes. Because the null set exists, even the outcomes of an impossible event can be described with a set.
Values of a set don't always have to be numerical.
Conventionally use UPPERCASE for SET and lowercase for elements.
Example: X is a set of even, x is a single value (elements) of set X.
Any set can be empty Ø or has values in it.
Non-empty sets: Non-empty sets can be finite or infinite
Part of a set:
x∈A'x is an element of the set A, or x is in A'
Not a part of a set:
x ∉ A'x is not an element of the set A, or x is not in A'
Generalize Statement about multiple elements:
∀'for all/ any'∀ x∈A'for all x in A'∀ x∈A: x is even'for all x in A such as x is even', the colon:means such as∀ x∈A: x ≥ 0'for any non-negative value x, within the set X'
The Subset, a set fully contained in another set
Example: A is a subset of B means every element of A is also an element of B
A ⊆ B
Remember: Every set contain at least two sets: the set itself and the null set
A ⊆ AandØ ⊆ A
If a ∈ A, b∈B, and B ⊆ A, we can say b is an element of A
Sets and Events - Ways Sets Can Interact
Intersecting: two events can occur at the same time
Example: in a desk, event A is a Queen, while event B is a Diamond. They can intersect each other
Subsets: one event can only ever occur if the other one does as well
Example: event A is a Red card and event B is a Diamond (or heart). Meaning we can only get a Diamond only if we get a Red card.
However, event B (Diamond) occur not guarantee event A (Red) occur. We also have a heart as event B
In sort:
If an outcome is not a part of a set, it cannot be part of any of its subsets.
An outcome not being part of some subset does not exclude it from the entirety of the greater set
Intersection
The intersection part of two events consists of the outcomes that are favorable for both event A and event B simultaneously
A ⋂ B 'A intersect B'
♥⋂ ♦ = ØThe intersection of all Hearts and all Diamonds is the EMPTY SET, there is no outcome that satisfies both events simultaneously♦ ⋂ Q = Q♦The Queen of Diamonds is the only one that satisfies being a Queen and being a Diamond at the same timeRED ⋂ ♦ = ♦orA ⋂ B = Bis where Diamond is a subset of Red cards or B is a subset of A
Remember: we use intersections to denote instances where both events A and B happen simultaneously
Non-positive and Non-negative numbers intersect in zero.
Union of sets
The combination of all outcomes preferred for either A or B
A ⋃ B: 'A union B'. The ⋃ sign is the capital U in Union
- If
A B = Ø,A ⋃ B = A + B - If
A ⋂ B,A ⋃ B = A + B - A ⋂ B - If
A ⊆ B,A ⋃ B = B
There are 8 blond and 10 brown-eyed people in the office. If only Jason and Eve have both features, how many people represent the union of blond and brown-eyed people?
- 16. Since Jason and Eve have both features, then the intersection of the two sets would only contain two people. Applying the formula we introduced in the lecture, the union is equal to the sum of the two sets, minus their intersection. Therefore, 8 + 10 – 2, or 16 people in the office posses at least one of the features
Mutually Exclusive Sets
Sets, which are not allowed to have any overlapping elements
Mutually exclusive sets have the empty set as their intersection
If the intersection of any number of sets is the empty set, then they must be mutually exclusive
Mutually exclusive is a statistical term describing two or more events that cannot happen simultaneously. It is commonly used to describe a situation where the occurrence of one outcome supersedes the other.
If two sets A and B are mutually exclusive:
A ⋂ B = Ø
A ⋃ B = A + B
Complement Sets
A' - complement
Complements are ALWAYS mutually exclusive, but NOT all mutually exclusive sets are complements
Example:
A is a set of all even numbers, B is a set with all numbers ending of 5 - {5, 15, 25, 35}
A → all even, A' → all odd, however A' ≠ B because 13 ∈ A' but 13 ∉ B
Winning and game and Drawing a game is a pair of sets which are mutually exclusive but not complements. Because you can not simultaneously win and draw the same game. However, you can also lose this game, so the two are not complements.
Dependence and Independence of Sets
Independent Events: the theoretical probability remains unaffected by other events.
Example
Flipping a coin. The outcome is 1 / 2. It does not matter if we have a head or a tail, they will be independent of each other. The probability of an event remains unaffected by another event
There are two events, event A - the card being a Diamond while event B - the card being an Ace. Event A and event B are independent because our chances of drawing an Ace are 1/13, regardless of whether we drew a diamond or not.
Dependent Events: probabilities of dependent events vary as conditions change.
Example
The probability of drawing the Queen of Spades. The outcome is 1 / 52.
If we know that the card we drew was a Spade, the new outcome would be 4 / 52 or 1 / 13.
If we already know the card is Queen, the new outcome is 1 / 4
Notations
P(A | B) 'A given B' - the probability of getting A, if we are given that B has occurred.
We call this a Conditional Probability to distinguish dependent from independent events.
Compare the Conditional and Unconditional probabilities (e.g. P(A) and P(A|B)) to determine whether the events in each pair are independent. If P(A) = P(A|B), then the two are independent and if not, A and B are dependent.
Conditional Probability Formula
Conditional probability is the likelihood of an event occurring, assuming a different one has already happened.
P(A | B) = P(A ⋂ B) / P(B) If P(B) > 0
If P(B) = 0, event B would NEVER occur
P(A|B) ≠ P(B|A)
Two Coin Flips:
A → HEAD; B → previous HEAD
P(A) = P(A|B)
P(A ⋂ B) = P(A) * P(B) If any two events are independent, the probability of the intersection of the two events is the product of the individual probabilities
Queen of Spades:
A → Q♠ ; B → ♠ ; C → Q
P(A) = 1/52
P(A|B) = 1/13
What can you conclude about events A and B, given that P(A) = P(A|B)? By definition, if the occurrence of one event does not alter the likelihood of the other one occurring, then the two are independent.
Applying the Conditional Probability Formula, what is the probability of event A occurring, given event B has occurred if the likelihood of getting their intersection is 0.15 and the likelihood of event B is 0.6? According to the formula, P(A|B) = 0.15/0.6 = 0.25.
What is the difference between P(A|B) and P(B|A)? One indicates the probability of getting A, given B has occurred, while the other indicates the likelihood of getting B, given A has occurred
The Law of Total Probability
A = B1 ⋃ B2 ⋃ B3 ⋃ ... U Bn
P(A) = P(A | B1) * P(A | B2) * ... * P(A | Bn)
The Additive Rule
P(A ⋃ B) = P(A) + P(B) - P(A ⋂ B)
The probability of the union of two sets is equal to the sum of the individual probabilities of each event, minus the probability of their intersection
The Multiplication Rule
P(A | B) * P(B) = P(A ⋂ B)
How do we interpret the multiplication rule formula for events A and B?
The probability of both events happening equals the product of the likelihood of A occurring and the conditional probability that B occurs, given A has already occurred.
Imagine you are watching your favorite British soccer team and want to know how likely it is that they will win, and your favorite player Mohamed Salah will score. You know that this season, the team has won in 80% of the games he has scored in. Furthermore, you know that he has scored in 70% of all the games the team has played this season.
0.56. If event A is Liverpool winning the game and event B is Salah scoring, then we know that P(A|B)=0.8 and P(B)=0.7. Plugging in these values into the formula gives us P(A∩B)= P(A|B)×P(B)=0.8×0.7=0.56.
The Bayes' Law
P(A | B) = P(B | A) * P(A) / P(B)
What is the value of P(A|B), knowing P(B|A) = 0.6, P(A) = 0.4 and P(B) = 0.3?
0.8 - Plugging the values we have into Bayes’ Law gives us P(A|B)=(P(B│A)×P(A))/P(B) =(0.6×0.4)/0.3=0.8. Thus, the conditional probability of getting A, given B equals 0.8.