STAT101 - Introduction to Probability Theory and Statistics
The Basics of Probability & The Rules of Probability (Sections 4.1 - 4.2)
Every statistical decision you'll ever make is built on probability. This chapter gives you the vocabulary (experiment, sample space, event) and the toolbox (addition rule, conditional probability, multiplication rule) that every later chapter assumes you already know cold. If you can confidently answer "is this OR or AND?" and "is this independent or dependent?" you have mastered 80% of this chapter's exam value.
Probability — the chance that a particular event will occur. Probability values always fall in the range 0 to 1 (or 0% to 100%).
Experiment — a process that produces a single outcome whose result cannot be predicted with certainty (e.g., drilling a well, hiring an employee, rolling a die).
Sample Space (SS) — the collection of all outcomes that can result from a selection, decision, or experiment.
Event — any subset of outcomes from the sample space that you are interested in (e.g., "rolling an odd number").
Scenario: A new car dealer wants to know the chance that a new business venture succeeds.
Notice the pattern: the experiment is the "verb" (what you do), the sample space is every possible "answer," and the probability is the number attached to one particular answer.
Step 1: Define the experiment (e.g., a single sale at a burger restaurant).
Step 2: Define the outcomes for one trial (e.g., e1 = Hamburger, e2 = Cheeseburger, e3 = Bacon Burger).
Step 3: Define the sample space SS. For one sale: SS = {e1, e2, e3}. If the experiment involves two sales, there are 3 × 3 = 9 possible outcomes: SS = {o1, o2, ..., o9} where o1 = {e1,e1}, o2 = {e1,e2}, ..., o9 = {e3,e3}.
Scenario: Three students are each asked whether they like statistics ("Yes" or "No"). Define the sample space for all three trials.
Step 1 (experiment): Ask three students Yes/No.
Step 2 (outcomes per trial): {Yes, No}
Step 3 (tree diagram): Branching Yes/No three times in a row gives 2 × 2 × 2 = 8 total outcomes:
{No,No,No}, {No,No,Yes}, {No,Yes,No}, {No,Yes,Yes}, {Yes,No,No}, {Yes,No,Yes}, {Yes,Yes,No}, {Yes,Yes,Yes}
A tree diagram is the fastest way to enumerate a sample space when you have a sequence of trials — draw one branch point per trial and multiply the number of branches at each level.
If a trial has k possible outcomes and you repeat it n times, the sample space has kn outcomes (assuming order matters and repeats are allowed). 3 students × 2 outcomes each = 23 = 8. Two sales × 3 menu items = 32 = 9. This shortcut saves you from drawing the whole tree on an exam.
Mutually Exclusive Events — two events are mutually exclusive if the occurrence of one event precludes the occurrence of the other. They cannot both happen at the same time (no overlap).
Independent Events — two events are independent if the occurrence of one event in no way influences the probability of the occurrence of the other.
Dependent Events — two events are dependent if the occurrence of one event impacts the probability of the other event occurring.
1. Mutually Exclusive: Event A = "oil well produces oil," Event B = "oil well does not produce oil." If A occurs, B cannot also occur → mutually exclusive.
2. Independent: Event A = "business in North Carolina succeeds," Event B = "business in Oregon succeeds." Whether the Oregon business succeeds is in no way influenced by the North Carolina outcome → independent.
3. Dependent: A company has 10 employees (3 male, 7 female). Two are selected at random. Event A = "first person selected is female," Event B = "second person selected is male." Because removing the first person changes the pool for the second draw, the probability of B depends on whether A happened → dependent events.
Students very often mix these up. They are opposite ideas in an important sense: if two events are mutually exclusive (and each has nonzero probability), they are automatically dependent — because if one happens, the probability of the other drops to zero! Mutually exclusive events can never be independent (unless one has probability 0). Ask yourself: "Can both happen at once?" (mutually exclusive question) is a completely different question from "Does one affect the chances of the other?" (independence question).
There are three ways to assign a numeric probability to an event:
Based on the ratio of the number of ways an event of interest can occur to the total number of ways any outcome can occur, when all individual outcomes are equally likely:
Q1: A company has 10 employees — 7 female, 3 male. What is the probability a randomly selected employee traveling to a convention is female?
P(Female) = 7 / 10 = 0.70
Q2: A car dealer has 5 GM, 6 Ford, 3 Toyota, 8 Nissan, and 2 BMW cars (24 total). What is the probability a randomly selected car is a Nissan?
P(Nissan) = 8 / 24 = 1/3 ≈ 0.333
Defines probability as the number of times an event occurs divided by the total number of times an experiment is performed, based on observed historical data (a large number of trials):
Q1: A local pub tracked 500 beer purchases; 120 were "light" beers. What's the probability the next beer ordered is light?
P(Light) = 120 / 500 = 0.24
Q2: A basketball player has taken 90 free throws and made 68. What's the probability the next free throw is made?
P(Made) = 68 / 90 ≈ 0.756
Defines the probability of an event as reflecting a decision maker's state of mind / personal belief that an event will occur, based on judgment rather than counting outcomes or historical frequency data. Examples: a manager estimating whether a new supplier's shipment arrives on time (weighing reputation, weather, order size), or a juror assessing the probability a defendant is at fault based on testimony and evidence. There is no formula — it is an informed opinion.
Classical → problem gives you counts of equally-likely outcomes ("dealer has 5 GM, 6 Ford..."), often phrases like "at random," "one item is selected." Relative frequency → problem gives you historical data ("of the past 500 purchases..." "has taken 90 free throws..."). Subjective → problem asks someone to "assess" or "estimate" a belief with no hard counting data (managers, jurors, experts).
| Approach | Basis | Requires Equally Likely Outcomes? | Requires Historical Data? | Example Clue Words |
|---|---|---|---|---|
| Classical | Counting outcomes | Yes | No | "randomly selected," "dealer has X of each" |
| Relative Frequency | Observed data over many trials | No | Yes | "of the past N," "has attempted/made" |
| Subjective | Personal judgment/belief | No | No | "assess," "believes," "estimates the chance" |
1. Identify the experiment, the sample space, and the event being asked about.
2. Decide which of the three assessment approaches applies (classical/relative frequency/subjective)
using the clue-word table above.
3. If a tree diagram is available or can be drawn quickly, use it to enumerate the sample space rather
than trying to list outcomes from memory.
4. Double-check any given probability distribution against Rules 1 and 2 before using it further (values
in [0,1], sum to 1).
There are 9 numbered probability rules in this course. The most heavily tested are Rules 1, 2, 4 (general addition), 5 (mutually exclusive addition), 6 (conditional probability), 7 (conditional probability for independent events), 8 (general multiplication) and 9 (multiplication for independent events). Master the "OR = add, AND = multiply" mental shortcut below before memorizing formulas.
The word "OR" means ADD. The word "AND" means MULTIPLY. Whenever you see "or" in a probability question, you are almost always looking at an addition rule (Rule 4 or 5). Whenever you see "and," you are almost always looking at a multiplication rule (Rule 8 or 9) or reading a joint probability straight off a contingency table.
Rule 1: For any event Ei, 0 ≤ P(Ei) ≤ 1 for all i (0 = no chance, 1 = 100% certain).
Rule 2: The sum of the probabilities of all possible outcomes in the sample space is 1.
A sales manager displays three products. The probability distribution for the number sold in the first 30 minutes is:
| x = number sold | P(x) |
|---|---|
| 0 | 0.2 |
| 1 | 0.3 |
| 2 | 0.4 |
| 3 | 0.1 |
Check Rule 1: every value is between 0 and 1. ✓ Check Rule 2: 0.2 + 0.3 + 0.4 + 0.1 = 1.00 ✓. This table is a valid probability distribution.
The probability of an event Ei equals the sum of the probabilities of the individual outcomes forming Ei.
Q: Roll a 6-sided die. What is the probability the value is an odd number?
E = {1, 3, 5}. P(E) = 1/6 + 1/6 + 1/6 = 3/6 = 1/2
Q: Using the sales table above, what is the probability that 2 or more products sell?
E = {2, 3}. P(E) = 0.4 + 0.1 = 0.5
Q: A real estate investment has these possible returns: -$100,000 (P=0.10), $0 (P=0.20), +$20,000 (P=0.50), +$500,000 (P=0.20). What is the probability of making at least $20,000?
E = {$20,000, $500,000}. P(E) = 0.50 + 0.20 = 0.70
The complement of event E is the collection of all possible outcomes not contained in E.
Q: An inspector checks 7 products for defects. The distribution of the number of defects is:
| x = defects | P(x) |
|---|---|
| 0 | 0.03 |
| 1 | 0.12 |
| 2 | 0.23 |
| 3 | 0.27 |
| 4 | 0.20 |
| 5 | 0.10 |
| 6 | 0.04 |
| 7 | 0.01 |
What is the probability of 1 or more defects? Instead of adding P(1) through P(7), use the complement: the complement of "1 or more" is "0 defects."
P(x ≥ 1) = 1 − P(x = 0) = 1 − 0.03 = 0.97
Whenever a question asks for "at least 1," "1 or more," "at least one," or "not," it is almost always faster to compute 1 minus the complement (usually "exactly 0" or "none") rather than summing many individual terms.
Scenario: Drawing one card from a standard deck of 52 cards. What is P(Red or Ace)?
| Type \ Color | Red | Black | Total |
|---|---|---|---|
| Ace | 2 | 2 | 4 |
| Non-Ace | 24 | 24 | 48 |
| Total | 26 | 26 | 52 |
Reading a contingency table: row totals and column totals give you the marginal (single-event) probabilities. Cell values give you joint (both-events) probabilities.
P(Red) = 26/52. P(Ace) = 4/52. P(Red and Ace) = 2/52 (the 2 red aces — the cell where the Ace row meets the Red column).
The 2 red aces live in both the "Red" count and the "Ace" count. If you just added 26/52 + 4/52 you would count them twice — that's exactly why we subtract the intersection.
The #1 addition-rule error: computing P(E1) + P(E2) and stopping there. Whenever the two events can overlap (i.e., are not mutually exclusive), you must subtract P(E1 and E2) or you will double-count outcomes that belong to both events — like counting the 2 red aces twice in the example above.
When two events cannot occur together, P(E1 and E2) = 0, so the general addition rule simplifies to:
Q: A retail store has three styles of work boots: 6 pairs of style A, 12 pairs of style B, 2 pairs of style C (20 total). A clerk randomly picks one pair to display. What is P(style A or style C)?
Because a single pair can only be one style, these events are mutually exclusive — no subtraction needed.
Conditional probability is the probability that an event will occur given that some other event has already happened.
Look for the word "given" (or phrases like "given that," "if we know," "assuming that already occurred"). Whatever comes right after "given" goes in the denominator — it's the event you already know happened, so it shrinks your sample space down to just that subset.
Scenario: Of the cars on a used car lot, 70% have air conditioning (AC), 40% have a CD player (CD), and 20% of the cars have both. What is the probability that a car has a CD player, given that it has AC?
| CD | No CD | Total | |
|---|---|---|---|
| AC | 0.20 | 0.50 | 0.70 |
| No AC | 0.20 | 0.10 | 0.30 |
| Total | 0.40 | 0.60 | 1.00 |
Reading the table: "given AC" tells you to restrict attention to the AC row (0.70 total). Within that row, 0.20 of all cars have CD, so the conditional probability re-scales 0.20 against the 0.70 row total.
Q: A company has 500 employees; 300 hold college degrees. Two employees are randomly selected for a front-row parking space (without replacement). What is the probability the second person selected has a college degree, given that the first person selected also has a college degree?
If the first person (with a degree) has already been removed, 299 degree-holders remain out of 499 total employees.
For independent events E1, E2:
The conditional probability of one event, given a second independent event has already occurred, is simply the (unconditional) probability of the first event — because independence means "knowing about E2 tells you nothing new about E1."
Scenario: Is the time an employee has been with a company independent of the employee's gender?
| Gender \ Tenure | Under 1 yr | 1-3 yrs | 4-10 yrs | 10-20 yrs | Over 20 yrs | Total |
|---|---|---|---|---|---|---|
| Male | 5 | 10 | 8 | 12 | 15 | 50 |
| Female | 15 | 30 | 24 | 36 | 45 | 150 |
| Total | 20 | 40 | 32 | 48 | 60 | 200 |
To check independence: compare P(Under 1 yr | Male) to P(Under 1 yr) overall. P(Under 1 yr | Male) = 5/50 = 0.10. P(Under 1 yr overall) = 20/200 = 0.10. They match! Repeating this check for every cell in this table confirms gender and tenure are independent here (every row is proportional to the total row).
To test whether events A and B are independent, check whether P(A | B) = P(A). If they're equal (or very close), the events are independent. If P(A|B) ≠ P(A), the events are dependent.
Used when you don't directly know the joint probability but do know a conditional probability:
Scenario: Vehicles use gasoline (E1, P = 0.8) or diesel (E2, P = 0.2). Given the fuel type, vehicles are trucks, cars, or SUVs with these conditional probabilities: P(Truck|Gas)=0.2, P(Car|Gas)=0.5, P(SUV|Gas)=0.3; P(Truck|Diesel)=0.6, P(Car|Diesel)=0.1, P(SUV|Diesel)=0.3.
Multiply along each branch of the tree:
Check: 0.16+0.40+0.24+0.12+0.02+0.06 = 1.00 ✓ (Rule 2 confirms the tree is complete.)
Q: A boutique hotel has 20 rooms: 5 suites, 15 traditional. What is the probability the first two rooms booked (randomly, no replacement) are both suites?
P(1st is Suite) = 5/20. Given the first was a suite, 4 suites remain out of 19 rooms: P(2nd is Suite | 1st is Suite) = 4/19.
The joint probability of two independent events is simply the product of their individual probabilities:
Q: At a small airport, 60% of passengers fly United, 30% fly Delta, 10% fly Southwest. Separately, 70% of fliers travel for pleasure and 30% for business. Assuming trip type and airline are independent, what is the probability a flier uses United and is a business traveler?
Because "airline used" and "trip purpose" are stated to be independent, we can multiply the plain (unconditional) probabilities directly — no need to find a conditional probability first.
Rule 9 (P(E1) × P(E2)) only works for independent events. If the events are dependent (e.g., selecting people without replacement, or gender/experience combos that are related), you must use the general multiplication rule (Rule 8): P(E1 and E2) = P(E1) × P(E2|E1), where the second term uses the conditional, not the plain, probability.
1. See "or" → Addition rule. Ask: "Can both happen together?" If no (mutually exclusive) → just add. If yes → add then subtract the overlap.
2. See "and" or a joint cell in a table → Multiplication rule (or read straight off the table if you already have the joint probability). Ask: "Does one event affect the other?" If no (independent) → just multiply the plain probabilities. If yes (dependent) → multiply the first probability by the conditional probability of the second.
3. See "given" → Conditional probability. Whatever follows "given" becomes the denominator/reduced sample space.
| Rule | Name | Formula | When to Use |
|---|---|---|---|
| 1 | Possible Values | 0 ≤ P(Ei) ≤ 1 | Checking a probability is valid |
| 2 | Summation | ΣP(ei) = 1 | Checking a full distribution is valid |
| 3 | Addition for Individual Outcomes | P(Ei) = ΣP(ej) | Event is a set of several base outcomes |
| — | Complement Rule | P(Ē) = 1 − P(E) | "At least one," "1 or more," "not" |
| 4 | Addition — Any Two Events | P(E1 or E2) = P(E1)+P(E2)−P(E1 and E2) | "Or" question, events CAN overlap |
| 5 | Addition — Mutually Exclusive | P(E1 or E2) = P(E1)+P(E2) | "Or" question, events CANNOT overlap |
| 6 | Conditional — Any Two Events | P(E1|E2) = P(E1 and E2)/P(E2) | "Given" question, general case |
| 7 | Conditional — Independent Events | P(E1|E2) = P(E1) | "Given" question, events are independent |
| 8 | Multiplication — Any Two Events | P(E1 and E2) = P(E1)×P(E2|E1) | "And" question, events dependent |
| 9 | Multiplication — Independent Events | P(E1 and E2) = P(E1)×P(E2) | "And" question, events independent |
Scenario: A survey of 300 shoppers records whether they made a purchase and whether they used a coupon:
| Purchased | No Purchase | Total | |
|---|---|---|---|
| Used Coupon | 90 | 30 | 120 |
| No Coupon | 60 | 120 | 180 |
| Total | 150 | 150 | 300 |
(a) P(Purchased): 150/300 = 0.50 (a marginal/row-column total probability)
(b) P(Used Coupon and Purchased): 90/300 = 0.30 (a joint probability — read straight from the intersection cell)
(c) P(Purchased | Used Coupon): restrict to the "Used Coupon" row (120 total); 90 of those purchased.
(d) Are "Used Coupon" and "Purchased" independent? Compare P(Purchased|Coupon) = 0.75 to P(Purchased) overall = 0.50. Since 0.75 ≠ 0.50, coupon use and purchasing are dependent — coupon users are considerably more likely to purchase.
(e) P(Used Coupon or Purchased): events can overlap (Rule 4).
When a question asks for P(A | B), you restrict to row/column B and divide by B's total — not the grand total. A frequent slip is dividing the joint cell by the grand total (giving the joint probability) instead of by the row/column total (which gives the conditional probability). Always ask: "Am I being asked for a joint probability (divide by grand total) or a conditional probability (divide by the row/column total of the 'given' event)?"
1. A process that produces a single outcome whose result cannot be predicted with certainty is called a(n):
2. Two events are ________ if the occurrence of one precludes the occurrence of the other.
3. A weather forecaster says "I believe there is a 70% chance of rain tomorrow" based on years of experience reading patterns, with no formal count of outcomes. Which probability approach is this?
4. If P(A) = 0.4, P(B) = 0.5, and P(A and B) = 0.2, what is P(A or B)?
5. If P(A) = 0.3, what is P(Ā), the complement of A?
6. For independent events E1 and E2, P(E1 and E2) equals:
7. The phrase "given that" in a probability problem is your cue to use:
8. A bag has 4 red and 6 blue marbles. One marble is drawn at random. What is the probability it is red?
9. A survey shows 120 of 300 shoppers used a coupon, and of those, 90 purchased. What is P(Purchased | Coupon)?
10. Which pair of formulas correctly distinguishes the two addition rules?
Q1. Of 200 employees, a contingency table shows 90 are managers and of those, 30 have an MBA. Overall, 60 employees (managers and non-managers combined) have an MBA. What is P(MBA | Manager)?
Answer: "Given Manager" restricts our sample space to the 90 managers. Of those, 30 have an MBA.
Q2. A deck of 52 cards. What is P(King or Spade)?
Answer: P(King) = 4/52. P(Spade) = 13/52. P(King and Spade) = 1/52 (the King of Spades — don't double count it).
Q3. Two fair coins are tossed. What is the probability of getting at least one head?
Answer: Use the complement rule. The only way to get zero heads is Tails-Tails: P(TT) = 0.5 × 0.5 = 0.25 (independent tosses, Rule 9).
Classical Probability:
Relative Frequency Probability:
Rule 1 — Possible Values:
Rule 2 — Summation:
Rule 3 — Addition for Individual Outcomes:
Complement Rule:
Rule 4 — Addition Rule for Any Two Events:
Rule 5 — Addition Rule for Mutually Exclusive Events:
Rule 6 — Conditional Probability for Any Two Events:
Rule 7 — Conditional Probability for Independent Events:
Rule 8 — Multiplication Rule for Any Two Events:
Rule 9 — Multiplication Rule for Independent Events:
| Term | Definition |
|---|---|
| Experiment | A process that produces a single outcome whose result cannot be predicted with certainty. |
| Sample space | The collection of all outcomes that can result from a selection, decision, or experiment. |
| Event | A subset of outcomes from the sample space that you're interested in measuring. |
| Mutually exclusive | Two events that cannot occur at the same time — the occurrence of one precludes the other. |
| Independent events | Two events where the occurrence of one in no way influences the probability of the other occurring. |
| Complement | The collection of all possible outcomes not contained in a given event E, written Ē; P(Ē) = 1 − P(E). |
| Conditional probability | The probability an event will occur given that some other event has already happened: P(E1|E2). |
| Joint probability | The probability that two events both occur together, written P(E1 and E2); read directly from the intersection cell of a contingency table. |