Probability Theory Fundamentals
From basic probability rules to complex distributions and real-world scenarios. Master the mathematics of uncertainty.
Ready to calculate?
Try our Probability Calculator
Introduction
Probability is the mathematical study of uncertainty and randomness. It provides a framework for quantifying how likely events are to occur, making it fundamental to statistics, data science, machine learning, finance, and countless other fields.
Understanding probability helps us make better decisions under uncertainty, assess risks, and interpret statistical results. Whether you're predicting weather, analyzing medical test results, or playing games, probability theory provides the tools you need.
What You'll Learn
- • Basic probability concepts and terminology
- • Rules for combining probabilities
- • Conditional probability and Bayes' theorem
- • Counting techniques (permutations and combinations)
- • Common probability distributions
Basic Concepts
Key Terminology
Experiment
A process that produces observable outcomes. Example: Rolling a die.
Sample Space (S)
The set of all possible outcomes. Example: S = {1, 2, 3, 4, 5, 6} for a die roll.
Event
A subset of the sample space. Example: "Rolling an even number" = {2, 4, 6}.
Probability
A number between 0 and 1 representing the likelihood of an event.
Probability Formula
Example: Drawing an Ace from a deck
P(Ace) = 4 aces / 52 cards = 4/52 = 1/13 ≈ 0.077
Probability Scale
0
Impossible
0.5
Equally likely
1
Certain
Probability Rules
Complement Rule
The probability of an event NOT happening equals 1 minus the probability of it happening.
Example: If P(rain) = 0.3, then P(no rain) = 1 - 0.3 = 0.7
Addition Rule
For mutually exclusive events (can't happen together), P(A and B) = 0.
Example: Drawing a King OR a Heart from a deck
P(King) = 4/52, P(Heart) = 13/52, P(King of Hearts) = 1/52
P(King or Heart) = 4/52 + 13/52 - 1/52 = 16/52 ≈ 0.308
Multiplication Rule
For independent events, P(B|A) = P(B), so P(A and B) = P(A) × P(B).
Example: Flipping two heads in a row
P(H and H) = 0.5 × 0.5 = 0.25
Conditional Probability
Conditional probability is the probability of an event occurring given that another event has already occurred. It's denoted P(A|B), read as "probability of A given B."
Formula
Example: Medical Testing
A disease affects 1% of the population. A test is 90% accurate for positive cases and 95% accurate for negative cases.
Given Information
- • P(Disease) = 0.01
- • P(Positive | Disease) = 0.90
- • P(Negative | No Disease) = 0.95
Question
What is P(Disease | Positive)?
This requires Bayes' Theorem...
Bayes' Theorem
Bayes' Theorem allows us to reverse conditional probabilities. It's crucial in machine learning, medical diagnosis, spam filtering, and many other applications.
Bayes' Theorem Formula
P(A|B)
Posterior probability
P(B|A)
Likelihood
P(A)
Prior probability
P(B)
Evidence
Solving the Medical Test Example
Using Bayes' Theorem to find P(Disease | Positive test):
P(Disease | Positive) = [P(Positive | Disease) × P(Disease)] / P(Positive)
First, find P(Positive) using total probability:
P(Positive) = P(Pos|Disease)×P(Disease) + P(Pos|No Disease)×P(No Disease)
= (0.90 × 0.01) + (0.05 × 0.99) = 0.009 + 0.0495 = 0.0585
P(Disease | Positive) = (0.90 × 0.01) / 0.0585 = 0.154 (15.4%)
Insight: Even with a positive test, there's only a 15.4% chance of having the disease! This counterintuitive result is called the base rate fallacy.
Permutations & Combinations
Counting techniques are essential for calculating probabilities. The key question is: Does order matter?
Permutations
Order DOES matter
Example: Arranging 3 people in 5 chairs
P(5,3) = 5!/(5-3)! = 120/2 = 60
Combinations
Order does NOT matter
Example: Choosing 3 people from 5
C(5,3) = 5!/(3!×2!) = 120/12 = 10
When to Use Each
Use Permutations for:
- • Arranging people in a line
- • Assigning positions (1st, 2nd, 3rd)
- • Creating passwords/PINs
- • Race finishing orders
Use Combinations for:
- • Forming committees
- • Choosing lottery numbers
- • Selecting pizza toppings
- • Poker hands
Probability Distributions
A probability distribution describes all possible outcomes and their probabilities. Here are the most common ones:
Binomial Distribution
Models the number of successes in n independent trials, each with probability p of success.
Example: Probability of getting exactly 3 heads in 5 coin flips.
Normal Distribution
The famous "bell curve." Many natural phenomena follow this distribution.
Examples: Heights, test scores, measurement errors.
Poisson Distribution
Models the number of events occurring in a fixed interval when events happen at a constant average rate.
Examples: Customer arrivals per hour, emails received per day.
Real-World Applications
🏥 Medicine
Diagnostic test interpretation, clinical trial design, disease risk assessment.
💰 Finance
Risk assessment, options pricing, portfolio optimization, fraud detection.
🤖 Machine Learning
Bayesian classifiers, neural networks, recommendation systems.
🎮 Gaming
Expected value calculations, game theory, optimal strategies.
Summary
Key Takeaways
- 1.Probability ranges from 0 (impossible) to 1 (certain).
- 2.Addition rule for "or" events; multiplication rule for "and" events.
- 3.Bayes' Theorem reverses conditional probabilities.
- 4.Use permutations when order matters, combinations when it doesn't.
- 5.Different distributions model different types of random phenomena.