What is Probability? A Data Science & AI Intro

Understand probability: the core concept for data science, AI, and machine learning. Learn its range and importance in predicting event likelihood.

7.1 What is Probability?

Probability is a fundamental concept in statistics and mathematics that quantifies the likelihood of a specific event occurring within a random experiment or uncertain situation. It provides a numerical measure to predict how likely an event is to happen.

Probability Range

The value of probability always falls within a specific range:

  • Probability = 0: This signifies that the event is impossible to occur.
  • Probability = 1: This signifies that the event is certain or guaranteed to occur.

Therefore, the value of probability, denoted as P(A), always lies between 0 and 1, inclusive: $0 \le P(A) \le 1$

Basic Probability Formula

The fundamental formula for calculating probability involves the ratio of favorable outcomes to the total number of possible outcomes within a sample space:

$P(A) = \frac{n(A)}{n(S)}$

Where:

  • $P(A)$: The probability of event A occurring.
  • $n(A)$: The number of favorable outcomes for event A.
  • $n(S)$: The total number of possible outcomes in the sample space (S).

Example: Rolling a Die

Problem: What is the probability of getting a 6 when rolling a standard six-sided die?

Solution:

  • Favorable Outcomes ($n(A)$): There is only one side with a '6'. So, $n(A) = 1$.
  • Total Outcomes ($n(S)$): A standard six-sided die has six possible outcomes (1, 2, 3, 4, 5, 6). So, $n(S) = 6$.

Using the formula: $P(\text{getting a 6}) = \frac{1}{6}$

Therefore, the probability of rolling a 6 is $1/6$.

Key Terminologies in Probability

Understanding the core terminology is crucial for grasping probability concepts.

  1. Event: An event is a specific outcome or a collection of outcomes from a random experiment. Each distinct result of an experiment is considered an event.

  2. Sample Space (S): The sample space is the set of all possible outcomes that can occur in a random experiment. It is typically denoted by the letter 'S'.

    • Example: If you toss a coin once, the sample space is $S = {\text{Heads, Tails}}$.
  3. Sure Event: A sure event is an event that is guaranteed to happen. Its probability is always 1. $P(\text{Sure Event}) = 1$

  4. Impossible Event: An impossible event is an event that cannot occur under any circumstances. Its probability is always 0. $P(\text{Impossible Event}) = 0$

  5. Favorable Outcome: A favorable outcome refers to any outcome within the sample space that matches the specific event of interest.

  6. Independent Events: Two or more events are considered independent if the occurrence or non-occurrence of one event does not affect the probability of the other events occurring.

    • Example: Rolling a die and flipping a coin are independent events. The outcome of the die roll does not influence the outcome of the coin flip.
  7. Mutually Exclusive Events: Two events are mutually exclusive if they cannot happen at the same time. If one event occurs, the other event cannot occur. The intersection of mutually exclusive events is an empty set. $P(A \cap B) = 0$

    • Example: Getting a '2' and getting a '5' on the same roll of a single die. These are mutually exclusive because a single roll cannot result in both outcomes simultaneously.
  8. Overlapping Events: Overlapping events (also known as dependent or non-mutually exclusive events) are events that can occur together. They share common outcomes. When calculating the probability of either of two overlapping events occurring, the formula accounts for the shared outcomes to avoid double-counting: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Summary Table

TermDefinitionProbability Value/Formula
ProbabilityLikelihood of an event occurring
FormulaRatio of favorable outcomes to total outcomes$P(A) = \frac{n(A)}{n(S)}$
Range of ProbabilityThe bounds within which probability values lie$0 \le P(A) \le 1$
Sample Space (S)The set of all possible outcomes of a random experiment
Sure EventAn event that is guaranteed to occur$P = 1$
Impossible EventAn event that cannot occur$P = 0$
Favorable OutcomeAn outcome that matches the specific event of interest
Independent EventsEvents whose occurrences do not influence each other$P(A \cap B) = P(A) \times P(B)$ (for independent events)
Mutually ExclusiveEvents that cannot happen simultaneously (no common outcomes)$P(A \cap B) = 0$
Overlapping EventsEvents that may happen together (share common outcomes)$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Potential Interview Questions

  • What is probability, and how is it defined in mathematical terms?
  • What is the range of possible probability values?
  • How do you calculate the probability of an event using the basic formula?
  • Explain the concept of a sample space with a clear example.
  • What is the fundamental difference between a sure event and an impossible event?
  • Define independent events and provide a real-life example.
  • What are mutually exclusive events? Give an illustrative example.
  • How do overlapping events differ from mutually exclusive events?
  • What constitutes a favorable outcome in probability calculations?
  • If two events are known to be independent, how does this characteristic affect the calculation of their combined probability?