7.5 Multiplication Theorem: Statistical Independence in AI

Master the Multiplication Theorem for AI & ML. Understand probability, joint probability, independent & dependent events for accurate statistical analysis.

7.5 Multiplication Theorem of Probability: Independent and Dependent Events

The Multiplication Theorem is a fundamental concept in probability used to determine the likelihood of two or more events occurring simultaneously. It is particularly useful for calculating the joint probability of multiple events. The application of the theorem depends on whether the events are independent or dependent.

1. Independent Events

Two events, A and B, are considered independent if the occurrence of one event does not affect the probability of the other event occurring.

Formula for Independent Events

For two independent events A and B, the probability that both occur is the product of their individual probabilities:

P(A ∩ B) = P(A) × P(B)

Where:

  • P(A ∩ B) is the probability that both event A and event B occur.
  • P(A) is the probability of event A occurring.
  • P(B) is the probability of event B occurring.

For three independent events A, B, and C:

P(A ∩ B ∩ C) = P(A) × P(B) × P(C)

Example: Independent Events

Scenario: Sam and John are taking a quiz independently.

  • The probability that Sam solves the quiz is P(Sam) = 0.6.
  • The probability that John solves the quiz is P(John) = 0.4.

Since their performance is independent:

P(Sam ∩ John) = P(Sam) × P(John)
              = 0.6 × 0.4
              = 0.24

Answer: There is a 24% chance that both Sam and John solve the quiz.

2. Dependent Events

Two events are considered dependent if the occurrence of one event does affect the probability of the other event occurring. In such cases, we use conditional probability.

Formula for Dependent Events

For two dependent events A and B, the probability that both occur is the probability of event A occurring multiplied by the probability of event B occurring given that event A has already occurred:

P(A ∩ B) = P(A) × P(B | A)

Alternatively, this can be expressed as:

P(A ∩ B) = P(B) × P(A | B)

Where:

  • P(B | A) is the conditional probability of event B occurring given that event A has already occurred.
  • P(A | B) is the conditional probability of event A occurring given that event B has already occurred.

For three dependent events A, B, and C:

P(A ∩ B ∩ C) = P(A) × P(B | A) × P(C | A ∩ B)

Example: Dependent Events

Scenario: A company has 90 employees.

  • 48 are men.
  • 42 are women.

Among the 42 women:

  • 12 are under 30.
  • 20 are between 30 and 45.
  • The remaining women are over 45 (42 - 12 - 20 = 10 women).

Question: What is the probability that a randomly selected employee is a woman and that woman is under 30?

Let:

  • Event A = The selected employee is a woman.
  • Event B = The selected employee is under 30.

We need to find P(A ∩ B).

  • The probability of selecting a woman is P(A) = 42 / 90.
  • Given that the selected employee is a woman (Event A has occurred), the probability that she is under 30 is P(B | A) = 12 / 42 (since there are 12 women under 30 out of a total of 42 women).

Using the formula for dependent events:

P(A ∩ B) = P(A) × P(B | A)
          = (42 / 90) × (12 / 42)
          = 12 / 90
          = 2 / 15

Answer: The probability that a randomly selected employee is a woman under 30 is 2/15.

Statistical Independence

Two events are statistically independent if the occurrence of one event has no influence on the occurrence of the other. This is the formal definition that underlies the multiplication theorem for independent events.

Mathematical Condition for Statistical Independence

Let E₁ and E₂ be two events. They are statistically independent if and only if:

P(E₁ ∩ E₂) = P(E₁) × P(E₂)

If this condition is not met (i.e., P(E₁ ∩ E₂) ≠ P(E₁) × P(E₂)), then the events are not statistically independent (they are dependent).

Example: Testing for Statistical Independence

Scenario: A fair six-sided die is rolled once.

  • Sample Space (S): {1, 2, 3, 4, 5, 6}

Let:

  • E₁ = The event that the number rolled is even.
    • E₁ = {2, 4, 6}
    • P(E₁) = 3 / 6 = 1/2
  • E₂ = The event that the number rolled is greater than 3.
    • E₂ = {4, 5, 6}
    • P(E₂) = 3 / 6 = 1/2

Now, let's find the intersection of E₁ and E₂:

  • E₁ ∩ E₂ = The event that the number rolled is both even and greater than 3.
    • E₁ ∩ E₂ = {4, 6}
    • P(E₁ ∩ E₂) = 2 / 6 = 1/3

Check for Independence:

We compare P(E₁ ∩ E₂) with P(E₁) × P(E₂):

  • P(E₁) × P(E₂) = (1/2) × (1/2) = 1/4
  • P(E₁ ∩ E₂) = 1/3

Since 1/3 ≠ 1/4:

Conclusion: The events E₁ (rolling an even number) and E₂ (rolling a number greater than 3) are not statistically independent. The probability of both occurring is different from the product of their individual probabilities.