11.2 Bernoulli Distribution Formula

The Bernoulli distribution is a fundamental discrete probability distribution used to model experiments with exactly two possible outcomes, conventionally labeled as "success" (represented by 1) and "failure" (represented by 0).

Notation

The Bernoulli distribution is commonly denoted as:

$X \sim \text{Bernoulli}(p)$

Where:

• $X$: A random variable representing the outcome of the experiment.
• $p$: The probability of success.
• $1 - p$ (often denoted as $q$): The probability of failure.

The parameter $p$ must satisfy $0 < p < 1$.

Key Functions

The Bernoulli distribution is characterized by its Probability Density Function (PDF) and Cumulative Distribution Function (CDF).

I. Probability Density Function (PDF)

The PDF of a Bernoulli distribution gives the probability that the random variable $X$ takes on a specific value. For the Bernoulli distribution, the possible values for $X$ are 0 and 1.

The formula for the Bernoulli PDF is:

$$ P(X = x) = \begin{cases} p & \text{if } x = 1 \ 1 - p & \text{if } x = 0 \end{cases} $$

This can also be expressed concisely using exponents:

$$ P(X = x) = p^x (1 - p)^{1 - x}, \quad \text{for } x \in {0, 1} $$

Explanation:

• Probability of Success (x = 1): If $x = 1$, the formula simplifies to $P(X = 1) = p^1 (1 - p)^{1 - 1} = p^1 (1 - p)^0 = p \times 1 = p$. This means the probability of observing a "success" is directly given by the parameter $p$.

• Probability of Failure (x = 0): If $x = 0$, the formula simplifies to $P(X = 0) = p^0 (1 - p)^{1 - 0} = 1 \times (1 - p)^1 = 1 - p$. This means the probability of observing a "failure" is $1 - p$.

Example: Suppose a coin flip has a probability of landing heads (success) $p = 0.7$.

• The probability of getting heads is $P(X = 1) = 0.7$.
• The probability of getting tails (failure) is $P(X = 0) = 1 - 0.7 = 0.3$.

II. Cumulative Distribution Function (CDF)

The CDF of a Bernoulli distribution, denoted by $F(x)$, calculates the probability that the random variable $X$ is less than or equal to a specific value $x$.

The formula for the Bernoulli CDF is:

$$ F(x) = P(X \le x) = \begin{cases} 0 & \text{if } x < 0 \ 1 - p & \text{if } 0 \le x < 1 \ 1 & \text{if } x \ge 1 \end{cases} $$

Explanation:

• For $x < 0$: Since the only possible outcomes are 0 and 1, the probability of $X$ being less than any value below 0 is impossible, hence $P(X \le x) = 0$.

• For $0 \le x < 1$: In this range, the only outcome that satisfies $X \le x$ is $X = 0$. Therefore, $P(X \le x) = P(X = 0) = 1 - p$.

• For $x \ge 1$: In this case, $X$ can be either 0 or 1, both of which are less than or equal to $x$. The sum of probabilities for all possible outcomes is always 1, so $P(X \le x) = P(X = 0) + P(X = 1) = (1 - p) + p = 1$.

Example: Let $p = 0.65$ for an experiment.

• $F(x)$ when $x = -1$: Since $-1 < 0$, $F(-1) = 0$.
• $F(x)$ when $x = 0.5$: Since $0 \le 0.5 < 1$, $F(0.5) = 1 - p = 1 - 0.65 = 0.35$. This is the probability of failure ($P(X=0)$).
• $F(x)$ when $x = 2$: Since $2 \ge 1$, $F(2) = 1$. This is the probability of getting a success or failure ($P(X=0) + P(X=1)$).

Summary of Bernoulli Distribution Formulas

Applications

The Bernoulli distribution is foundational in probability and statistics, serving as a building block for more complex distributions like the Binomial distribution. It is widely used in:

• Data Science: Modeling binary outcomes such as click-through rates, customer churn, or the success of an A/B test.
• Quality Control: Assessing the probability of a product being defective or conforming.
• Binary Classification: As the underlying model for the outcome of a single prediction.

Related Concepts

• Bernoulli Trial: A single experiment with only two possible outcomes.
• Binomial Distribution: The sum of independent Bernoulli trials.

---

Interview Questions

Here are some common interview questions related to the Bernoulli distribution:

1. What is the formula for the Probability Distribution Function (PDF) of the Bernoulli distribution?

   • Answer: $P(X = x) = p^x (1 - p)^{1 - x}$ for $x \in {0, 1}$.

2. How do you interpret the PDF of a Bernoulli distribution?

   • It tells us the probability of a single trial resulting in success (if $x=1$) or failure (if $x=0$).

3. What is the Cumulative Distribution Function (CDF) of a Bernoulli distribution?

   • Answer: $F(x) = P(X \le x)$, which is 0 for $x < 0$, $1-p$ for $0 \le x < 1$, and 1 for $x \ge 1$.

4. How does the Bernoulli CDF behave for values less than 0, between 0 and 1, and greater than or equal to 1?

   • Less than 0: $F(x)=0$ (impossible event).
   • Between 0 and 1: $F(x)=1-p$ (probability of failure).
   • Greater than or equal to 1: $F(x)=1$ (probability of success or failure).

5. Can you provide a real-world example illustrating the PDF of the Bernoulli distribution?

   • A coin flip: $p=0.5$ for heads (success), $1-p=0.5$ for tails (failure). Or a survey question with a Yes/No answer.

6. How is the probability of failure represented in the Bernoulli distribution formula?

   • It's represented by $1-p$, or $q$.

7. Why is the Bernoulli distribution considered a discrete distribution?

   • Because its random variable $X$ can only take on a finite number of distinct values (0 and 1).

8. How do you calculate the probability of success and failure using the Bernoulli PDF?

   • Probability of success ($X=1$) is $p$. Probability of failure ($X=0$) is $1-p$.

9. What role does the Bernoulli distribution play in binary classification problems?

   • It models the probability of a specific class being predicted for a single data point. Many classification algorithms aim to estimate this probability $p$.

10. How can the Bernoulli distribution CDF be used in decision-making or quality control?

    • In quality control, the CDF can tell you the probability that a product's defect count is below a certain threshold. In decision-making, it can help assess the likelihood of achieving at least a certain number of successes in a single trial.