20. Inferential Statistics

Inferential statistics involves using sample data to make inferences, predictions, or generalizations about a larger population. It's a crucial aspect of data analysis that allows us to draw conclusions beyond the immediate data we have.

20.1 Overview of Inferential Statistics

Inferential statistics builds upon descriptive statistics. While descriptive statistics summarizes and describes the characteristics of a dataset, inferential statistics aims to understand the underlying population from which the sample was drawn. Key goals include:

• Hypothesis Testing: Determining if there's enough evidence in a sample to reject a null hypothesis about a population.
• Estimation: Estimating population parameters (like the mean or proportion) based on sample statistics.
• Prediction: Using sample data to predict future outcomes or values.

20.2 Degrees of Freedom

Degrees of freedom (df) represent the number of independent values in a data sample that are free to vary when estimating a parameter. It's a critical concept in many statistical tests.

Concept: When we calculate a sample statistic (like the sample mean) that will be used to estimate a population parameter, we lose one degree of freedom because the sample mean is fixed. The remaining observations can then vary freely.

Example: If you have a sample of $n$ observations and you've calculated the sample mean, you can freely choose $n-1$ of those observations, but the last observation is determined by the sample mean and the values of the other $n-1$ observations.

The degrees of freedom vary depending on the specific statistical test being used.

20.3 Central Limit Theorem (CLT)

The Central Limit Theorem (CLT) is a fundamental concept in inferential statistics. It states that, regardless of the original population's distribution, the distribution of sample means will tend to be normally distributed, provided the sample size is sufficiently large.

Key Implications:

• Normality of Sample Means: Even if the population is skewed, the sampling distribution of the mean will approach a normal distribution as the sample size ($n$) increases.
• Mean of Sample Means: The mean of the sampling distribution of the mean is equal to the population mean ($\mu$).
• Standard Deviation of Sample Means (Standard Error): The standard deviation of the sampling distribution of the mean (known as the standard error) is equal to the population standard deviation ($\sigma$) divided by the square root of the sample size ($\sqrt{n}$).

The CLT is essential for many inferential statistical methods, particularly those that rely on the assumption of normality, such as $t$-tests and confidence intervals.

20.4 Parameters vs. Test Statistics

It's important to distinguish between population parameters and sample test statistics.

• Parameter: A numerical characteristic of a population. Parameters are usually unknown and are what we aim to infer. They are often represented by Greek letters.

  • Examples:
    • $\mu$ (mu): Population mean
    • $\sigma$ (sigma): Population standard deviation
    • $p$: Population proportion

• Test Statistic: A numerical value calculated from sample data that is used to make inferences about a population parameter. Test statistics are used to test hypotheses. They are often represented by Roman letters.

  • Examples:
    • $\bar{x}$ (x-bar): Sample mean
    • $s$: Sample standard deviation
    • $\hat{p}$ (p-hat): Sample proportion

The goal of inferential statistics is to use sample test statistics to estimate or make decisions about population parameters.

20.5 Test Statistics

A test statistic is a value computed from sample data in hypothesis testing. It quantifies how far the sample result deviates from what would be expected under the null hypothesis. The value of the test statistic is then compared to a critical value from a sampling distribution or used to calculate a $p$-value.

Commonly used test statistics include:

• z-statistic: Used when the population standard deviation ($\sigma$) is known or when the sample size is large ($n \ge 30$) and $\sigma$ is unknown (approximated by the sample standard deviation $s$). $$ z = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}} $$ or $$ z = \frac{\bar{x} - \mu_0}{s / \sqrt{n}} $$ where:

  • $\bar{x}$ is the sample mean
  • $\mu_0$ is the hypothesized population mean under the null hypothesis
  • $\sigma$ is the population standard deviation
  • $s$ is the sample standard deviation
  • $n$ is the sample size

• t-statistic: Used when the population standard deviation ($\sigma$) is unknown and the sample size is small ($n < 30$), assuming the population is approximately normally distributed. $$ t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}} $$ where:

  • $\bar{x}$ is the sample mean
  • $\mu_0$ is the hypothesized population mean under the null hypothesis
  • $s$ is the sample standard deviation
  • $n$ is the sample size
  • The distribution of the $t$-statistic depends on the degrees of freedom ($n-1$).

• Chi-squared ($\chi^2$) statistic: Used for tests of independence and goodness-of-fit for categorical data.

• F-statistic: Used in ANOVA (Analysis of Variance) to compare means of two or more groups.

The specific formula for a test statistic depends on the type of data and the hypothesis being tested.

20.6 Estimation

Estimation is the process of using sample data to approximate the value of an unknown population parameter. There are two main types of estimation:

1. Point Estimation:

   • A single value is calculated from the sample to estimate the population parameter.
   • The sample mean ($\bar{x}$) is a point estimate for the population mean ($\mu$).
   • The sample proportion ($\hat{p}$) is a point estimate for the population proportion ($p$).
   • Limitation: A point estimate doesn't convey the uncertainty associated with the estimate.

2. Interval Estimation (Confidence Intervals):

   • A range of values is calculated from the sample data, within which the population parameter is likely to lie, with a certain level of confidence.
   • This method provides a measure of the precision of the estimate.

20.7 Standard Error

The Standard Error (SE) is the standard deviation of the sampling distribution of a statistic. It measures the variability of sample statistics obtained from different samples of the same population.

• For the Sample Mean ($\bar{x}$): The standard error of the mean (SEM) is calculated as: $$ SEM = \frac{\sigma}{\sqrt{n}} $$ If the population standard deviation ($\sigma$) is unknown, we use the sample standard deviation ($s$) as an estimate: $$ SEM \approx \frac{s}{\sqrt{n}} $$ A smaller standard error indicates that sample means are clustered more tightly around the population mean, suggesting a more reliable estimate.

• For the Sample Proportion ($\hat{p}$): The standard error of a proportion is calculated as: $$ SE_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}} $$ If the population proportion ($p$) is unknown, we use the sample proportion ($\hat{p}$) as an estimate: $$ SE_{\hat{p}} \approx \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} $$

The standard error is a crucial component in constructing confidence intervals and performing hypothesis tests.

20.8 Confidence Interval

A Confidence Interval (CI) is a range of values, derived from sample statistics, that is likely to contain the value of an unknown population parameter. It is defined by a confidence level, which represents the probability that the interval contains the true population parameter.

Structure of a Confidence Interval:

$$ \text{Point Estimate} \pm (\text{Critical Value} \times \text{Standard Error}) $$

Common Confidence Levels:

• 90% confidence
• 95% confidence (most common)
• 99% confidence

Interpretation: A 95% confidence interval means that if we were to take many random samples and construct a confidence interval from each sample, approximately 95% of those intervals would contain the true population parameter. It does not mean there is a 95% probability that the true parameter falls within a specific calculated interval.

Example (Confidence Interval for a Mean):

Suppose we want to estimate the average height of adult males in a city. We take a sample of 100 men and find a sample mean height of 175 cm with a sample standard deviation of 7 cm. We want to construct a 95% confidence interval for the population mean height.

• Point Estimate: $\bar{x} = 175$ cm
• Standard Error (SE): $SE = \frac{s}{\sqrt{n}} = \frac{7}{\sqrt{100}} = \frac{7}{10} = 0.7$ cm
• Critical Value: For a 95% confidence level and large sample size (or degrees of freedom), the z-critical value is approximately 1.96.
• Margin of Error (ME): $ME = \text{Critical Value} \times SE = 1.96 \times 0.7 = 1.372$ cm

Confidence Interval: $$ 175 \pm 1.372 $$ This gives us an interval of $(173.628, 176.372)$ cm.

Conclusion: We are 95% confident that the true average height of adult males in this city lies between 173.63 cm and 176.37 cm.