4.8 Measurement of Skewness

Skewness is a fundamental statistical measure that quantifies the asymmetry of a data distribution. It indicates whether the data points are equally distributed around the mean, or if they lean more towards one side, either the right (positive skew) or the left (negative skew).

Understanding skewness is crucial in data analysis because many statistical methods and machine learning algorithms assume a normal (symmetric) distribution. Skewness highlights deviations from this assumption, which can significantly impact the reliability and interpretation of analytical results.

Why Measure Skewness?

Measuring and understanding skewness is essential for several reasons:

• Understanding Data Shape: To gain insights into the overall shape and distribution of a dataset, identifying patterns beyond just central tendency and spread.
• Detecting Outliers and Imbalance: To identify potential outliers or an imbalance in the data that might not be apparent from other measures.
• Model Selection: To inform the choice of appropriate statistical tests or machine learning models, as many models perform optimally with symmetric data.
• Data Preprocessing: To guide data transformation strategies (e.g., logarithmic or square root transformations) aimed at normalizing skewed data for improved model performance.

Interpretation of Skewness Values

The value of the skewness coefficient provides a clear indication of the data's asymmetry:

• Skewness = 0: Indicates a perfectly symmetrical distribution. The data is balanced around the mean. This is characteristic of a normal distribution.
• Skewness > 0 (Positive Skew): Indicates that the tail of the distribution extends further to the right. The majority of data points are clustered on the left side, with a few high-value outliers pulling the mean to the right. The mean will typically be greater than the median.
• Skewness < 0 (Negative Skew): Indicates that the tail of the distribution extends further to the left. The majority of data points are clustered on the right side, with a few low-value outliers pulling the mean to the left. The mean will typically be less than the median.

Summary Table of Skewness

Example Use Cases

Skewness plays a significant role in various domains:

• Finance: Analyzing financial returns is a common application. For instance, stock returns might exhibit positive skewness, indicating that while most days have small gains or losses, there's a possibility of rare but significant positive returns (a long right tail).
• Healthcare: In medical studies, skewness can highlight the presence of abnormal or rare conditions. For example, a disease incidence rate might be negatively skewed if most people are healthy, but a few individuals have very high incidences.
• Machine Learning: Understanding the skewness of features can guide crucial data preprocessing steps. Knowing that a feature is highly skewed might prompt the application of transformations (like log or box-cox) to normalize the distribution, which can improve the performance of algorithms sensitive to feature scaling or distribution assumptions.

Related Concepts

Skewness is closely related to the measures of central tendency:

• Mean: The average of the data.
• Median: The middle value in a sorted dataset.
• Mode: The most frequent value in a dataset.

In a perfectly symmetrical (normal) distribution, the mean, median, and mode are all equal. However, in skewed distributions:

• Positively Skewed: Mean > Median > Mode
• Negatively Skewed: Mode > Median > Mean

Addressing heavily skewed data often involves employing data transformation techniques to reduce asymmetry, making the data more amenable to standard statistical analyses and machine learning models.