Descriptive statistics
What would you like to know about descriptive statistics?
Descriptive Statistics
1. Types of Data
| Type | Description | Example |
|---|---|---|
| Nominal | Categories with no order | Blood type, gender |
| Ordinal | Categories with order, unequal intervals | Pain scale (1-10), education level |
| Interval | Equal intervals, no true zero | Temperature (°C), IQ score |
| Ratio | Equal intervals + true zero | Height, weight, income |
2. Measures of Central Tendency
Mean (Arithmetic Average)
- Formula: x̄ = Σx / n
- Sum all values, divide by the count.
- Best for: Interval/ratio data with no extreme outliers.
- Weakness: Sensitive to outliers. One extreme value pulls the mean significantly.
Median
- The middle value when data is sorted in order.
- If n is even, the median = average of the two middle values.
- Best for: Skewed distributions or ordinal data.
- Strength: Not affected by outliers. (e.g., median house price is more meaningful than mean)
Mode
- The most frequently occurring value.
- A dataset can be unimodal, bimodal, or multimodal.
- Best for: Nominal data (e.g., most common blood type).
- Can be used for any data type.
Relationship Between Mean, Median, Mode
| Distribution | Relationship |
|---|---|
| Symmetric (normal) | Mean = Median = Mode |
| Positively skewed (right) | Mode < Median < Mean |
| Negatively skewed (left) | Mean < Median < Mode |
3. Measures of Dispersion (Spread)
Range
- Formula: Range = Max - Min
- Simple but highly sensitive to outliers.
Interquartile Range (IQR)
- Formula: IQR = Q3 - Q1
- Covers the middle 50% of data.
- Robust to outliers. Used in box plots.
- Q1 = 25th percentile, Q3 = 75th percentile
Variance (s² or σ²)
- The average of the squared deviations from the mean.
- Sample variance: s² = Σ(x - x̄)² / (n - 1)
- Population variance: σ² = Σ(x - μ)² / N
- Note: Dividing by (n-1) for sample variance corrects for bias (Bessel's correction).
Standard Deviation (SD)
- Formula: s = √s² (square root of variance)
- Returns to the original units of the data, making it more interpretable than variance.
- Empirical Rule (Normal Distribution):
- ~68% of data falls within ±1 SD of the mean
- ~95% within ±2 SD
- ~99.7% within ±3 SD
Coefficient of Variation (CV)
- Formula: CV = (s / x̄) × 100%
- Expresses SD as a percentage of the mean.
- Useful for comparing spread across datasets with different units or scales.
4. Measures of Shape
Skewness
- Describes the asymmetry of the distribution.
- Positive (right) skew: Long tail to the right. Mean > Median.
- Negative (left) skew: Long tail to the left. Mean < Median.
- Skewness = 0 → perfectly symmetric.
Kurtosis
- Describes the "peakedness" or tail weight of the distribution.
- Leptokurtic (kurtosis > 3): Sharper peak, heavier tails.
- Platykurtic (kurtosis < 3): Flatter peak, lighter tails.
- Mesokurtic (kurtosis = 3): Normal distribution.
- Excess kurtosis = kurtosis - 3 (so normal = 0).
5. The Five-Number Summary
- Minimum
- Q1 (25th percentile)
- Median (Q2, 50th percentile)
- Q3 (75th percentile)
- Maximum
6. Data Visualization Methods
| Chart Type | Best Used For |
|---|---|
| Frequency table / histogram | Distribution of continuous data |
| Bar chart | Comparing categories (nominal/ordinal) |
| Box plot | Comparing spread and detecting outliers |
| Pie chart | Proportions of a whole (use sparingly) |
| Scatter plot | Relationship between two continuous variables |
| Stem-and-leaf plot | Small datasets - shows distribution and raw values |
Identifying Outliers
- Lower fence: Q1 - 1.5 × IQR
- Upper fence: Q3 + 1.5 × IQR
- Any value outside these fences is considered an outlier.
7. Quick Reference - Which Statistic to Use?
| Goal | Use |
|---|---|
| Describe center of symmetric data | Mean |
| Describe center of skewed data | Median |
| Describe most common category | Mode |
| Describe spread, same units as data | Standard Deviation |
| Compare spread across different scales | Coefficient of Variation |
| Detect outliers | IQR method |
| Summarize a distribution | Five-number summary + box plot |
Example
| Statistic | Value |
|---|---|
| Mean | (5+7+7+9+10+12+15) / 7 = 9.29 |
| Median | 9 (middle value) |
| Mode | 7 (appears twice) |
| Range | 15 - 5 = 10 |
| Q1 | 7 |
| Q3 | 12 |
| IQR | 12 - 7 = 5 |
| Variance (sample) | 11.57 |
| SD | 3.40 |
A short summary
Descriptive Statistics - Short Summary
- Mean - arithmetic average; sensitive to outliers
- Median - middle value; better for skewed data
- Mode - most frequent value
- Range - max minus min
- IQR - middle 50% of data (Q3 - Q1); outlier-resistant
- Standard Deviation - average distance from the mean; most commonly used
- Skewness - how asymmetric the distribution is (left or right tail)
- Kurtosis - how peaked or flat the distribution is
Show me examples with real datasets
python3 << 'EOF' import statistics import math # ── DATASET 1: NBA Player Heights (inches) - 15 players, 2023-24 season sample nba_heights = [69, 72, 73, 74, 75, 75, 76, 77, 78, 79, 80, 81, 82, 84, 87] # ── DATASET 2: US Household Incomes ($000s) - skewed sample incomes = [28, 32, 35, 38, 41, 44, 47, 51, 55, 60, 68, 75, 90, 120, 250] def describe(label, data): n = len(data) mean = sum(data) / n median = statistics.median(data) try: mode = statistics.mode(data) except: mode = "No unique mode" variance = statistics.variance(data) # sample variance sd = math.sqrt(variance) sorted_d = sorted(data) q1 = statistics.median(sorted_d[:n//2]) q3 = statistics.median(sorted_d[(n+1)//2:]) iqr = q3 - q1 min_v = min(data) max_v = max(data) rng = max_v - min_v cv = (sd / mean) * 100 # Outlier fences lower_fence = q1 - 1.5 * iqr upper_fence = q3 + 1.5 * iqr outliers = [x for x in data if x < lower_fence or x > upper_fence] # Skewness (Pearson's second coefficient) skew = 3 * (mean - median) / sd print(f"\n{'='*50}") print(f" {label}") print(f"{'='*50}") print(f" Data: {data}") print(f" n: {n}") print(f"\n --- Central Tendency ---") print(f" Mean: {mean:.2f}") print(f" Median: {median:.2f}") print(f" Mode: {mode}") print(f"\n --- Spread ---") print(f" Min: {min_v}") print(f" Max: {max_v}") print(f" Range: {rng}") print(f" Q1: {q1}") print(f" Q3: {q3}") print(f" IQR: {iqr}") print(f" Variance: {variance:.2f}") print(f" Std Dev: {sd:.2f}") print(f" CV: {cv:.1f}%") print(f"\n --- Shape ---") print(f" Skewness: {skew:.3f} ({'right/positive skew' if skew > 0.2 else 'left/negative skew' if skew < -0.2 else 'approx symmetric'})") print(f"\n --- Outliers (IQR method) ---") print(f" Lower fence: {lower_fence:.1f}") print(f" Upper fence: {upper_fence:.1f}") print(f" Outliers: {outliers if outliers else 'None'}") describe("DATASET 1: NBA Player Heights (inches)", nba_heights) describe("DATASET 2: US Household Income ($000s)", incomes) EOF
Dataset 1: NBA Player Heights (inches)
| Statistic | Value |
|---|---|
| Mean | 77.47 in |
| Median | 77.00 in |
| Mode | 75 in |
| Std Dev | 4.81 in |
| IQR | 7 (Q1=74, Q3=81) |
| CV | 6.2% |
| Skewness | +0.291 (slight right skew) |
| Outliers | None |
- Mean and median are very close (77.47 vs 77.00), meaning the distribution is nearly symmetric. Either is a fair summary.
- SD of 4.81 inches is small relative to the mean (CV = 6.2%) - NBA players are a homogeneous group in height.
- No outliers - heights are tightly clustered. A player at 69" (point guard) or 87" (center) is unusual but still within normal range.
Dataset 2: US Household Income ($000s)
| Statistic | Value |
|---|---|
| Mean | $68,930 |
| Median | $51,000 |
| Mode | $28,000 |
| Std Dev | $55,740 |
| IQR | $37,000 (Q1=38k, Q3=75k) |
| CV | 80.9% |
| Skewness | +0.965 (strong right skew) |
| Outliers | $250,000 |
- The mean ($68,930) is much higher than the median ($51,000) - a classic sign of right skew. The $250k earner pulls the mean up.
- The median is the better summary here. This is exactly why "median household income" is used in news and government reports, not the mean.
- CV of 80.9% vs 6.2% for heights - income is far more variable as a population characteristic.
- $250,000 is a confirmed outlier by the IQR method (upper fence = $130,500).
Key Takeaway from Comparing Both
| Heights | Incomes | |
|---|---|---|
| Shape | Near-symmetric | Strongly right-skewed |
| Best center measure | Mean | Median |
| Best spread measure | SD | IQR |
| Outliers | None | Yes (high earner) |