Non-Response Bias

Definition

Non-response bias (also called participation bias) occurs when individuals who do not respond to a survey or study differ systematically from those who do respond, making the resulting sample unrepresentative of the target population. It is one of the most common and serious sources of error in survey research.

Two Types of Non-Response

Type	Description	Example
Unit non-response	An entire person/unit does not participate	A sampled household never returns the survey
Item non-response	A participant skips specific questions	Refusing to answer income or sexual behavior questions

Causes

Poor survey design - confusing, too long, or poorly worded questions
Sensitive topics - questions about drugs, infidelity, sexual behavior, or finances that people prefer to skip
Wrong target audience - mismatched content (e.g., asking elderly home residents about extreme sports)
Inability to contact - participants with no fixed address, no phone, or no internet
Language or technical barriers - survey not translated or requires technology the respondent lacks
Longitudinal attrition - in multi-wave studies, participants drop out over time (also called attrition bias)
Lack of motivation or incentive - respondents see no reason to participate

Why It Matters

Non-response bias creates two main statistical problems:

Biased estimates - if non-respondents differ on the variable of interest, the sample mean or proportion will be wrong
Increased variance - a smaller effective sample size widens confidence intervals, reducing precision

A classic example: in an AIDS-related survey, those who refused to participate tended to be "older, attend church more often, less likely to believe in confidentiality of surveys, and have lower sexual self-disclosure" - meaning the sample systematically under-represented that demographic.

The Response Rate Myth

A common assumption is that a higher response rate automatically reduces non-response bias. Research has challenged this:

A meta-analysis of 30 methodological studies by Robert Groves found that response rate explained only 11% of the variance in non-response bias (R² = 0.11) - a very weak predictor.
Methods that boost response rates (like incentives or prior notification) do not necessarily reduce bias and can sometimes worsen it.

The key issue is who is not responding, not how many.

How to Detect Non-Response Bias

Method	How It Works
Wave analysis	Compare early vs. late respondents; late responders approximate non-respondents
Follow-up studies	Contact a subsample of non-respondents to compare their characteristics
Comparison to known population data	Check if sample demographics match census or registry data
Administrative records	Compare available data on respondents vs. non-respondents

How to Reduce Non-Response Bias

Improve survey design - shorter, clearer, more engaging surveys
Multiple contact attempts - follow up by phone, email, mail, or in person
Offer incentives - monetary or non-monetary rewards for participation
Ensure anonymity/confidentiality - especially for sensitive topics
Mixed-mode surveying - offer online, paper, and phone options to reach different groups
Statistical adjustment - post-hoc weighting (raking, propensity score adjustment) to match known population distributions
Translation and accessibility - remove language and technical barriers

Non-Response Bias vs. Response Bias

These are related but distinct:

Non-response bias - the bias introduced by who doesn't participate
Response bias - the bias introduced by how participants answer (e.g., social desirability, acquiescence)

Both can distort results, but through different mechanisms.

Relation to Other Biases

Non-response bias is a subtype of selection bias, alongside sampling bias, volunteer bias, survivorship bias, and undercoverage bias. In longitudinal studies, it overlaps with attrition bias.

This is a shared conversation. Sign in to Orris to start your own chat.