Missing Data in Survey Research: Delete, Impute, or Report as a Limitation?

Missing data is one of the most common problems in survey research. It is also one of the most commonly mishandled.

At first, missing data may seem like a minor inconvenience. A few respondents skipped a question. Some people started the survey but did not finish it. One scale has a handful of blank responses. A few demographic variables are incomplete. It can be tempting to ignore the problem, delete the incomplete cases, or write a quick sentence saying that missing data was a limitation.

Sometimes that is reasonable. Sometimes it is not.

The right approach depends on how much data are missing, why the data are missing, which variables are affected, how the analysis will be conducted, and whether the missingness could bias the results. In dissertation research, journal manuscripts, program evaluations, and applied survey projects, the way missing data are handled can affect the sample size, statistical power, estimated effects, p-values, confidence intervals, and credibility of the conclusions.

This post explains how to think about missing data in survey research, including when it may be defensible to delete cases, when imputation may be useful, and when missing data should be reported as a limitation.

Why Missing Data Matter

Missing data are not just empty cells in a spreadsheet. They can change the meaning of the analysis.

Suppose a survey asks employees about job satisfaction, burnout, workload, supervisor support, and intent to leave the organization. If the employees with the highest burnout are also the most likely to stop answering the survey halfway through, the final dataset may understate the true level of burnout. If people with lower income are less likely to report their income, any analysis involving income may be biased. If participants skip sensitive questions about trauma, substance use, mental health, or workplace conflict, the missingness itself may be related to the topic being studied.

In other words, missing data can create at least three problems:

Loss of sample size: Fewer usable cases can reduce statistical power.
Biased estimates: Results may be distorted if the missingness is related to the variables being analyzed.
Confusing reporting: Different analyses may be based on different numbers of cases, making the results harder to explain.

Because of this, missing data should be addressed intentionally. The goal is not always to eliminate every missing value. The goal is to use a method that is appropriate, transparent, and defensible.

Common Types of Missing Data in Surveys

Survey data can be missing for many different reasons. Some are relatively harmless. Others are more serious.

Item Nonresponse

Item nonresponse occurs when a respondent answers some survey questions but skips others. For example, a participant may answer questions about job satisfaction but skip questions about income or age.

This is common in survey research, especially when questions are sensitive, confusing, optional, or placed late in a long survey.

Unit Nonresponse

Unit nonresponse occurs when a selected person does not participate in the survey at all. This is usually harder to address because there may be little or no information about the nonrespondent.

For example, if 1,000 people are invited to complete a survey and only 150 respond, the dataset contains information about respondents but not about the 850 people who did not participate. This raises questions about response bias and generalizability.

Dropout or Attrition

Attrition occurs when participants begin a study but do not complete later parts of it. This is especially common in longitudinal surveys, pretest/posttest designs, intervention studies, and long online questionnaires.

Attrition can be a major issue if the people who drop out differ from those who remain.

Missing by Design

Sometimes data are missing because the survey was intentionally designed that way. For example, respondents may only see certain questions if they answer a prior screening question in a particular way.

This is not necessarily a problem. If only parents are asked questions about children, then nonparents should have missing values for those items. In this case, the missingness reflects the survey design rather than a failure to respond.

Invalid or Unusable Responses

Data may also become missing during cleaning. For example, a response may be impossible, inconsistent, duplicated, or clearly invalid. A respondent may complete the survey too quickly to have read the questions. A free-text response may show that the person was not eligible for the study.

In these cases, missingness may be created by data-cleaning decisions, and those decisions should be documented.

The Three Missing Data Mechanisms: MCAR, MAR, and MNAR

When deciding what to do with missing data, it helps to think about the likely missing data mechanism. The three common terms are MCAR, MAR, and MNAR.

Mechanism	Meaning	Example	Why it matters
MCAR	Missing completely at random	A software error randomly skips a question for some respondents	Deletion may be less problematic, although sample size is still reduced
MAR	Missing at random, conditional on observed data	Younger respondents are less likely to answer an income question, but age is measured	Methods such as multiple imputation may be useful if the model includes relevant observed variables
MNAR	Missing not at random	People with the highest income are less likely to report income because of the income itself	The missingness can be difficult to correct and may need to be treated as a serious limitation

These categories are sometimes misunderstood. “Missing at random” does not mean the data are missing for no reason. It means that the probability of missingness can be explained by variables that are observed in the dataset. For example, if income is more often missing among younger respondents, and age is available, then the missingness may be related to observed information.

“Missing not at random” means the probability of missingness depends on the missing value itself or on unobserved information. For example, income may be missing because people with very high or very low incomes are less willing to report it. Trauma history may be missing because people with certain experiences do not want to disclose them. In these situations, the missingness may be directly related to the value that is missing.

In real research projects, the true mechanism is usually not known with certainty. The best you can do is examine patterns, use subject-matter knowledge, and choose a method that is reasonable given the evidence.

First Step: Diagnose the Missing Data Problem

Before deleting or imputing anything, begin by describing the missing data.

Useful questions include:

How many cases are in the raw dataset?
How many cases are usable for the main analysis?
Which variables have missing values?
What percentage of each variable is missing?
Are some respondents missing many items?
Are missing values concentrated in certain survey sections?
Are demographic groups different in their rates of missingness?
Do people with missing data differ from people with complete data on key variables?
Is the missingness due to skip logic, nonresponse, attrition, or data cleaning?

This diagnostic step is often more important than people realize. You cannot choose a defensible method until you understand the pattern of missingness.

For example, 5% missing data spread randomly across a large dataset may not be a serious issue. But 5% missing data concentrated entirely in the dependent variable for one subgroup could be a much bigger problem.

Option 1: Delete Cases With Missing Data

The simplest approach is deletion. This usually means analyzing only cases with complete data for the variables included in a particular analysis.

There are several forms of deletion.

Listwise Deletion

Listwise deletion removes a case from an analysis if the case is missing any variable required for that analysis.

For example, if a regression model includes burnout, workload, supervisor support, years of experience, and job satisfaction, a respondent must have valid data on all five variables to be included. If one value is missing, the case is excluded from that regression.

Listwise deletion is easy to understand and easy to report. Many statistical programs use it by default for common analyses. It may be reasonable when the amount of missing data is small, the missingness is plausibly unrelated to the study variables, and the remaining sample size is adequate.

However, listwise deletion has two major drawbacks. First, it can sharply reduce sample size, especially when many variables are included. Second, it can bias results if the complete cases differ systematically from the incomplete cases.

Pairwise Deletion

Pairwise deletion uses all available data for each pair of variables. For example, a correlation between anxiety and depression might use all participants with valid anxiety and depression scores, while a correlation between anxiety and stress might use a slightly different set of participants.

This can preserve more data, but it can also make the analysis harder to interpret because different parts of the output may be based on different sample sizes. In some settings, pairwise deletion can also produce inconsistent or mathematically problematic results.

Available Case Analysis

Available case analysis is a broader term for using all cases that have the necessary data for a specific statistic or model. Like pairwise deletion, it can preserve more information than listwise deletion, but it must be reported clearly.

When Is Deletion Defensible?

Deletion can be defensible when:

The percentage of missing data is small.
The missingness appears unrelated to the outcome or key predictors.
The complete-case sample remains large enough for the analysis.
The excluded cases do not differ meaningfully from included cases on observed variables.
The missingness is due to clearly irrelevant or planned survey structure.

Deletion is more questionable when:

A large percentage of cases would be excluded.
The missingness is concentrated in important variables.
The people with missing data differ systematically from those with complete data.
The final sample becomes too small for the proposed model.
The missingness is likely related to sensitive or substantive values.

Deletion is not automatically wrong. But it should not be used simply because it is the default option in the software.

Option 2: Impute the Missing Values

Imputation means replacing missing values with estimated values. The purpose is not to pretend the data were actually observed. The purpose is to use the information available in the dataset to make a more reasonable estimate than simply deleting the case.

There are many imputation methods, and they are not equally good.

Mean Substitution

Mean substitution replaces missing values with the mean of the observed values for that variable.

For example, if a respondent is missing a depression score, the missing value might be replaced with the average depression score from the rest of the sample.

This method is simple, but it is usually a poor choice. It can reduce variability, distort relationships among variables, and make the data look more precise than they really are. In most serious analyses, mean substitution should be avoided unless there is a very specific and limited reason for using it.

Single Imputation

Single imputation uses one estimated value for each missing value. This could be based on regression, hot-deck methods, last observation carried forward, or other approaches.

Single imputation can be better than mean substitution, but it still has a major limitation: it treats the imputed value as if it were known with certainty. This can understate uncertainty and make standard errors too small.

Multiple Imputation

Multiple imputation is often a stronger approach. Instead of creating one completed dataset, it creates multiple completed datasets. Each dataset contains plausible imputed values based on the observed data. The analysis is then run across the imputed datasets, and the results are pooled.

The advantage is that multiple imputation reflects uncertainty about the missing values. It does not simply fill in one “best guess” and move on.

Multiple imputation can be useful when the amount of missing data is meaningful, the missingness is plausibly related to observed variables, and the imputation model can include variables that help explain the missingness.

However, multiple imputation is not magic. It depends on the quality of the imputation model and the plausibility of the assumptions. If the data are missing not at random in a way that cannot be captured by observed variables, imputation may not solve the problem.

Full Information Maximum Likelihood

Full information maximum likelihood, often abbreviated FIML, is another method for handling missing data in certain statistical models, especially structural equation modeling and related approaches. Rather than filling in missing values directly, FIML estimates model parameters using all available information under the model assumptions.

FIML can be a useful option when the statistical model and software support it. Like multiple imputation, it depends on assumptions about the missingness mechanism and the model being estimated.

When Is Imputation Appropriate?

Imputation may be appropriate when:

Important variables have missing data.
Deleting cases would substantially reduce sample size.
The missingness is plausibly related to observed variables.
The dataset contains variables that can help predict the missing values.
The analysis plan and software support a defensible imputation method.
The imputation method can be explained clearly in the methods and results sections.

Imputation may be inappropriate or less useful when:

The sample size is extremely small.
The missingness is mostly in the outcome variable and cannot be explained by observed data.
The missingness is likely missing not at random.
The imputation model would be weak or poorly specified.
The amount of missing data is so severe that the results would be mostly model-based speculation.

A common misconception is that imputation “fixes” missing data. It does not. Imputation is a modeling strategy. It can reduce bias and preserve power under certain conditions, but it does not create information out of nothing.

Option 3: Report Missing Data as a Limitation

Sometimes the most honest answer is that the missing data cannot be fully corrected. In those cases, missing data should be reported as a limitation.

However, reporting missing data as a limitation should not be used as a substitute for analysis. It is not enough to say, “Missing data were a limitation,” and then ignore the issue. A good report should explain what was missing, how much was missing, how it was handled, and how it may affect the conclusions.

For example, a weak statement would be:

Missing data were a limitation of the study.

A stronger statement would be:

Because respondents with incomplete surveys were excluded from the regression analyses, the analytic sample was smaller than the full respondent pool. If participants with incomplete responses differed systematically from participants with complete responses, the results may not fully represent the original sample. Therefore, the findings should be interpreted with caution.

An even stronger report would include the number of cases excluded, the variables affected, and any comparison between included and excluded cases.

When Should Missing Data Be Treated as a Limitation?

Missing data should usually be discussed as a limitation when:

The amount of missing data is substantial.
The missingness affects key variables.
The final analytic sample is much smaller than the original sample.
The missingness may be related to the outcome or predictors.
The study has low response rates or high attrition.
The data do not support a more sophisticated missing-data method.
Different analytic choices lead to different conclusions.

Reporting missing data as a limitation is not an admission that the study is worthless. It is a way of being transparent about uncertainty. In many applied projects and dissertations, that is exactly what is needed.

What About Scale Scores With a Few Missing Items?

Survey studies often use multi-item scales. For example, a burnout scale might contain 10 items, and a respondent may answer 9 of them but skip 1.

In these cases, the first place to look is the scoring guidance for the instrument. Some scales provide explicit rules for how many items must be answered to compute a valid score. For example, a scale manual may allow a score to be calculated if at least 80% of the items are answered.

If no formal guidance is available, researchers sometimes calculate the mean of the completed items when only a small number of items are missing. This is different from replacing a missing variable with the overall sample mean. The idea is to compute a respondent’s scale score based on the items that person did answer, but only when enough items are present to make the score meaningful.

For example, if a respondent answers 9 of 10 items on a job satisfaction scale, it may be reasonable to calculate the person’s average across the 9 answered items, depending on the scale and research context. If the respondent answers only 3 of 10 items, the score is much harder to defend.

Whatever rule is used should be stated clearly.

Do Not Confuse Skip Logic With Missing Data

Survey skip logic can create blank cells that are not really missing data in the usual sense.

For example, suppose a survey asks:

Have you ever supervised employees?

Respondents who answer “No” are not shown the follow-up questions about supervision style. Those blank cells are expected. They do not mean the respondent failed to answer. They mean the questions did not apply.

This distinction matters because skip-logic missingness should not be treated the same way as accidental nonresponse. Depending on the analysis, these values may need to be coded as “not applicable,” left structurally missing, or handled by analyzing only the subgroup for whom the questions apply.

Before deciding how to handle missing data, confirm whether blank values are true missing responses or expected blanks created by the survey design.

How Missing Data Affect Dissertation Results Chapters

In dissertation research, missing data should usually be addressed in both the methods chapter and the results chapter.

In the methods chapter, explain the planned approach. This might include the criteria for excluding incomplete responses, the rules for scoring scales, the treatment of skip-logic items, and any planned imputation procedures.

In the results chapter, report what actually happened. This should usually include:

The number of responses originally collected.
The number of responses excluded during data cleaning.
The final analytic sample size.
The amount of missing data on key variables.
The method used to handle missing data.
Any sensitivity analyses, if applicable.

The sample size should also be clear in each analysis. If one regression uses 120 cases and another uses 96 cases because of missing data, that difference should be visible in the reporting.

Should You Use Sensitivity Analysis?

When missing data could affect the conclusions, a sensitivity analysis can be useful. A sensitivity analysis examines whether the results change when the missing data are handled in a different reasonable way.

For example, a researcher might compare:

Results using listwise deletion.
Results using a scale-score rule for partial responses.
Results using multiple imputation.
Results with and without a small group of incomplete cases.

If the conclusions are similar across reasonable methods, the findings may be more robust. If the conclusions change substantially, that should be reported and discussed.

Sensitivity analysis is especially useful when the missing-data decision is not obvious. It shows that the researcher did not simply choose the method that produced the most favorable result.

A Practical Decision Framework

Here is a practical way to decide what to do with missing survey data.

Situation	Possible approach	Important caution
Very small amount of missing data, apparently random	Listwise deletion may be acceptable	Still report the final analytic sample size
Missing values are due to survey skip logic	Treat as structurally missing or not applicable	Do not impute values for questions that did not apply
Multi-item scale has a few skipped items	Use instrument scoring rules or a defensible minimum-item rule	Do not compute scores when too much of the scale is missing
Deleting cases would greatly reduce sample size	Consider multiple imputation or FIML if appropriate	Imputation must be justified and reported clearly
Missingness appears related to observed variables	Consider methods that use those observed variables	The method depends on the quality of the model
Missingness is likely related to unobserved or sensitive values	Report as a limitation; consider sensitivity analysis	No method fully removes the concern
Key outcome variable is missing for many cases	Evaluate whether the research question remains answerable	The study may have reduced power or biased conclusions

This framework does not replace statistical judgment, but it helps clarify the decision.

Common Mistakes to Avoid

Mistake 1: Automatically Deleting All Incomplete Surveys

Deleting incomplete surveys may be simple, but it can waste usable data. A respondent who skipped one demographic item may still have valid responses for the main outcome and predictors.

Mistake 2: Using Mean Substitution Without Justification

Mean substitution is easy, but it can distort the data. It usually makes the dataset look cleaner than it really is and can reduce variability in a misleading way.

Mistake 3: Ignoring Different Sample Sizes Across Analyses

If each analysis uses a different number of cases, readers need to know that. Otherwise, the results may appear more consistent than they actually are.

Mistake 4: Treating “Prefer Not to Answer” as the Same as Accidental Missingness

A “prefer not to answer” response may carry meaning, especially for sensitive items. It should not automatically be treated as identical to an accidentally skipped question.

Mistake 5: Imputing Values for Questions That Do Not Apply

If a question was skipped because it was not applicable, imputation may be inappropriate. For example, nonparents should not be imputed answers to parenting-style questions that they were never supposed to answer.

Mistake 6: Reporting Only Percent Missing Without Explaining the Consequences

It is useful to report how much data are missing, but that is not enough. The report should explain how the missing data affected the analysis, sample size, and interpretation.

How to Write About Missing Data

A clear missing-data statement might look like this:

Of the 184 respondents who began the survey, 152 provided sufficient data for the primary analyses. Missing data were reviewed by variable and by case. Cases missing the primary outcome variable were excluded from inferential analyses involving that outcome. For multi-item scales, scale scores were computed when respondents completed at least 80% of the items; otherwise, the scale score was treated as missing. The final sample size for each analysis is reported with the corresponding results.

For a study using multiple imputation, the statement might look like this:

Missing data were handled using multiple imputation. The imputation model included the variables used in the primary analyses as well as demographic variables associated with missingness. Analyses were conducted across the imputed datasets, and pooled estimates are reported. Complete-case results were also reviewed as a sensitivity check.

For a study where missing data are a major limitation, the statement might look like this:

Because a substantial number of respondents did not complete the final section of the survey, analyses involving those variables were based on a smaller analytic sample. Respondents with incomplete data may have differed from respondents with complete data in ways not fully captured by the available variables. Therefore, findings involving these measures should be interpreted cautiously.

The exact wording should match the study, but the principle is the same: explain what happened, what you did, and how it affects interpretation.

Delete, Impute, or Report as a Limitation?

The choice is not always one or the other. In many projects, the best approach combines more than one strategy.

For example, a researcher might:

Exclude cases that failed eligibility screening.
Treat skip-logic blanks as not applicable.
Use scale scoring rules for partial item nonresponse.
Use listwise deletion for a small amount of remaining missingness.
Report reduced sample size as a limitation.

Another researcher might:

Describe missingness by variable.
Compare complete and incomplete cases.
Use multiple imputation for missing predictors.
Run a complete-case sensitivity analysis.
Discuss uncertainty in the limitations section.

The decision depends on the research question, the dataset, and the assumptions you can reasonably defend.

Final Thoughts

Missing data are a normal part of survey research. The presence of missing data does not automatically invalidate a study, but ignoring missing data can weaken the analysis and make the results harder to defend.

Deleting cases can be appropriate when missingness is limited and unlikely to bias the results. Imputation can be useful when deletion would waste substantial information and the assumptions are reasonable. Reporting missing data as a limitation is necessary when the missingness affects interpretation or cannot be fully corrected.

The most important point is transparency. A strong analysis does not hide missing data. It describes the problem, uses a defensible method, reports the final sample sizes, and explains how the missingness may affect the conclusions.

If you are working on a dissertation, thesis, manuscript, survey, or applied research project and are unsure how to handle missing data, dissertation statistical analysis support or broader statistical consulting services can help you decide whether deletion, imputation, sensitivity analysis, or a limitation statement is the most defensible approach for your study.