Response rates are considered one of the most common indicators of survey data quality in population-based
surveys, and concerns exist regarding the quality of estimates generated from the Office of
People Analytics (OPA) surveys as response rates continue to decline. Between 2004 and 2018,
response rates for DoD active duty surveys have declined from 40% to about 15%.
OPA frequently receives the question, “Are the results still valid from a survey with a low response
rate.” While there is no scientifically backed threshold upon which population-based survey results
move from valid to invalid, OPA statisticians consider nonresponse bias (NRB) the primary source of
error in military surveys. OPA periodically conducts NRB analysis on its surveys to confirm the
validity of weighted estimates generated. These NRB studies have evidenced limited issues with NRB,
due to the sophisticated weighting methods employed on OPA population-based surveys. That said, if
response rates continue to decline, survey results are at greater risk of NRB and the results could
eventually be deemed invalid. While no current evidence exists to question the validity of estimates on
OPA surveys based on low response rates, OPA statisticians recommend that military survey
organizations take aggressive and immediate action to change the landscape of military surveying to
improve response rates.