OPA Resources Sign In Request Access
account_circle expand_more expand_less
In Group 2019 Health and Resilience (H&R) Division Methodology Studies

Details

Response rates are considered one of the most common indicators of survey data quality in population-based surveys, and concerns exist regarding the quality of estimates generated from the Office of People Analytics (OPA) surveys as response rates continue to decline. Between 2004 and 2018, response rates for DoD active duty surveys have declined from 40% to about 15%. OPA frequently receives the question, “Are the results still valid from a survey with a low response rate.” While there is no scientifically backed threshold upon which population-based survey results move from valid to invalid, OPA statisticians consider nonresponse bias (NRB) the primary source of error in military surveys. OPA periodically conducts NRB analysis on its surveys to confirm the validity of weighted estimates generated. These NRB studies have evidenced limited issues with NRB, due to the sophisticated weighting methods employed on OPA population-based surveys. That said, if response rates continue to decline, survey results are at greater risk of NRB and the results could eventually be deemed invalid. While no current evidence exists to question the validity of estimates on OPA surveys based on low response rates, OPA statisticians recommend that military survey organizations take aggressive and immediate action to change the landscape of military surveying to improve response rates.
Produced By: H&R
Document Type: Executive Note
Product ID: 2019-077