How to deal with randomly selected people who chose not to take a survey?

[moonman239]

I have a question. Let's say I wanted to estimate the median income of all adults in my church. So I randomly select individual stakes and send surveys out to the presidencies to distribute to members within their stakes, otherwise known as "cluster sampling." Cluster sampling has its disadvantages, but my church doesn't release contact information for any individual outside my ward(smaller group within a stake)/branch(similar to a ward, but consists of less members)/stake, with the exception of ward bishoprics, branch and stake presidencies.

Note: "Stakes" and "stake presidencies" also mean "districts," which are like stakes but smaller.

Here's my question: Let's say that 30% of members did not return the survey. I cannot contact those members. Which would be the best choice? 1) making my estimate using the data the respondents provided, acknowledging there were a few who did not return the survey 2) collect data on the income of all adults living in their countries, then estimate how much money the non-respondents earn, acknowledging that that part of the data was estimated.

[zli034]

Since you are talking about clustering sampling, I assume you know about the sampling weights. To adjust the selection bias, that is produced by the 30% non respondents, you can adjust the sample weights. There are many ways of doing that, for example, weighting class adjusting, CHAID tree analysis, postratification, raking. As long as you did not bias your result during the sending survey process these methods would be proven unbiased.

For estimating median you might want to try bootstrap method of estimation which is really good.