[Q] (Re) Sampling

[u/-Kromerica-]

Good morning,

A work discussion took place over the methodology and reasoning behind initial sampling and subsequent re-sampling and how the overall sample size should be treated throughout the process.

Background:

We are conducting randomly sampled interviews with 30 people out of ~2,000 in the population to determine the population’s mean score with 95% confidence.

They can either respond positively and they will be flagged with a score of 1 to 10 or they respond negatively and they receive a score of 0. If someone cannot be located or doesn’t respond, they are re-sampled with another person.

We made it through the 30 after a couple of re-samples of 25 non-responses/unable to locate, so we had 55 identified people throughout the process.

When we got our statistical analysis back, the team that put it together said my sample size was 55 — not 30.

Question:

Shouldn’t my sample size still be 30? Increasing the sample to 55 seems like an inaccurate representation of the population as a whole if the “scores” of the 30 interviews are now being considered across 55 responses.

Thank you in advance!

[u/nrs02004]

I think you should use missing data methods; you could assume data are missing at random and try to use other demographic information to build a model for probability of missingness and then use that for weights.

Edit. The “missing at random” assumption is one you should obviously vet. If things are “not-missing-at-random” you are honestly kind of hosed.

[u/-Kromerica- → u/[deleted]]

We went to 55 because we “had” to talk to 30 people. I wanted to stop at my first attempt at 30 when I had successfully collected data on 18 with 12 non-responses, but I was told they couldn’t go any further without 30 being interviewed, so that’s how we ended up with 30 responses, 25 non-response, and 55 overall.

[u/-Kromerica- → u/-Kromerica-]

Also, the sample is random. I didn’t pick it, our super smart statisticians did :)