Vent: a guy at a city council meeting said my 300 person survey was junk, and now I can't look at small polls the same way | Debate the Data - Vivisector
Vent: a guy at a city council meeting said my 300 person survey was junk, and now I can't look at small polls the same way
Last month I stood up at our council meeting in Spokane and showed a survey I ran, 312 people, and I was pretty proud of it. Some guy in the back row, retired stat teacher I guess, asked me one thing: what's your response rate and who didn't answer. I didn't have either number, and everyone in that room knew it. He said a survey of 312 out of a city with 220k people is fine if you show the math, but you can't just wave the number around like it settles the whole thing. So now I always write down how many people I sent it to and who ignored it, and I put the margin of error on the top of the page. Half the people I show it to think that's overkill, the other half respect it more. My side of it: small samples are still useful when you're honest about the limits. Other side: if you can't say who's missing, the number is just decoration. Where do you all land on this?