Margin of Sampling Errors: What Do They Mean?

What does it mean when a survey reports that there is a margin of sampling error of plus or minus 3%? Why not four, or two? Certainly it’s possible for the sample to be off by more than 3%, so stating the sampling error (without specifying the corresponding level of confidence) is simply misleading the audience into a sense of false confidence, because in reality, “there is a margin of sampling error of plus or minus 3%” means “we’re pretty sure we’re correct to 3%, but good luck if you want to know how sure!”

Every assertion of a specific margin of sampling error in surveys must be accompanied by the corresponding confidence level so that the public can see a more informative survey and develop a more realistic understanding of the survey results. If there is a standard (95%, maybe?), publish that with each survey; otherwise, develop a standard.

Gallup, you start first.