Evaluating statistical claims means judging whether a study's design supports its conclusion. Two ideas dominate.
A sample represents a population only when it is chosen at random from that whole population, so a survey of people at one location, or of volunteers who chose to respond, leans toward a particular group. Cause and effect can be concluded only from an experiment in which subjects are randomly assigned to treatments; an observational study shows association alone.
Even a small margin of error cannot repair a biased sample. Answer choices typically describe a limitation, and the right one names the actual flaw in how the data were gathered.
Sample questions
Pick an answer to see whether it is right — nothing is saved, and nothing needs an account.
Question 1Easier
A city council wants to estimate the proportion of all city residents who favor extending public transit hours. A staff member surveys people waiting at a downtown bus stop one morning. Which of the following best explains a problem with using this survey to estimate the opinion of all city residents?
Show answer
Answer: B — The sample overrepresents people who already use public transit.
People waiting at a bus stop are more likely to already use public transit and may have different opinions than residents overall, so the sample is not representative of the full population. A survey need not include every resident to estimate a proportion; conducting the survey on only one morning does not prevent estimating a proportion from the responses; and a proportion can be estimated from categorical (yes/no) opinions.
Question 2Easier
A survey of adults in a city reports a margin of error of 0.5 percentage point, but the sample was drawn only from people who visited one specific website. Which of the following best describes a limitation of this survey’s results?
Show answer
Answer: B — The sample may not represent the population the survey describes.
The sample includes only visitors to one website, so it may not represent the adults in the city that the survey describes. A reported margin of error of 0.5 percentage point does not by itself make the results too imprecise to be useful. Reporting the margin of error in percentage points rather than as a proportion is not a limitation. The margin of error is calculated from the sample size rather than from the number of adults in the city.
Question 3Easier
A student wants to estimate the percent of all students at a large high school who walk to school. She stands at the main entrance and asks the first 60 students she sees whether they walked. Of those 60, 9 say yes. Which of the following best describes a limitation of using her result to estimate the school-wide percent?
Show answer
Answer: B — The sample was not randomly selected, so it may not represent all students.
Convenience sampling at one entrance is not a random sample of all students, so generalizing the 9/60=15% result to the entire school is unreliable.
Study on the fly is an independent study tool and is not affiliated with, sponsored by, or endorsed by any testing organization, school, or government body.
SAT® is a registered trademark of the College Board, which is not affiliated with, and does not endorse, this product.
All practice questions and passages are original works created for Study on the fly and are not reproduced from any official test.