Quota Sampling

Non-probability sampling means researchers subjectively choose the sample instead of using random selection. Not everyone has an equal chance to take part.

Researchers assign quotas to subgroups of the population. Each quota represents that subgroup’s share of the target population (the complete group the study is trying to draw conclusions about) as a whole.

Common quota characteristics include gender, age, residency, education level, and income. Once the subgroups are set, researchers use their own judgment to select participants from each one until the quotas are filled.

Correct proportions matter. If the wider population is 65% female and 35% male, the sample should reflect those percentages too.

Sampling Concepts Behind Quota Sampling

Target Population, Sampling Frame and Sample

Every sampling exercise depends on the target population being defined narrowly, rather than assumed to be everyone.

A study of student wellbeing, for example, might target all undergraduates at one university, not young people in general.

Researchers then draw a sample from a sampling frame: the accessible list of population members they can actually reach, such as a customer database.

The frame is only a stand-in for the population. It rarely matches perfectly.

When the frame leaves people out, that gap is called under-coverage.

A database that only holds email subscribers, for instance, misses everyone who buys without signing up, biasing any sample drawn from it.

Quota sampling sits at the final link in this chain.

Its whole purpose is generalisation: reasoning from the sample back to the target population with enough confidence that the findings apply beyond the people actually studied.

Probability vs Non-Probability Sampling

Sampling methods split into two families.

Probability sampling gives every member of the population a known, non-zero chance of selection, which is what lets researchers calculate sampling error and generalise with quantifiable confidence.

Simple random, systematic, stratified and cluster sampling all work this way. Cluster sampling, for example, randomly selects whole naturally occurring groups, such as schools or regions, and then studies the people inside them.

Neyman (1934) formally set out this distinction. His classic paper contrasted random, probability-based selection with purposive selection.

It showed that only the probability method lets a researcher measure how far a sample estimate is likely to depart from the true population value.

Non-probability sampling is different. It does not give everyone a known chance of being chosen.

It relies instead on convenience, availability or the researcher’s own judgement, and quota sampling belongs firmly in this camp.

That trade-off cuts both ways: quicker and cheaper, but far less secure for statistical generalisation, because the selection probabilities are unknown.

Why This Matters: Generalisation and Bias

The whole point of sampling is generalisation: drawing confident conclusions about the target population from the sample studied.

The more representative the sample, the more confident a researcher can be that the results generalise.

The chief threat to that confidence is sampling bias: a systematic difference between who is included and who is excluded.

Quota sampling is not immune to this.

A survey run only at the start of a lecture, for instance, misses latecomers.

If latecomers differ from everyone else, the sample is biased.

Quota sampling tries to guard against this at the group level, by fixing proportions for each subgroup in advance.

Whether it succeeds still depends on how those quotas are filled, which is where the technique’s own limitations begin.

Applications

Quota sampling is used when…

• Time is limited as quota sampling is a quick method of sampling.
• The budget is tight as it is cheaper than other sampling methods.
• Researchers have specific criteria or constraints for conducting their research.
• Researchers want to monitor the number of participants allowed to complete a survey depending on characteristics such as age, gender, or race.
• Researchers rarely have access to an entire population, so they draw the sample from a sampling frame: the accessible list of people they can select from (e.g. a customer database).

The sampling method matters most for external validity: whether the results generalise beyond the people actually studied. Random allocation to conditions is a different thing. It protects internal validity, but it is the sampling method that decides who the causal conclusion applies to.

A tightly controlled experiment run entirely on a quota sample can therefore have excellent internal validity yet weak external validity. That happens whenever the sample’s demographic quotas fail to capture everything that matters for the finding.

How to Use

Here’s a basic outline of how quota sampling might work in a study:

Identify Strata and Proportions

Before identifying strata, researchers must narrowly define the target population itself, rather than assuming it means everyone. A company surveying customer satisfaction, for instance, might target only customers who bought in the last year, not everyone who has ever purchased from it.

The first step in quota sampling is to identify the strata of the population. Strata are subgroups or categories within the population.

The researchers would then determine the proportions of these strata within the population, which would be the sample’s target proportions.

This is a real constraint: quota sampling only works if the researcher already knows the population’s true proportions.

Select sample size

Several factors, including the population size, the margin of error, the confidence level, and the expected response distribution, determine the sample size in a research study.

In quantitative research there is a principled way to choose this number. It is called a power analysis. Statistical power is the probability that a study will detect a real effect. It rises with the sample size, the effect size, and the significance criterion used (Cohen, 1992).

Cohen (1992) supplied benchmark effect sizes and the sample sizes needed to detect them with adequate power, conventionally 0.80.

Detecting a small effect takes a large sample. Under-powered studies miss real effects and, when they do reach significance, tend to exaggerate them.

Quota sampling is not a probability method. So it cannot support the formal sampling-error calculations a power analysis assumes. Many quota-sampling studies still borrow a target sample size from one anyway, as a defensible starting point rather than a guess.

Select Participants

The researchers then select participants from each stratum until its quota is filled.

This can be done in several ways. Researchers might randomly draw names from a sampling frame list, approach whoever is accessible, or send out a survey and use the first responses that arrive.

Because the final choice is usually left to the researcher, not chance, the same interviewer bias described under Limitations can creep in here too.

Advantages

Quota sampling has four practical strengths:

• Quick and Easy: It needs no full sampling frame and no random draw, so data collection is faster and more convenient than stratified sampling.
• Cheap: Researchers survey a small sample rather than every individual, and stop recruiting a group as soon as its quota is full.
• Matched Proportions: Quotas stop any one group being over-represented, so the sample mirrors the population on the chosen characteristics.
• Flexible Recruitment: It suits situations where availability changes, such as by time of day, because researchers can fill each quota as the right people become available.

Limitations

Large potential for bias

Because selection within each quota is non-random, the interviewer chooses who to approach. This can introduce interviewer bias: researchers tend to pick people who look approachable or convenient, rather than a true cross-section of the subgroup.

This selection bias also means quota sampling cannot support formal statistical inference about the wider population.

Not generalizable to the population

While this sampling method can be very representative of the quota-defining characteristics, other important characteristics may not be represented in the final sample group.

The logic here is asymmetric. A representative sample licenses generalisation, but an unrepresentative one does not just weaken it. It can make a confident generalisation actively misleading, because the size and direction of the bias are usually unknown.

Improving other parts of the study does not rescue this. A highly reliable, consistent measure given to a biased quota sample still generalises poorly, because reliability is about consistency of measurement, not about who was actually measured.

Cannot calculate sampling error

Because quota sampling is not a probability sampling method, researchers cannot calculate the sampling error.

Sampling error is the ordinary, random gap between a sample statistic and the true population value.

It shrinks as samples grow.

It can only be quantified as a margin of error when every member’s chance of selection is known.

Quota sampling never meets that condition.

Selection within each quota is left to the interviewer, not to chance.

This is different from sampling bias, which is systematic and does not shrink as the sample grows.

Bias does not wash out with size.

A third category, non-sampling error, covers mistakes unrelated to selection itself, such as poor question wording or data-entry errors.

It can affect even a full census, and quota sampling is no more or less exposed to it than any other method.

WEIRD Samples and the Replication Crisis

Quota sampling can hit its own proportions perfectly and still deliver a skewed sample.

That is because the traits it quotas for are rarely the only ones that matter.

The quotas themselves can hide this.

Henrich, Heine and Norenzayan (2010) showed that most published psychology draws on WEIRD participants: Western, Educated, Industrialised, Rich and Democratic societies.

On measures from visual perception to fairness and moral reasoning, these participants are frequent outliers rather than a neutral human default.

That is not a minor caveat.

The Open Science Collaboration (2015) later tested how well psychology’s published findings hold up.

A quota sample that misses important traits carries the same risk as any other convenience-based method.

Contemporary Research

Online Panels and Crowdsourcing

Much data collection has moved to online participant panels such as Amazon Mechanical Turk (MTurk) and Prolific. These give fast, cheap access to larger and somewhat more diverse samples than the campus subject pool.

Buhrmester, Kwang and Gosling (2011) showed that MTurk can yield data at least as reliable as traditional methods. The samples are another matter.

Paolacci and Chandler (2014) reviewed MTurk as a participant pool. They documented that its workers are not representative, skewing in age, income and attitudes.

The workers are also non-naïve. Many have seen common manipulations and attention checks repeatedly, which can distort results.

Peer et al. (2017) compared platforms directly. They found meaningful differences in data quality, attention and honesty between MTurk and alternatives such as Prolific.

Online panels widen the pool without solving the representativeness problem.

Quotas can balance a panel sample on age or gender, but they cannot control who chooses to join the panel.

Pre-Registered Sampling Plans

A second reform answers the replication crisis. Pre-registration publicly fixes the design and sampling plan before data collection: a target sample size justified by power analysis, stopping rules and inclusion criteria.

Flexibility over when to stop collecting data inflates false-positive rates (Simmons et al., 2011). Nosek et al. (2018) argue that pre-registration makes sampling and analysis decisions transparent and confirmatory rather than post hoc.

Example Situation

Suppose we are conducting a study on the reading habits of high school students in a district. The district’s high school population is 45% freshmen, 25% sophomores, 20% juniors, and 10% seniors.

1. Identify Strata and Proportions: We identify the grade level as our stratum. The proportions are 45% freshmen, 25% sophomores, 20% juniors, and 10% seniors.
2. Select Sample Size: We decide to survey 500 students in total, based on the size of the district’s high school population.
3. Select Participants: Our quotas require 225 freshmen, 125 sophomores, 100 juniors, and 50 seniors, selected by approaching students from each grade until each quota is filled.

Real-Life Examples

• Ensuring that an adequate number of midlife women were recruited from the targeted ethnic groups in an Internet-based study (Im & Chee, 2011).
• Recruiting at-risk women for microbicide research and ensuring adequate representation of specific sample characteristics (Morrow et al., 2007).
• Obtaining a representative sample of pregnant women to study trends in smoking during pregnancy in England (Owen, McNeill, & Callum, 1998).
• Recruiting respondents to participate in an interview about stress levels with quotas based on sex, age, working status, residential location, housing tenure, and ethnicity (Sedgwick, 2012).
• Monitoring national trends of tobacco smoking in France (Guignard et al., 2013).
• Quantifying the use of sunbeds in children across England and identifying geographical variation to study the rise of malignant melanoma (Thomson et al., 2010).

A Landmark Example: Gallup and the 1936 US Election

Quota sampling has one especially famous success story. Ahead of the 1936 US presidential election, the magazine Literary Digest mailed about ten million straw-vote ballots. The ballots were drawn from car-registration, telephone and club-membership lists, and the magazine predicted a landslide win for the Republican candidate, Alf Landon.

Only around 2.3 million of those ballots were ever returned.

That low return rate added a second bias on top of the skewed mailing list. People willing to post back a ballot were not a random slice of everyone who received one.

The magazine’s sample was enormous but badly skewed: car and telephone owners were disproportionately wealthy, so the poorer, Democratic-leaning voters of the Depression era were largely excluded. Landon lost in a historic landslide.

In the same election, researcher George Gallup correctly predicted the result using a quota sample of only around thirty thousand people, built to match the voting population’s known proportions.

The episode is still used to teach a key lesson in sampling. A smaller, well-targeted quota sample can outperform a huge but biased one, because representativeness, not sheer size, is what licenses generalisation.

Quota Sampling vs Stratified Sampling

Quota sampling and stratified sampling both involve dividing a population into mutually exclusive subgroups and sampling a predetermined number of individuals from each.

However, the most significant difference between these two techniques is that quota sampling is a non-probability sampling method, while stratified sampling is a probability sampling method.

In a stratified sample, individuals within each stratum are selected at random, while in a quota sample, researchers choose the sample as opposed to randomly selecting it.

Key Terms

• A sample is the participants you select from a target population (the group you are interested in) to make generalizations about. An entire population tends to be too large to work with, so a smaller group of participants must act as a representative sample.
• Representative means the extent to which a sample mirrors a researcher’s target population and reflects its characteristics (e.g., gender, ethnicity, socioeconomic level). In an attempt to select a representative sample and avoid sampling bias (the over-representation of one category of participant in the sample), psychologists utilize various sampling methods.
• Generalisability means the extent to which their findings can be applied to the larger population of which their sample was a part.

References

Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality data? Perspectives on Psychological Science, 6(1), 3–5. https://doi.org/10.1177/1745691610393980

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155

Guignard, R., Wilquin, J.-L., Richard, J.-B., & Beck, F. (2013). Tobacco smoking surveillance: Is quota sampling an efficient tool for monitoring national trends? A comparison with a random cross-sectional survey. PLoS ONE, 8(10), Article e78372. https://doi.org/10.1371/journal.pone.0078372

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X

Im, E. O., & Chee, W. (2011). Quota sampling in internet research: Practical issues. CIN: Computers, Informatics, Nursing, 29(7), 381–385. https://doi.org/10.1097/NCN.0b013e3181f9dc45

Morrow, K. M., Vargas, S., Rosen, R. K., Christensen, A. L., Salomon, L., Shulman, L., Barroso, C., & Fava, J. L. (2007). The utility of non-proportional quota sampling for recruiting at-risk women for microbicide research. AIDS and Behavior, 11(4), 586–595. https://doi.org/10.1007/s10461-007-9213-z

Neyman, J. (1934). On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society, 97(4), 558–625. https://doi.org/10.2307/2342192

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), Article aac4716. https://doi.org/10.1126/science.aac4716

Owen, L., McNeill, A., & Callum, C. (1998). Trends in smoking during pregnancy in England, 1992–7: Quota sampling surveys. BMJ, 317(7160), 728–730. https://doi.org/10.1136/bmj.317.7160.728

Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184–188. https://doi.org/10.1177/0963721414531598

Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163. https://doi.org/10.1016/j.jesp.2017.01.006

QuestionPro. (2021, July 19). Quota sampling: Definition, types & free examples. https://www.questionpro.com/blog/quota-sampling/

Sedgwick, P. (2012). Proportional quota sampling. BMJ, 345, Article e6336. https://doi.org/10.1136/bmj.e6336

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Thomson, C. S., Woolnough, S., Wickenden, M., Hiom, S., & Twelves, C. J. (2010). Sunbed use in children aged 11–17 in England: Face to face quota sampling surveys in the National Prevalence Study and Six Cities Study. BMJ, 340, Article c877. https://doi.org/10.1136/bmj.c877

Further Information

• Taherdoost, H. (2016). Sampling methods in research methodology; how to choose a sampling technique for research. How to Choose a Sampling Technique for Research (April 10, 2016).