Background Surveys play an important role in providing public health data for researchers and policy-makers. Traditional modes of survey data collection are subject to declining response rates and increasing costs. With the spread of the internet among the population, web surveys potentially provide a cost-effective alternative mode. Volunteer web panels are now widely used for market research/opinion polling, but less for academic/government research due to concerns about their representativeness. Various methods attempt to make web panels more “representative” of the population. We compared results from four UK web panels with a national probability survey.

Methods A shortened Natsal3 questionnaire was included on four web panels: two used standard demographic quotas, and two were ‘modified’ using variables correlated with key outcomes as additional quotas. After weighting for age and sex, comparisons were made with Natsal3 for demographic characteristics, key behaviours and attitudes, to examine whether modified quotas ‘improved’ the results.

Results All four web panels gave significantly different results from Natsal3 on a majority of the variables. There were more differences among men than women for all the web panels. There were more differences between the web panels and Natsal3 questions asked face-to-face (CAPI) than in self-completion format (CASI). The web panels also differed significantly from each other. One of the modified quota web panels produced estimates closer to Natsal3 than the standard quota panels, but still differed on three-fifths of the variables. Moreover, meeting the modified quotas proved difficult, and the quotas had to be relaxed in both cases.

Conclusions When measuring sensitive sexual behaviours in the UK population, volunteer web panels provided significantly different estimates than a probability CAPI/CASI survey. Adjusting web panel quotas did not lead to much improvement.