Article Text
PDF
Abstract
Objective To examine sociodemographic and behavioural differences between men who have sex with men (MSM) participating in recent UK convenience surveys and a national probability sample survey.
Methods We compared 148 MSM aged 18–64 years interviewed for Britain's third National Survey of Sexual Attitudes and Lifestyles (Natsal-3) undertaken in 2010–2012, with men in the same age range participating in contemporaneous convenience surveys of MSM: 15 500 British resident men in the European MSM Internet Survey (EMIS); 797 in the London Gay Men's Sexual Health Survey; and 1234 in Scotland's Gay Men's Sexual Health Survey. Analyses compared men reporting at least one male sexual partner (past year) on similarly worded questions and multivariable analyses accounted for sociodemographic differences between the surveys.
Results MSM in convenience surveys were younger and better educated than MSM in Natsal-3, and a larger proportion identified as gay (85%–95% vs 62%). Partner numbers were higher and same-sex anal sex more common in convenience surveys. Unprotected anal intercourse was more commonly reported in EMIS. Compared with Natsal-3, MSM in convenience surveys were more likely to report gonorrhoea diagnoses and HIV testing (both past year). Differences between the samples were reduced when restricting analysis to gay-identifying MSM.
Conclusions National probability surveys better reflect the population of MSM but are limited by their smaller samples of MSM. Convenience surveys recruit larger samples of MSM but tend to over-represent MSM identifying as gay and reporting more sexual risk behaviours. Because both sampling strategies have strengths and weaknesses, methods are needed to triangulate data from probability and convenience surveys.
- GAY MEN
- SEXUAL BEHAVIOUR
- SEXUAL HEALTH
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
Introduction
The emergence of the HIV epidemic in the 1980s prompted an unprecedented medical and social science interest in the sexual behaviour of men who have sex with men (MSM). Currently in Britain, an estimated 3% of men aged 16–74 years report sex with one or more men in the past 5 years.1 In 2013, 61% of HIV infections acquired in the UK were among MSM.2 ,3
To inform health promotion for this population in the UK, various surveys have been undertaken. Currently, all large surveys of MSM in the UK recruit using convenience sampling. Convenience surveys have traditionally been venue-based such as at Gay Pride events, or in gay bars and clubs in the case of the London Gay Men's Sexual Health Survey (London-GMSHS) (S Wayal et al. Temporal trends in HIV testing and undiagnosed HIV in community sample of men who have sex with men in London, UK 2000–13: an observational study. Lancet HIV 2015 (in review).) and Scotland's Gay Men's Sexual Health Survey (Scotland-GMSHS).4 More recently, web-based convenience surveys are being used, such as the European MSM Internet Survey (EMIS), which includes British men.5 These surveys recruit certain subgroups of MSM but the proportion and particularities of the MSM population represented in such surveys is unknown. Probability sample surveys like Britain's National Survey of Sexual Attitudes and Lifestyles (Natsal) should recruit a more representative sample of MSM. However, because the proportion of men having sex with men is relatively low,1 the sample of MSM in general population surveys like Natsal is small, precluding anything but relatively rudimentary analyses. Nonetheless, Natsal enables assessment of the proportion of MSM who attend gay bars and clubs or use the internet to find a sexual partner, which are potentially useful in assessing the selection biases inherent in convenience surveys of MSM recruited through venues and websites.
Previous research has shown that MSM participating in venue-based convenience surveys were more likely to be younger, report greater sexual risk behaviours and sexually transmitted infection (STI) diagnoses than MSM who participated in Natsal-2.6 ,7 This is consistent with some international comparisons, which found greater risk behaviour reported by MSM in convenience surveys.8 However, some international studies have suggested that some convenience surveys provide similar estimates of sexual risk behaviours as probability surveys.9 Because the recruitment methods used by venue-based and web-based convenience surveys change as do MSM's use of venues and websites, regular comparisons between convenience surveys and probability sample surveys are needed.
This study is the first to compare several convenience surveys of MSM in Britain carried out in 2010–2012 with a population probability survey (Natsal-3) carried out contemporaneously. We begin by drawing on Natsal-3 data to calculate the proportion of MSM who use gay venues and seek a sexual partner on the internet. We then compare the three convenience surveys with Natsal-3 to calculate differences in sociodemographic characteristics, drug use, sexual behaviour and sexual health characteristics. In addition, we investigate the influence of gay identity on the observed differences.
Methods
This paper compares data from a national probability sample survey, Natsal-3, with data from three surveys that used convenience sampling: EMIS London-GMSHS and Scotland-GMSHS. To be included in these analyses, research participants were required to have reported at least one male sexual partner in the year prior to data collection, being resident in Britain and being aged 18–64 years. Our analyses involved comparison between the surveys where questions on sociodemographics, drug use, sexual behaviour and sexual health had similar wordings (table 1). Further details of each survey are reported below.
Question wording used in the different surveys
Natsal-3
The Natsal-3 survey used a multistage, stratified random probability sample design.1 ,11 Using addresses from the comprehensive Small User Postcode Address File as the sampling frame, households in Great Britain were selected at random, and one individual aged 16–74 was randomly selected from each household. Individuals aged 16–34 years were oversampled. Data collection occurred between September 2010 and August 2012. Participants were interviewed face-to-face using computer-assisted personal interviewing (CAPI) and computer-assisted self-interviewing (CASI) for the more sensitive topics. Sociodemographics were assessed in the CAPI and drug use, sexual behaviour and sexual health in the CASI. With a response rate of 57.7% (interviews completed from eligible addresses) and a co-operation rate of 65.8% (interviews completed from eligible addresses contacted), Natsal-3 achieved a total sample size of 15 162 participants. A total of 148 MSM met the inclusion criteria for the analyses reported here.
EMIS
EMIS 2010 was a self-completion online sexual health needs assessment survey.5 The survey was promoted on over 230 websites aiming to appeal to gay and other MSM, including Gaydar, Manhunt, Gay Romeo and Terence Higgins Trust, as well as via posters and postcards distributed at gay venues. Conducted across 38 countries in 25 languages, data collection ran from June 2010 to August 2010. Over 180 000 men aged between 18 and 88 years across Europe participated, including 18 435 MSM resident in England, Scotland, Wales and Northern Ireland. MSM from Northern Ireland participants were excluded from the analyses reported here to increase comparability with Natsal-3. A total of 15 500 men met the inclusion criteria for our analyses.
London-GMSHS
Men attending gay bars, clubs and saunas across London were recruited in 2011 (S Wayal et al. Lancet HIV 2015 (in review).). Participants were given a self-completion pen-and-paper questionnaire. A total sample of 1185 men, aged between 18 and 81 years, was recruited (response rate 61%). This survey did not record the country of residence of people living outside of London. Therefore, only London residents (797 men) were included in our analyses.
Scotland-GMSHS
Conducted every 3 years for a 2-week period,4 participants were recruited from 15 gay bars and two saunas across Edinburgh and Glasgow. Recruitment occurred at two time-points in the evening each day of the week. All men present at the time of recruitment were approached and asked to self-complete a pen-and-paper questionnaire. In 2011, with a response rate of 65.2%, a total sample of 1515 men, aged between 18 and 83 years, was recruited. The analyses reported here were restricted to a total of 1234 men resident in Scotland.
Statistical methods
Analyses were conducted using the complex survey functions in Stata 13.1. Natsal-3 data were weighted to account for differential probability of selection and non-response by age, sex and region. For each variable, frequencies are reported for all surveys; 95% CIs are reported only for Natsal-3 since these are not appropriate in the case of prevalence estimates from convenience samples as they were narrow and so contribute little to the comparison between surveys. We first estimated the proportion of MSM in Natsal-3 who reported use of gay venues and seeking sex via the internet. We then compared each convenience survey with Natsal-3 individually on all our variables. First, survey-equivalent χ2 tests were used to test for differences in sociodemographic characteristics. Then forward stepwise regression was used to identify the sociodemographic differences associated with participating in each convenience survey compared with Natsal-3. Logistic regression was then used to compare drug use, sexual behaviour and sexual health in the convenience surveys compared with Natsal-3, crude ORs and ORs after adjusting for sociodemographic differences. Finally, we repeated these comparisons, restricting analyses to those MSM identifying as gay. Due to the small sample size in Natsal-3, resulting in insufficient power, we were unable to formally test this as an interaction.
Results
Estimating the proportion of MSM using gay venues and seeking sex on the internet
Among MSM in Natsal-3, 52.4% (95% CI 42.1% to 62.4%) reported visiting a gay pub, bar or club at least once in the past year; while 41.4% (95% CI 32.3% to 51.1%) reported using the internet to find a sexual partner in the past year.
Sociodemographic characteristics
Compared with those participating in Natsal-3, MSM in the convenience surveys tended to be younger and more likely to report education to at least higher education (table 2). MSM in EMIS were more likely to report living in London. No significant differences in employment were found comparing Natsal-3 with EMIS and with Scotland-GMSHS. However, men in the London-GMSHS were more likely to be employed and also to be of non-white ethnicity. With respect to sexual identity, 62.4% (95% CI 52.0% to 71.7%) of MSM in Natsal-3 reported identifying as gay, but this was more commonly reported in all three convenience surveys: 84.7% (EMIS), 94.4% (London-GMSHS) and 90.8% (Scotland-GMSHS).
Demographic characteristics: convenience surveys relative to Natsal-3
Drug use
Recreational drug use in the past year (adjusted odds ratio (AOR): 3.62, 95% CI 2.33 to 5.61) and ever having taken amyl nitrates (AOR: 5.21, 95% CI 3.40 to 7.98) were more likely to be reported in EMIS than Natsal-3. No significant difference was found with respect to reporting injecting non-prescribed drugs in the past year.
Sexual behaviours
An estimated 26.4% (95% CI 18.2% to 36.5%) of MSM in Natsal-3 reported a same-sex sexual experience before age 16 (table 3), similar to that observed in EMIS (28.5%). Reporting at least five male sexual partners in the past year was more common among MSM participating in EMIS (AOR: 4.28, 95% CI 2.63 to 6.97), London-GMSHS (AOR: 4.06, 95% CI 2.40 to 6.88) and Scotland-GMSHS (AOR: 2.23, 95% CI 1.34 to 3.74) than Natsal-3. Conversely, reporting female sexual partner(s) was rarer among MSM participating in EMIS than Natsal-3. Same-sex anal sex in the past year was more likely to be reported among MSM participating in EMIS and London-GMSHS than Natsal-3, while no difference was found between Natsal-3 and Scotland-GMSHS after adjusting for sociodemographic differences. Unprotected anal intercourse (UAI) with multiple partners in the past year was more commonly reported among MSM participating in EMIS than Natsal-3 (AOR: 2.30, 95% CI 1.16 to 4.59) but no differences were found comparing Natsal-3 with London-GMSHS or Scotland-GMSHS.
Sexual behaviours, HIV testing, reported STI diagnoses and drug use: convenience surveys relative to Natsal-3
HIV testing
HIV testing in the past year was consistently more commonly reported among MSM participating in the convenience samples than Natsal-3. A similar relationship was found for ever having tested for HIV (table 3).
STI diagnoses
MSM participating in London-GMSHS were more likely to report attending a sexual health clinic in the past year than those participating in Natsal-3. The reported prevalence of gonorrhoea diagnosis in the past year was 0.5% (95% CI 0.1% to 3.2%) among Natsal-3 MSM, which was significantly lower than that found among MSM participating in EMIS (3.8%) or London-GMSHS (5.9%). No significant differences in prevalence of diagnoses of syphilis or chlamydia were found between surveys.
MSM who identify as gay
We undertook a subgroup analysis of MSM who reported identifying as gay in each survey, corresponding to sample sizes of 98 (Natsal-3), 13 088 (EMIS), 752 (London-GMSHS) and 1119 (Scotland-GMSHS). The age-group distribution was similar between Natsal-3 and EMIS, but age differences remained between Natsal-3 and the other surveys (see online supplementary table S1). We found a reduction in the magnitude of many of the differences observed between Natsal-3 and EMIS when the samples were restricted to gay-identified MSM, although no formal test was used and results may be due to small sample sizes. As expected we found no difference in reporting female partner(s) (OR: 0.77, 95% CI 0.26 to 2.27); furthermore, UAI in EMIS changed to that of no significant difference as did same-sex anal sex in EMIS and Scotland-GMSHS (figure 1). However, differences still remained with MSM who identify as gay in convenience surveys reporting greater numbers of same-sex sexual partners, HIV testing and sexual health clinic attendance in the past year.
OR (95% CI) for sexually transmitted infection (STI) diagnosis, testing and sexual practices and behaviours in the convenience samples relative to Natsal-3, in men who have sex with men (MSM) who identify as gay. EMIS, European MSM (men who have sex with men) Internet Survey; GMSHS, Gay Men's Sexual Health Survey; MSM, men who have sex with men; Natsal-3, National Survey of Sexual Attitudes and Lifestyles.
Supplementary table
Demographic characteristics of MSM who identify as gay: Convenience surveys relative to Natsal-3
Discussion
We estimated the proportion of MSM participating in Natsal-3 who report visiting gay venues and who searched for sexual partners online. The non-negligible proportion of MSM who did not do so illustrates the potential proportions of MSM who might be missed by convenience surveys that use venues and the internet for recruitment. However, it is important to recognise that MSM who do not report using the internet for seeking sex may still access gay-interest websites for other reasons. EMIS was promoted via a variety of websites, not all of which were dating sites, as well as being promoted via posters and postcards distributed at gay venues. It is estimated that 20% of participants were recruited via these other sources, for instance charities such as the Terrence Higgins Trust and GMFA (data not shown).
We then compared MSM participating in Britain's most recent national probability sample survey of sexual behaviour with participants in three major convenience surveys of MSM undertaken contemporaneously. Several sexual health indicators, specifically the number of male partners, anal sex, gonorrhoea diagnosis and HIV testing, were more commonly reported in the convenience surveys. This is most likely due to participants in Natsal-3 being recruited at home through a probability survey while convenience surveys are those of self-selected individuals in an environment where people typically look for a sexual partner. These differences remained in multivariable analyses, adjusting for sociodemographic differences between the surveys and Natsal-3. While greater similarity may exist between Natsal and convenience samples for MSM who identified as gay, some key differences remained.
There are several limitations to our study. The comparisons are predicated on the assumption that Natsal-3 provides an approximately representative sample of MSM. Natsal-3 achieved a response rate of 57.7%, in line with other major social surveys undertaken in Britain at the time,12 ,13 but if those who did not participate systematically differed from those who did then Natsal-3 estimates will be biased. However, previous research suggests that, overall, Natsal-3 participants were demographically similar to participants in the 2011 UK census.11 With respect to sexual behaviour characteristics, research has found that participants taking part in Natsal-3 reported greater sexual risk behaviours compared with participants in a population-based general health survey,14 although, methodological differences exist that may, on balance, make Natsal-3's estimates more robust.
As a national probability sample survey, Natsal-3 has a relatively small sample size of MSM resulting in large CIs for rarer outcomes such as gonorrhoea diagnoses. Natsal-3's small sample size may also in part explain why fewer statistically significant differences were observed when we restricted the sample to MSM who identified as gay, although there was insufficient power to formally test an interaction. Due to Natsal-3's small sample size of MSM, we were unable to make geographically focused comparisons with London and Scotland. It is therefore uncertain to what extent differences in sexual health characteristics observed were due to selection bias in the venues or geographical differences.
We compared characteristics with similar question wording wherever possible. However, wording was not always identical, which may have affected our comparisons. For instance, men in Natsal-3 were asked a single question about how many men they had had sex with, whereas EMIS participants were asked separately about the number of their steady and non-steady sexual partners (table 1). It is possible therefore that combining responses to separate questions may result in a higher total number of partners than a single question.
Furthermore, the surveys used different data collection modes that may result in differences in reporting. However, many of the sexual health questions in Natsal were asked in the CASI, which is similar to EMIS.
This is the first study to compare data from MSM recruited to a national probability sample survey and MSM recruited to multiple major UK convenience surveys in an attempt to identify general rather than survey-specific differences. Such comparisons are needed on a regular basis to monitor whether differences exist, the magnitude of these differences and to identify possible reasons for them.8 The finding of greater reporting of sexual risk behaviours in convenience surveys than Natsal-3, which remain after adjusting for sociodemographic differences between the surveys, is consistent with previous studies.6–8 This suggests that men who are recruited to convenience surveys via gay-interest venues and websites continue to be different from MSM who do not. It is likely that data collected by such convenience surveys reflect a particular cross-section of MSM who are more likely to report greater risk behaviours, STI outcomes and HIV testing than the overall population of MSM and so most likely to benefit from health interventions.
A strength of online surveys is that they are able to collect data from a large sample and geographically broader target population more quickly and cheaply than venue-based surveys. This is a recruitment method that is continually growing in popularity, for instance, recruitment via social media and smartphone applications were recently used elsewhere.15 Furthermore, research has shown that participants in online surveys are less likely to report a gay identity, male-only partnerships and recent HIV testing than venue-based sampling,16–20 and as such there may be potential for estimates from online surveys of MSM to more accurately represent the heterogeneity of the whole MSM population than venue-based surveys. Online surveys also benefit from enabling participants to complete the questionnaire in an environment with greater anonymity, which may minimise social desirability bias compared with venue-based pen-and-paper questionnaire surveys,21 ,22 although research of this benefit is inconclusive.22 Future research should examine the impact of different data collection methods for convenience surveys to ascertain, which results in the best data quality in terms of overall survey response, item non-response and prevalence estimates.
Applying adjustment weights based on demographic differences to convenience survey data could potentially account for some selection bias.6 However, the data presented here suggest that this may not be all that effective as adjusting for demographic differences between Natsal-3 and each individual convenience survey made little impact. In addition, weighting-up data from MSM who identify as bisexual or heterosexual in convenience surveys may not be statistically efficient due to their small number in these surveys.
Convenience samples also have the advantage that they can efficiently recruit MSM engaged in greater sexual risks who may be most likely to benefit from risk reduction interventions. They are also able to ask detailed questions about same-sex sexual behaviours and sexual health needs to inform the design and delivery of STI/HIV prevention interventions and policies. While these surveys are therefore essential, they do under-represent MSM less engaged in sexual risk behaviours and less engaged with sexual health services, who may have unmet sexual health needs. As MSM's use of the internet and other forms of communication technology develop, for example, apps, it is important that convenience surveys develop new ways of recruiting men, which reflect these changes in order to recruit representative (or where appropriate, targeted) samples.
To inform health service planning, it is important to triangulate the different sources of information. An example is the synthesis of multiple sources of data including Natsal and convenience surveys to estimate numbers of MSM with undiagnosed HIV in the whole MSM population.23 ,24 However, further research is needed to develop triangulation methods and consider modifications to surveys so as to maximise the utility of data collected by probability and convenience surveys, providing added-value and strengthening the evidence-base for interventions that promote well-being in MSM.
Key messages
Convenience surveys to date have tended to sample men in gay-orientated venues and so represent only a proportion of the men who have sex with men (MSM) population in Britain, who report greater risk behaviour.
In contrast, probability sample surveys by definition are better placed to generate estimates representative of all MSM although based on smaller samples of MSM typically, and for a smaller number of behaviours.
Differences between convenience and probability surveys reduce for some behaviours when focusing on MSM who identify as gay, but are not eliminated.
As both sampling strategies have strengths and weaknesses, methods should be developed to triangulate data from probability and convenience surveys.
References
Footnotes
Handling editor Jackie A Cassell
Twitter Follow Lisa McDaid at @mcd_li
Contributors The paper was conceived by CB, CHM, PP, FH, LMM and AN. PP wrote the first draft with further contributions from AJC, CHM, FH, CB, LMM, AMJ, SW, SC, PS, AN, BE, AJC, JR and PW. Statistical analysis was done by PP, with support from CHM, CB, JR and AJC. All authors contributed to data interpretation, reviewed successive drafts and approved the final version of the manuscript.
Funding Natsal-3: Natsal-3 is a collaboration between University College London, the London School of Hygiene and Tropical Medicine, NatCen social research, Public Health England (formerly the Health Protection Agency) and the University of Manchester. The study was supported by grants from the Medical Research Council (G0701757); and the Wellcome Trust (084840); with contributions from the Economic and Social Research Council and Department of Health. EMIS: The EMIS project was funded by Executive Agency for Health and Consumers, EU Health Programme 2008–2013; CEEISCat—Centre d'Estudis Epidemiològics sobre les ITS/HIV/SIDA de Catalunya; Terrence Higgins Trust (CHAPS) for Department of Health for England; Maastricht University; Regione del Veneto; Robert Koch Institute; BZgA (Bundeszentrale für gesundheitliche Aufklärung, Köln); German Ministry of Health; Finnish Ministry of Health; Norwegian Institute of Public Health; Swedish Board of Health and Welfare. London–GMSHS: This study was funded by Public Health England, Colindale, UK. Scotland–GMSHS: The 2011 survey was funded by the UK Medical Research Council (MRC) as part of the Sexual Health programme (MC_U130031238/MC_UU_12017/2) at the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow and Health Protection Scotland. JR and LMM are funded by the MRC (MC_U130031238/MC_UU_12017/2/MC_UU_12017-11) at the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow.
Competing interests AMJ has been a Governor of the Wellcome Trust since 2011.
Ethics approval The Natsal-3 study was approved by the Oxfordshire Research Ethics Committee A (reference: 09/H0604/27). EMIS 2010 was approved by the Research Ethics Committee of the University of Portsmouth, UK (REC application number 08/09:21). London-GMSHS was approved by the University College London Research Ethics Committee. Scotland-GMSHS was approved by University of Glasgow College of Social Sciences Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.