You are here

  • Home
  • Archive
  • Volume 95, Issue 2
  • Post-test comparison of HIV test knowledge and changes in sexual risk behaviour between clients accessing HIV testing online versus in-clinic

Article Text

Article menu

Download PDFPDF

Health services research
Original article
Post-test comparison of HIV test knowledge and changes in sexual risk behaviour between clients accessing HIV testing online versus in-clinic
  1. Travis Salway1,2,
  2. Kimberly Thomson1,2,
  3. Darlene Taylor3,
  4. Devon Haag1,
  5. Elizabeth Elliot4,
  6. Tom Wong5,
  7. Christopher K Fairley6,
  8. Troy Grennan1,7,
  9. Jean Shoveller2,
  10. Gina Ogilvie1,2,
  11. Mark Gilbert1,2
  1. 1 Clinical Prevention Services, British Columbia Centre for Disease Control, Vancouver, British Columbia, Canada
  2. 2 School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
  3. 3 School of Nursing, University of British Columbia, Okanagan, Kelowna, British Columbia, Canada
  4. 4 College of Registered Nurses of British Columbia, Vancouver, British Columbia, Canada
  5. 5 Indigenous Services Canada, Ottawa, Ontario, Canada
  6. 6 Central Clinical School, Monash University, Melbourne, Victoria, Australia
  7. 7 Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
  1. Correspondence to Dr Mark Gilbert, BC Centre for Disease Control, Vancouver, BC V5Z 4R4, Canada; mark.gilbert{at}bccdc.ca

Abstract

Objective Internet-based HIV testing offers the potential to address privacy-related barriers to testing and increase frequency of testing but may result in missed opportunities related to sexual health education and prevention that typically occur in face-to-face encounters. In this study, we assessed the HIV test knowledge and sexual risk behaviour of clients testing for HIV through GetCheckedOnline, an internet-based sexually transmitted and bloodborne infection testing platform inclusive of HIV testing, in comparison to clients testing through a large sexual health clinic.

Methods We concurrently recruited GetCheckedOnline clients and clinic clients from Vancouver, Canada, over the course of a 10-month period during 2015–2016. Participants completed baseline and 3-month questionnaires, anonymous and online. A six-item score was used to estimate knowledge of HIV test concepts typically conveyed during an HIV pretest encounter in a clinic. We used multiple regression to estimate associations between testing modality (online vs clinic based) and two outcomes—HIV test knowledge and change in condom use pre/post-test—with adjustment for relevant background factors.

Results Among 352 participants, online testers demonstrated higher HIV post-test knowledge than clinic-based testers (mean score 4.65/6 vs 4.09/6; p<0.05); this difference was reduced in adjusted analysis (p>0.05). Men who have sex with men, clients with a university degree, those who have lived in Canada >10 years and English speakers had higher HIV post-test knowledge (p<0.05). Eighteen per cent of online testers and 10% of clinic-based testers increased condom use during the 3 months post-test (p>0.05).

Conclusions In this comparative study between online and clinic-based testers, we found no evidence of decreased HIV test knowledge or decreased condom use following HIV testing through GetCheckedOnline. Our findings suggest that with careful design and attention to educational content, online testing services may not lead to missed opportunities for HIV education and counselling.

  • sexual health
  • HIV testing
  • intervention studies
  • patient education

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0

https://doi.org/10.1136/sextrans-2018-053652

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    Internet-based (or online) testing for sexually transmitted and bloodborne infections (STBBI) is considered a convenient, private and low-barrier alternative to clinic-based testing. Relatively few online STBBI testing programmes have included HIV tests.1–4 Online HIV testing offers the potential to reach individuals who have never tested before or who face barriers to testing access (eg, due to HIV-related stigma or inaccessibility of in-person testing), and to increase test frequency among individuals who are at risk of acquiring HIV due to engagement in high-risk behaviours.5 At the same time, however, online testing may result in missed opportunities for education and prevention that often occur during face-to-face clinical encounters.6 7

    The concern that online HIV testers will fail to receive important educational messages stems from long-standing guidelines for HIV screening, which recommend that HIV testing encounters include: explanation of the meanings of positive and negative results (in particular the window period and its effect on interpretation of a negative result), an assessment of the client’s preparedness to receive a positive test result, information about the implications of a positive result (ie, mandatory reporting) and the provision of counselling—all of which may vary by country or setting.8–10 Current guidelines also emphasise not mandating behavioural counselling as a requirement of testing in order to reduce barriers to testing for clients unwilling or unprepared to participate in counselling.9 10

    While missed opportunities regarding HIV education are perceived as a potential risk of online testing, to our knowledge, there is no empirical evidence to support or refute this assumption.8 Face-to-face clinical counters no doubt offer a quality of interaction that cannot easily be replicated online; however, cost and capacity pressures limit the amount of time clinicians can spend with clients, which in turn often makes it difficult for providers to address all aspects of sexual health in a single visit.11 12 In addition, individual clients will have distinct learning styles (eg, visual vs auditory), oral and written language abilities, and abilities to retain information during face-to-face clinical encounters.13 14 The risk of missed educational opportunities could be mitigated through careful and thorough design of online services that attends to educational messages that clients would otherwise receive in a face-to-face clinical encounter.

    As described in detail elsewhere, GetCheckedOnline.com (GCO, http://getcheckedonline.com) is an online STBBI testing service that was developed and launched in 2014 in British Columbia (BC), Canada.4 6 The objectives of this programme include increasing uptake and frequency of testing (leading to earlier diagnosis), by reaching populations with higher prevalence of STBBIs—many of whom face barriers to accessing testing.5

    In this study, we assessed the HIV test knowledge and sexual risk behaviour of clients testing for HIV through GCO in comparison to clients testing in-person through the Provincial STBBI Clinic. The purpose was to understand whether clients testing through GCO may be missing opportunities for education and counselling provided during clinic-based interactions with testing providers. Given the extensive efforts to incorporate key pretest HIV messages into the website described below, we hypothesised that there would be no post-test differences in HIV test knowledge or pre/post-test changes in sexual risk behaviour between online and clinic-based testers.

    Methods

    Development of educational messages in GCO

    GCO (http://getcheckedonline.com) is an online testing service for HIV, chlamydia, gonorrhoea, syphilis and hepatitis C, launched in September 2014 in Vancouver, Canada. Development of the intervention is described in detail elsewhere.4 Receiving an STBBI test through GCO involves five steps: (1) create an account; (2) complete an assessment of sexual history and risk; (3) consent to testing and print laboratory requisition form; (4) present in person to provide specimens (blood, urine, swabs) at a designated collection centre; and (5) receive results, online if all are negative, or by phone if any result is positive or indeterminate. Educational information about HIV was accessible through GCO at all online steps. During step 3, clients reviewed a page summarising key HIV concepts and then provided informed consent (online supplementary figure S1). In step 5, clients receiving negative HIV test results were shown information about the meaning of positive and negative results, including information about window periods.

    Supplemental material

    The educational content about HIV testing on GCO was developed by nurses and health educators working at the BC Centre for Disease Control (BCCDC) and was based on key concepts necessary for an individual to provide informed consent for HIV testing. In addition to reviewing existing testing and consent guidelines, we interviewed the following individuals to elicit detailed feedback about best practices for delivering relevant HIV information through GCO: clinicians who work in sexual health clinics,8 individuals working with professional practice regulatory bodies,4 authors of HIV testing guidelines, ethicists15 and end users.7 15 16 Once educational content was finalised, we incorporated it into an online informed consent page. We then pilot-tested prototypes of the online informed consent page with potential end users to understand how best to deliver this information in order to achieve adequate informed consent for HIV testing online.15 The following information was specifically incorporated into the informed consent page: information about the tests clients will receive; benefits and harms of testing; follow-up of positive STBBI results by public health nurses; descriptions of test window periods; implications of positive and negative results; electronic storage of test results; availability of support services; and STBBI disease information and prevention messages (online supplementary figure S1).15 Online materials were written with a Flesch-Kincaid readability score at or below grade 8.17 18 At the time of this study, GCO was only offered in English.

    Comparator group: clinic-based versus online testing modalities

    The comparator group for GCO were clients who accessed HIV testing at the Provincial STBBI Clinic at the BCCDC, a public health clinic in Vancouver, British Columbia, that delivers approximately 13 000 STBBI testing encounters per year. Both GCO and the Provincial STBBI Clinic are publicly funded, free of cost to the user and allow clients to test using a pseudonym (ie, no identification is required). The Provincial STBBI Clinic is staffed by specially trained certified STI practice registered nurses and physicians who routinely incorporate STBBI risk assessments and counselling into STBBI testing visits; in this regard, the Provincial Clinic offers more extensive pretest education than the average HIV testing encounter. Provincial Clinic staff were involved in the development of GCO educational materials—described above—thus ensuring that online educational content mirrored those messages typically delivered in-clinic.

    Study design

    Design and recruitment details for this study are described in a separate report published concurrently in this issue.5 A baseline questionnaire (completed +2 weeks after their test), described in our parallel publication,5 contained questions related to HIV testing knowledge (outcome 1) and sexual risk behaviour, in addition to questions on experiences of testing barriers (online supplementary text S1). In the present study, we also drew on a follow-up questionnaire, administered 3 months after their test to detect any change in condom use behaviour since the baseline (outcome 2; online supplementary text S1). While both GCO and Provincial Clinic clients may have received pretest information both before and on the day of testing, some GCO clients may have had a longer time lapse between receipt of pretest information and the date of testing because they may not have gone to the lab to provide specimens (step 4, described above) immediately after viewing the informed consent page and printing the requisition (step 3) (online supplementary figure S2). For the present study, the sample was restricted to HIV-negative participants who received an HIV test (90% of total survey respondents). All study participants provided informed consent before undergoing any study activities.

    Supplemental material

    Supplemental material

    Measurement

    In the absence of an existing HIV knowledge scale, we conducted a pilot study to develop a measure of knowledge related to key HIV testing-related concepts, based on existing clinical guidelines (online supplementary text S2).9 10 19–24 An HIV test knowledge score was calculated by summing the number of correct responses in the six-item questionnaire (outcome 1, baseline survey). In addition, the questionnaire included a behavioural measure as a proxy for HIV sexual risk: ‘Have you had anal or vaginal sex without condoms in the past 3 months?: yes/no’ (outcome 2). The condom use question was asked during the baseline and 3-month surveys to describe self-reported pretest and post-test sexual behaviour, respectively.

    Supplemental material

    The following explanatory variables were included in analysis. The primary comparison of interest was testing modality, that is, receiving an HIV test via GCO or at the Provincial STBBI Clinic. We used directed acyclic graphs to identify the minimally sufficient set of background variables to control for confounding of the effects of testing modality on both outcomes of interest (HIV knowledge and changes in sexual risk behaviours), as shown in figure 1.25 Pretest HIV test knowledge was not measured; therefore, we selected background covariates to serve as a proxy for this construct. Specifically, we hypothesised that gay, bisexual and other men who have sex with men (gbMSM) would be more likely to use GCO due to promotional activities directed at gbMSM during the study period, and that gbMSM would have higher levels of HIV test knowledge, given the high prevalence of HIV and testing rates among communities of gay and bisexual men.26 gbMSM status, age and HIV testing history were conceived as common causes of both outcomes, and choice of testing modality. Level of education completed, language spoken at home (English vs other) and immigration history (immigrated to Canada in previous 10 years) were conceived as having potential causal effects on pretest knowledge about HIV testing.

    Figure 1
    Figure 1

    Conceptual model for analysis of HIV test knowledge and condom use by testing modality: online versus clinic.

    Analysis

    First, we compared characteristics of online and clinic-based testers using t-test and χ2 test, as appropriate (p<0.05 statistically significant). Second, we used multiple regression models to evaluate the associations between testing modality and each of the outcomes. HIV test knowledge (outcome 1) at baseline was analysed as a continuous six-point variable. Change in condom use—that is, an increase in condom use, as would be expected in the context of effective HIV test counselling related to behaviour change (outcome 2)—was calculated as reporting condomless anal sex or condomless vaginal sex (CAS/CVS) at baseline (during the 3 months before the test) but not reporting CAS/CVS at follow-up (during the 3 months following the test). Linear regression was used to evaluate the relationship between testing modality and HIV test knowledge, before and after adjustment for background covariates, in a sample restricted to those who completed the HIV knowledge items. Poisson regression with robust variance estimation was used to evaluate the relationship between testing modality and change in condom use, before and after adjustment for background covariates, in a sample restricted to those who completed CAS/CVS questions at both time points. Zou’s method estimated the relative risk (RR) of increased condom use.27 95% CIs were considered statistically significant if they excluded 0 for outcome 1 (β), and if they excluded 1 for outcome 2 (RR). All analyses were conducted in R V.3.4.1; exploratory factor analysis (described above) was performed using the psych package, and modified Poisson regression was performed using the geepack package.

    Results

    A total of 68 eligible GCO clients and 284 eligible clinic clients completed the study. Online testers were older than clinic-based testers (mean: 36 years vs 33 years), more online testers were gbMSM (43% vs 17%) and fewer immigrated to Canada in the previous 10 years (4% vs 16%) (all comparisons statistically significant; table 1). (These differences are consistent with those reported in the parallel publication describing differences in testing-related barriers between online and in-clinic testers, though the exact numbers differ due to differences in the analytic sample—ie, present analyses are restricted to HIV-negative clients who received an HIV test.)

    View this table:
    Table 1

    Characteristics of survey respondents by testing modality, n=352

    In unadjusted analysis, online testers demonstrated higher HIV post-test knowledge at baseline (mean score: 4.65, 95% CI 4.36 to 4.93 vs 4.09, 95% CI 3.94 to 4.23) (table 2). This difference reduced and became non-significant in adjusted analysis. gbMSM and clients with a university degree, who have lived in Canada >10 years and speak English at home all had a significantly higher post-test HIV test knowledge following adjustment. Lower HIV test knowledge scores were observed for first-time testers compared with individuals previously tested, however this difference also reduced and became non-significant in adjusted analysis (data not shown).

    View this table:
    Table 2

    HIV test knowledge score by testing modality and covariates, n=321

    At baseline, self-reported condomless sex in the past 3 months was high within both groups (71% of online testers and 69% of clinic testers). Condom use increased among clients using both modalities by the 3-month survey; 18% of online testers and 10% of clinic-based testers increased condom use (ie, reported CAS/CVS pretest but not post-test) (table 3). The unadjusted RR comparing the two modalities was 1.87 (95% CI 0.94 to 3.74). None of the hypothesised factors shown in figure 1 were associated with a statistically significant change in condom use.

    View this table:
    Table 3

    Condomless anal or vaginal sex (CAS/CVS) by testing modality and covariates, n=271

    Discussion

    In this comparative study between online and clinic-based testers, we found no evidence of decreased HIV test knowledge or increased sexual risk behaviour following testing for HIV through GCO. In fact, online testers demonstrated significantly higher HIV test knowledge after their test, though this difference was removed after adjustment for potential confounding variables, suggesting that online HIV testing may initially have higher uptake among population subgroups who are already informed about the HIV testing process (ie, gbMSM, and those with higher educational attainment, who speak English as a first language, or who were born in Canada). To our knowledge, no other studies of online STBBI testing services have addressed this research question.

    With the exception of age and testing history, our hypothesised background covariates were significantly associated with HIV knowledge score in adjusted analyses, further supporting the construct validity of our HIV knowledge measure. Knowledge scores were significantly higher among gbMSM, individuals with a university degree, English speakers and long-term residents of Canada. Conversely, non-English speakers, recent immigrants and individuals with lower educational attainment demonstrated lower knowledge of HIV test concepts, suggesting the need to specifically adapt and tailor information for these clients, as has also been identified in qualitative research related to GCO.7 8 28 Future development of GCO will include translation into other languages common in British Columbia, as well as considering alternative methods for conveying information, such as videos.

    We did not observe any significant differences between pretest and post-test reports of condomless sex by testing modality, or by age, gbMSM status or testing history. Our findings here are in accordance with prior research where the effectiveness of behavioural counselling during HIV test visits in reducing an individual’s sexual risk of HIV acquisition or transmission has been demonstrated for HIV-positive clients but not for those who test negative.29

    We acknowledge the following limitations of our study. We were unable to assess pretest HIV knowledge due to the GCO and provincial STBBI service pathways (ie, consent to be contacted for research in the Provincial STBBI Clinic is collected at the time of the pretest encounter). Prior studies on the effectiveness of different pretest and post-test HIV counselling interventions on HIV knowledge are typically based on randomised controlled designs with assessment of both preintervention and postintervention knowledge. Individually randomised designs, however, are generally not appropriate for evaluations of public health digital interventions like GCO, which have already been implemented. While we have adjusted for variables hypothesised to affect pretest knowledge, there may be other unmeasured characteristics confounding our results relating to HIV knowledge, or that further explain the relatively high levels of test knowledge observed among online testers. However, our use of a comparative clinic which routinely offers comprehensive pretest counselling (the ‘gold standard’) is a strength of our study, considering our finding of higher knowledge among online testers. We also may not have had sufficient sample size to detect significant differences, particularly for testing history, which was not significantly associated with HIV knowledge in our final adjusted analysis. In a sensitivity analysis among first-time testers, there was no difference in post-test HIV test knowledge between online and clinic-based testers; however, the sample size was limited (n=4 first-time testers online; n=39 first-time testers in-clinic). That testing history was statistically associated with HIV knowledge scores in unadjusted analysis suggests this is a clinically important difference and that additional resources within the GCO service for first-time testers may be needed. Finally, while we examined two potential missed opportunities associated with online STBBI testing (ie, HIV testing-related education and counselling), others remain to be examined; for example, qualitative interviews with GCO users, published in a separate report in this issue, highlighted the identification of co-occurring mental health and substance use concerns (eg, anxiety, depression, addiction) as another potential missed opportunity in online versus in-person clinical STBBI testing encounters.30

    This study adds to a small but growing body of knowledge regarding the impacts and role of online STBBI testing services in comparison to clinical testing services; few comparative studies between online and clinic-based testers have been performed.5 Our findings suggest that with careful design and attention to educational content, online testing services in which individuals access HIV testing and receive negative HIV results online may not lead to missed opportunities for HIV education and counselling. In fact, online platforms offer an opportunity to standardise health education messages. Users of GCO may consistently receive specific messages—for example, those explaining window periods in relation to negative tests—compared with clients accessing testing in-clinic, where language, communication styles and timing may vary between providers or from visit to visit. The results of this study and others related to GCO4 7 15 underscore the importance of thorough consultation—with end users, clinicians and other sexual health stakeholders—and clinical service integration in order to implement online sexual health services that reduce barriers to testing while simultaneously delivering critical STBBI prevention messages at the time of testing.

    Key messages

    • Internet-based HIV testing reduces barriers related to matters of privacy and convenience but may result in missed opportunities for education typically provided in clinical encounters.

    • Among 352 adults completing HIV tests, we found no evidence of decreased HIV test knowledge or condom use among online testers, relative to clinic-based testers.

    • Careful and deliberate design of online educational content can ensure that internet-based HIV and STI testing platforms adequately address clients’ HIV/STI prevention needs.

    References

    1. ↵
      2. Gaydos CA ,
      3. Dwyer K ,
      4. Barnes M , et al
      . Internet-based screening for Chlamydia trachomatis to reach non-clinic populations with mailed self-administered vaginal swabs. Sex Transm Dis 2006;33:451–7.doi:10.1097/01.olq.0000200497.14326.fb
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      2. Levine DK ,
      3. Scott KC ,
      4. Klausner JD
      . Online syphilis testing--confidential and convenient. Sex Transm Dis 2005;32:139–41.doi:10.1097/01.olq.0000149783.67826.4d
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. Koekenbier RH ,
      3. Davidovich U ,
      4. van Leent EJ , et al
      . Online-mediated syphilis testing: feasibility, efficacy, and usage. Sex Transm Dis 2008;35:764–9.doi:10.1097/OLQ.0b013e31816fcb0a
      OpenUrlCrossRefPubMed
    4. ↵
      2. Gilbert M ,
      3. Haag D ,
      4. Hottes TS , et al
      . Get checked… where? the development of a comprehensive, integrated internet-based testing program for sexually transmitted and blood-borne infections in British Columbia, Canada. JMIR Res Protoc 2016;5:e186.doi:10.2196/resprot.6293
    5. ↵
      2. Gilbert M ,
      3. Thomson K ,
      4. Salway T , et al
      . Differences in experiences of barriers to STI testing between clients of the internet-based diagnostic testing service GetCheckedOnline.com and an STI clinic in Vancouver, Canada. Sex Transm Infect 2018:sextrans-2017-053325.doi:10.1136/sextrans-2017-053325
    6. ↵
      2. Gilbert M ,
      3. Salway T ,
      4. Haag D , et al
      . Use of getcheckedonline, a comprehensive web-based testing service for sexually transmitted and Blood-Borne infections. J Med Internet Res 2017;19:e81.doi:10.2196/jmir.7097
    7. ↵
      2. Hottes TS ,
      3. Farrell J ,
      4. Bondyra M , et al
      . Internet-based HIV and sexually transmitted infection testing in British Columbia, Canada: opinions and expectations of prospective clients. J Med Internet Res 2012;14:e41.doi:10.2196/jmir.1948
    8. ↵
      2. Gilbert M ,
      3. Hottes TS ,
      4. Chabot C
      . There are a million scenarios to consider’: Health care provider perspectives on internet-based testing for sexually transmitted infections, HIV and hepatitis C in British Columbia. In: STI & AIDS World Congress 2013. Vienna, Austria, 2013.
    9. ↵
      2. Branson BM ,
      3. Handsfield HH ,
      4. Lampe MA
      . Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Recomm reports Morb Mortal Wkly report Recomm reports 2006;55:1–17.
      OpenUrl
    10. ↵
      1. Public Health Agency of Canada
      . HIV screening and testing guide. Ottawa, Ontario, 2012.
    11. ↵
      2. Rietmeijer CA ,
      3. Alfonsi GA ,
      4. Douglas JM , et al
      . Trends in clinic visits and diagnosed Chlamydia trachomatis and Neisseria gonorrhoeae infections after the introduction of a copayment in a sexually transmitted infection clinic. Sex Transm Dis 2005;32:243–6.doi:10.1097/01.olq.0000149851.97269.91
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      2. Cassell JA ,
      3. Brook MG ,
      4. Mercer CH , et al
      . Maintaining patient access to GUM clinics: is it compatible with appointments? Sex Transm Infect 2003;79:11–15.doi:10.1136/sti.79.1.11
      OpenUrlAbstract/FREE Full Text
    13. ↵
      2. Baker DW
      . The meaning and the measure of health literacy. J Gen Intern Med 2006;21:878–83.doi:10.1111/j.1525-1497.2006.00540.x
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      2. Pashler H ,
      3. McDaniel M ,
      4. Rohrer D , et al
      . Learning Styles. Psychological Science in the Public Interest 2008;9:105–19.doi:10.1111/j.1539-6053.2009.01038.x
      OpenUrlCrossRefPubMed
    15. ↵
      2. Gilbert M ,
      3. Bonnell A ,
      4. Farrell J , et al
      . Click yes to consent: acceptability of incorporating informed consent into an internet-based testing program for sexually transmitted and blood-borne infections. Int J Med Inform 2017;105:38–48.doi:10.1016/j.ijmedinf.2017.05.020
      OpenUrl
    16. ↵
      2. Shoveller J ,
      3. Knight R ,
      4. Davis W
      . Online sexual health services: examining youth’s perspectives. Can J public Heal = Rev Can santé publique;103:14–18.
    17. ↵
      2. FLESCH R
      . A new readability yardstick. J Appl Psychol 1948;32:221–33.doi:10.1037/h0057532
      OpenUrlCrossRefPubMed
    18. ↵
      2. Eltorai AE ,
      3. Ghanian S ,
      4. Adams CA , et al
      . Readability of patient education materials on the american association for surgery of trauma website. Arch Trauma Res 2014;3:e18161.doi:10.5812/atr.18161
      OpenUrl
    19. ↵
      2. Taylor D ,
      3. Hottes T ,
      4. Lester R
      . Assessing HIV knowledge after pre-test counseling in clients attending a public health STI clinic (P131). In: John’s S , ed. 23rd Annual Canadian Conference on HIV/AIDS Research. Newfoundland, 2014.
    20. ↵
      2. Dilorio CK
      . Measurement in health behavior: methods for research and education. San Francisco: Jossey-Bass, 2005.
    21. ↵
      2. Gadermann A ,
      3. Guhn M ,
      4. Assessment BZ
      . Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assessment. Res Eval 2012;17.
    22. ↵
      2. Schreiber JB ,
      3. Nora A ,
      4. Stage FK , et al
      . Reporting structural equation modeling and confirmatory factor analysis results: a review. J Educ Res 2006;99:323–38.doi:10.3200/JOER.99.6.323-338
      OpenUrlCrossRefWeb of Science
    23. ↵
      2. Ziglio E
      . The Delphi Method and its Contribution to Decision-making. In: Adler M , Ziglio E , eds. Gazing Into the oracle: the delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers, 1996.
    24. ↵
      2. Streiner DL ,
      3. Norman GR ,
      4. Cairney J
      . Health measurement scales. Oxford University Press, 2014.
    25. ↵
      2. Greenland S ,
      3. Pearl J ,
      4. Robins JM
      . Causal diagrams for epidemiologic research. Epidemiology 1999;10:37–48.doi:10.1097/00001648-199901000-00008
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      2. Beyrer C ,
      3. Baral SD ,
      4. van Griensven F , et al
      . Global epidemiology of HIV infection in men who have sex with men. Lancet 2012;380:367–77.doi:10.1016/S0140-6736(12)60821-6
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      2. Zou G
      . A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004;159:702–6.doi:10.1093/aje/kwh090
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      2. Farrell J ,
      3. Hottes TS ,
      4. Haag D , et al
      . Health equity and British Columbia’s GetCheckedOnline program: How can we make an online testing service for sexually transmitted infections that works for everyone? Portland, Oregon: Gay and Lesbian Medical Association 33rd Annual Conference, 2015.
    29. ↵
      2. Weinhardt LS ,
      3. Carey MP ,
      4. Johnson BT , et al
      . Effects of HIV counseling and testing on sexual risk behavior: a meta-analytic review of published research, 1985-1997. Am J Public Health 1999;89:1397–405.doi:10.2105/AJPH.89.9.1397
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      2. Knight R ,
      3. Chabot C ,
      4. Carson A
      . Factors influencing the experiences of gay, bisexual and other men who have sex with men who use GetCheckedOnline, a comprehensive web-based testing service for HIV and other sexually transmitted and blood-borne infections. Sex Transm Infect 2018.

    Footnotes

    • Handling editor Jackie A Cassell

    • Contributors TS conducted analyses, wrote the first draft of the manuscript and led manuscript preparation. MG is the principal investigator of this study and led the overall conduct of the study. DT led the substudy to develop the HIV test knowledge instrument. KT assisted with data preparation and analysis. TS, KT, DH, EE, TW, CKF, TG, JS and GO contributed to the study design and interpretation of study findings. All authors have reviewed and contributed to the manuscript.

    • Funding This work was supported by the Canadian Institutes of Health Research grant numbers PHE-114129 and PHE-318068.

    • Competing interests TS, KT, DH, TG, JS, GO and MG report the above named grants from Canadian Institutes of Health Research during the conduct of the study.

    • Patient consent Not required.

    • Ethics approval University of British Columbia Clinical Research Ethics Board (H11-01168).

    • Provenance and peer review Not commissioned; internally peer reviewed.

    Linked Articles