Article Text

Original research
Estimating the percentage of European MSM eligible for PrEP: insights from a bio-behavioural survey in thirteen cities
Free
  1. Maddalena Cordioli1,
  2. Lorenzo Gios1,
  3. Jörg W Huber2,
  4. Nigel Sherriff2,
  5. Cinta Folch3,
  6. Ivailo Alexiev4,
  7. Sónia Dias5,
  8. Christiana Nöstlinger6,
  9. Ana Gama5,
  10. Emilia Naseva7,
  11. Danica Valkovičová Staneková8,
  12. Ulrich Marcus9,
  13. Susanne Barbara Schink9,
  14. Magdalena Rosinska10,
  15. Karel Blondeel11,12,
  16. Igor Toskin12,
  17. Massimo Mirandola1,2
  1. 1 Infectious Diseases Section, Department of Diagnostics and Public Health, University of Verona, Verona, Italy
  2. 2 School of Health Sciences, University of Brighton, Brighton, UK
  3. 3 Centre for Epidemiological Studies on Sexually Transmitted Infections and HIV/AIDS of Catalonia (CEEISCAT), Catalonia Public Health Agency (ASPCAT), Badalona, Spain - Biomedical Research Networking Centre in Epidemiology and Public Health (CIBERESP), Madrid, Spain
  4. 4 National Reference Confirmatory Laboratory of HIV, National Center of Infectious and Parasitic Diseases, Sofia, Bulgaria
  5. 5 NOVA National School of Public Health, Public Health Research Centre, Comprehensive Health Research Centre, Universidade Nova de Lisboa, Lisboa, Lisboa, Portugal
  6. 6 Department of Public Health, Institute of Tropical Medicine, Antwerp, Belgium
  7. 7 Ministry of Health, Program "Prevention and control of HIV/AIDS", Sofia, Bulgaria
  8. 8 NRC for HIV/AIDS Prevention, Slovak Medical University, Bratislava, Slovakia
  9. 9 Department of Infectious Diseases Epidemiology, Robert Koch Institute, Berlin, Germany
  10. 10 Department of Infectious Disease Epidemiology and Surveillance, National Institute of Public Health-National Institute of Hygiene, Warszawa, Poland
  11. 11 Faculty of Medicine and Health Sciences, Ghent University, Gent, Belgium
  12. 12 UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, Switzerland
  1. Correspondence to Dr Maddalena Cordioli, Infectious Diseases Section, Department of Diagnostics and Public Health, University of Verona, Verona 37129, Italy; maddalena.cordioli{at}univr.it

Abstract

Objectives This paper aims to estimate the percentage of European men who have sex with men (MSM) who may benefit from pre-exposure prophylaxis (PrEP), applying the three most widely used HIV risk indices for MSM (MSM Risk Index, Menza score, San Diego Early Test (SDET) score) and drawing on a large-scale multisite bio-behavioural survey (Sialon II).

Methods The Sialon II study was a bio-behavioural survey among MSM implemented in 13 European cities using either time-location sampling or respondent-driven sampling. Biological and behavioural data from 4901 MSM were collected. Only behavioural data of HIV-negative individuals were considered. Three widely used risk indices to assess HIV acquisition risk among MSM were used to estimate individual HIV risk scores and PrEP eligibility criteria.

Results 4219 HIV-negative MSM were considered. Regardless the HIV risk score used and the city, percentages of MSM eligible for PrEP were found to range between 5.19% and 73.84%. Overall, the MSM Risk Index and the Menza score yielded broadly similar percentages, whereas the SDET Index provided estimates constantly lower across all cities. Although all the three scores correlated positively (r>0.6), their concordance was highly variable (0.01<CCC<0.62).

Conclusion Our findings showed the impact of different scoring systems on the estimation of the percentage of MSM who may benefit from PrEP in European cities. Although our primary aim was not to compare the performance of different HIV risk scores, data show that a considerable percentage of MSM in each city should be offered PrEP in order to reduce HIV infections. As PrEP is highly effective at preventing HIV among MSM, our findings provide useful, practical guidance for stakeholders in implementing PrEP at city level to tackle HIV infections in Europe.

  • pre-exposure prophylaxis
  • sexual behaviour
  • HIV
  • preventive health services
  • public health

Data availability statement

Data can be obtained by the corresponding author on request, after the approval of the Sialon Editorial Board.

Statistics from Altmetric.com

Introduction

HIV transmission remains a challenging public health issue globally, despite intensive efforts to prevent new infections through health promotion and timely diagnosis and treatment. Preventive strategies and actions developed and delivered worldwide over the last 30 years among key populations have had a considerable impact on the epidemic. However, such a trend has not necessarily been seen among men who have sex with men (MSM), where infections have continued to increase accounting for 38% of all new HIV diagnoses in Europe in 2018.1

During the last decade, the use of antiretroviral treatment as prevention has been advocated for MSM at high risk of HIV infection.2 Pre-exposure prophylaxis (PrEP) is a biomedical preventive strategy that HIV-negative individuals can take prior to sexual contact to reduce their risk of HIV acquisition per single sexual act. Data indicate that PrEP is highly effective among MSM in reducing HIV incidence.2 3 In 2012, the US Food and Drug Administration approved the use of PrEP as a regimen to prevent HIV infection among people at risk. Similarly in Europe, recent studies have shown both the efficacy and cost-benefit of PrEP.2 4–7 Consequently, the European Centre for Disease Prevention and Control, WHO, the European AIDS Clinical Society and the European Medicines Agency have recommended the use of PrEP among MSM as part of national comprehensive HIV and other STIs prevention programmes.8–10

However the evidence, PrEP availability within the European Union (EU) continues to be limited. Indeed, recent studies suggest that almost 500 000 MSM are not able to access PrEP despite their desire to use it.11 This situation is partly due to different European Member States (MS) being at very different stages of policy and implementation. For instance, while some countries remain in the early stages of developing national guidelines for the implementation of PrEP, others are much more advanced and have already started to deliver it routinely as a part of their national HIV/STI prevention strategies. In particular, among the 13 cities where our Sialon II study was implemented, all but 4 cities (Bratislava, Bucharest, Sofia, Vilnius) currently have PrEP programmes available at the national level. Unsurprisingly perhaps, one of the main challenges in implementing PrEP is the allocation of adequate funding and the estimation of the percentage of potential PrEP users who may wish to access it. Early estimates regarding PrEP need have relied mainly on end-user intentions or expressions of willingness to use it. However, HIV self-perceived and clinically assessed risk may not necessarily converge; resulting in people at high risk of HIV acquisition not receiving PrEP. Therefore, the possibility of estimating the percentage of eligible PrEP users among MSM is an important step to ensure national prevention strategies are feasible and effective.

Over the years, a number of indices have been developed and validated (mainly in the USA) to assess an individual’s risk and identify MSM that could potentially be offered PrEP. The most widely accepted scoring systems include the 4-item Menza score,12 the 7-item HIV Incidence Risk Index for MSM (HIRI-MSM)13 and the 4-item San Diego Early Test (SDET) score.14

To our knowledge, these indices have never been used to estimate the percentage of potential PrEP users at community level. Using data from a European bio-behavioural survey (Sialon II) carried out in 13 cities among MSM populations and applying the above-mentioned HIV risk scoring systems, this current paper estimates and compares the percentage of MSM potentially eligible for PrEP at city level. The main objective of this paper therefore is to estimate the number of MSM who may benefit in each city of using PrEP independently from their risk self-assessment or request.

Materials and methods

A detailed description of the Sialon II methodology has been published elsewhere.15 Here, we present the main methodological aspects related to this secondary analysis.

Study design and setting

Sialon II was a multicentre bio-behavioural survey with the concomitant collection of biological (oral fluid (OF) or blood specimens) and behavioural data. The survey was implemented in 13 European cities during April 2013 to November 2014 using a dual-frame sampling strategy. Time-location sampling (TLS) or respondent-driven sampling (RDS) was used based on formative research. TLS was used in nine cities: Brussels, Sofia, Hamburg, Warsaw, Lisbon, Ljubljana, Barcelona, Stockholm and Brighton. RDS was used in four cities: Verona, Vilnius, Bucharest and Bratislava.

Sample size

Sample size estimates were calculated based on the former Sialon I project and other available studies.16 Assumed HIV prevalence in the target population of 15%, a precision of 5%, a significance level of 95% and a design effect of 2.0 provided a random clustered sample size of n=392 per study site. Accounting for possible invalid samples, a final target sample per city of n=408 (TLS) and n=400 (RDS) was planned.

Data sources and measurements

Behavioural data, sociodemographic characteristics, sexual and testing behaviour and HIV status were collected through a self-completion questionnaire (note: the questionnaire was not designed specifically to collect information relating to PrEP). To allow for sampling weight calculations, items were included on venues attendance (TLS) and network size (RDS). All participants irrespective of the sampling method were asked to provide a biological specimen (OF or blood).

Participants

Men were included if they had any kind of sex with another man during the 12 months before enrolment, provided written informed consent and agreed to donate an OF or blood specimen (TLS and RDS, respectively). Men younger than 18 years were excluded. For this analysis, participants with an OF/serum positive HIV test result were excluded to ensure estimates are based on actual potentially eligible PrEP users (ie, MSM who are HIV negative).

Calculation of HIV risk scores

Outcome variables were three indices to identify MSM at risk of acquiring HIV: a revised CDC version of the HIRI-MSM called the MSM Risk Index,13 17 the Menza score12 and the Amsterdam version of the SDET score.14 18 Individual scores were calculated on the basis of the items of the Sialon II questionnaire. Some minor adjustments for generating the HIV risk scores were needed as our questions differed slightly from the items used in the original indices (table 1). All changes were conservative; in other words, the estimates presented in this paper are likely to underestimate the percentage of MSM eligible for PrEP.

Table 1

HIV risk score items, associated scores and adaptations

Statistical analysis

Following the calculation of risk scores, analysis was carried out separately for all 13 participating cities. To calculate the sampling weights, a procedure was devised based on previous publications and methodological guidelines.15 19 20 All point estimates are reported with their respective sample size, 95% CIs and estimated design effect (DE). In order to assess the concordance between the percentage of PrEP eligible users based on the three HIV risk scores, Lin’s concordance correlation coefficient (CCC) and Bland-Altman limits of agreement were used.21 22 Pearson’s r correlation was used to estimate the linearity of the estimates.

Results

This analysis is based on 4219 HIV-negative MSM recruited across 13 European cities as part of the Sialon II study (TLS n=3010; RDS n=1209). A general description of the overall sample is available in the Sialon II core publications.23 24 Our estimates based on three HIV risk acquisition scores are presented in table 2 and figure 1, organised by city to illustrate how the risk scores perform across cities.

Figure 1

HIV risk scores among European men who have sex with men (MSM) enrolled in the Sialon II study (participants with positive HIV laboratory test results were excluded). CDC, Centre for Disease Prevention and Control.

Table 2

HIV acquisition risk scores among European MSM enrolled in the Sialon II study

Overall, our estimates of potentially eligible PrEP users are broadly similar for both the MSM Risk Index and the Menza score, although the SDET Index produces consistently lower estimates across all cities, compared with the other two. City comparisons show that Sofia is the city with the highest estimates of eligible PrEP users regardless of the risk index (64.69%, 73.84% and 23.57% for MSM Risk Index, Menza and SDET scores, respectively). In contrast, Bucharest, Verona and Vilnius show the lowest percentages, although not uniformly on all scores. The MSM Risk Index and Menza scores produce very similar estimates of eligible PrEP users in Bratislava, Bucharest and Vilnius, whereas Ljubljana is the only city where the estimates obtained using the Menza is lower than that using the MSM Risk Index (33.62% vs 46.28%). As far as the SDET Index is concerned, Ljubljana shows a potential PrEP user estimate far higher than other cities with comparable estimates obtained using Menza and the MSM Risk indices. Hamburg shows a considerable percentage of potential PrEP users when SDET and Menza indices are used, whereas the MSM Risk score estimate is much lower.

The CCC between the MSM Risk score and the Menza score is 0.62 (95% CI 0.31 to 0.92; r=0.73). The same coefficient between the Menza score and the SDET score yields 0.09 (95% CI 0.00 to 0.19; r=0.65), while between the MSM Risk score and the SDET score the CCC is 0.11 (95% CI 0.01 to 0.22; r=0.728). Figure 2 shows the concordance correlation graph and the limits of agreement plot comparing the different estimates based on the three HIV risk scores. Figure 2(A1) presents the agreement between the MSM Risk score and the Menza score. The latter seems to overestimate the percentage of potential PrEP users with the exception of few cities (Ljubljana, Vilnius and Verona). The Bland-Altman limits of agreement (figure 2(A2)) confirms the overall overestimate with all estimates laying within the 95% CI and Ljubljana right on the upper limit. Figure 2(B1) shows the agreement between the MSM Risk score and the SDET score. Compared with the SDET, the MSM Risk score overestimates the percentage of potential PrEP users for all cities. Although the Bland-Altman’s limits of agreement (figure 2(B2)) confirms this finding, all estimates but Sofia lay within the 95% CI with Hamburg on the lower limit. The comparison of SDET score versus the Menza score (figure 2(C1)) shows the overestimated percentages of MSM eligible for PrEP. In this case as well, Sofia lays outside the 95% CI (figure 2(C2)).

Figure 2

Estimates of men who have sex with men (MSM) potentially eligible for pre-exposure prophylaxis (PrEP) and their concordance correlation coefficient and Bland-Altman limits of agreement.

Discussion

Previous attempts to estimate the need for PrEP have often relied solely on end-user intentions to use PrEP. A recent paper reported that 17.4% or 500 000 MSM (95% CI 420 000 to 610 000) in EU countries who would be ‘very likely’ to use PrEP if they could access it.11 This estimate is based on a self-reported intention that could potentially be determined by an individual’s perception of their own risk of HIV acquisitions and/or their perception(s) of the benefit of using PrEP.

Recently, a Canadian study among MSM found that 48.3% HIV-negative participants who perceived themselves as at low risk, met a behaviourally based criterion for PrEP prescription.25 In Thailand, another study found even larger discrepancies.26 A Belgian study also showed that self-reported risk perception was not a good indicator for PrEP eligibility.27 This suggests that simply using self-perceived HIV risk as an indicator of PrEP eligibility can lead to widely varying estimates of potential PrEP beneficiaries. The consequences of such variation relate to potentially continued and/or increased HIV transmission, and may add to cost-effectiveness of public health expenditure, that is, allocating resources to implement PrEP at country level by not accurately targeting those most in need.

In order to reduce self-perception bias, in this paper we estimated the percentage of potential PrEP users on a population level, using a set of validated indices that are commonly used by clinicians to assess PrEP indication at individual level. Our data show the minimum percentages of MSM population, in each city, that should be offered PrEP to have an impact on HIV epidemic. These percentages represent the lower level of PrEP implementation at city-level, as WHO recommends access to PrEP for anyone who belongs to an high HIV incidence population, regardless of the use of HIV risk scoring systems.

Both the MSM Risk Index and the Menza scores identify large percentages of MSM who could be eligible for PrEP use, and even the SDET, with much lower estimates overall, identifies a significant percentage of MSM who met the criteria for PrEP prescription in all cities. Importantly, our estimates are higher than the percentage of HIV-negative MSM who reported to have ever taken PrEP in a survey carried out 4 years after Sialon II was conducted (European-MSM-Internet-Survey 2017).28 In addition, PrEP eligibility, as assessed using HIV risk scores, refers to a past risk. It has to be considered that MSM who may have not taken risk so far and, therefore, are not eligible for PrEP yet, would anyway benefit from PrEP use in terms of likelihood in engaging in HIV high-risk behaviours in the coming months. The objective estimates of MSM eligible for PrEP as provided in this paper, along with the larger percentage of MSM who may benefit from PrEP use and the recently described disparities in MSM’s PrEP uptake,29 indicates that, despite the concerted PrEP promotion efforts of civil society and public health organisations across EU member states, a large gap in PrEP access exists.11

The variation of the estimates by the three different indices in our analysis most likely reflects the different factors and the scoring algorithms used in their calculation. The CDC version of MSM Risk Index and the Menza score provided broadly similar ranges of percentages of MSM potentially eligible for PrEP compared with SDET. Both the similarity between MSM Risk Index and Menza score and their difference with the SDET are reflected in the size of concordance coefficients: the SDET Index performs very differently compared with the other two. These results are consistent with the different ways in which the scores have been developed and validated. In fact, the CDC version of MSM Risk Index and the Menza score are based on a common approach in terms of variables to be considered in the algorithm and reflect the validation samples. The characteristics of the MSM cohorts used to validate the scores, particularly related to age and drug use, may have had a significant impact. Only the MSM Risk Index considers age, inversely with scores increasing for younger adults. Neither Menza nor SDET consider age in their calculation algorithm. Thus, the MSM Risk Index will lead to higher estimates in cities with younger MSM populations. This is reflected in our data: in cities with <10% of Sialon II participants being older than 44 years (Bratislava, Bucharest, Ljubljana, Sofia and Warsaw23), our estimates are higher compared with cities with older participants (Barcelona, Hamburg, Lisbon).23 For the Menza score drug use is central, with the weight originally assigned being larger than the sum all other items. Thus, in Bratislava, Hamburg, Sofia, Stockholm, Vilnius and Warsaw with use of poppers being more frequent,30 estimates of those potentially benefiting from PrEP are higher. The SDET, again, takes a different approach, only considering sexual behaviour and previous bacterial infections. Given the well-established associations between the use of recreational drugs and HIV infection risk within MSM populations30 and the possibility that sexual risk behaviours considered by the score do not capture all aspects, the SDET estimates of MSM eligible for PrEP use may be underestimated. Therefore, in cities with larger percentages of MSM diagnosed with bacterial STI(s) in the previous year (Brussels, Bucharest, Hamburg, Sofia), a larger percentage of MSM would be identified as potentially benefiting from PrEP. Of all cities, the estimates for Sofia are highest regardless of the risk index considered. We suggest that this is due to the young age, extensive popper use and the frequent diagnosis of at least one bacterial STI in the previous year, resulting in high estimates for all three risk scores.

Our data show clearly that the choice of risk index has important implications for all stakeholders involved including policy makers, public health services and PrEP users themselves. Given the different risk score algorithms, different estimates are to be expected which implies that the direct comparisons of the percentage of eligible PrEP users are only meaningful when considering the characteristics of the populations and items used in the scoring systems. In addition, these scores have been developed for clinical purposes, primarily on a data-driven approach and without theoretical underpinnings from behavioural and social sciences. This means that they may reflect the specific sociocultural and historical context in which these indices were developed (US cohorts with mostly white non-Hispanic MSM). Future European studies addressing these wider issues of theoretical underpinning and validation are important issues not only for the European context.

In terms of the limitations of our analysis, it is important to note that the sampling methodology on which Sialon II was based (TLS and RDS) can result in quite different sample characteristics which may persist even after applying weighting corrections.15 In most Western European cities, the percentage of young MSM aged ≤25 years in the samples is smaller than in the central/eastern European cities. This could impact on the estimates of the scores, particularly when age is considered as in the MSM Risk Index. Another important limitation is that, at the time of the Sialon II survey (2013–14) PrEP was neither well known nor available outside of studies and trials, therefore the Sialon II questionnaire was not designed specifically to collect data necessary to calculate precisely the three indices used in this current article.

In order to estimate the various risk scores, some minor adaptations were required (table 1). These adaptations however, were made relatively conservatively so it is arguably likely that the percentage of eligible PrEP users is underestimated for all scores. Nevertheless, caution should be used when extending our findings to MSM of different cities and social contexts, as the survey was based on 13 European cities and contextual factors not measured in the survey might have had an impact on the data and related estimates.

Answering the question which risk index is more valid and whether asking directly to the individuals about their self-perceived risk leads to more accurate estimates, would require longitudinal validation studies. Independent of the index, health services and policy makers need to be mindful of local demographics (above all age) and behavioural (above all drug use) patterns among MSM if choosing one of the three indicators discussed here to assess eligibility for PrEP. The pattern of results confirms that roll-out of PrEP should be considered widely, however, this study also shows that adopting different approaches to risk assessment will lead to significantly different estimates of potential PrEP beneficiaries. Considering this, the possibility of combining the estimates provided by the three HIV risk indices according to the specific city/region demography and behaviours should be considered carefully.

A significant percentage of European MSM should be offered PrEP and even considering only the lowest estimates obtained in our study, the allocation of funding for PrEP should be increased significantly to meet the needs of this vulnerable population. The available evidence about PrEP efficacy, safety and scalability is now overwhelming. It is now a matter of political willingness, although many factors may play a role, to allocate sufficient funding to reduce the burden of HIV among MSM.

Key messages

  • Pre-exposure prophylaxis (PrEP) is highly effective at preventing HIV among MSM; however, there is still uncertainty on how best to identify those to whom PrEP provision should be prioritised.

  • The three most widely used HIV risk indices in the MSM population yielded different estimates with poor concordance, particularly when the San Diego Early Test score is considered.

  • Regardless of the HIV risk index used, a considerable percentage of European MSM, higher than that have had access so far, should be offered PrEP in order to reduce new HIV infections.

Supplemental material

Data availability statement

Data can be obtained by the corresponding author on request, after the approval of the Sialon Editorial Board.

Ethics statements

Patient consent for publication

Ethics approval

The Sialon II master protocol was approved by the WHO Research Project Review Panel (RP2) and the WHO Research Ethics Review Committee (WHO-ERC: RPC557). Locally adapted protocols were approved by the relevant ethics committees in each participating city.

Acknowledgments

The Sialon II Network (short list): Massimo Mirandola, Lorenzo Gios (Infectious Diseases Section, Department of Pathology-Verona University Hospital—Veneto Region); Christiana Nöstlinger, Wim Vanden Berghe (Institute of Tropical Medicine, ITG, Antwerp, Belgium); Ivailo Alexiev (National Center of Infectious and Parasitic Diseases, Sofia, Bulgaria); Ulrich Marcus, Susanne Barbara Schink (Robert Koch Institute, Berlin, Germany); Barbara Suligoi, Vincenza Regine (Centro Operativo AIDS, Istituto Superiore di Sanità, Rome, Italy); Saulius Caplinskas, Irma Caplinskiene (Centre for Communicable Diseases and AIDS, Vilnius, Lithuania); Magdalena Rosińska, Marta Niedźwiedzka-Stadnik (NIZP-PZH, Warsaw, Poland); Sónia Ferreira Dias (Global Health and Tropical Medicine, GHTM, Instituto de Higiene e Medicina Tropical, IHMT, Universidade Nova de Lisboa, Portugal); Alexandru Rafila, Daniela Pitigoi (National Institute for infectious Diseases), Matei Bals (Carol Davila University of Medicine and Pharmacy, Bucharest, Romania); Danica Staneková, Monika Hábeková (Slovak Medical University, Bratislava, Slovakia); Irena Klavs (Nacionalni Institut za Javno Zdravje, Ljubljana, Slovenia); Cinta Folch, Laia Ferrer (Centre for Epidemiological Studies on HIV/STI in Catalonia CEEISCAT, Agència de Salut Pública de Catalunya, Barcelona, Spain); Inga Velicko, Sharon Kühlmann-Berenzon (Public Health Agency of Sweden, Stockholm, Sweden); Igor Toskin (Department of Reproductive Health & Research World Health Organization, Geneva, Switzerland); Nigel Sherriff (School of Health Sciences, University of Brighton, Brighton, UK).

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Handling editor Jason J Ong

  • Contributors The Sialon II network participated in the design and implementation of the study. This analysis was conceived by MC, MM and LG. Data were analysed by MM. The first draft was jointly written by MC, NS, MM, LG and JH. All authors revised the manuscript for content. All authors read and approved the final manuscript.

  • Funding This manuscript is based on data from the Sialon II project, co-funded under the European Commission Second Programme of Community Action in the field of Health (2008–2013) (Work Plan 2010).

  • Disclaimer The sole responsibility lies with the authors of this manuscript and the Commission is not responsible for any use that may be made of the information contained therein. Authors IT and KB alone are responsible for the views expressed in this publication and do not necessarily represent the decisions or the policies of the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) or the World Health Organization (WHO).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.