Over the past decade, a resurgence of sexual risk behaviour, sexually transmitted infections (STI) and/or incident HIV infections have been reported among men who have sex with men (MSM).1^–3 These phenomena were first noticed in cities of the industrialised world with large gay communities in North America, Europe and Australia, and later Asian cities with emerging economies, such as Bangkok and Beijing, also noticed increases in HIV prevalence.4 5 As in the beginning of the HIV epidemic, MSM continue to be disproportionately affected.

San Francisco was one of the first cities to identify AIDS cases among MSM in the early 1980s.6 Since then, the HIV epidemic has been tracked through multiple studies, including cohorts pre-dating and overlapping the discovery of AIDS, HIV and AIDS case reporting, STI surveillance, cross-sectional behavioural and seroprevalence surveys and community-based programmatic data. From these data sources, a history of the epidemic among MSM in San Francisco can be charted. Data from a hepatitis B vaccine trial and back calculation of early AIDS cases show that HIV transmission began and accelerated in the late 1970s, with a peak incidence of new infections already evident in the early 1980s.7 By the mid to late 1980s, rapid declines in sexual risk behaviour, STI and HIV incidence among MSM were noted.8 Despite this apparent success, transmission continued, albeit at a lower level, into the 1990s. The pivotal year of 1995 saw the advent of highly active antiretroviral therapy (ART), which greatly improved survival with AIDS and brought hope that HIV transmission would be reduced by therapeutic viral suppression. However, the period of rapid scale up of highly active ART in San Francisco (1995–2001) was paralleled by a resurgence in sexual risk behaviour, STI and HIV incidence.9 The resurgence may have been driven by a number of factors, including the rising number of MSM living with HIV, the resumption of sexual activity among previously ill MSM with HIV, the increase in STI among MSM with HIV and the increase in sexual risk behaviour and STI among other MSM. “ART optimism”, the decreased concern over HIV infection as a result of treatment successes, may also have contributed to the resumption in sexual risk behaviour.10 We previously reported that this resurgence may have abated in San Francisco during the mid-2000s, possibly partly due to increased “serosorting” whereby MSM were choosing partners of concordant HIV status for unprotected sex.11 The potential role of serosorting was supported by a high level of HIV testing, continued rises in STI and overall unprotected anal intercourse (UAI), but declines in UAI with partners of unknown serostatus. The period was also marked by intensified prevention efforts, notably prevention programmes for HIV-positive MSM, innovative approaches to screening and treating STI and programmes addressing substance use associated with unsafe sexual behaviour.12

The questions now at hand are: what are the current trends in the HIV epidemic among MSM and what are their implications for future HIV prevention? Because of the high concentration of MSM and well-established systems for collecting data, San Francisco has been among the cities reporting early changes in the HIV epidemic in MSM.13 In this paper, we apply our previously used ecological approach to compile multiple HIV-related indicators to update and examine trends in HIV prevalence and incidence. In particular, we examine whether the current status of HIV should be classified as an epidemic, a continuing endemic, or a hyperendemic, by which we mean that HIV prevalence has stabilised at a very high level, 24.3%, by the most recent estimate.14

METHODS

Overall approach and study subjects

The general methods used in this analysis have been described previously.9 11 15 We examine temporal trends in multiple indicators of HIV risk, including sexual risk behaviour, STI and HIV incidence and prevalence. Indicators originate from diverse, existing sources of data, none of which was designed as a specific research study. For all indicators, MSM and MSM who inject drugs are included.

HIV incidence

Annual HIV incidence among MSM seeking voluntary HIV testing was estimated from 1998 to the first half of 2007 using the serological testing algorithm for recent HIV seroconversion (STARHS).16 STARHS identifies recent HIV seroconversions using a less-sensitive enzyme immunoassay methodology and permits an estimation of annual HIV incidence using a calibrated period of likely seroconversion (ie, in the preceding 170 days, 95% CI 145 to 200). Specimens confirmed positive for HIV antibodies in the San Francisco Department of Public Health Laboratory were re-tested using the bioMérieux Vironostika less sensitive enzyme immunoassay (bioMérieux, Marcy l’Etoile, France) following standard Centers for Disease Control and Prevention protocols. STARHS was applied to all HIV-positive serological specimens collected by San Francisco’s municipal STI clinic testing programme and the anonymous testing sites of the AIDS Health Project (AHP), the largest provider of HIV testing in the city from 1998 to the first half of 2007. HIV incidence was calculated among testers reporting male–male sexual behaviour or identifying as gay or bisexual and who were tested with serological specimens.

AIDS and HIV non-AIDS cases

The numbers of MSM newly diagnosed with HIV, dying with AIDS or HIV non-AIDS, living with AIDS or HIV non-AIDS and ever using ART originate from the city’s HIV/AIDS case registry. AIDS case reporting has existed in San Francisco since 1981 and is over 95% complete.17 Code-based HIV case reporting was initiated in July 2002 and replaced with a name-based HIV reporting system in April 2006; reporting from both systems is incomplete. Code-based reports from 2002 forward are re-reported by name. Variables collected include demographics, vital status, ART use and mode of transmission. Updated information is collected by review of medical records approximately every 18 months. Vital status is determined by weekly review of county death certificates, local newspaper obituaries and periodic matches with the national death index. Vital status is complete to 2005. Cases are defined as using ART if their medical chart indicated that a protease inhibitor or non-nucleoside reverse transcriptase inhibitor had been prescribed or if ART was documented but information regarding the type of ART was unavailable. MSM status is determined by the initial report made and by abstracting male–male sexual behaviour or gay/bisexual orientation from medical records. The first date of testing HIV positive is recorded from medical chart abstraction and, if earlier than the initial positive laboratory test reported to the case registry, it is used for the year of diagnosis. To control for reporting delays, we defined the date of new HIV diagnosis as the report of HIV infection up to one year after the first date of HIV infection. Therefore, new HIV diagnoses are presented to 2006 only.

STI data

The numbers of new cases of male rectal gonorrhoea and primary and secondary syphilis were obtained from the citywide surveillance system for 1998–2007. The system is based on reporting from physicians and clinical laboratories. MSM status originates from the reporting records or the municipal STI clinic medical records. Cases of male rectal gonorrhoea are considered to be in MSM regardless of reported behaviour or orientation. STI among MSM living with AIDS were identified through a match of the AIDS case registry and the STI surveillance data for the years 1996–2006. For each year, MSM living with AIDS were matched to cases of gonorrhoea, chlamydia, non-gonococcal urethritis, or primary or secondary syphilis diagnosed in that year.

Sexual risk behavioural data

Data on the proportion of MSM engaging in sexual risk behaviour originate from three sources: MSM interviewed in the course of outreach prevention activities conducted by the STOP AIDS Project; MSM seeking voluntary HIV testing at the municipal STI clinic and MSM seeking anonymous testing at the AHP.

The Stop AIDS Project, a community-based organisation serving MSM in San Francisco, collects information on HIV-related risk behaviour through short street-based intercept interviews conducted at gay-oriented events, outside of gay clubs or bars and in gay neighbourhoods. Every year from 1998 to 2007 over 1200 MSM were surveyed, ranging from 4954 in 1999 to 1205 in 2003. Respondents were asked about their HIV serostatus, whether they engaged in any UAI with men and the serostatus of their partners.9 15 Until 2005, respondents were asked whether they knew the serostatus of all partners with whom they had UAI. From 2005 onwards, the serostatus of their partners, as well as whether the activity was insertive UAI or receptive UAI, was ascertained. From the STOP AIDS Project data, we tracked any UAI from 1998 to 2007, UAI with one or more partners of unknown serostatus from 1998 to 2005 and UAI with one or more partners of unknown or opposite serostatus from 2005 to 2007 among HIV-positive and HIV-negative respondents. The last indicator was further stratified as insertive potentially discordant UAI among HIV-positive MSM and receptive potentially discordant UAI among HIV-negative MSM.

All persons receiving voluntary HIV counselling and testing at the municipal STI clinic and at AHP sites completed a sexual behaviour risk assessment form at their pretest counselling session. MSM status was defined as men reporting any male partners or by self-reported gay/bisexual orientation. Any insertive UAI, any receptive UAI, as well as both of these indicators with a known HIV-positive partner, were tracked from 2001 to 2006. The sample of MSM tested ranged from 1000 in 2001 at the municipal sexually transmitted disease clinic to 4005 in 2004 at the AHP sites.

Statistical methods

For data that are counts of all such known cases (ie, HIV/AIDS case reporting, STI case reporting), we do not apply a statistical test of trend as these data do not represent probability-based samples of a larger population. For the data originating from the HIV testing sites (ie, STARHS incidence data, sexual risk behaviour data) and from the STOP AIDS Project, we make the assumption that the data each year approximate independent samples of the MSM population of San Francisco. For these data, we use the Cochrane–Armitage test to assess the trend over the total time period of available data.

RESULTS

Between 1998 and the first half of 2007, HIV incidence among MSM testing at AHP anonymous sites as determined by STARHS fluctuated, with a high of 4.1% per year in 1999 (95% CI 2.0 to 7.1), a low of 1.7% in 2003 (95% CI 0.7 to 3.6) and no overall temporal trend (p = 0.27; table 1 and fig 1). HIV incidence among MSM testing at San Francisco’s municipal STI clinic was higher than at the AHP, but also showed no overall temporal trend (p = 0.40). HIV incidence at the STI clinic was highest in 1999 (4.9%, 95% CI 2.3 to 8.3) and lowest in 2006 (2.7%, 95% CI 1.4 to 4.8).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 HIV incidence among men who have sex with men seeking HIV testing using the serological testing algorithm for recent HIV seroconversion (STARHS) assay, San Francisco, 1998–2007. STD, sexually transmitted disease.

Controlling for a reporting delay of one year, the number of new HIV infections reported and diagnosed among MSM from 2002 to 2006 was highest in 2004, 590 cases, and lowest in 2006, 396 cases (table 1 and fig 2). Deaths among MSM with HIV/AIDS have remained stable, ranging from 228 in 2002 and 2003 to 153 in 2006. The number of MSM known to be living with AIDS in San Francisco has steadily increased since 1998 from 6433 to 7809 by 2007. In 2002, the first year HIV non-AIDS cases were available, 4673 MSM were known to be living with HIV non-AIDS in San Francisco, increasing to 5392 by 2007. ART use among MSM with AIDS remained stable between 1998 and 2007 (range 88% and 91%).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2 Men who have sex with men living with HIV/AIDS, new HIV diagnoses and AIDS deaths, San Francisco, 2002–7.

New diagnoses of STI among MSM living with AIDS increased each year since 2000, from 164 cases to 328 cases by 2006 (table 1 and fig 3). Newly diagnosed cases of male rectal gonorrhoea recorded in the STI surveillance system increased from 271 cases in 1998 to 827 in 2006, with 690 reported in 2007 (table 1 and fig 3). Primary and secondary syphilis cases among MSM rapidly increased from six in 1998 to a peak of 373 in 2003 (table 1 and fig 3). By 2007, cases of syphilis declined to 195.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3 New sexually transmitted infection (STI) diagnoses among men who have sex with men (MSM) living with AIDS, male rectal gonorrhoea and primary and secondary syphilis in MSM, San Francisco, 1998–2007.

Trends in indicators of sexual risk behaviour among MSM in San Francisco were mixed. Overall, UAI among MSM from the STOP AIDS Project increased from 31% in 1998 to 46% in 2007 (p<0.001; table 1 and fig 4). UAI with a partner of unknown serostatus decreased among HIV-negative MSM from 1998 to 2004 (p = 0.001) while showing no overall trend among HIV-positive MSM during the same period (p = 0.48; table 1 and fig 4). UAI with a partner of unknown or opposite serostatus showed no temporal trend among HIV-positive MSM from 2005 to 2007 (p = 0.12) yet there was an increase among HIV-negative MSM during the same period (p = 0.001; table 1 and fig 4). Increases were noted among HIV-positive MSM reporting insertive UAI with a partner of unknown or HIV-negative serostatus and among HIV-negative MSM reporting receptive UAI with a partner of unknown or HIV-positive serostatus between 2005 and 2007 (p = 0.06 and 0.07, respectively; table 1 and fig 4). Among MSM seeking HIV counselling and testing, increases were noted in insertive UAI at AHP (p<0.001) and the STI clinic (p = 0.03), in receptive UAI at AHP (p<0.001) and in receptive UAI with a known HIV-positive partner at the STI clinic (p = 0.06; table 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4 Any unprotected anal intercourse (UAI), UAI with a partner of unknown HIV serostatus and UAI with a partner of opposite or unknown serostatus, men who have sex with men, San Francisco, 1998–2007.

DISCUSSION

Temporal trends from 1998 to 2007 in multiple biological and behavioural indicators describe a hyperendemic state of HIV infection among MSM in San Francisco. By “hyperendemic” we mean given the current state of the epidemic, HIV prevalence has stabilised at an unacceptably high level. Several factors lead us to conclude that in the absence of new and effective prevention strategies, a hyperendemic HIV state will persist. New HIV infections more than replace the number of MSM dying from AIDS. HIV incidence rates observed at the counselling and testing sites suggest a very high lifetime probability of an MSM in San Francisco becoming infected.18 The benefits of ART contribute to the growing number of MSM living with HIV/AIDS and to the potential for transmission of infection to others. Sexual risk behaviours, particularly UAI between potentially serodiscordant partners, remain common and may be increasing. STI among MSM, particularly among MSM living with AIDS, also remain elevated, further contributing to the potential for ongoing transmission through the co-factor effect of secondary STI.19 Although individual biological and behavioural indicators presented here show mixed trends, taken together and over the long term, these data indicate that HIV prevalence will persist at a high level among MSM in San Francisco for years to come. Despite well-funded innovative prevention initiatives, it has not been possible to reduce the HIV incidence rate consistently low enough to decrease HIV prevalence in the MSM community.

Decreasing the number of unrecognised HIV infections and decreasing HIV infectivity through the use of ART20 may also decrease new infections. Unfortunately, the effects of both strategies in San Francisco may have reached a plateau. In the national HIV behavioural surveillance surveys, San Francisco had the highest rate of ever testing for HIV (96%), the lowest level of previously unknown infection (23%) and the lowest HIV incidence (1.2%) of five cities included.1 A recent random digit dial telephone survey of MSM suggested that previously unrecognised infection may be even lower.21 In San Francisco HIV care and treatment is provided free or at very low cost. Persistent HIV transmission among MSM in San Francisco, despite high rates of therapy and care, suggests that increases in risk behaviour and STI have overwhelmed any decreases in infectivity due to ART.

We recognise that the ecological approach we employed has limitations. We are foremost vulnerable to an ecological fallacy as these are not data from longitudinal studies of the same individuals and therefore causal inference is limited. Also, each source of data comes with its own potential biases and limitations. For example, incidence data include only individuals seeking HIV testing and are, therefore, influenced by the testing frequency in the MSM population. STI surveillance may undercount cases to varying degrees over time and among different pathogens and temporal trends may be influenced by changing screening practices. HIV case reporting is incomplete, has undergone methodological changes over the study period and does not include undiagnosed individuals. Nonetheless, our overall approach is consistent with UNAIDS/WHO principles of the second generation of HIV surveillance,22 particularly in the use of multiple data sources and the focus on trends over time.

Of course, we cannot conclude that HIV prevention efforts have failed. Without HIV prevention efforts, rates of unsafe behaviour and HIV incidence might be higher. Alternatively, it is fair to say that existing behavioural HIV prevention efforts are not sufficient, meaning they have not been adequately implemented, or their intensity and coverage have been too low. Recent setbacks in HIV prevention research studies, several of which included San Francisco, have been disappointing. A randomised controlled trial of an intensive behavioural prevention effort failed to decrease HIV incidence when compared with usual care;23 long-term therapeutic herpes simplex virus type 2 suppression was found inefficacious in preventing HIV acquisition;24 rectal microbicides may be a remote possibility given the failure of recent vaginal microbicides25 and HIV vaccine progress remains slow.26 Pre-exposure prophylaxis holds promise; however, its efficacy and ultimate public health utility remain to be determined.

The current state of the HIV epidemic, or hyperendemic, calls for intensification of effective prevention programmes as well as vigorous pursuit of promising, novel approaches. One is the expansion of early HIV diagnosis, from rapid HIV testing in outreach settings, to ensuring routine and frequent testing of all sexually active MSM, to “opt out” testing in diverse healthcare settings where blood is screened for other conditions. Pooled RNA testing, currently done at the municipal STI clinic, could be extended to other testing sites and may identify an additional 5–10% of HIV-infected individuals undergoing regular testing, a key population as high viral load during the acute infection phase increases transmission risk.27 HIV partner services for newly diagnosed cases may be used to identify sexual networks and venues where transmission is high and inform individuals who are unaware of their infection. Evidence that some persons still remain unaware of their serostatus for many years can be found in the 100–150 individuals each year presenting with AIDS within 12 months of their HIV diagnosis in San Francisco.28

Another strategy that may need to be scaled up is prevention with positives as the numbers of MSM living with infection will extend over decades. This argument is strengthened by a recent meta-analysis demonstrating the effectiveness of prevention interventions among individuals living with HIV.29 Although access to HIV care among HIV-positive individuals in San Francisco is high, prevention in clinical settings remains underemphasised and much work remains to integrate prevention and care further. Personal cognitive counselling has been shown to be efficacious among HIV-uninfected MSM30 and is currently being studied among HIV-infected MSM, will need to be evaluated. Whereas most prevention for positive interventions has focused on reducing risk behaviours, other outcomes, such as increasing treatment adherence to lower viral loads and thereby potentially reducing infectivity, should be evaluated from a community-wide prevention perspective.

Rapidly identifying the subpopulations at highest risk of infection and addressing behavioural, biological and environmental co-factors of transmission remain priorities. For example, methamphetamine use among MSM in San Francisco still remains magnitudes higher than in the general population12 and facilitates a large proportion of new infections.31 Internet-based prevention interventions deserve further exploration to determine their efficacy as this venue is used by many MSM to meet sex partners.32 More resources are needed to translate and validate prevention programmes for the minority MSM communities with the highest prevalence of HIV, as efficacious programmes for the majority population cannot be assumed to be effective for all. Finally, addressing social biases may help reduce HIV transmission.33 For example, federal and state recognition of same-sex marriage would signal societal support for primary relationships among MSM, which may be correlated with reduced numbers of partners.

Some may contend that San Francisco is unique in the relative size, high visibility and political engagement of its gay community. However, changes in the HIV epidemic identified in San Francisco over the past 20–30 years have preceded those reported worldwide. By declaring HIV hyperendemic among MSM in San Francisco, we mean to declare the current state unacceptable and hope to stimulate a debate on the need for different prevention strategies than those that have been used in the epidemic years.