Article Text
Statistics from Altmetric.com
The Global HIV Prevention Working Group recently stated: “We should be winning in HIV prevention. There are effective means to prevent every mode of transmission; political commitment on HIV has never been stronger; and financing for HIV programs in low-and-middle income countries increased six fold between 2001 and 2006 … If comprehensive HIV prevention were brought to scale, half of the infections projected to occur by 2015 could be averted.”1
Since the recognition of HIV as the cause of AIDS, considerable resources—both human and economic—have been allocated to the development and evaluation of efficacious interventions to halt the transmission of the virus and the progression of the disease. For an even longer period, researchers have worked to develop and evaluate efficacious interventions to prevent the transmission of other sexually transmitted pathogens and to treat and cure the diseases caused by them. The interventions studied have included behavioural interventions, treatments, vaccines, vaginal microbicides and male circumcision. A recent review of trials of interventions to prevent sexual transmission of all sexually transmitted infections (STI) including HIV identified 83 trials of individual, group or community-level interventions and concluded that, although many interventions have been found to be effective against STI including HIV, few have been replicated, widely implemented or carefully evaluated for effectiveness in other settings.2
In order to stop the spread of an infection in populations, the right intervention must be delivered to the right people at the right scale, the delivery of interventions must be sustained and the adherence of individuals must be ensured. There is a considerable gap between the development and evaluation of an efficacious intervention and the implementation of the correct mix of interventions at the right scale in populations in order to achieve population-level impact.
Even in the case of individual-level biomedical interventions such as drug therapy, efficacy under clinical trial conditions may differ importantly from effectiveness in real-life usual clinical conditions. A number of factors may contribute to differences between efficacy under ideal trial conditions and effectiveness under real-life conditions. Many people may be screened before a few who fit the inclusion criteria are chosen to be included in a study, yet the results of the study are often applied to the very people who were excluded.3 The population studied in trials tends to be young, male and white; they tend to suffer from a single condition and use a single treatment. Most patients do not fit the description of study populations. They often have multiple illnesses, take multiple medications and may be younger or older or a different gender or ethnicity than trial subjects. A treatment is effective if it works in real-life non-ideal circumstances. In real life, medications will be used in doses and frequencies which were never studied and in patient groups which were never assessed in the trials. Drugs may be used in combination with other medications that have not been tested for interactions. “Effectiveness cannot be measured in controlled trials because the act of inclusion into a study is a distortion of usual practice”.3 It is important to note that drugs are licensed for use based on the results of controlled trials, but withdrawn from use because of observational data that would not be acceptable for licensing purposes.
The gap between efficacy and effectiveness of interventions may be even wider for social and behavioural, group or community-level interventions.
EVALUATING EFFICACY, EFFECTIVENESS AND IMPACT
The strongest evidence for the efficacy of an intervention is provided by randomised controlled trials (RCTs) with appropriate controls and randomisation algorithms, adequate statistical power and appropriate statistical analysis.4 Randomised trials have been generally accepted as the gold reference standard, not only for biomedical interventions but also for behavioural and social interventions.5 Some experts—while accepting RCTs as the gold standard in the evaluation of medical technologies such as medications and vaccines—suggest that randomised trials may not be the most appropriate evaluation approach for public health interventions.6 Randomised trials of HIV prevention are difficult to conduct; nearly all such trials have failed to show protection from HIV infection and a few have shown increases in risk of HIV acquisition.7 In some of these trials the intervention itself was not efficacious or was potentially harmful.8 9 Other trials assessed the efficacy of intercourse-related interventions such as condoms, diaphragms or microbicides, and compliance with intercourse-related interventions is often poor.7
Compliance is also poor with other user-dependent interventions. A trial of daily tenofovir pre-exposure prophylaxis reported product use for only 69–74% of study days.10 More recently, a trial of a diaphragm and lubricant gel with condoms failed to show efficacy; the incidence of HIV in the intervention group was not lower than in the control (condoms only) group.11 In this study, compliance with self-reported use of the diaphragm and gel was around 73%. In addition, self-reported condom use at last intercourse was substantially higher in the control group (85%) than in the intervention group (54%). Poor compliance (or adherence) compromises statistical power of an experimental design to show efficacy. It is noted that assessment of a user-dependent method of prevention is driven by both efficacy and actual use of the product11 and by the effects of using the product on other protective or risk behaviours.7 Often, the use of one prevention method results in discontinued use of other prevention methods, a phenomenon referred to as risk compensation. Condom migration is a well known example of this phenomenon.
The importance of adherence to intervention in the evaluation of intervention efficacy has become increasingly clear in recent years as additional efficacy trials of HIV prevention failed to decrease HIV acquisition. These findings suggest that one-time provider-delivered interventions that are temporally removed from sexual arousal and intercourse may be more successful in decreasing HIV acquisition. Male circumcision is one example of such interventions and has been shown to be efficacious in three randomised trials.12–14
Adherence is an important issue in the evaluation of efficacy of interventions—but it is not the only one. Recruitment of study subjects, retention of study subjects through the course of the trial, adequate coverage for follow-up measurement and avoidance of contamination between intervention and control conditions are all important parameters which, if not adequately implemented, may threaten successful measurement of efficacy.
Cluster-randomised trials where pairs of groups or sometimes whole communities are randomly assigned to intervention and control conditions introduce additional concerns in the measurement of intervention efficacy. Population movements which threaten the “closed population” status of clusters, population movement between intervention and control communities and contamination across intervention and control communities emerge as important considerations in this context.15 In addition, depending on the intervention and the sexual structure and transmission dynamics, the timeframe of the trial might be insufficient to fully measure the impact at a population level.
In real-life conditions, evaluation of intervention effectiveness often involves a different set of important parameters. Effectiveness of an efficacious intervention in real-life settings is influenced by fidelity to the original intervention, adaptation of the intervention to the particular population or context as needed and without sacrificing fidelity, achievement of adequate scale-up, maintenance of the intervention over time and uptake of the intervention by the target population.16 A focus on effectiveness in real-life situations necessarily shifts attention from internal validity as the predominant concern (which is the case in efficacy trials) to external validity. Recently, public health literature has reflected an increasing interest in effectiveness of interventions; one example of such interest is the RE-AIM model.17 The RE-AIM model includes Reach, Effectiveness, Adoption, Implementation and Maintenance as its key dimensions; it is a multidimensional conceptual model that intends to focus attention on important steps and key elements in the successful translation of research to practice. Evaluation of effectiveness involves monitoring the key parameters of programme evaluation including inputs, activities, outputs and outcomes.
Evaluation of the impact of interventions at the population level is more complicated. “Impact” is not clearly defined in the literature; the definition apparently varies across authors. Most experts in the field agree that impact refers to the overall effect of an intervention at the population level, including direct effects on persons exposed to the intervention and indirect effects on others in the community whose health outcomes are affected by the health outcomes of persons exposed to the intervention.18 Thus, following wide-scale implantation, the population-level impact of an intervention depends on coverage, magnitude of indirect effects which may vary by the characteristics of the population and the transmission dynamics of the infection.15 In the STI/HIV area, the impact of prevention has been defined as a function of the efficacy of the intervention in a specific subpopulation, contribution of the specific subpopulation to the health outcome of the population and the achieved effective coverage of the intervention in the specific subpopulation.19
Evaluating the population-level impact of an intervention is complicated since, at any time point, multiple interventions are implemented to curb the spread of HIV or other STIs. It is difficult to attribute any changes in the incidence or prevalence of HIV to any one intervention. In the presence of multiple interventions it is important to consider synergistic or antagonistic interactions between interventions, potential duplication across interventions, potential saturation, incremental impact of each additional intervention and the potential for diminishing marginal returns.20
Recently, broader approaches to public health evaluation and health impact assessment have been proposed.21 Health impact assessment is a combination of methods to examine formally the potential health effects of a proposed policy, programme or project. Over the past decade it has received attention internationally as a cross-sectoral approach to promoting health.21
This issue of Sexually Transmitted Infections is a compilation of articles which address various aspects of the issues of efficacy, effectiveness and impact discussed above. Debate has ensued about the relative effectiveness on STIs of interventions to reduce sex partners compared with interventions to increase condom use. By applying a transmission dynamics model, the first paper in this special issue by Garnett et al explores the impact on the incidence of STI at the population level of changing partner acquisition rates and using condoms. Using a variety of biological and behavioural scenarios related to STIs, the researchers observe complex relationships among factors related to interventions that affect the numbers of infections in a population. While they confirm that both reduction of sexual partners and increased condom use lower the incidence of STIs, no simple rule allows calculation of efficiency of any single intervention at the population level.
In the second paper White et al look at the potential impact of episodic and suppressive herpes simplex virus-2 (HSV-2) therapy on the incidence of HIV at the population level. Using one low and one high HIV prevalence city in sub-Saharan Africa as examples, the authors set up a mathematical model to simulate the impact on HIV of clinician- and patient-initiated HSV-2 episodic therapy. They also modelled use of suppressive therapy, regardless of symptoms. Episodic therapy yielded little or no effect. However, the model does show that long-term suppressive HSV-2 therapy with high coverage in both types of cities could potentially have a population-level impact on the incidence of HIV.
Two papers focus on the important population of female sex workers (FSWs). The first by Blanchard, Bhattacharjee and colleagues focuses on strategies to scale up prevention programmes for FSWs in a southern Indian state. Researchers examined implementation of the prevention programme entitled Sankalp (Sanskrit for “determination”) in urban areas with a high percentage of the FSW population in the Sankalp-designated districts. FSW peer educators and outreach workers designed outreach and implementation plans to deliver the intervention. Service delivery indicators included amount of contact with FSWs by peer educators, clinic visits and condoms delivered. The researchers concluded that geographically defined coverage and micro-level outreach planning is effective in increasing all these measures. Programmes must go beyond “best practices” and focus on sufficient and effective scale and coverage. In the second paper, Blanchard, Khan and Bokhari describe the size and distribution of FSWs as well as their encounters in seven large cities in Pakistan through the use of mapping and surveys. An unexpectedly high number of sexual encounters per month identified in these areas signals the importance of assessing the size of bridge or client population in determining the focus of targeted interventions. Furthermore, variations in client-sex worker contact patterns ranged from a relatively even distribution of clients among sex workers to a highly skewed distribution. The authors conclude that programmes should consider the size and distribution of sex workers and sex worker-client dynamics in identifying and targeting interventions.
Boily et al ask new questions about the role of male circumcision (MC) as an HIV prevention strategy. Based on the fact that MC reduces STI transmission, the researchers set up a stochastic model to simulate the incremental benefits of reduced prevalence of STI due to MC on HIV. Using a generalised HIV epidemic sub-Saharan country, both individual- and population-level effects were examined. The authors found that MC-influenced protection against STIs did not substantially contribute to the overall effect of MC on HIV. They conclude that the epidemiology of STIs in a community does not change the consistency with which MC protects against HIV.
A paper by Kirby examines the reasons for the dramatic reduction in the early 1990s of HIV prevalence in Uganda, a country with a generalised HIV epidemic. Using seven different types of evidence, the author asks which sexual behaviours changed, when they changed and which of these behaviours contributed most to reduced HIV transmission. Consistent with basic theories about STI transmission, the evidence consistently showed that Ugandans first restricted their sexual activity outside long-term relationships and subsequently used condoms if they had remaining casual sexual partners. The combination of breaking up sexual networks by decreasing sexual partners and then using condoms can significantly reduce HIV transmission. These findings can be instructive for other countries with generalised HIV epidemics.
Deviating from a focus on changes in risk behaviours, Walker et al use a deterministic mathematical model to explore declines in HIV prevalence as they relate to natural dynamics between groups of people. Such dynamics include patterns of recruitment of individuals into high-risk behaviour groups, population growth and migration between regions with different types of epidemics. Their model suggests that a decline in the prevalence of HIV can be affected by HIV status and risk status of both those migrating and the population they are migrating. It also points out the limitations of looking at data in the aggregate since individual subregional declines may be masked. The model brings into question the effectiveness of individual behaviour change interventions. The authors conclude that understanding patterns of adoption of risk behaviours and patterns of migration is important in interpreting declines in the prevalence of HIV.
Finally, two papers focus on the role of contraception in prevention of mother-to-child transmission of HIV infection. Through their study, Reynolds et al estimate the number of HIV-positive births currently averted because of contraceptive use in President’s Emergency Plan for AIDS Relief (PEPFAR) focus countries. Annual cost savings of such births were also estimated. The authors conclude that the number of HIV-positive births averted range from 178 to >120 000 with annual cost savings of up to $2.2 million per year in a single country. Provision of antiretroviral prophylaxis to HIV-positive pregnant women is considerably more costly than pregnancy prevention in these women. They conclude that a comprehensive approach to prevention of mother-to-child HIV transmission which includes increased contraceptive use will maximise prevention of HIV transmission. Wilcher et al offer a commentary on issues related to translating knowledge about contraceptive efficacy on prevention of HIV-positive births into programmes at international, national and service delivery levels. Although policy support for integration of family planning into HIV programmes is substantial, HIV programmes continue to overlook family planning for HIV-positive pregnant women as an intervention. Furthermore, separate funding streams for family planning and HIV undermine the coordination and ability of providers to offer both services. Contraception is clearly underused as an intervention to reduce mother-to-child transmission of HIV. Integrated service delivery practice models are needed to guide scale-up of this effective intervention.
Although at an early stage, discussions about efficacy, effectiveness and population impact are critical for stemming the tide of STI and HIV epidemics across the globe. We hope this compilation of articles can help to move thinking about these issues forward.
Acknowledgments
The authors thank Patricia Jackson for her outstanding support in the preparation of this article.
REFERENCES
Footnotes
Competing interests: None.
The findings and conclusions in this paper are those of the author(s) and do not necessarily represent the views of the Centers for Disease Control and Prevention. The first part of this introduction will also appear in: Fenton KA, Wolitski RJ, Lyles CM, Aral SO. Adapting successful research studies in the public health arena: going from efficacy trials to effective public health interventions. In: Mayer KH, Pizer H, eds. HIV prevention, a comprehensive approach. Academic Press, 2008 (in press).