Published Online First 2 April 2007
Funding: Funding for the work was provided by the Health Protection Agency and the Department of Health (England).
Conflict of interest: none.
Statistics from Altmetric.com
- CER, cost effectiveness ratio
- EP, ectopic pregnancy
- ICER, incremental cost effectiveness ratio
- MO, major outcome
- MOA, major outcomes averted
- NCSP, National Chlamydia Screening Programme
- PID, pelvic inflammatory disease
- PN, partner notification
- QALY, quality adjusted life year
- TFI, tubal factor infertility
The prevalence of genital chlamydial infection is 3–10% in women under 25 years old in England.1 Since many cases are asymptomatic, chlamydia screening is a way of identifying undiagnosed infection so that individuals and their partners can be treated. Earlier treatment prevents complications and reduces onward transmission. The National Chlamydia Screening Programme (NCSP) is currently being implemented in England.2,3 However, questions remain regarding its cost effectiveness and that of alternative screening strategies. A transmission dynamic model of chlamydial infection in a sexually active population was used previously to estimate the impact of different screening strategies on chlamydia prevalence.4,5 These results were used in this analysis to estimate the complications from untreated chlamydial infection, and the costs associated with acute infection, clinical complications, and screening activities. A cost effectiveness analysis was performed to compare different screening strategies, in the context of limited resources.
Transmission dynamic model
A stochastic, individual based, dynamic sexual network model was developed to simulate sexual behaviour and chlamydia transmission in England. The full methodology is explained elsewhere, including details of the extensive fitting process used to model sexual behaviour and chlamydia epidemiology realistically.4 The model simulated a heterosexual population of 20 000 men and 20 000 women aged 16–44. The rate of sexual partner change was highest in the youngest cohorts and decreased with age. Infection in the model was transmitted within discordant partnerships, assuming no acquired immunity to chlamydia. Symptomatic infection was assumed to have a shorter average duration than asymptomatic infection (1 month vs 6 months), because of active treatment seeking. Without screening effective partner notification (notification plus treatment of infected partners) was assumed to be 20%. The overall chlamydia prevalence in the model was 3.2% among all individuals, highest in 16–19 year olds and decreasing with age.4
Three opportunistic screening strategies were modelled, targeting different age groups (<20, <25, <30, <35, <40 years old):
Strategy 1 Offer an annual screen to women
Strategy 2 Offer an annual screen to women and if they have changed their partner in the last 6 months
Strategy 3 Offer an annual screen to women and men.
It was assumed that 85% of the population attended a healthcare site annually.5 Of those eligible for screening, a proportion (50% at baseline) accepted the screen. Thus, under strategy 1 the minimum interval between screens was 1 year. Once eligible, individuals attend approximately twice a year, but accept 50% of the time, hence the average time between screens was 2 years. Each subsequent screening offer was assumed to be independent of previous offers or acceptances.
Evidence from a recent study indicated that women have a greater risk of infection and reinfection if they have acquired a new partner.6 Strategy 2 extends screening eligibility based on sexual behaviour, to target those at highest risk. The NCSP recommendation of an annual screen for men and women under 25 years old2 (strategy 3) was chosen as the baseline screening strategy for sensitivity analyses. Results of the sensitivity analyses performed on screening effectiveness have been reported previously.5 The probability of accepting a screen when offered was changed for both men and women from 50% (baseline) to 10%, 30%, and 70%. An additional, pessimistic scenario of 10% of women and 1.4% of men accepting was also modelled, which roughly approximates the number of screens performed in men and women in the NCSP in 2004–5.7 The efficacy of partner notification (PN) and treatment with screening introduction was changed from 20% to 50% (applied to partners of those screened and those actively seeking treatment). A final scenario examined the cost effectiveness when individuals only accepted a screen once, since evidence suggested that acceptance declines after the first screen acceptance.8
Cost effectiveness model
A cost effectiveness model was constructed in Excel to estimate the costs of acute infection, the number of complications and their associated costs, and the costs of interventions under different screening strategies compared to no screening. The number of female and neonatal complications was modelled using a decision tree (Precision Tree, Palisade software) (fig 1).
Only symptomatic pelvic inflammatory disease (PID) was modelled, as there is evidence from Westrom et al that the severity of PID symptoms is directly related to the probability of further complications such as ectopic pregnancy (EP) and tubal factor infertility (TFI).9 Furthermore, the causal link between undetected asymptomatic PID and TFI is weak. There is conflicting evidence about the proportion of chlamydia cases that result in PID.10–12 Therefore, three scenarios were run for no screening and each screening strategy with a PID progression probability of 1%, 10%, and 30%. To determine which assumption may be closest to the actual value, the number of cases of PID estimated by the model (no screening) was compared to an estimate of the annual incidence of PID in 16–44 year olds of between 1500 and 2400 per 100 000 women, from a GP based study.13 This included all clinical diagnoses of PID from any cause, and also those who might have been misdiagnosed (none of these cases were confirmed laproscopically).
The dynamic model output the incident cases of symptomatic and asymptomatic chlamydial infection in men and women, and acute complications (symptomatic PID in women and epididymitis in men), by year for each simulation. These cases in women were then used to estimate the number of cases of EP, TFI, neonatal conjunctivitis, and neonatal pneumonia. The probabilities of complications are given in table 1 and supporting evidence in the appendix (available at http://sti.bmj.com/supplemental). Because of the stochastic nature of infection within the dynamic model, each simulation of the dynamic model resulted in a different number of cases of infection. The dynamic model was run 100 times for each scenario, and the average of these was input into the model to get base case results.
The model was constructed and parameterised from the perspective of the National Health Service in England, and included the direct costs of infection, complications, and screening. Unit costs were derived from standard data sources14–16 and other published studies. Costs to the patient and wider society were not included in this analysis as recommended by the National Institute for Health and Clinical Excellence (NICE).17 Estimates of the average costs of acute conditions, complications, and interventions are given in table 2 and further details including how they were derived are given in the appendix (see http://sti.bmj.com/supplemental). Costs estimated in previous time periods were inflated to 2004 pounds sterling (£) using the Hospital and Community Health Services Pay and Prices Index.14 All costs and effects were discounted at an annual rate of 3.5% in the base case. Sensitivity analyses were done using no discount rate for costs and effects, and 6% for costs and effects, and 3.5% for costs and no discounting of effects.
Two outcomes were considered in the analysis: the number of major outcomes averted (MOA) and quality adjusted life year (QALY) gained. The MOAs included cases of epididymitis, PID, EP, TFI, and neonatal conjunctivitis and neonatal pneumonia. Details of the QALY estimates for each condition are given in the appendix (on the STI website, see http://sti.bmj.com/supplemental).
The average cost effectiveness ratio (CER) was used to compare each strategy with no screening. The CER was calculated as: (difference in costs)/(difference in benefits), between screening and no screening, where the benefits are either MOAs or QALYs gained. However, as recommended by NICE, an incremental cost effectiveness ratio (ICER) analysis was also done to assess the relative cost effectiveness of alternative screening strategies.17 The ICER was calculated by ranking the programmes in order of net costs, and the additional benefits and additional costs of each programme compared with the previous strategy (excluding dominated ones) were estimated. Programmes were dominated if they cost more than the previous strategy and resulted in fewer benefits. Both the CER and ICER were estimated separately for each assumption about the progression to PID.
The time horizon for analysing the effects of screening was 10 years. Chronic complications in women (EP, TFI) and the associated costs that occurred until a woman was 44 years old were also included.
A probabilistic multivariate sensitivity analysis was performed to assess the uncertainty of model assumptions using @Risk (version 4.5, Palisade Corporation) running within Excel (version 2000, Microsoft). For each dynamic model simulation result (100 total for each screening strategy), the economic model was run 100 times, and for each realisation a different value for input parameters was randomly sampled from their distributions (through Latin hypercube sampling). Details of the distributions are given in the tables and the appendix (see http://sti.bmj.com/supplemental). For the multivariate sensitivity analysis, PID progression was assumed to be 10%. The ICER was estimated for the costs and effects of no screening and the top four screening strategies.
The assumption about the probability of progression to PID had a large impact on the results. The average annual incidence of predicted by the model was 58 (PID = 1%), 581 (PID = 10%), and 1750 (PID = 30%) per 100 000 women for a PID progression of 1%, 10%, and 30%, respectively. A study PID found 30% (42/140) of PID cases had evidence of ever being exposed to chlamydial infection.25 If that is applied to the numbers seen in GP surgeries, then an estimated maximum of between 450 and 720 cases of PID per 100 000 annually seen in GP surgeries may be caused by chlamydia. This suggests an estimate of around 10% progression to PID is the most likely of the PID scenarios modelled.
Under the baseline scenario without screening, in a model population of 40 000 sexually active individuals, there were on average 1392 major outcomes and 65 QALYs lost over 10 years (assuming a PID progression probability of 10%). For different PID progression probabilities there were on average 393 (1% PID) and 3529 (30% PID) MOs, corresponding to 10 and 156 QALYs lost, respectively.
The average cost effectiveness of different screening strategies (screening versus no screening) is presented in figure 2 and in tables 3–5 (results are ranked according to increasing costs). Strategy 1 was the least effective strategy, but most cost effective (that is, lowest average cost per MOA or QALY gained). Strategies 2 and 3 yielded similar results and were less cost effective than strategy 1. Extending a strategy to include older ages resulted in smaller increases in health than costs, thereby increasing the CER. The average CER of the NCSP strategy under baseline assumptions and 10% PID progression was £27 269. None of the screening programmes modelled were cost saving.
Results of the incremental cost effectiveness analyses comparing alternative strategies are given in tables 3–5. A high ICER corresponds to a small increase in benefit over the screening programme above it but with a relatively large additional cost. The rank order of screening scenarios was the same in the incremental analysis for all assumptions about PID progression. If PID progression were 1%, the ICER was very high (over £80 000 per QALY gained) for any screening programme compared to no screening. For PID progression of 10% or higher, the incremental cost per QALY gained when strategies 1, 2, and 3 (aged under 20 years) were added was below £20 000–£30 000 per QALY gained. However, adding screening of older age groups resulted in high ICERs (over £50 000).
The sensitivity of the estimated cost effectiveness to the intervention assumptions given the NCSP strategy (strategy 3, <25 years) is presented in table 6. Low acceptance resulted in a higher CER compared to the baseline of 50% acceptance. Increasing the effective partner notification rate from 20% to 50% reduced the cost effectiveness ratio by about 10%. Offering men and women aged under 25 years a single screening test was more cost effective than continuous screening, mainly because of the much lower costs. The impact of changing the discount rate is given in table 7.
Figure 3 illustrates the range of likely results from the probabilistic sensitivity analysis on the ICER (PID progression = 10%). There is considerable uncertainty, even in the no screening scenario, particularly in the QALYs lost from chlamydia (the spread in the horizontal axis is greater than in the vertical). It is clear from figure 3 that strategy 1 (<20 years) results in large incremental QALY gains and has a high probability of falling below £20 000 per QALY gained (at 10% PID progression). Moving to strategy 2 (<20 years) results in almost half the points lying above the £30 000 per QALY gained line. Including men (strategy 3, <20 years) results in small additions to the cost of the programme and small additional benefits over strategy 2, and about half of the simulations fall below £20 000 per QALY gained. Increasing the programme further (strategy 1, <30 years), would result in large additional costs and few additional benefits, with nearly all results falling above £30 000 per QALY gained.
Estimates of the costs and cost effectiveness of different chlamydia screening strategies including the current strategy recommended by the NCSP (strategy 3, annual screening offer to women and men aged under 25 years) are presented. None of the screening strategies modelled were cost saving, but all resulted in better health and fewer major outcomes.
The most influential parameter was the probability of cases progressing to PID. Most other cost effectiveness studies of chlamydia screening have used an estimate of around 25–30% progression to PID (including both symptomatic and asymptomatic PID).26 However, a recent study by van Valkengoed et al based on Dutch data concluded that the risk of PID after a chlamydial infection is likely to be less than 1%.12 Another study by Morré et al followed up 30 asymptomatically infected women and none developed PID after 1 year of follow up.11 If 30% of women with asymptomatic chlamydial infection progress to PID, we would expect a much higher reported incidence of PID in general practice than is observed. Although some cases may be undiagnosed, the number of reported cases of PID in general practice is likely to be a reasonable upper bound on the number of cases caused by chlamydial infection, since this is PID from all causes including misdiagnosis.13 In fact the number of reported cases is inconsistent with progression greater than about 10%. This has major implications for the results of the cost effectiveness analysis (tables 3–5).
If we were to consider solely the NCSP strategy compared to no screening, the average cost effectiveness ratio is about £27 000 when PID progression is 10%. NICE suggest that programmes with an ICER of greater than £20 000–£30 000 per QALY gained are unlikely to be accepted on cost effectiveness grounds.17 Therefore, the NCSP strategy appears to be acceptable on cost effectiveness grounds if we ignore other screening strategies. However, NICE recommends that the incremental cost effectiveness ratio of alternative strategies is also explored.17 This indicates that the NCSP strategy involves a relatively high expected cost compared to the additional expected benefits. If PID progression were 10% or higher, then the full incremental analysis suggests that screening men and women aged under 20 years should be recommended. If only 1% of infected women develop PID, then none of the screening strategies appeared to be acceptable on cost effectiveness grounds.
The sensitivity analyses highlighted how the current strategy could be made more cost effective. Increased acceptance rates result in more favourable cost effectiveness results compared to baseline (table 6). The high CER for low acceptance occurs from the costs not only for those who accept screening but who also do not accept a screen,27 in addition to the costs of complications. Efforts could be made to raise awareness about chlamydia and the benefits of regularly obtaining a screen to improve acceptance rates. Additionally, results from the third year of the NCSP indicate that 33% of partners were treated,3 which is higher than our baseline assumption of 20% and would make screening more cost effective. Finally, the model used in this analysis was fitted to data from a review of chlamydia prevalence studies in women, but no equivalent data were available on male prevalence.1 New evidence from the NCSP and surveillance from STI clinics suggest that the peak prevalence is in men aged 20–25.3,28 Future analyses could include new data to reflect these changes, which may in turn impact on the results.
A few papers have estimated the cost effectiveness of chlamydia screening using dynamic models20,29,30; however, most studies have used static models, which are incapable of including population level effects.26,31 Welte et al20 used an appropriate dynamic model similar to ours to examine the cost effectiveness of screening in the Netherlands. They estimated that screening might be cost saving after 10 years. The disparity in these results from ours is likely to be due to three key differences in their assumptions in both their dynamic and cost effectiveness models. Firstly, they assumed a high proportion of individuals being treated as symptomatic cases before screening introduction (∼40% compared with under 5% in our model4), thereby effectively removing them from developing complications. Secondly, they assumed a high probability of PID progression (25%). Thirdly, costs for most complications were much higher than those assumed in our model. For example, they assumed that 25% of PID cases will be admitted to hospital inpatient care, including an 11 day hospital stay, yielding an average estimated cost that was over six times higher than ours. The costs of other complications (EP, TFI, neonatal complications, epididymitis) were also higher than our estimates.
The screening costs in the current analysis were taken from a chlamydia screening pilot study.27,32 The initial set-up costs of a national chlamydia screening programme are likely to include costs not modelled in this analysis, including training costs, computerisation costs, personnel, etc. Therefore this analysis may underestimate the true costs of a screening programme, thereby making screening appear more favourable than it may be. Additionally, in accordance with the NICE guidelines, in this study only the direct medical and screening costs were examined. Another large population based chlamydia screening study is being conducted which includes an analysis of patient costs.33 These could be included in further analyses, along with other societal costs. Finally, costs associated with false positive or false negative tests were not considered in this analysis. False positive tests result in costs due to treatment and partner follow-up. If chlamydia prevalence declines, the probability of false positive results increases. Individuals with false positive tests may incur psychological and social costs associated with disclosure of diagnosis to sex partners and stigma attached to STIs, with no compensating benefit resulting from treatment gained by those infected.34–37 Therefore, there may be QALY loss from screening itself, which should be further investigated.
In this analysis two outcomes were used: MOAs and QALYs gained. MOAs are an intermediate outcome, and it is difficult to compare results with other health interventions. However, only one other cost effectiveness study has used QALYs for chlamydia screening.26,38 Hu et al also used the Institute of Medicine values,21,38 as these are the only estimates currently available. The QALY estimates could be improved in future studies to better understand the health loss from chlamydial infection, complications, and screening.
This study used a dynamic model to estimate the likely cost effectiveness of chlamydia screening strategies. Results can be used to inform decisions about which screening strategies may be the most beneficial in the context of limited healthcare resources. It suggests that offering an annual screen to men and women under 25 years of age result in ICERs above the normally accepted levels when compared with screening only those aged under 20 years (although this strategy may be deemed cost effective when compared with “no screening”). Results suggest that increasing screening acceptance and effective partner notification may yield a more favourable cost effectiveness ratio owing to greater benefits without a large relative increase in costs. Since one of the greatest uncertainties that impacts on the results is the probability of progression to PID, future work should focus on understanding its natural history. Monitoring the incidence of PID as screening is introduced nationally should be a research priority.
All chlamydia screening strategies modelled are likely to improve health and cost money.
Screening men and women aged under 25 years (NCSP strategy) appears to be cost effective compared to no screening; results suggest that a less inclusive strategy may be more acceptable on cost effectiveness grounds.
The results depend largely on the assumed progression from chlamydial infection to PID, which is likely to be lower than that used previously in cost effectiveness studies.
A realistic model of infection, disease progression, and costs can be used to estimate the likely cost effectiveness of a national chlamydia screening programme which in turn can be used to advise public health policy decisions.
We would like to thank Iona Martin for advice about laboratory testing, Alessia Melegaro for HES extraction and help analysing the data, members of the STI Unit and Modelling & Economic Unit of the HPA, and members of the NCSP team including Lynsey Emmett, Teresa Battison, and Ian Simms for comments on the paper.
Contributors: EJA was the primary author, constructed the cost effectiveness model and analysed and interpreted the results. KMET coded changes to the dynamic simulation model, helped interpret the results, and revised the manuscript. WJE assisted with model development, interpretation of results and revised the manuscript.
Adams et al cast doubt on the cost-effectiveness of opportunistic chlamydia screening, as implemented in the English National Chlamydia Screening Programme.1 A recent economic evaluation of a proactive postal screening approach has shown that this is also unlikely to be a cost-effective approach.2 These two studies, using state-of-the-art individual-based transmission dynamic network models,2,3 contrast sharply with almost all other published evaluations. Their results contradict not only those using an inappropriate modelling approach,4 but also more recent studies that used dynamic modelling and found screening to be cost effective5 or even cost saving.6,7
Adams et al’s study is an important contribution to the debate about the appropriateness of chlamydia screening programmes.8 They clearly show that a chlamydia screening intervention can appear cost effective or not, depending on the assumption made about the probability of endocervical chlamydia progressing to pelvic inflammatory disease. They concluded that opportunistic screening was not cost effective if the progression rate was below 10% and that available epidemiological data were incompatible with a higher progression rate. The recent economic evaluations using individual-based dynamic modelling that found screening to be cost saving used figures of 20% to 25%.6,7
Decision makers such as the National Institute for Health and Clinical Excellence (NICE) use the incremental cost-effectiveness ratio (ICER) for decision making. In this case, the ICER refers to the additional cost per additional unit of benefit of screening compared to the alternative of no screening. Unfortunately, Adams et al misleadingly refer to the results of their main evaluation and the sensitivity analysis as average cost-effectiveness ratios (average CERs). Their definition, “(difference in costs)/(difference in benefits) between screening and no screening” is, however, the standard definition for the ICER.1,9 The standard unit of benefit is the quality adjusted life-year (QALY) which is a single measure summarising health improvements resulting from changes in both quality and quantity of life. The only QALY estimates for chlamydia and its complications were derived from an expert panel meeting rather than primary research,10 so their validity is not known.4 Most economic evaluations of chlamydia screening have therefore estimated the costs of screening per major outcome averted, typically including pelvic inflammatory disease, ectopic pregnancy, tubal infertility and neonatal complications.2,5–7 Such studies can be compared among themselves, but are less useful for decision making because there are no agreed thresholds for cost-effectiveness measured in “natural units” such as these.
Adams et al used both measures of cost-effectiveness. They estimated that the current strategy being implemented in the National Chlamydia Screening Programme in England had an ICER of £27 269 per QALY gained and £1348 per major outcome averted, compared to no screening, assuming about 40% of sexually active women and men under 25 years would be screened every year, and that 10% of untreated chlamydia cases result in pelvic inflammatory disease. They suggest that such a programme could be accepted on cost-effectiveness grounds because the ICER was below £30,000 per QALY. In fact, NICE guidance states that “Above a most plausible ICER of £20,000/QALY, judgements about the acceptability of the technology as an effective use of NHS resources are more likely to make more explicit reference to factors including: the degree of uncertainty around the calculation of the ICERs …” (section 18.104.22.168; page 33).11 Given the considerable uncertainty about the QALY estimates used by Adams et al this ICER, in QALY terms, exceeds the acceptable threshold approved by NICE and should be subjected to much greater scrutiny. Furthermore, the coverage of chlamydia screening assumed in the baseline model has not been achieved in any area of England that has implemented the screening programme. If coverage were to reach 10% of both women and men every year, the ICER would be £57 000 per QALY.
The clear uncertainty about the economic benefits of organised chlamydia screening programmes reinforce doubts about their long-term effectiveness.8 It seems paradoxical for Adams et al to suggest that the priority for research is to determine the progression rate of chlamydia to pelvic inflammatory disease after a screening programme has been introduced. Adequate understanding of the natural history of the condition is a basic prerequisite to introducing a screening programme.12 Until this happens, however, the doubt and debate will continue.
We thank Roberts and Low for their commentary on our paper and would like to take the opportunity to clarify one or two points. Roberts and Low rightly point out that the National Institute for Clinical Excellence (NICE) recommends using incremental cost-effectiveness ratios (ICER) when assessing health interventions.1 An incremental cost-effectiveness analysis compares each intervention with the next best option. We evaluated a large number of potential interventions (“no screening”, screening men and women, or just women, targeting different age ranges, etc) and, in comparing all of these with one another, we were able to suggest which of the strategies should be adopted.2 Note that this is a much more comprehensive analysis than just comparing the current National Chlamydia Screening Programme (NCSP) strategy of screening men and women aged under 25 years with “no screening” and working out whether it is worth doing. That is not to say that this information is not provided by our analysis – it is. The NCSP strategy can be thought of as being a bundle of sub-strategies (targeting just women aged under 20, adding men aged under 20, adding women and men aged over 20), and the ICER of this strategy compared with “no screening” is the average of the ICERs of the sub-strategies. Hence, our use of the term average cost-effectiveness ratio (CER) to describe this, which follows standard health economics terminology.3 As we are careful to explain this in the text and present the results in both ways, we cannot see how they are misleading. Our more detailed analysis suggests that the current NCSP strategy should be amended, not (as Roberts and Low claim) because the average CER for this strategy (compared with no screening) is close to the “threshold” value of £30 000/Quality-Adjusted Life Years (QALY) gained, but because the incremental cost-effectiveness of expanding screening from those aged under 20 to those aged under 25 is likely to be far in excess of £30 000/QALY gained.
Roberts and Low correctly point out that there is considerable uncertainty in the QALY estimates used in our study. We accounted for this by assigning distributions to the QALY estimates in our probabilistic sensitivity analysis (see fig 3 in our paper). We acknowledge that this is not a perfect solution, but we still believe that it is worth using QALYs as an outcome measure rather than Major Outcomes Averted (MOAs) for two reasons. First, the QALY is the common currency by which decisions are made on health programmes in the UK and other countries. Second, the use of MOAs implies that all of the “major” outcomes are equal. So a case of epididymitis carries the same weight as a case of ectopic pregnancy. We doubt that many people would agree with this. Indeed, the use of MOAs in the CLaSS study leads to the seemingly perverse conclusion that screening men is more cost-effective than screening women because the probability of men developing epididymitis is greater (in their study) than the probability of women developing complications.4 Since most previous cost-effectiveness studies have used MOAs as their outcome measure, we also provide this information.
We agree with Roberts and Low that more research is needed in the two key areas of progression to pelvic inflammatory disease and QALY values for chlamydia states. However, it is not paradoxical, as they suggest, to call for better understanding of the natural history of chlamydia infection, when many studies have subsequently cast doubt on the earliest empirical estimates of pelvic inflammatory disease progression (on which initial decisions to screen were based). The purpose of science is to move knowledge forward and inform best practice. We hope that our paper has helped achieve this.
Published Online First 2 April 2007
Funding: Funding for the work was provided by the Health Protection Agency and the Department of Health (England).
Conflict of interest: none.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.