Objectives To compare the effectiveness and cost-effectiveness of a promising new point-of-care (POC) chlamydia test with traditional nucleic acid amplification testing (NAAT), and to determine the characteristics that would make a POC test most cost-effective.
Methods A decision tree was constructed to model chlamydia screening visits to a sexually transmitted disease clinic by a hypothetical cohort of 10 000 women. The model incorporated programmatic screening costs, treatment costs and medical costs averted through prevention of pelvic inflammatory disease (PID) and its sequelae. Parameter values and costs were estimated for each node in the decision tree based on primary data, published data and unpublished health data.
Results For the base-case scenario (POC sensitivity 92.9%; 47.5% of women willing to wait 40 min for test results; test cost $33.48), POC was estimated to save US$5050 for each case of PID averted compared with NAAT. One-way sensitivity analyses indicated that POC would dominate NAAT if the POC test cost is <US$41.52 or if POC sensitivity is ≥87.1%. In a probabilistic sensitivity analysis (Monte Carlo simulations, 10 000 iterations), 10.8% of iterations indicated that the POC strategy dominated the NAAT strategy. The mean incremental cost-effectiveness ratio indicated that the POC strategy would save US$28 in total, and avert 14 PID cases.
Conclusions A promising new chlamydia POC test is likely to be cost-effective compared with traditional NAAT. The POC test sensitivity, cost and proportion of women willing to wait for the POC test result are key elements to determining the cost-effectiveness of any new POC test strategy.
- Chlamydia Trachomatis
Statistics from Altmetric.com
Chlamydia trachomatis infection is the most common bacterial sexually transmitted infection (STI) in the USA, with >2 800 000 new cases estimated to occur annually.1 In untreated women, chlamydia infection can result in pelvic inflammatory disease (PID), which can cause serious and costly sequelae (infertility, ectopic pregnancy and chronic pelvic pain).1 Since the majority of chlamydia infections in women are asymptomatic,2 ,3 tubal damage can occur without her knowledge.
With the advent of nucleic acid amplification tests (NAATs), which can use non-invasive specimens and have high sensitivity and specificity, improved detection and treatment of chlamydia infection has been possible.4 However, these laboratory tests involve a delay between specimen submission and receipt of test results. This delay may lead to overtreatment (if patients are treated presumptively at their initial visit), postponed treatment (if the follow-up visit occurs days/weeks later), or even lack of treatment (if patients are lost to follow-up).5
Recently, several companies have developed point-of-care (POC) tests that provide rapid results for the detection of chlamydia at the index visit. This can dramatically reduce the time between testing and treatment, as well as increase treatment rates.3 However, sensitivities of the three US Food and Drug Administration-approved POC chlamydia tests (25%–65%) preclude more widespread use in clinical settings.6 Nevertheless, POC tests can be beneficial in settings where patients do not reliably return for treatment.2 ,3 Based on focus group discussions with clinicians, opinion leaders and public health professionals, and a recent clinician survey, an ideal POC test should have ≥90% sensitivity, require ≤30 min to process, and cost ≤US$20.7
We conducted a comparative effectiveness research (CER) study to examine differences in costs and outcomes between a promising new chlamydia POC test in development and traditional chlamydia NAATs. We also evaluated the characteristics that would result in making future POC tests more cost-effective than NAATs.
Materials and methods
Decision analysis model
A decision tree (appendix 1) was constructed to model visits to a sexually transmitted disease (STD) clinic by a hypothetical cohort of 10 000 sexually active women. We compared the clinical effectiveness and economic impact of two chlamydia screening strategies: a vaginal swab POC test currently in development versus a vaginal swab NAAT assay. Parameter values and costs were estimated for each node in the decision tree based on primary, published and unpublished data.
Medical outcomes included PID resulting from untreated chlamydia infection and the three most serious sequelae of PID: chronic pelvic pain, ectopic pregnancy and tubal infertility. The model incorporated programmatic screening costs, treatment costs and medical costs averted through prevention of PID and its sequelae. We used commonly accepted time horizons for each of the sequelae: 10 years for infertility, 5 years for ectopic pregnancy and 2 years for chronic pelvic pain.8–10 In our analyses, we considered time-to-treatment effects, but not transmission effects, because most US STD services do not have the resources to trace partners of chlamydia-infected individuals. We also did not consider the potential effects of presumptive treatment for patients screened with NAAT because most of the surveyed clinicians reported that they infrequently treat women presumptively. We did, however, consider false positive and false negative results that could influence treatment decisions.
Costs were adjusted to US dollars (in 2011), and future costs were discounted at an annual rate of 3%. The analyses were conducted from a public healthcare perspective and included only direct medical costs.
Primary data were derived from a convenience sample of 154 eligible women who were recruited from the Baltimore, Maryland, STD clinics between April 2010 and February 2011 for an evaluation of a new chlamydia POC test in development. Inclusion criteria were: age ≥18 years, no antibiotic treatment within the past 21 days, ≥1 h since last urine void, and requiring a pelvic examination on the day of their visit. One participating clinic did not perform pelvic examinations on pregnant women, so at that clinic, pregnant women were excluded. For each study subject, cervical, self-collected vaginal and urine specimens were collected for chlamydia screening with a NAAT assay (Aptima Combo 2, Gen-Probe Inc, San Diego, California, USA). An additional, self-collected vaginal swab was obtained for testing with the experimental POC chlamydia test. For study purposes, a woman was considered to be chlamydia infected if ≥2/3 NAAT results were positive.
All women were given 2-week follow-up appointments. As per standard clinical protocol, patients received results at the follow-up visit, plus antibiotic treatment if infected. A disease intervention specialist (DIS) attempted to contact all infected women who did not return for their 2-week follow-up visit. Demographic and other data about study visits were extracted from electronic health records. A self-reported questionnaire, assessing patients’ perceptions about the ideal characteristics of a POC test, was completed by another set of 118 women during their index visit. Study procedures were approved by the Johns Hopkins University Institutional Review Board.
Parameter value estimates
The chlamydia prevalence estimate for the hypothetical cohort was based on the chlamydia infection rate among study women. Return rate, treatment rate and need for DIS assistance for the NAAT strategy of the decision tree were modelled using data from this study sample. Based on the performance of the new POC test, the waiting time to obtain the POC test result would be ∼40 min. For the base-case scenario, the proportion of women willing to wait 40 min to receive the POC test result was based on the self-reported questionnaires. Given that some commercially available POC tests require 20 min to perform, but also recognising that more sensitive tests may require longer processing times, we conducted sensitivity analyses using a variety of estimates for the proportion of women who would remain in the clinic for POC test results.
Based on current clinical practice patterns with POC tests for other STIs, all infected women in the POC strategy of our model who waited for their POC test result were presumed to receive treatment at their index visit. For those not willing to wait, we assumed that the same rate of infected women would receive treatment (with DIS assistance) as with the NAAT strategy. According to DIS experience, we estimated that 1 h of labour would be required to track each chlamydia-infected woman who did not receive treatment at her index visit (POC strategy), or failed to return for her 2-week follow-up visit (NAAT strategy).
The sensitivity and specificity of vaginal swab NAAT were based on studies reported in the literature and the package insert.9 ,11 ,12 We also used published literature to derive estimates for the azithromycin treatment success rate,4 pre-existing rate of PID among chlamydia-positive cases at the time of diagnosis,13 ,14 incident rate of PID among chlamydia-positive cases not treated until 2 weeks after the index visit,15 rate of PID among untreated chlamydia-positive cases,16–19 proportion of silent PID,8 proportion of inpatient PID treatment20 ,21 and rates of key sequelae.22–28 The estimated rate of PID among untreated chlamydia-positive women was based on results from earlier classical studies, which used actual laparoscopic or histological findings and reported asymptomatic and symptomatic PID. Parameter value estimates for all variables are provided in table 1.
The average cost to screen a woman for chlamydia in the clinic setting was calculated using direct clinician labour costs at the Baltimore STD clinics for a 40 min visit, supply costs for the visit, and NAAT laboratory processing costs. The laboratory cost of a NAAT was based on the Medicare reimbursement rate received by the Baltimore STD clinic. We assumed that a new POC test with improved sensitivity would be priced competitively and, therefore, used the same point estimate for POC test cost as for NAAT cost. The State of Maryland Public Health cost for 1 g azithromycin to treat chlamydia infection was used. DIS labour costs were based on actual Baltimore STD clinic salaries. Based on clinician estimates, treatment of women with a positive POC test at the index visit would require an additional 10 min of clinician time and treatment of women at a 2-week follow-up visit would require 20 min of clinician time.
The cost associated with inpatient PID treatment was based on hospital direct costs and the reimbursement rate for one 25 min outpatient follow-up visit. The cost associated with outpatient PID treatment was based on one 40 min initial visit, one 25 min follow-up visit, and medication costs. To calculate the cost associated with treatment of PID-related sequelae, we summed the proportion of women estimated to develop each sequela multiplied by the cost to treat the sequela and discounted by 3% for each year of delay in developing PID-related sequelae. Additional details of this approach have been published previously.9 ,10 Cost estimates are presented in table 1.
We used SAS V.9.2 (Cary, North Carolina, USA) to analyse our primary data. We report the POC sensitivity, specificity and positive and negative predictive values, all with 95% CI.
We used TreeAge Pro 2009 decision analysis software (TreeAge Software, Williamstown, Massachusetts, USA) to perform the cost-effectiveness analyses. The primary outcome measures were the number of cases of PID averted, and the incremental cost incurred or saved per episode of PID averted (including costs for PID-related sequelae: infertility, ectopic pregnancy and chronic pelvic pain). The incremental cost-effectiveness ratio (ICER) was calculated using NAAT as the comparator strategy.
We conducted sensitivity analyses for parameter values that were uncertain. Threshold analyses were conducted to determine the parameter value at which recommendations would change. Scenario analyses were performed to evaluate the characteristics that would make POC tests more cost-effective than NAAT based on variations among three key parameters identified by one-way sensitivity analyses. The values of the three key parameters used in our scenario analyses were based on our pilot study results and clinical needs assessment survey results. All other base-case scenario parameter values were held constant during scenario analyses.
We also conducted probabilistic sensitivity analyses (PSAs) using Monte Carlo simulations (10 000 iterations) with β distributions for binomial variable and γ distribution for cost data (appendix 2).
Of 154 eligible participants, one woman withdrew, and four did not have valid cervical NAAT results, leaving 149 women in the analysis. Participant median age was 27 years, and 99.3% were African–American. The majority of women indicated that their reason for visiting the clinic was because they were due a checkup (53.0%), or had symptoms (34.9%). Overall, 62.8% of women reported ≥1 symptom, and 80.5% had a history of STI.
The sensitivity and specificity of the experimental vaginal POC test were 92.9% (95% CI 66.1% to 99.8%) and 98.5% (95% CI 94.6% to 99.8%), respectively (table 1). The positive and negative predictive values were 86.7% (95% CI 59.5% to 98.3%) and 99.2% (95% CI 95.8% to 100%), respectively.
The chlamydia prevalence rate was 9.4%. Of chlamydia-infected women at the STD clinic, 93.7% received treatment; however, 59.2% of these women required DIS intervention. Of the 118 women surveyed during an STI screening visit to assess perceptions about the ideal characteristics of an STI POC test, 47.5% said they would be willing to wait in clinic for 40 min for the POC test result (table 1).
In the base case for this hypothetical cohort of 10 000 women visiting a typical STD clinic, the POC strategy was predicted to prevent 14 additional cases of PID and save US$70 706 in direct medical costs, compared with the NAAT strategy (table 2, scenario A).
One-way sensitivity analyses conducted on the base-case scenario demonstrated that the three key parameters driving the results were POC test sensitivity, proportion of women willing to wait for POC test results, and POC test cost. These analyses also revealed that changes in the chlamydia prevalence (0%–15%; appendix 3A), proportion of women receiving treatment with or without DIS assistance (0%–100%; appendix 3B), and PID rate among untreated chlamydia-infected women (0%–50%; appendix 3C) would alter the magnitude of the outcomes, but not their direction. Threshold analyses demonstrated that the POC strategy dominated the NAAT strategy when the POC test cost was <US$41.52, or when sensitivity was ≥87.1%.
A two-way sensitivity analysis assessing the relationship between POC test sensitivity and proportion of women willing to wait for their result (two of the three key parameters) demonstrated that if ≥50% of women were willing to wait, POC test sensitivity needed to reach 86.7% in order to be at least as effective as NAAT. This relationship was investigated further by substituting four different values of the third key parameter (POC test cost), and these two-way sensitivity analyses are represented in figure 1. Figure 1C most closely approximates the base-case scenario, in which the POC test cost is the same as the NAAT cost (US$33.48). The effects of higher and lower POC test costs on this relationship are illustrated by the other panels in figure 1.
In addition to the base-case scenario, table 2 presents the best-case (B), worst-case (E), and two intermediate-case (C and D) scenarios using different combinations of the three key parameters: POC test sensitivity, cost of POC test, and proportion of women willing to wait for the POC test result. For the best-case scenario (B), which corresponds to parameter estimates recommended for an ideal POC test,7 the POC strategy would prevent 20.4 additional cases of PID and save US$236 342 in direct medical costs compared with the NAAT strategy. Parameter values for the worst-case scenario (E) correspond to the lowest POC test sensitivity (70%), the longest processing time (90 min), and the highest price (US$50) that might still be acceptable. For this worst-case scenario, the NAAT strategy would be more effective and less expensive than the POC strategy (table 2). Parameter values for the two intermediate-case scenarios (C and D) were chosen to assess the effect of an intermediate POC test sensitivity (80%) and a tradeoff between intermediately higher (US$40) or lower (US$20) cost and intermediately shorter (40 min) or longer (60 min) test processing times.
In the PSA, 10.8% of iterations indicated that POC would dominate NAAT; 5.5% indicated that POC would not save money but would avert more PID cases; and 47.8% indicated that POC would save money but avert fewer PID cases. The mean ICER indicated that the POC strategy would save US$28 and avert 14 PID cases. Figure 2 shows the cost-effectiveness acceptability curve.
This analysis of a hypothetical cohort of 10 000 women visiting a typical STD clinic demonstrated that, for the base case (POC sensitivity 92.9%; 47.5% willing to wait 40 min for results; test cost US$33.48), POC would be more effective and less expensive than NAAT. We chose to conduct a cost-effectiveness analysis form of CER because such studies examine differences in costs and outcomes, and aim to identify the approach that offers the better value, rather than the lowest cost.
This study has some limitations. First, there is a lack of consensus on the incidence of PID in women with untreated chlamydia. Nevertheless, one-way sensitivity analyses demonstrated that differences in PID rates among untreated chlamydia-infected women do not change the direction of the findings, only the magnitude (appendix 3C). Second, the primary data were only based on 149 women, and there is inherent uncertainty about estimates for several of the variables. However, we conducted sensitivity analyses that explore high and low values for such parameters, as well as PSAs, which will assist clinicians and policymakers in determining whether POC or NAAT would be best in their setting. Third, we did not consider transmission effects to sexual partners because most of the costs and morbidity of untreated chlamydia infection are borne by women. Last, our data may not be generalisable to other clinical settings.
Despite these limitations, our data show that if new POC tests have good sensitivity, a short processing time, and are priced competitively, they are likely to prevent more PID while saving money, compared with NAAT. Our results are consistent with a previous decision analysis that supported the use of moderate-sensitivity POC tests for chlamydia in scenarios where a high proportion of women are not expected to return for treatment.2 Gift et al concluded that the rapid test (ie, POC) would treat more infected cases than NAAT if the treatment return rate was <65%.2
With adequate funding, treatment rates augmented by DIS can be quite high, as demonstrated by our finding that 93.7% of women with NAAT-detected chlamydia infections received treatment. In our model, we accounted for the DIS labour cost needed to achieve such a high treatment rate. If, however, economic pressures lead to decreased DIS funding, then fewer NAAT-screened women will be treated, and the proportion of women willing to wait for a POC test result will have an even stronger influence on the comparative effectiveness of POC over NAAT.
In our base case, the promising POC test required a 40 min processing time. Very little data exist on how long patients are willing to wait in clinic to receive a POC test result. Our survey indicated that 47.5% of women would be willing to wait 40 min. Gift et al used 90% as the point estimate for women willing to wait for BioStar Chlamydia OIA test results, although they did not specify how long the wait was.2 Mahilum-Tapay et al reported that 93.1% of respondents were willing to wait for 30 min (75.0% for 0.5–2 h; 10.9% for >2 h; 7.2% for >1 day) for their results.29 If future refinements in POC tests lead to shorter processing times without a decrease in sensitivity, use of POC will prevent even more PID and become even more cost-effective.
Our model demonstrates that a promising new vaginal swab POC test is likely to be cost-effective compared with a traditional NAAT strategy in the typical STD clinical setting. With additional refinements in future POC tests, the public health benefit of this cost-effective POC strategy could be even greater. More research will be required to determine the true impact of improved POC tests on the chlamydia epidemic. POC test sensitivity, cost and proportion of women willing to wait for the POC test result at their index visit will be key factors.
The potential clinical and economic implications of a promising new point-of-care (POC) chlamydia test were compared with a traditional nucleic acid amplification test (NAAT).
Vaginal swab POC tests are likely to be cost-effective compared with traditional NAAT screening in the typical sexually transmitted disease clinic setting.
The POC test sensitivity, cost and proportion of women willing to wait for the POC test result are key factors in evaluating comparative effectiveness.
Funding the project described was supported by NIH HPTN U-01 AI06813 and Award Number U-01 U54EB007958 and 3U54EB007958-03S1, issued under the American Recovery and Reinvestment act of 2009, from the National Institute of Biomedical Imaging and Bioengineering.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Contributors All the authors had full access to the data in the study and take responsibility for the integrity of the data and the accuracy of the data analyses. DRB, CAG and WH contributed to study concept and design. MRB and MJ-G contributed to acquisition of data. DRB and WH contributed to analysis and interpretation of data. WH contributed to drafting of the manuscript. DRB, CAG and WH contributed to critical revision of the manuscript for important intellectual content. WH contributed to statistical analysis. MRB and MJ-G contributed to administrative, technical or material support. DRB and CAG contributed to study supervision.
Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Biomedical Imaging and Bioengineering or the National Institutes of Health.
Competing interests None.
Patient consent Obtained.
Ethics approval Study procedures were approved by the Johns Hopkins University Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.