Objectives We aimed to examine the impact of a point-of-care (POC) test on overtreatment and undertreatment of sexually transmitted infections (STIs) by comparing treatment patterns for gonorrhoea (detected with nucleic acid amplification testing (NAAT)) with trichomoniasis (detected by POC test) for young women seen in an emergency department (ED).
Methods We reviewed the database of a quality improvement (QI) project that aims to improve follow-up care for STIs in the ED. Data included the test result and antibiotic given (if any) during visits by women age 14–21 for whom an STI test was ordered. We generated Shewhart control charts and compared overtreatment and undertreatment rates for gonorrhoea and trichomoniasis using χ2 testing.
Results Of 1877 visits over 18 months, 8.8% of women had gonorrhoea and 16.5% had trichomoniasis. Overtreatment was higher for women with gonorrhoea than trichomoniasis (54% vs 23%, p<0.001). Overtreatment for gonorrhoea decreased from 58% to 47% (p<0.01) and overtreatment for trichomoniasis decreased from 24% to 18% (p<0.01), which corresponded to improvements in patient follow-up for the QI project. Undertreatment was higher for women with gonorrhoea than trichomoniasis (29% vs 21%, p=0.03), and did not change over time.
Conclusions A POC test improves the accuracy of STI care in an ED compared with NAAT testing. An unanticipated benefit of QI efforts to improve patient follow-up is the observed decrease in antibiotic use in the ED. Given the ability of gonorrhoea to develop antibiotic resistance, future efforts should focus on development of an accurate POC test for gonorrhoea.
- CLINICAL STI CARE
Statistics from Altmetric.com
Over 300 000 people become infected with Neisseria gonorrhea (NG) each year and in 2010, adolescent women aged 15–19 had the highest rate per age group of this sexually transmitted infection (STI).1 Despite the tests available, accurately identifying and treating this infection in emergency departments (EDs) remains a challenge. In non-ED settings, clinical judgment and algorithms based on signs, symptoms and history perform poorly in detecting STIs, with a reported 29–70% sensitivity and 30–85% specificity.2 ,3 In our prior work, we showed that a clinical algorithm was over 70% sensitive and specific to identify STIs in adolescent women in an ED.4 However, many of the variables that we identified are not routinely obtained (ie, vaginal gram stain, patient report of partner's discharge). The poor predictive value of clinical judgment presents a dilemma for providers when determining treatment at the point of care (POC), especially in an ED where length of stay is limited and follow-up may be difficult.
Accurate treatment for STIs avoids overtreatment and undertreatment but is difficult to achieve. Undertreatment (defined as no antibiotics given to those whose test is positive) increases the risk for costly and dangerous sequelae, such as reproductive complications, HIV acquisition and ongoing STI transmission. Prior studies in EDs report undertreatment rates ranging from 38% to 64%.5–9 Overtreatment (defined as giving antibiotics to those whose test is negative) comes with consequences such as allergic and gastrointestinal reactions, antibiotic resistance and unnecessary costs and anxiety for the patient. STI overtreatment for ED patients ranges from 24% to 54%.5–9 Overtreatment and undertreatment are of particular concern when dealing with NG due to the organism's ability to develop antibiotic resistance.10
The best methods to detect NG are culture or nucleic acid amplification testing (NAAT), which requires about 24–48 h for the results.11 Thus, ED providers are unaware of NG test results at the time of the patient visit. In contrast to NG, for which there is no reliable POC test available, an accurate POC test is available for Trichomonas vaginalis (TV), another common STI. The rapid antigen test takes 10 min to process, and is 90% sensitive and 100% specific compared with NAAT.12
Several authors have reported that even imperfect POC tests for STIs would improve STI control in many settings.13 A mathematical modelling study reported that a POC test for gonorrhoea (sensitivity of 80% and specificity of 90%) could improve the cost effectiveness of screening for gonorrhoea in an ED setting over NAAT testing.14 However, because there are no currently available accurate POC tests for NG, the real-world effect of using POC versus NAAT for NG testing and the impact of a POC test on the treatment of NG has not been evaluated.
Annually, the ED at Cincinnati Children's Hospital Medical Center (CCHMC) sees over 10 000 visits from women age 14–21 years, and each month about 100 of these visits include an STI test. The prevalence of STIs in this population is high: 8% have NG and 15% have TV.15 Because of the high volume and high prevalence in the ED, we implemented a quality improvement (QI) project to improve follow-up care for adolescent women diagnosed with an STI. This project enabled us to increase the proportion of infected women who were notified/treated within 7 days of their visit from 45% at baseline to over 85% at present (Reed, personal communication, 2012).15 QI project results are shared with clinicians in real time.
During this QI project, we collected data on any treatment given in the ED, and can determine treatment patterns for NG and TV. The primary aim of the current study is to use real world data to compare the levels of overtreatment and undertreatment for a POC test (TV) versus traditional testing with delayed results (NG using NAAT testing) over time to estimate the role that a POC test plays in accuracy of STI care in an ED. We chose NG as the comparator rather than chlamydia because emerging gonorrhoea resistance has increased the urgency to develop more POC tests for NG.10 The secondary aim is to compare overtreatment and undertreatment for NG by type of specimen (urine vs swab).
Using a database developed for the ongoing QI project, we abstracted data for all female ED patients who were tested for TV or NG. The project originated in July 2008, but because of the substantial procedural changes we accomplished over time, we limited our analyses to the most recent 18 months of data available (January 2011–June 2012). The database included all ED visits by women age 14–21 for whom an STI test was ordered and the following variables: date of visit, STI tests ordered, specimen source (swab or urine), test results and antibiotics given (if any). STI tests included Chlamydia trachomatis and NG NAAT (BD Probetek ET, Becton Dickinson, Sparks, Maryland, USA) and a rapid antigen test for TV (OSOM TV, Sekisui Diagnostics, Inc., Cambridge, Massachusetts, USA). Specimens were transmitted to the clinical laboratory and tested according to the manufacturer's direction. For this study only the NG and TV data were analysed. NG specimen sources included urine or swabs. In 2011, only endocervical swabs were accepted for NG NAAT testing. In January 2012, the laboratory added vaginal swabs for NG NAAT testing. The type of swab (vaginal or endocervical) is not distinguished in the database. For TV, vaginal swabs could be obtained by a provider during a pelvic exam or be self-collected from the patient. The database did not distinguish self-collected versus provider-collected swabs. Thus for analyses, the specimen source is dichotomised as urine or swab. The CCHMC Institutional Review Board reviewed the protocol for the QI project and approved these additional analyses with a waiver of written informed consent.
For NG, overtreatment was defined as a visit when NG NAAT was negative but the patient received ceftriaxone 250 mg intramuscularly or cefixime 400 mg orally. Undertreatment was defined as an ED visit when NG NAAT testing was positive but the patient did not receive either ceftriaxone or cefixime. For TV, overtreatment was defined as an ED visit when a POC TV test was negative but metronidazole 2 g orally was administered. Undertreatment was an ED visit when a POC TV test was positive but the patient did not receive metronidazole 2 g orally.
To evaluate the percentage of women overtreated and undertreated for TV and NG each month, we used Shewhart control charts (P charts) to evaluate each outcome.16 On each P chart, we plotted the percentage of failures (overtreatment or undertreatment) on the vertical axis versus the month on the x axis. The control limits for P charts are set at three SDs from the mean, based on the binomial distribution of proportions. We used the following established special cause rules to determine when a change occurred: a single point outside the control limits; a run of six to eight points in a row above or below the centreline; a trend of six consecutive points increasing or decreasing; and two of three consecutive points in the outer third of a control limit.16 Sentinel interventions or system changes that occurred during the QI project were annotated on each chart. These included a Best Practice Alert embedded in the electronic medical record; an increase in the proportion of women who were contacted within 7 days of their ED visit; and resolution of laboratory errors. The Best Practice Alert initiated in June 2011 (annotation 1 on figure 1) was a reminder embedded in the electronic health record that instructed clinicians to record a patient's cell/confidential telephone number when an STI test was ordered. The second change was a documented improvement in our ability to contact women within 7 days of their ED visit (from 70% to 85%) that occurred in August 2011 (annotation 2 on figure 1). The third event was in regards to laboratory handling of TV tests (annotation 3 on figure 1). The QI team discovered that the mean turnaround times for TV testing had increased and some positive results were not being faxed to the ED as required. Interventions resulted in laboratory improvement in January 2012. All of these changes were shared with clinicians and ED personnel at regular intervals.
In addition to QI methodology, we confirmed our results using χ2 testing to compare overtreatment rates over time. We also used χ2 testing to compare overtreatment and undertreatment rates by type of STI and by specimen source (for NG).
During the study period (January 2011–June 2012), there were 1877 visits by adolescent women with NG testing and 1492 of these included TV testing. The prevalence of NG was 8.8% and for TV was 16.5%. Figure 1 displays the control charts for overtreatment for NG over time. Initially, the mean overtreatment rate for NG was 58%; this decreased to 47% after August 2011, and remained at the lower rate for the duration of the study (figure 1). The decrease met criteria for significance by special cause variation rules (six to eight points below the mean) as well as by χ2 testing (p<0.01, see table 1). This change occurred 1 month after the QI team increased the proportion of women successfully contacted from 70% to 85% (annotation 2 on figure 1).
For TV (figure 2), the initial mean overtreatment rate was 24%; this decreased to 18% after December 2011, and remained at the lower rate for the duration of the study. The decrease met criteria for significance by special cause variation rules (six to eight points below the prior mean) and by χ2 testing (table 1, p=0.03). This change occurred about the same time that the laboratory errors were resolved (annotation 3). The last point (June 2012) was excluded as a special cause variation.
Undertreatment for NG and TV are displayed in figure 3, with the same annotations described previously. The mean undertreatment rate for NG was 29.5% and for TV was 20.2%. Because of the smaller sample size (n=166 positive for NG; n=247 positive for TV) the control limits are much wider, reflecting the less precise estimate of proportions. There were no trends for undertreatment over time for either TV or NG.
Table 1 summarises the treatment patterns for NG compared with TV. Overall, women with NG testing were more likely to be overtreated and undertreated than women with TV testing. Overtreatment was significantly higher for NG (895/1877, 52%) than for TV (279/1492, 22.4%), p<0.001. In contrast, undertreatment was lower for TV than for NG. Of the 247 women positive for TV, 50 (20.2%) were undertreated compared with 49 (29.5%) of 166 positive for NG (p=0.03). Thus, correct treatment at the ED visit (defined as antibiotics if test positive and no antibiotics if test negative) was more likely with a POC TV test than for NG with NAAT (77.9% vs 49.7%, p<0.001). Stated alternatively, of the 1012 doses of cephalosporin dispensed, 895 (88%) were given to women with a negative NG test result. When we compared the differences over time seen in the control charts, χ2 testing confirmed that overtreatment was significantly higher for NG than for TV at time 1 (58% vs 24%, p<0.01) and at time 2 (47% vs 18%, p<0.01). In terms of number of antibiotic doses, this corresponds to a decrease from 442 excess doses during time 1 (8 months, 55 doses/month) to 450 doses during time 2 (10 months, 45 doses/month).
We further investigated treatment for NG based on sample source (table 1). Of 1877 visits with testing, NG testing was performed on urine specimens at 604 (32%) visits. NG prevalence was lower in urine than swab specimens (6.4% vs 10%, p=0.012). Of 565 women with a negative NG test on a urine sample, 126 (22.3%) were overtreated compared with 768 of 1146 (67.1%) with a negative NG swab sample (p<0.001). Undertreatment was more common in those whose urine was positive than for those whose swab was positive for NG (51.3% vs 22.8%, p=0.001). Thus, those with a urine NG test were more likely to receive correct treatment than those with a swab (75.8% vs 37.4%, p<0.001).
Our results support prior mathematical modelling studies that POC tests increase the accuracy of STI treatment in an ED to about 78% compared with an accuracy of about 50% with NAAT testing.13 ,14 ,17 When ED providers must rely on clinical judgment rather than a POC test to distinguish women with and without NG, more than half of women with negative tests are treated unnecessarily, and more than a quarter of those infected do not receive treatment in the ED. Fortunately, in our ED, we have shown that we can reach untreated women and achieve successful treatment in a timely manner.15
The undertreatment rate of 29% that we found for NG is lower than the rates of 38–64% reported in other studies, while our overtreatment rate of 52% is slightly higher than the 24–32% reported.5–9 ,18 This difference may be explained by the population studied, local disease prevalence and provider practice. The study by Levitt et al6 included over 1200 ED visits by women, but only 44% of the sample were under age 30 and the STI prevalence was 6.4%. In this study overtreatment was 32% and undertreatment was 38%. One small retrospective study from a paediatric ED detected 38 women who were STI positive under age 18 (prevalence 12%) of whom 20 (52%) were untreated, and the overtreatment rate was 32% (40/125).7
There are several explanations for the lower undertreatment and higher overtreatment rates that we found using traditional NAAT testing. First, our entire population is women under the age of 25 who are at high risk for STIs. Also, we have published our high local STI prevalence in several prior studies, which drives our current QI efforts.4 ,12 ,15 ,19 ,20 In addition, providers are likely following guidelines that recommend empiric STI treatment in some situations. For example, the 2010 STD Treatment guidelines recommend cervicitis treatment with antibiotics to cover C trachomatis for women 25 years or younger, especially if follow-up cannot be ensured, and antibiotics to cover NG if the prevalence of this infection is >5%.21 Similarly, several authors recommend a low threshold for empirical treatment in the ED.7 ,8 ,18 ,22
We theorise that this world view—that adolescent women are high risk for STIs, STI test results take too long, clinical algorithms are imprecise and adolescent women seen in an ED are difficult to follow up—likely maintains the high proportion of women who are overtreated and the lower percentage who go untreated in our institution. Several observations support this theory. Although the prevalence of TV was higher than NG in this high-risk population, when a POC test is used only 20% of infections were missed in the ED and only 22% were overtreated. Overtreatment was decreased when providers ordered a urine NG test compared with a swab, suggesting that providers ordered a urine test for women with milder symptoms. Furthermore, during a QI project that demonstrated our improved ability to follow up women with an STI in a timely manner, providers responded by decreasing the use of empirical antibiotics for NG.
While overtreatment and undertreatment were lower for TV, the fact that 20% of patients who were TV positive left the ED untreated is likely due to the way the POC test is handled in our institution, with longer turnaround time for results than expected. If the POC test was performed near the bedside and the results obtained in 10 min rather than processed in the central lab and the results released in 30–90 min, we would expect undertreatment to be closer to zero.
There may be other unmeasured reasons for the NG and TV treatment patterns that we found. For example, new residents starting at the end of June may have been unaware of the rapid TV test, leading to overtreatment. Further, our limited database did not include a final diagnosis or any physical exam findings. For example, ceftriaxone and metronidazole are listed as outpatient oral therapy for pelvic inflammatory disease (PID). It is well known that the clinical diagnosis of PID is imprecise and therapy is warranted even without a positive STI test result.23 We expect that some proportion of women that we deemed ‘overtreated’ were in fact treated appropriately based on a clinical diagnosis of cervicitis or PID. However, our methods using antibiotic treatment recorded per positive or negative test results are consistent with how other authors have measured overtreatment and undertreatment.6 ,9 ,18
Although the prevalence of NG in the USA has decreased over the past several decades, the ability of NG to develop antibiotic resistance threatens this decline.10 NG first developed resistance to sulfonamides in the 1940s, then to penicillin and tetracyclines in the 1980s.24 ,25 When NG developed resistance to fluoroquinolones in 2007, cephalosporins became the last effective treatment left against the infection.26 NG strains resistant to cephalosporins and even azithromycins have now been identified.27 ,28 During our project (January 2011–June 2012), cefixime and ceftriaxone treatment were the standard treatment for NG. In August 2012, the Centers for Disease Control and Prevention changed its recommendations for NG treatment to dual therapy with ceftriaxone plus azithromycin due to the decline in effectiveness of cefixime.29 The threat of antibiotic resistance for NG is greater now than ever, making the need for accurate treatment of this STI more pressing than for other STIs.10 ,30 POC tests could dramatically improve the accuracy of treatment and the development of improved POC tests for NG should be promoted.
The development of antibiotic resistance is not limited to NG; it is also a serious problem with urinary pathogens and Staphylococcus aureus. Microorganisms can transfer resistance between species, therefore overuse of antibiotics in any situation can contribute to resistance and treatment failure.31 Thus, while we need to develop more accurate STI tests and new antibiotics, we must also develop clinical programmes that make therapy more accurate and allow us to be more responsible stewards of antibiotic use.
Neisseria gonorrhea (NG) can be difficult to accurately treat due to its asymptomatic nature and lack of point-of-care (POC) testing.
Rates of accurate treatment were consistently higher for POC tests compared with nucleic acid amplification tests in an emergency department over an 18-month period.
Accurate treatment increased during a quality improvement study focused on increasing patient follow-up.
POC tests for NG are needed to improve the rates of accurate treatment while reducing the risks of antibiotic resistance.
We gratefully acknowledge the guidance of Maria Britto, MD, and the technical support provided by Frances Saad.
Contributors JH: conception and design, drafting of the manuscript, critical revision, statistical expertise, administrative, material support and grant funding. RT: data acquisition, drafting of the manuscript and critical revision. SSC: drafting of the manuscript, critical revision and technical support. EH: drafting of the manuscript, critical revision and technical support. JR: conception and design, drafting of the manuscript, critical revision and grant funding.
Funding The project was funded with divisional resources (JSH), internal funding (Place Outcomes Research award, JLR; Summer Undergraduate Research Fellowship, SJS) and NIH (BIRCWH K12HD051953-06, JLR).
Competing interests JSH has received speakers fees (honorarium) from Sekisui Diagnostics, Inc.
Ethics approval Cincinnati Children's Hospital IRB.
Provenance and peer review Commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.