Article Text

Original article
Point-of-care management of urogenital Chlamydia trachomatis via Gram-stained smear analysis in male high-risk patients. Diagnostic accuracy and cost-effectiveness before and after changing the screening indication at the STI Clinic in Amsterdam
  1. M Bartelsman1,
  2. M S van Rooijen1,2,3,
  3. S Alba4,
  4. K Vaughan5,
  5. W R Faber1,6,
  6. M Straetemans2,
  7. H J C de Vries1,6,7
  1. 1STI Outpatient Clinic, Public Health Service of Amsterdam (GGD Amsterdam), Amsterdam, The Netherlands
  2. 2Department of Research, Public Health Service of Amsterdam (GGD Amsterdam), Amsterdam, The Netherlands
  3. 3Public Health Laboratory, Public Health Service of Amsterdam (GGD Amsterdam), Amsterdam, The Netherlands
  4. 4KIT Biomedical Research, Royal Tropical Institute, Amsterdam, The Netherlands
  5. 5KIT Health, Royal Tropical Institute, Amsterdam, The Netherlands
  6. 6Department of Dermatology, Academic Medical Center (AMC), University of Amsterdam, Amsterdam, The Netherlands
  7. 7Center for Infection and Immunology Amsterdam (CINIMA), Academic Medical Center (AMC), University of Amsterdam, Amsterdam, The Netherlands
  1. Correspondence to Professor Henry de Vries, Public Health Service of Amsterdam, Cluster of Infectious Diseases, STI Outpatient Clinic, Weesperplein 1, Amsterdam 1018 WZ, The Netherlands; h.j.devries{at}amc.nl

Abstract

Objectives To measure the effect of changing the point-of-care (POC) testing algorithm of urogenital chlamydia for all male high-risk patients to those with only symptoms with respect to: diagnostic accuracy, loss to follow-up, correctly managed consultations and costs.

Methods Retrospective comparison of the diagnostic accuracy and cost-effectiveness of Gram-stained urethral smear analysis for the POC management of urogenital Chlamydia trachomatis infections. Between 2008 and 2009 Gram-stained urethral smear analysis was offered to all men irrespective of symptoms; between 2010 and 2011 only to those with symptoms. The Aptima CT assay was the reference diagnostic test.

Results The number of examined Gram-stained smears in the two periods was respectively 7185 (2008–2009 period) and 18 852 (2010–2011 period). The sensitivity of the Gram stain analysis was respectively 83.8% (95% CI 81.2% to 86.1%) and 91.0% (95% CI 89.5% to 92.3%) (p<0.001). The specificity was respectively 74.1% (95% CI 73.0% to 75.2%) and 53.1% (95% CI 51.8% to 54.4%) (p<0.001). The positive predictive value was low in both periods, respectively 31.7% (95% CI 29.8% to 33.6%) and 35.6% (95% CI 34.1% to 37.1%) (p=0.002), whereas the negative predictive value was high, respectively 97.0% (95% CI 96.4% to 97.4%) and 95.4% (95% CI 94.6% to 96.1%) (p=0.002). The loss to follow-up rate between 2008–2009 and 2010–2011 was, respectively, 1.8% (95% CI 1.0% to 2.9%) vs 2.3% (95% CI 1.7% to 3.0%) (p=0.36). There was a small difference in overtreatment, 68.0% (95% CI 66.0% to 69.8%) vs 64.1% (95% CI 62.6% to 65.5%) (p=0.001). The cost per correctly managed consultation was 14.3% lower in the 2010–2011 period (€94.31 vs €80.82). The percentage of delayed treated infections was significantly lower in the 2008–2009 period (10.5%) compared with the 2010–2011 period (22.8%) (p<0.001).

Conclusions With a high sensitivity in male high-risk patients, the Gram-stained urethral smear is a useful POC test to detect urogenital C. trachomatis. When offered only to men with urogenital symptoms the specificity decreases but the cost per correctly managed consultation is reduced with 14.3% without a significant difference in loss to follow-up but with a significantly higher rate of delayed treatment.

  • COST-EFFECTIVENESS
  • TESTING
  • CHLAMYDIA TRACHOMATIS

Statistics from Altmetric.com

Introduction

Chlamydia trachomatis (CT) urogenital infection is a common sexually transmitted infection (STI) causing life threatening conditions like ectopic pregnancy and a high burden of morbidity like infertility and pelvic inflammatory disease.1 In 2005, according to WHO, there were approximately 98 million adults infected with CT and it was estimated that yearly 101 million new cases of CT occurred globally.2

Point-of-care (POC) tests are important in the management of STIs because they allow the clinician to provide immediate test results and treatment.3 Recently, several companies have developed lipopolysaccharide based POC tests that provide rapid results for the detection of CT. However, low sensitivity (25–65%) precludes more widespread use in clinical settings.4 ,5

In most STI clinics male patients are treated promptly for CT if a non-gonococcal urethritis (NGU) is diagnosed. NGU in men is characterised by discharge and urethral symptoms such as dysuria or urethral itching, but may be asymptomatic.6 NGU is based on the microscopic analysis of a smear from the urethra or urine sediment. NGU can be indicative for CT infections but other causative organisms such as Mycoplasma genitalum (MG), Ureaplasma urolyticum, Trichomonas vaginalis, anaerobic colon flora or herpes simplex virus may be involved. In many settings NGU is diagnosed if more than five polymorph nucleated leucocytes (PMNL) per high power field (hpf) are seen in a urethral Gram-stained smear, in the absence of intracellular negative diplococci. In the Netherlands >10 PMNLs/hpf is used as a threshold to diagnose NGU. For female high-risk patients a Gram-stained smear of genital samples is not a suitable POC test for urogenital CT because of the low positive predictive value.7

Irrespective of symptomatology all high-risk male patients who visited the STI Outpatient Clinic of the Public Health Service of Amsterdam before February 2010 were offered a Gram-stained smear as a POC test for CT and Neisseria gonorrhoeae (NG). Since February 2010, only symptomatic high-risk patients were offered a Gram-stained smear, because of structural understaffing of the POC laboratory. Here we compared the Gram stain analysis in the POC management of urogenital CT in male high-risk patients in these two time periods. We evaluated the diagnostic accuracy, loss to follow up, percentage correctly managed consultations and the costs. The data set used in the current study was collected and described in an earlier study where we investigated the Gram stain analysis for POC management of urogenital NG.8

Methods

Study setting

The STI Outpatient Clinic in Amsterdam, the Netherlands, is a nurse-led clinic that offers free STI screening and treatment. Annually approximately 38 000 screenings are performed.9 During on-line enrolment patients are stratified into different risk groups. Patients are classified as high-risk if one of the following criteria is met: having STI-related symptoms, notified of an STI by a sexual partner, paid for sexual contact, men who had sex with men or uninsured patients from sub-Saharan Africa. More information about the study population can be found in the earlier published study about POC management of urogenital NG.8

Study design and selection of patients

We performed a retrospective analysis of data from the electronic patient file. Since all data were collected for routine purposes and anonymised before the analysis, ethical clearance was not sought. Only male high-risk patients were selected. The male low-risk patients were excluded from the study because this group was not offered Gram stain examination. Some patients visited the STI clinic more than once and were offered a new standard testing procedure each time. The number of patients used in our calculations refers to the number of consultations and not to the number of unique patients.

Consultations with a missing or failing Gram stain and/or confirmation test result were excluded from the analysis. Consultations that took place on days that Gram stain examination was not available (because of laboratory understaffing) were also excluded. NGU diagnoses based on urine sediment were excluded from the diagnostic accuracy analysis. In the period between 1 January 2008 and 31 December 2009 (referred to as the 2008–2009 period) urethral Gram-stained smears for light microscopic examination were obtained from all male patients who were identified as high-risk irrespective of signs and/or symptoms. Between 12 February 2010 and 31 December 2011 (referred to as the 2010–2011 period), Gram stain analysis was performed only in male high-risk patients with urogenital signs or symptoms (discharge, painful and/or frequent urination).

Gram stain analysis and confirmation testing

If >10 PMNL/hpf were seen (in the absence of intracellular Gram-negative diplococci) in at least 3 different hpfs under a light microscope a presumptive diagnosis of NGU was made. In the case of a negative result, a first-void urine sample was examined under a light microscope after centrifugation at 1000 rpm for 3 min (urine sediment) and if more than 10 PMNL/hpf were seen in the urine also the presumptive diagnosis of NGU was made. Those diagnosed with NGU were given an instant oral dose of 1000 mg azithromycin plus post-test counselling and contact tracing. If Gram-negative diplococci were seen in the PMNLs, patients were presumptively treated for NG with ceftriaxone 500 mg intramuscularly and for CT with azithromycin 1000 mg orally. Patients with a Gram stain result of <10 PMNLs/hpf were not treated at the initial visit but were managed 1 week later when the definite results (serological, culture and nucleic acid amplification test) became available.

Aptima CT assay (Genprobe, USA) was used as the standard reference test for urogenital CT. Samples for reference testing were obtained from a first void urine sample. More information about the management of high-risk patients is described in the online supplementary data and also in an earlier published study.8

Confirmed and treated infections, prompt and delayed treatment, loss to follow-up and overtreatment

The percentage of confirmed (by standard test) and treated infections was calculated as all confirmed CT infections treated at our clinic, out of all confirmed CT infections.

The percentage of promptly tested and correctly treated infections was calculated as all confirmed CT infections treated at the initial visit out of all confirmed CT infections. The proportion of delayed treated infections was calculated as all infections treated at the return visit out of all confirmed CT infections.

The percentage of loss to follow-up was calculated as the proportion of confirmed CT infections that were not treated at our clinic within 12 weeks after the definite diagnosis was available (upon three attempts to inform the patient), out of all confirmed CT infections. The percentage of overtreatment was calculated as those who received treatment upon a false positive Gram stain (negative Aptima CT assay) out of all infections that had to be treated upon a positive Gram stain.

Costs per consultation and per correctly managed consultation

Costs of the consultations were estimated from a health services perspective. Costs included were direct staff time (salary plus benefits), clinic space, supplies, overhead and medication. Costs for the patient, like loss of productivity due to waiting hours, were not accounted for. We calculated the cost per consultation by dividing the total costs by all consultations. To combine both the costs and the diagnostic accuracy in one outcome the cost per correctly managed consultation was calculated by dividing the total costs by the number of correctly managed consultations. Incorrect management of CT was defined as delayed treatment (treatment after the first visit), no treatment at all (loss to follow-up) or overtreatment (a positive Gram stain result without confirmed CT). The remaining consultations fell into the group of correctly managed consultations, that is, treated presumptively at the clinic upon a confirmed (by standard test) CT infection or those not treated with a negative standard test result. More information about the costs calculations can be found in the online supplementary data and table S1.

Urethritis caused by NG

Gram-stained smear analysis was also used for the detection of urogenital NG. In case a presumptive NG diagnosis was made, patients were treated with ceftriaxone plus azithromycin to cover NG and CT. During the 2010–2011 period, Gram-stained smear analysis was offered only to symptomatic patients. Since in men urogenital NG is more frequently symptomatic then urogenital CT, we calculated the prevalence of NG in both periods to exclude possible bias caused by the presumptive treatment of NG infections.

Analysis

Statistical analyses were done using Stata/SE V.12.1 for Windows and IBM SPSS Statistics V.21. Differences in proportions between the two study periods with 95% CIs were tested for the equality of proportions using large-sample statistics (binomial approximation) in the two study periods. Data of the consultation costs were analysed in Excel.

Results

In the 2008–2009 period 30 079 consultations were performed in men, of which 20 492 (68.1%) were considered high-risk (figure 1). After exclusion, the final analysis set consisted of 7185 (23.9%) high-risk consultations.

Figure 1

Flow chart of the management of urogenital Chlamydia trachomatis among high-risk male patients, STI outpatient clinic, Amsterdam, The Netherlands, 2008–2011. *Already treated for other reason (positive urine sediment, urogenital gonorrhoea, anorectal chlamydia/gonorrhoea, epididymitis, partner with chlamydia). †Four patients were not treated at the first visit (one patient was treated at the second visit, three patients were loss to follow-up). ‡Eight patients were not treated presumptively for unknown reason. §Four patients did read their results on internet, two patients didn't. ¶Nine patients were not treated at the first visit (six patients were treated during the second visit and three patients were loss to follow-up). µFifteen patient were not treated presumptively at the first visit for unknown reason. ¥Five patients did read their results on internet, five patients didn't. ψFourteen patients did read their results on internet, 14 patients didn't. STI, sexually transmitted infection.

In the 2010–2011 period 30 460 consultations were performed in men, of which 20 349 (66.8%) were considered high-risk. After exclusion the final analyses set consisted of 18 852 (61.9%) high-risk consultations.

Although the percentage of consultations included in the analysis is much higher in the 2010–2011 period as compared with the 2008–2009 period (mainly because of exclusion of the consultations in the 2008–2009 period because of days with lack of laboratory staff), the general characteristics like sex, age, nationality and various risk factors in those included and excluded are comparable (see online supplementary table S2).

Diagnostic accuracy

The sensitivity of the Gram stain for CT was 83.8% (95% CI 81.2% to 86.1%) in the 2008–2009 period and 91.0% (95% CI 89.5% to 92.3%) in the 2010–2011 period (p<0.001).

The specificity of the Gram stain for CT was 74.1% (95% CI 73.0% to 75.2%) in the 2008–2009 period and 53.1% (95% CI 51.8% to 54.4%) in the 2010–2011 period (p<0.001).

The positive predictive value was low in both periods; 31.7% (95% CI 29.8% to 33.6%) in the 2008–2009 period and 35.6% (95% CI 34.1% to 37.1%) in the 2010–2011 period. In both periods there was a comparable high negative predictive value of 97.0% (95% CI 96.4% to 97.4%) and 95.4% (95% CI 94.6% to 96.1%), respectively. Detailed information about diagnostic accuracy can be found in the online supplementary table S3.

Confirmed and treated infections, prompt and delayed treatment, loss to follow-up and overtreatment

The proportion of confirmed CT infections treated at the clinic was comparably high in both periods: 98.2% vs 97.7% (p=0.26) (table 1). In the 2008–2009 period the percentage of promptly treated infections based on Gram stain POC management was significantly higher compared with the 2010–2011 period, respectively, 87.7% and 74.9% (p<0.001). Consequently the percentage of delayed treated infections was significantly lower in the 2008–2009 period (10.5%) compared with the 2010–2011 period (22.8%) (p<0.001).

Table 1

Confirmed* and treated urogenital Chlamydia trachomatis (CT) infections, promptly treated and delayed treated infections, loss to follow-up and overtreated, in high-risk male patients, STI Outpatient Clinic, Amsterdam, the Netherlands, 2008–2011

There was no statistically significant difference in the loss to follow-up percentage between both periods: 1.8% (95% CI 1.0% to 2.9%) vs 2.3% (95% CI 1.7% to 3.0%) (p=0.36), but a small statistically significant difference in the percentage overtreated, 68.0% (95% CI 66.0% to 69.8%) vs 64.1% (95% CI 62.6% to 65.5%) (p=0.001) (table 1).

Costs per consultation and per correctly managed consultation

The average cost per consultation was estimated to be €71.60 in the 2008–2009 period versus €67.20 in the 2010–2011 period, a savings of €4.40 or 6.2% per consultation. On the basis of 75.9% correctly managed CT infections in the first period and 83.2% correctly managed CT infections in the second period, the cost per correctly managed consultation was €94.31 in the first period compared with €80.82 in the second period, a savings of €13.49 or 14.3% per correctly managed consultation (table 2).

Table 2

Cost per consultation and per correctly managed consultation of urogenital Chlamydia trachomatis (CT), among high-risk male patients, STI Outpatient Clinic, Amsterdam, the Netherlands, 2008–2011

Urethritis caused by NG

In the 2008–2009 period 10.7% (95% CI 8.6% to 13.2%) (81/775) patients with a true positive Gram stain for CT had a co-infection with NG compared with 13.8% (95% CI 12.0% to 15.6%) (201/1461) of the patients with a true positive Gram for CT in the 2010–2011 period (p=0.043).

In the 2008–2009 period 14.1% (95% CI 12.4% to 15.9%) (229/1627) of the patients with a false positive Gram stain for CT turned out to have an NG infection compared with 20.6% (95% CI 19.1% to 22.2%) (546/2645) of the patients with a false positive Gram stain for CT in the 2010–2011 period (p<0.001).

Discussion

POC management is a highly valued public health principle to prevent sequelae, loss to follow-up consultations and ongoing transmission.3 Faced with budget cuts we had to economise our POC laboratory around 2010. Whereas before 2010 all high-risk patients were offered a Gram stain smear testing to diagnose urogenital CT promptly, only those with urogenital symptoms received POC management from 2010 onwards. This measure resulted in a cost reduction of 14.3% per correctly managed consultation of urogenital CT.

In this study we found a relatively high sensitivity of Gram stain analysis for urogenital CT in both periods (83.8% in the 2008–2009 period and 91.0% in the 2010–2011 period) compared with the sensitivity reported by other studies, ranging from 23% to 71%.10 ,11 The relative high sensitivities in our study could be due to a selective study population that consisted of high- risk patients of which the majority were men who have sex with men (respectively 56.5% and 64.8% of the study populations) compared with more general populations in other studies. Also different analysis techniques and thresholds used for a prompt NGU diagnosis (5 PMNLs/hpf in other studies vs 10 PMNLs/hpf in the present study) make a comparison on outcomes difficult.12 According to national guidelines the outpatient clinic in Amsterdam uses more than 10 PMNLs/hpf as a threshold for NGU whereas most international guidelines advise to use more than 5 PMNLs/hpf as a threshold. The higher threshold in our study could have resulted in a higher specificity but lower sensitivity.

The higher sensitivity of the Gram stain to detect CT in the 2010–2011 period (91.0%) compared with the 2008–2009 period (83.8%) could be explained by fewer false negative outcomes when urethral smear analysis was performed solely in symptomatic men. Probably symptomatic CT infections are more likely to cause Gram stain positive smear results as opposed to asymptomatic CT infections.

On the other hand, the specificity was relatively low in both periods (74.1% and 53.1%). This can be partly explained by infection with NG which may have caused symptomatic urethritis in many patients. The lower specificity in the 2010–2011 period (53.1%) compared with the 2008–2009 period (74.1%) is remarkable. A possible explanation is that CT infections are mostly asymptomatic in contrast with infections with urogenital NG. That also explains the higher percentage of NG infections found among the patients with a false positive Gram stain for CT in the 2010–2011 period when the Grams stained smear was only offered to symptomatic patients (20.6% vs 14.1%, p<0.001).

Moreover, other micro-organisms we did not test for routinely could have caused positive smear results and so could be responsible for the high amount of false positivity in both periods. NGU is reported to be caused by CT in 15–40% of cases, MG in 15–25%, T. vaginalis in 5–15% and less commonly, herpes simplex virus and adenovirus in 2–4%. Ureaplasma urealyticum has been associated in some but not all studies.7 ,13 ,14 However in 20–50% of NGU cases the aetiology remains unknown.15 Although not excluded, most MG infections are symptomatic; the organism is found in only 5–6% of asymptomatic men.16 ,17 Likewise, urogenital infections caused by herpes simplex virus and adenovirus are to cause symptoms more frequently than on average in bacterial related NGU.5 These non-detected pathogens can further explain the low specificity found in the study.

The cost per correctly managed consultation was 14.3% lower (a difference of €13.49 per consultation) in the 2010–2011 period compared with the 2008–2009 period. This difference can be partially explained by the higher sensitivity in the 2010–2011 period resulting in a higher percentage of correctly managed consultations. Moreover there was less overtreatment in the 2010–2011 period. Also the relative decrease in the amount of Gram stains analyses performed in the 2010–2011 period contributed in the reduction of the cost per correctly managed consultation. Yet the percentage of delayed treatment doubled. In a future mathematical modelling study we want to quantify ongoing transmission due to delayed treatment.

The strength of our study is the analysis of a large data set of comparable study populations over both periods. The outcomes of this study are in line with the outcomes of an earlier published study in which we compared the cost-effectiveness of the Gram stain in detecting urogenital NG in both periods.8 In that study we also reported a lower cost of urethral Gram stain analysis when offered solely to symptomatic high-risk patients as opposed to all patients irrespective of symptomatology.

A limitation of both our studies is that we focused only on CT and NG as possible causative agents of urethritis. We could not evaluate urethral infections by other micro-organisms known to cause urethritis. It is still debateable if micro-organisms like MG should be routinely screened for, or only in case of symptoms. More prospective studies are needed to give additional insight in the pathology and treatment of MG and other causative micro-organisms of NGU.16 ,17

To summarise, Gram stain smear analysis as a POC test for urogenital CT in symptomatic high-risk men only is more cost-effective compared with the analysis of all men, irrespective of symptoms. Screening symptomatic men only, saved 14.3% per correctly managed consultation, resulted in a higher sensitivity but a lower specificity, less overtreatment and a comparable loss to follow-up. Since there is no accurate and affordable pathogen-specific POC test for CT available, Gram stain smear analysis remains the preferred test for the prompt management of urogenital CT in high-risk men.

Key messages

  • Microscopic analysis of Gram-stained urethral smears is the most reliable point-of-care (POC) test to date for the presumptive management of urogenital chlamydia in men.

  • The Gram-stained smear POC system is accurate for the presumptive management of urogenital chlamydia in high-risk men.

  • When offered only to high-risk patients with urogenital symptoms, the cost per correctly managed consultation is reduced by 14.3% with less overtreatment and comparable loss to follow-up but a higher rate of delayed treatment.

Acknowledgments

The authors wish to thank Dewi Usmany, Arjen Speksnijder, Caspar Signet and Ineke Linde of the Public Health Laboratory for providing data of diagnostic costs, Sander Hoogenkamp of the financial department of the PHS of Amsterdam for providing data of human resources costs and Ente Rood of the Royal Tropical Institute for helping setting up the design of the study.

References

Supplementary materials

  • Abstract in Dutch

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Handling editor Jackie A Cassell

  • Contributors HJCdV, MSvR and MS were the authors responsible for the design of the study. MSvR and MB collected and interpreted the data of the diagnostic outcomes. SA did the statistical analysis of the diagnostic outcomes. KV collected and analysed the data of the costs. MB drafted the paper. HJCdV, MS and WRF supervised the overall study. All authors reviewed and approved the final article.

  • Funding This study was financed by the Research and Development fund of the Public Health Service of Amsterdam, no 2382.

  • Competing interests None.

  • Ethics approval All data used in this study was collected as part of routine management and anonymised before the analysis. Therefore ethical approval was not considered necessary.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.