Article Text
PDF
Abstract
Background WHO estimates that 131 million new cases of urogenital Chlamydia trachomatis (CT) infections occur globally every year. Most infections are asymptomatic. Untreated infection in women can lead to severe complications. Screening and treatment of at-risk populations is a priority for prevention and control.
Objectives To summarise systematic reviews of the performance characteristics of commercially available point-of-care tests (POCT) for screening and diagnosis of urogenital CT infection.
Methods Two separate systematic reviews covering the periods 2004–2013 and 2010–2015 were conducted on rapid CT POCTs. Studies were included if tests were evaluated against a valid reference standard.
Results In the first review, 635 articles were identified, of which 11 were included. Nine studies evaluated the performance of eight antigen detection rapid POCTs on 10 280 patients and two studies evaluated a near-patient nucleic acid amplification test (NAAT) on 3518 patients. Pooled sensitivity of antigen detection tests was 53%, 37% and 63% for cervical swabs, vaginal swabs and male urine, and specificity was 99%, 97% and 98%, respectively. The pooled sensitivity and specificity of the near-patient NAAT for all specimen types were >98% and 99.4%, respectively. The second review identified two additional studies on four antigen detection POCTs with sensitivities and specificities of 22.7%–37.7% and 99.4%–100%, respectively. A new two-step 15 min rapid POCT using fluorescent nanoparticles showed performance comparable to that of near-patient NAATs.
Conclusions The systematic reviews showed that antigen detection POCTs for CT, although easy to use, lacked sufficient sensitivity to be recommended as a screening test. A near-patient NAAT shows acceptable performance as a screening or diagnostic test but requires electricity, takes 90 min and is costly. More affordable POCTs are in development.
- Chlamydia trachomatis
- Systematic Reviews
- Chlamydia Infection
- Diagnosis
© World Health Organization [2017]. Licensee BMJ Publishing Group Limited. This is an open access article distributed under the terms of the Creative Commons Attribution IGO License (http://creativecommons.org/licenses/by/4.0/igo), which permits use, distribution, and reproduction for non-commercial purposes in any medium, provided the original work is properly cited. In any reproduction of this article there should not be any suggestion that WHO or this article endorse any specific organization or products. The use of the WHO logo is not permitted. This notice should be preserved along with the article’s original URL.
Statistics from Altmetric.com
Introduction
According to WHO estimates in 2012, urogenital chlamydial infection (etiological agent: Chlamydia trachomatis, CT) is the most common bacterial STI and approximately 131 million new cases occur globally every year.1 The highest number of estimated cases was in the WHO Western Pacific Region (61 million), followed by the WHO American Region (25 million), WHO South-East Asian Region (14 million), WHO African Region (12 million), WHO Eastern Mediterranean Region (10 million) and WHO European Region (9 million).1 Most CT infections are asymptomatic. Undetected, CT infection may result in severe complications such as pelvic inflammatory disease, ectopic pregnancy, infertility, and enhanced transmission and acquisition of HIV. Screening and treatment of at-risk populations is therefore a priority for prevention and control.
Highly accurate nucleic acid amplification tests (NAATs) are available in the developed world but since they require robust laboratory infrastructure and trained personnel, these NAATs are neither affordable nor accessible to patients in the developing world, where access to laboratories is limited and the STI burden remains high despite syndromic management. In the absence of screening programmes for asymptomatic infections and improved access to accurate laboratory diagnosis, the number of detected and reported cases is substantially lower than the number of real cases.1
NAATs are costly, technically demanding and laboratory based, requiring patients to come to the clinic to be screened and return for the result, or to mail in self-collected samples (vaginal swabs for women and urine specimens for men) and waiting for results to be sent by phone or internet.2 Point-of-care tests (POCTs) that are affordable, sensitive, specific, user-friendly, rapid and robust, equipment-free and delivered to end-users (ASSURED) would have considerable implications for STI control in less resourced settings and in well-resourced settings. Using accurate and rapid CT POCTs, patients can be promptly diagnosed and appropriately treated at presentation, preventing complications, and ongoing transmission, and offering opportunities for counselling and contact notification. Simple rapid POCTs for the diagnosis and screening of CT infections are commercially available, but there are limited data on their performance.4 5 The WHO STI point-of-care (POC) diagnostic initiative, coordinated by the Department of Reproductive Health and Research at WHO, including the UNDP/UNFPA/Unicef/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction, aims to facilitate and support access to quality-assured STI POCTs within national STI programmes through providing advice to WHO Member States and other relevant public health institutions on the performance and operational characteristics of new commercially available STI diagnostic tests that can be used at the POC across all countries.
This paper describes the findings from two systematic reviews of the performance and operational characteristics of commercially available POCTs for the diagnosis of urogenital and extragenital CT infection.
Methods
The first systematic review
The first systematic review was conducted in 2013–2014 by authors from the London School of Hygiene and Tropical Medicine, UK, in collaboration with McGill University, Canada, in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, which aim to advance the reporting of systematic reviews and meta-analyses by improving the transparency and completeness of information.6 Only rapid and simple POCTs that met the ASSURED criteria3 were included.
Search terms and strategy
MEDLINE and GLOBAL HEALTH databases were searched from 1 January 2004 to 31 December 2013. Search terms were as follows: (chlamydia AND Chlamydia trachomatis AND C. trachomatis) AND (point-of-care OR POC test OR POCT OR rapid test* OR rapid assay* OR diagnos* OR near patient OR screening OR urine dipstick) AND (evaluation OR performance characteristics OR validation OR performance OR sensitivity OR specificity).
Systematic reviews were identified first. If no systematic reviews were published on the topic, case–control, cross-sectional or cohort studies were identified.
Inclusion criteria
Eligibility criteria were defined using PICOS (Population, Interventions, Comparisons, Outcomes, Study Design) criteria as shown in table 1. Studies evaluating the accuracy and/or precision of any CT POCT commercially available at the time of the review were considered for inclusion.
Inclusion criteria (PICOS criteria)
Exclusion criteria
Studies were excluded if the test was not a rapid test for CT, if diagnostic accuracies were not compared with an appropriate reference standard, and if the studies did not report data to allow for calculation of diagnostic accuracy. Studies reporting POCT analytical performance and POCTs in development were excluded.
Data extraction
Two reviewers independently extracted data from the included studies meeting the inclusion criteria. Items for data extraction for each POCT included: study (journal, author, year); location (country, healthcare level where study was performed); field/laboratory location where tested (peripheral/reference); test method; reference/gold standard; specimen type; sample size; population; age range; genital symptoms; sensitivity; specificity; positive predictive value; negative predictive value; receiver operating characteristics: number of steps; major equipment required for test; time to result; and CT prevalence. Disagreements were resolved by consensus or external advisors.
Data synthesis and statistical analysis
Descriptive analyses were performed using STATA V.14 (STATA, College Station, TX, USA). For each study, the sensitivity and specificity along with 95% CIs, compared with the reference standard, were calculated. Forest plots were generated to display sensitivity and specificity estimates.
Meta-analysis
The heterogeneity in the forest plots was assessed by visually examining the CIs of individual studies, and in summary Hierarchical Summary Receiver Operating Characteristic (HSROC) plots (by examining the width of the prediction region, with a wider prediction region suggesting more heterogeneity). Heterogeneity in terms of the sample types by index test for CT and also prespecified subgroups such as different specimen types were examined. A bivariate random-effects model was used and meta-analyses were carried out in STATA. For this review of CT POCTs, a meta-analysis for a predefined sample type was only carried out if at least four studies were available.
Metaregression
Additional heterogeneity was anticipated with respect to samples, patient population groups and prevalence within the prespecified subgroups. Therefore, a bivariate metaregression model in STATA was selected for use under the assumption that the pooled sensitivity and specificity were different in each subgroup, but not the between-study variance-covariance matrix. The metaregression assessment was performed only on studies with the same reference standard. It was also presumed that the effect of the covariate would not differ between the different reference standards.
Sensitivity analysis
Sensitivity analyses could not be performed due to lack of additional covariate data.
Formal assessment of publication bias using methods such as funnel plots or regression tests was not performed because such techniques are not considered to be valid for diagnostic accuracy reviews.
Data quality
Quality of the studies included was assessed according to the STARD Criteria and Checklist.7
The second systematic review
The second systematic review was conducted in 2015 by authors from the University of California, Los Angeles, USA, for papers published from January 2010 to August 2015.4
Search terms and strategy
The search was conducted according to PRISMA guidelines6 in PubMed using search terms as follows: sexually transmitted diseases or sexually transmitted infection∗ and chlamydia* and (point-of-care and (rapid test or diagnostic or screening or test)).
Abstracts of all search results and the full text of all potentially eligible articles were reviewed. This search yielded 61 articles whose abstracts were evaluated to determine whether they fit the inclusion criteria.
Inclusion criteria
The inclusion criteria were (1) publications including Chlamydia as STIs; (2) publications that date from January 2010 through August 2015; (3) publications relating to diagnostics; (4) publications published in English; and (5) original research.
Exclusion criteria
The exclusion criteria were (1) publications not covering CT; (2) publications including those infections but not in the sexually transmissible form; (3) publications published before 2010; and (4) publications not evaluating POC diagnostics using a valid reference standard assay.
Articles were sorted into the following subject categories and stratified based on: (1) performance evaluations, (2) cost analyses, (3) acceptability and feasibility trials and (4) proof of concept studies.
Results
The first systematic review identified a total of 635 papers, of which 557 were excluded based on their title and abstract. An additional 68 articles were excluded after a review of the full text, leaving 10 articles for data extraction. An additional article was identified by a study adviser (figure 1).
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram for study selection in systematic review 1.
The second review by Herbst de Cortina et al identified 33 articles, of which 8 were on the performance of tests to detect genital CT infection.4 Of the eight studies, two evaluated the performance of Gram-stained urethral smears and one evaluated an automated urine flow cytometry compared with NAATs. Since microscopy and flow cytometry are not POCTs, they are not included in this review. Of the five studies included in this review, three were already identified in the first review and two studies published in 2015 and 2016 were added to the data extraction for this review.
Operational characteristics of POCTs included in the review
The two systematic reviews described the evaluation of nine brands of antigen detection POCTs and one NAAT that can be labelled as near patient as it is an automated sample-in answer-out assay that requires 2 min of hands-on time and is designed for both laboratories and clinic settings (table 2). The antigen detection tests included six immunochromatographic tests in a lateral format, one test using optical detection, one using enzyme detection and one using fluorescent nanoparticles.
Chlamydia trachomatis POCTs identified in the two systematic reviews
Data extracted from the studies
The two systematic reviews identified 11 studies that evaluated the performance of 9 brands of CT antigen detection POCTs on 11 889 study participants from 10 countries. Two studies described evaluation of the performance of a near-POC NAAT on 3568 study participants from two countries. Reference standard assays used included PCR assays (Roche Molecular Systems, USA), ligase chain reaction assay (Abbott Diagnostics, USA), strand displacement amplification assay, ProbeTec ET assay (Becton Dickinson, USA) and transcription-mediated amplification assay (Aptima, GenProbe, now Hologic, USA).
Table 3 displays data extracted from the studies evaluating the performance of CT antigen detection POCTs and near-POC NAAT in both systematic reviews.
Data from studies evaluating the performance of Chlamydia trachomatis antigen detection POCT
Antigen detection rapid POCTs exhibited high specificity across all specimen types (range 97%–100%), the pooled sensitivity was 37% for vaginal swabs (95% CI 22.9% to 52.9%; range 17.1%–74.2%), 53% for endocervical swabs (95% CI 34.7% to 70.8%; range 22.7%–87%) and 63% for urine (95% CI 43.2% to 78.5%; range 49.7%–88.2%) (table 4). The aQcare Chlamydia TRF kit, which is a fluorescent nanoparticle-based lateral flow assay, was the best performing antigen detection POCT, with sensitivities and specificities comparable to that of near-patient NAATs.8
Pooled performance of the POC antigen detection and near-patient NAATs for different specimen types
Although CT antigen detection rapid POCTs exhibited high specificity across all specimen types (range 97%–100%), the pooled sensitivity was 37% for vaginal swabs (95% CI 22.9% to 52.9%; range 17.1%–74.2%), 53% for endocervical swabs (95% CI 34.7% to 70.8%; range 22.7%–87%) and 63% for urine (95% CI 43.2% to 78.5%; range 49.7%–88.2%) (table 4). The aQcare Chlamydia TRF kit, which is a fluorescent nanoparticle-based lateral flow assay, was the best performing antigen detection POCT, with sensitivities and specificities comparable to that of near-patient NAATs.8 The best performing test overall was the Xpert CT/NG, a Food and Drug Administration-approved real-time PCR assay.
The sensitivity of the Cepheid GeneXpert assay showed no significant difference between self-collected vaginal swabs (98.7%), cervical swabs (97.4%), female urine (97.6%) and male urine specimens (97.5%) with specificities ranging from 99.4% to 99.9%. The sensitivity and specificity of this assay for rectal swabs are 86.0% and 99.2% respectively. In particular, for POCTs we are interested in specimens that are easy to collect. The overall sensitivity and specificity of antigen detection POCTs for vaginal swabs from the two systematic reviews are graphically represented in figure 2A,B and those for urine specimens in figure 3A,B.
(A) Meta-analysis of Chlamydia trachomatis antigen detection POC test performance using vaginal swabs in 10 studies. (B)HSROC for C. trachomatis antigen detection POC test performance using vaginal swabs in 10 studies. AUC, area under the curve; POC, point of care; SENS, sensitivity; SPEC, specificity.
(A) Meta-analysis of Chlamydia trachomatis antigen detection POC test performance using urine samples in five studies among men. (B) HSROC for C. trachomatis antigen detection POC test using urine samples in 5five studies among men. AUC, area under the curve; POC, point of care; SENS, sensitivity; SPEC, specificity.
According to the STARD criteria7 relevant to POCTs, the quality of the papers included was acceptable, with noted omissions in the categories describing whether operators were trained to perform the tests, mechanism for blinding between the results of the index and reference tests and documentation of the frequency of indeterminate results (table 5).
Assessment of the quality of the studies included in the first systematic review
Discussion
The two systematic reviews described in this paper showed that while the specificity of most CT antigen detection POCTs was >97%, their sensitivities were suboptimal, especially when used with vaginal swabs. Nevertheless, they are being used, especially in countries lacking the capacity for stringent regulatory review and approval. Women who had false negative test results due to low sensitivity would not be treated and could subsequently develop long-term reproductive sequelae such as pelvic inflammatory disease, ectopic pregnancy and tubal infertility. Using a model, Gift et al 9 showed that a POCT with a sensitivity of 65% can lead to more patients being treated for genital chlamydial infections than a more accurate NAAT because only 50% of patients who were screened for chlamydia returned for their test results within 3 weeks. Moreover, by the time they returned, 3% of women had developed pelvic inflammatory disease. Hence, POCTs offer an important opportunity to treat any infected patient and initiate partner notification in the same clinic visit.
Although the sensitivity of POCTs is higher for cervical swabs than vaginal swabs, POCTs are best used with specimens that are easy to collect such as urine for men or vaginal swabs, which are self-collected or collected by a healthcare provider. Hurly et al 10 compared the performance of the Chlamydia Rapid Test (Diagnostics for the Real World, Cambridge, UK) and ACON CT test for men and women, and found lower sensitivities for urine from men (41.4%–43.8%) compared that for vaginal swabs (66.7%–74.2%) for both POCTs. The authors attributed this difference to the CT load in these specimen types. Wisniewski et al 11 showed that for urine samples, there is significantly more chlamydia in the first 4–5 mL of the void than subsequent aliquots.
A single study on a POCT based on fluorescent nanoparticles requiring only 15 min turnaround time showed promising performance characteristics that are comparable to near-patient NAATs. More evaluations of this POCT would be necessary to determine if these performance characteristics are reproducible.
Gaydos et al 12 evaluated both symptomatic and asymptomatic women at reproductive health clinics, and showed excellent accuracy for the Cepheid Xpert CT/NG test. In contrast to the antigen detection POCTs, the Xpert CT/NG did not show any significant difference in performance between specimen types. Goldenberg et al 13 showed that the Xpert test has adequate performance with rectal swabs. Although more accurate, near-patient NAATs are more expensive than antigen detection POCTs. However, the Cepheid platform has two features which are advantages. First, it allows random access in that detection of different pathogens on its test menu can be initiated any time as each cell is an independent nucleic acid amplification and detection reaction. Second, the Cepheid platform, like most other near-patient molecular platforms, is polyvalent in that the equipment can be used with cartridges for over 20 pathogens. This makes the testing more cost-effective.
In selecting a test for screening of asymptomatic populations or diagnosis of symptomatic patients, health providers and control programmes need to consider the trade-off between accuracy and affordability for their epidemiological setting and what patients are willing to pay and whether they are willing to wait for 90 min versus 15–30 min. Gift et al described the rapid test paradox in a model of chlamydial screening in which a rapid test of 65% sensitivity led to more infected patients being treated than using a NAAT with higher accuracy because of low patient return rates for results.9 Adams et al 14 examined the need to modify patient pathways to take full advantage of near-patient NAATs, which can reduce cost and clinician time and may lead to more efficient and appropriate care for patients compared with standard of care which is off-site laboratory testing and having to return for test results and treatment. In a simulation of 1.2 million patients seeking STI care across the UK, it was estimated that POC testing can be cost-saving and reduce overtreatment of patients who are diagnosed using a syndromic approach.15 POC testing in this scenario can prevent 189 cases of pelvic inflammatory disease and 17 561 cases of onward transmission.
The quality of the studies included in these systematic reviews is generally satisfactory (table 5). The main shortcomings include omission in describing training in specimen storage and processing, how indeterminate results are managed, and blinding between index and reference test results.
Our review had several limitations. First, the first systematic review only used MEDLINE and GLOBAL HEALTH databases and the second systematic review only used PubMed to find relevant articles. Second, there are existing commercial POCTs which have been approved for use but were not evaluated in any publications between January 2000 and August 2015, preventing their inclusion in this paper. A number of studies that are proof of concept studies or analytical performance studies from promising new technology platforms have not been included. They were described in the second systematic review.4
Emerging new technologies, including isothermal amplification technologies, promise major advancements in the field of rapid POCTs for STIs in the near future.16 A number of novel POC molecular platforms have been developed for HIV early infant diagnosis and viral load and are now being adapted for the diagnosis of STIs.17 18 As more of these molecular POCTs become available, the cost to produce, distribute and use these tests will also decrease, thus increasing accessibility and affordability in less resourced settings, where STI prevalence is highest and the burden of adverse outcomes is greatest.19
Conclusions
The systematic reviews showed that antigen detection POCTs for CT, although easy to use, lacked sufficient sensitivity to be recommended as screening tests. Currently available near-POC NAATs have acceptable performance characteristics to be used as screening and diagnostic tests but need a source of electricity, have a relatively long turnaround time of approximately 90 min and are too costly for widespread use, especially in low resource settings. Other novel POC molecular assays are under development and may soon be available to improve chlamydial screening and diagnosis in less resourced settings as well as more well-resourced settings. However, before the introduction of these novel POCTs it is crucial to evaluate their performance and operational characteristics and their acceptability to patients and healthcare facilities. This is a high priority for the WHO STI POC diagnostic initiative in the coming years as countries strive to reduce the burden of STIs.
Key messages
Diagnostic tests are needed for detecting genital chlamydial infections but there are limited data on their performance and operational characteristics for use in the developing world. Systematic reviews show that antigen detection tests that can be used at the point of care (POC) have good specificity but suboptimal sensitivity.
Near-patient molecular assays are highly accurate but require electricity, 90 min turnaround time and are too costly for use in low resource settings.
Promising novel POC technologies that are accurate have shorter turnaround time and are less costly in development.
Acknowledgments
We are grateful to Rachel Chater for assistance with the preparation of the manuscript.
References
Footnotes
Contributors RWP and IT designed the review. HK and CEMC conducted the searches and data extraction. NPP performed the analyses. All authors contributed to the finalisation of the manuscript.
Funding This work was funded by the UNDP-UNFPA-Unicef-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), a cosponsored programme executed by WHO (Contract No SPHQ13-APW-4095).
Disclaimer The author(s) is(are) staff member(s) of the World Health Organization. The author(s) alone is(are) responsible for the views expressed inthis publication and they do not necessarily represent the views,decisions or policies of the World Health Organization.
Competing interests None declared.
Provenance and peer review Commissioned; externally peer reviewed.