Objectives A new molecular test for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) (GeneXpert CT/NG) has been demonstrated to be as accurate as conventional nucleic acid amplification tests (NAAT), but performance has not been evaluated in routine primary care, performed at the point of care by clinicians. We aimed to examine its diagnostic performance when used by clinicians in remote community health services in Australia with high prevalences of CT and NG infection. The trial was registered with the Australian and New Zealand Clinical Trials Registry (#12613000808741)
Methods At 12 health services, training was provided to 99 clinicians in the use of the GeneXpert CT/NG assay who tested specimens from all patients undergoing STI screening. Specimens were also sent in parallel for conventional laboratory-based NAATs and the concordance of results was evaluated.
Results Clinicians conducted 2486 tests: CT concordance was 99.4% (95% CI 99.1 to 99.7) with a positive concordance of 98.6% (95% CI 95.9 to 99.7) and negative concordance of 99.5% (95% CI 99.1 to 99.8); NG concordance was 99.9% (95% CI 99.7 to 100.0) with a positive concordance of 100.0% (95% CI 97.5 to 100.0) and negative concordance of 99.9% (95% CI 99.7 to 100.0).
Conclusions In this first study reporting routine point-of-care use of GeneXpert CT/NG by primary care clinicians, we found excellent concordance with conventional NAATs. The use of the GeneXpert CT/NG at the point of care could potentially transform management and control of these infections in many endemic settings, including low/middle-income countries.
- Chlamydia trachomatis
- Neisseria gonorrhoea
- primary care
- molecular techniques
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Accurate diagnosis and timely treatment are critical to the control of infectious diseases including STIs. In many settings where disease burden is greatest, diagnostic technology is unavailable so clinicians rely on syndromic management guidelines to diagnose infection and make treatment decisions.1 This approach has substantial limitations due to the high proportion of STIs that are asymptomatic and the non-specificity of syndromes potentially leading to both inadequate treatment and overtreatment, thereby increasing the risk of antimicrobial resistance.2 Even in well-resourced settings the reliance on laboratory-based diagnosis can generate delays in time to treatment and even non-treatment in the context of highly mobile populations who cannot be contacted when a positive result is obtained days (or, sometimes weeks) after the specimen collection. There has therefore been a long-recognised need for an easy-to-use, accurate point-of-care (POC) test for treatable STIs.3
Infections with chlamydia (Chlamydia trachomatis; CT) and gonorrhoea (Neisseria gonorrhoeae; NG) represent a significant global public health burden, with an estimated 256 million cases among adults aged 15–49 years in 2012.4 Without treatment, their complications include pelvic inflammatory disease (PID), infertility and ectopic pregnancy,5 as well as adverse pregnancy outcomes,5 they contribute to a range of psychosocial consequences, and are associated with an increased risk of HIV transmission.6
Until recently, the only commercially available POC tests for CT and NG were based on lateral flow platforms which are inexpensive but require numerous specimen preparation steps7 and have very poor sensitivity.7 8 In 2013, the GeneXpert CT/NG assay (Cepheid, Sunnyvale, CA) became the first nucleic acid-based test for CT/NG available for POC use. It has been shown to have high analytical9 and clinical10 accuracy compared with established commercial nucleic acid amplification tests (NAAT) in highly controlled laboratory environments with dedicated staff. However, the ultimate demonstration of a POC test’s performance is not in this ideal setting, but rather in the hands of clinicians in front-line settings where patient care takes place.11
The world’s first demonstration of the GeneXpert’s potential to revolutionise infectious disease diagnostics was in the detection of tuberculosis with subsequent demonstration of its POC potential when used by clinicians in dedicated clinics.12 For CT/NG, the assay has been successfully adopted for POC use at a specialist, urban genitourinary clinic in the UK, but all testing is conducted by a designated technician.13 A pilot study in remote primary health services in Australia suggested the GeneXpert CT/NG assay was feasible and highly accurate for POC use by research staff.14 These findings are encouraging, but do not exclude the possibility that processing errors may arise if clinicians are required to integrate testing into their busy and demanding routine workload. The TTANGO (Test, Treat and Go) trial,15 taking place in 12 remote Australian Aboriginal communities, is the first investigation of the GeneXpert CT/NG test’s integration into routine primary health practice performed by clinicians. From the TTANGO trial we have previously reported on the high levels of acceptability and policy implications of this technology in these settings16 17 and a novel quality control (QC) method implemented.18 We now report on its operational performance compared with conventional laboratory testing when used by clinicians in these remote communities.
Remote Australian Aboriginal communities are typically geographically isolated across central and northern regions of the country. All have primary care services, mostly staffed by nurses and Aboriginal health practitioners with visiting doctors, and regular but often non-daily transport of diagnostic specimens to reference laboratories that may be thousands of kilometres away. Remote communities experience very high rates of CT and NG infections (807 and 532 per 100 000 respectively) compared with major urban settings (327 and 101 per 100 000 respectively).19 Prevalence estimates for CT and NG are high (11% and 9% respectively) in these remote communities, reaching 26% for CT among women and 21% for NG for men aged 16–19 years.20 With delays in return of laboratory results and a highly mobile population, there can be substantial delays to treatment (average of 3 weeks).21 Rates of hospital diagnoses of PID in Aboriginal women are reported to be double that of their non-Indigenous counterparts.22
TTANGO trial and study design
The TTANGO trial15 aimed to evaluate the public health and health service impact of POC testing for STIs. ‘POC test’ and ‘POC testing’ here refer to GeneXpert and its use by clinicians at the POC, under the definition that includes diagnostics providing accurate results and facilitating treatment within the same clinical visit,23 which accurately describes the intent here.
In this trial context, we conducted a prospective study to assess the operational performance of the GeneXpert CT/NG test in the hands of primary care clinicians (non-laboratory staff providing direct patient clinical care; including doctors, but mostly nurses and Aboriginal health workers/practitioners in this setting) compared with routine conventional laboratory-based tests. Evaluations of the GeneXpert CT/NG test’s analytical9 and potential field performance,14 which were conducted prior to the TTANGO trial, informed the final choice of POC test for this trial.
Under the cluster randomised crossover trial design of TTANGO,15 12 remote health services were enrolled and randomly assigned to either standard care (routine laboratory testing) or standard care plus POC testing for 1 year before crossing to the other modality for another year. Service eligibility criteria are described elsewhere.15 The recommended target group for testing followed clinical guidelines (annual screening of 15–30 year-olds),24 however all patients presenting to the health service and offered STI testing at the clinician’s discretion were eligible. During the POC testing year, routinely collected specimens (self-collected urine or self-collected lower vaginal swabs depending on individual health service practice) from this convenience series were tested using the GeneXpert by clinicians on the day of consultation. All specimens continued to be sent for conventional laboratory testing throughout the trial and results were not available to health service staff at the time of POC testing.
For patients who had a POC test, a positive result informed clinical management following local guidelines, initiated ideally during the same consultation. For patients with a negative POC result but positive laboratory NAAT, or not having a POC test or a valid result (including insufficient specimen for repeat test or repeated errors), the laboratory-based results informed further management when available. Current local guidelines for these remote settings recommend syndromic and presumptive treatment for those presenting with symptoms of an STI or recent risk behaviour(s). In these settings, first-line treatment for uncomplicated CT is azithromycin 1 g and NG is amoxicillin 3 g plus probenecid 1 g.24
POC test training
Study coordinators delivered standardised training to 99 nominated clinicians, of whom 61 were nurses, 34 Aboriginal health workers or practitioners (a professional category providing healthcare in services with primarily Aboriginal patients) and 4 doctors. Training was conducted in person, on-site and included a competency assessment. Study coordinators monitored POC testing remotely (test numbers and results, including errors) and made six monthly site visits to discuss STI testing and encourage best practice STI management.
GeneXpert CT/NG test
The GeneXpert CT/NG test was approved for diagnostic testing by the Australian Therapeutic Goods Administration (TGA) in March 2013. Each health service was provided a four-module GeneXpert platform and cartridges during the POC testing period to be used according to manufacturer’s instructions for CT/NG testing (Xpert Ct/NG assay. 201-0234 Revision B. January 2013, Cepheid, Sunnyvale, CA). The test takes approximately 90 min, and objectively yields a positive/negative result separately for CT and NG, or an error result. Specimens generating an error result were retested if sufficient specimen remained.
All services participated in an internal QC and external quality assessment (EQA) testing programme, consistent with quality processes undertaken in accredited laboratories, supported by National Reference Laboratory Australia. Quality testing samples were externally provided, with QC testing conducted monthly (one sample with a known bacterial load) and EQA testing conducted every 6 months (panel of four samples).
All specimens were collected and sent using standard protocols by the health services with requests for conventional STI testing to their respective diagnostic laboratories. When POC testing was performed, an aliquot was collected from the original specimen prior to being sent to the laboratory, or an additional swab was collected at the same time. As the study aimed to reflect real-world implementation we compared the POC test result with the NAAT result generated routinely by the six laboratories providing pathology services to the 12 health services. The laboratory-based NAATs used for CT and NG detection were: Aptima Combo 2 (Gen-Probe, San Diego, CA, USA), Cobas4800 (Roche Diagnostics, Pleasanton, CA, USA), or in-house CT and NG assays.25 Laboratories were blinded to POC test results. Conventional laboratory test results as routinely reported were cross-checked against the POC test results by clinicians. For specimens with discordant results, the laboratory was notified to store the specimen for additional investigation.
Additional laboratory testing of discordant specimens
Stored samples with discordant results were transported to the study reference laboratory in Melbourne (Royal Women’s Hospital) for investigation. Urine or samples received in Cobas transport media were tested using Cobas 4800 (for both CT and NG), in-house OmpA CT assay,26 PorA and Opa NG assays.27 28 Any swab or Aptima sample received was extracted on MagNA pure 96 (Roche Diagnostic) and tested with the in-house OmpA CT assay, PorA and Opa NG assays. A final reference laboratory ‘positive’ result was assigned to any sample in which CT or NG was detected by either the repeat Cobas 4800 assay or the in-house CT or NG assays.
Data collection and statistical analyses
POC test results were compared with corresponding initial laboratory NAAT results as reported to the health service. Positive, negative and overall per cent concordance was determined separately for CT and NG, along with 95% CI, by standard methods.29 As both the GeneXpert test and laboratory NAATs use similar molecular amplification techniques, are highly accurate and TGA approved, the laboratory NAATs were considered to be a ‘non-reference’ standard for the purposes of these analyses, hence concordance (or agreement) was measured.29 Results from additional laboratory reference testing of discordant specimens were compared with corresponding POC test results and the initial laboratory test result.
POC test result cycle thresholds were analysed using rank-sum and t-tests to assess the difference in the median and mean cycle threshold values between specimens that had yielded concordant and discordant samples, respectively (Stata: Release 12. College Station, TX).
GeneXpert test error results were investigated by study coordinators by telephone interview with operators. Determination of likely causes and remedial actions were recorded.
Sample size calculation
Sample size calculations for the overall TTANGO trial are described elsewhere.15 For this operational performance assessment, assuming a positivity of 10% for each of CT and NG20 and POC test concordance of 95% or above, we aimed to include a minimum of 1000 POC tests (100 POC test positives). Estimated CIs around positive concordance were ±5% and negative concordance ±2%.
TTANGO was approved by the West Australian Aboriginal Health Ethics Research Committee (HREC#396); Kimberley Aboriginal Health Planning Forum (HREC#2012-003); West Australian Community Health Board (HREC#2012/16); Townsville (HREC/12/QTHS/133)/Cairns and Hinterland Hospital and Health Service (HREC#12/QCH/89–810); and Aboriginal Health Council South Australia (HREC#04-13-500). Patient consent was not required. Deidentified data were transferred for study analyses.
A GeneXpert CT/NG test was performed on 2509 specimens from 1 July 2013 to 30 April 2015 at the 12 participating sites, with 2426 (96.7%) producing a valid result at first test, as recorded by the device. Of the 83 specimens for which the result was recorded as an error at first test, 60 produced a valid result on repeat test (two CT detected; one NG detected), 8 produced a second error and 15 had no repeat test conducted (figure 1). The 2486 results classed as valid (2426 at first test and 60 on retest) formed the basis of the analyses reported here.
Among the 2486 samples included in the concordance analysis 724 were tested using Aptima Combo 2, 878 with Cobas4800 and 883 with in-house CT and NG assays. The laboratory NAATs gave a positive CT result in 212 (8.5%) and a positive NG result in 145 (5.8%). Overall CT concordance was 99.4% with positive concordance of 98.6% and negative concordance of 99.5%. Overall NG concordance was 99.9% with positive concordance of 100.0% and negative concordance of 99.9% (table 1).
A total of 16 (0.6%) discordant results were identified. Discordant results occurred in both swab (n=6) and urine (n=10) specimens and all three laboratory NAATs (Aptima Combo 2, Cobas4800 and in-house assays). The majority of the discordant results identified were for CT tests (n=14), with 11 positive on GeneXpert and negative on laboratory NAAT and three negative on GeneXpert but positive on laboratory NAAT.
The mean cycle threshold for the 11 GeneXpert CT positive discordant results was higher (35.3) than the 209 concordant (29.1) results (p<0.001). There were two discordant NG results, both positive on GeneXpert but negative on the laboratory NAAT. The mean cycle thresholds were not compared due to small numbers.
Ten of 16 discordant samples had sufficient remaining volume for additional reference testing. Reference laboratory testing was in final agreement with GeneXpert in six samples and in disagreement for four samples (table 2).
There was a median of 10 errors (IQR 5–12) across the 12 services. Almost one quarter of all errors (21/83) occurred at one site in a 1-month period due to faulty cartridges. The remaining errors appeared to be primarily related to the test operator skill including inadequate volume of specimen (<1 mL) in the GeneXpert cartridge.
In what we believe to be the first report of the GeneXpert CT/NG assay’s accuracy for routine POC diagnosis as performed by primary care clinicians, we found the GeneXpert CT/NG test to have extremely high concordance with results from conventional NAAT testing. The study took place in remote communities in Australia, settings in which clinicians are primarily nurses and Aboriginal health practitioners, dealing with numerous competing clinical priorities. The high level of accuracy demonstrates for the first time the robustness of the test in this extreme setting, with results that match its performance in the hands of technicians working in conventional diagnostic pathology laboratories under highly controlled conditions.9 14 30
The GeneXpert device has enabled molecular testing to take place in settings that lack conventional laboratory facilities,12 however in many low/middle-income countries this new technology remains confined to hospital laboratories, generally for administrative and logistical issues. There is also a residual preference by many clinicians for treating without testing, as a matter of long-standing necessity.12
Very few (0.6%) discordant results were identified in our study. The higher GeneXpert cycle thresholds observed in specimens that yielded discordant results suggests the role of lower organism loads, potentially at the limit of test detection. Further retesting of 10 discordant specimens did not produce a consistent finding with regard to agreement between the primary laboratory NAAT and the additional reference NAAT, further supporting this premise.
Around 3% of the GeneXpert tests led to an error result when first performed, but the vast majority (60 out of 68) returned a valid result when repeated, consistent with a previous report.30 Although detailed analysis of specific error types was not possible, they were generally related to operator inexperience (eg, inadequate sample volume with poor pipetting technique) although faulty cartridges were associated with a cluster of errors at one service. The outcome of these investigations highlights the benefit of good communication and logistics support to minimise disruptions to testing and maintaining staff confidence in the POC test.
Although POC testing was integrated into routine clinical practice at services, our results may not be representative of all settings or if widely scaled up. Our study provided standardised training to clinicians on-site, supported with regular telephone contact and site visits by study coordinators. This support, as well as the engagement of Aboriginal health workers/practitioners as POC test operators, likely assisted in maintaining a relatively low error rate despite the considerable turnover of other clinicians. Before wider scale-up, sustainable training and supervision processes will need to be determined, such as certified train-the-trainer models or web-based formats.
Remote, real-time monitoring of results allowed for the rapid identification of problems and remedial actions. Trial quality assurance (QA) was supported through partnerships with conventional laboratories and provided a complementary objective quality snapshot of GeneXpert use. Quality management is an essential component of any further expansion of STI POC testing both in pilot programmes and beyond.3
This study was conducted in a population with a very high prevalence of CT and NG in very remote settings. The use of highly sensitive testing in populations with a very low prevalence of NG can lead to greater numbers of false positive results.31 An advantage of the GeneXpert in NG detection is the inclusion of two targets on different genes, avoiding the need for a second, confirmatory test for NG as currently is recommended for conventional laboratory NAATs, adding time and cost. Another potential advantage of the GeneXpert is in the context of surveillance for NG antimicrobial resistance, an issue of mounting global public health concern.2 Widespread use of POC tests may contribute to the spread of resistance,32 however rapid detection of NG cases at the POC may allow for better targeting and prioritised collection of specimens for transport to laboratories capable of resistance testing. A POC test that provides both a diagnostic result with NG antibiotic resistance information would be ideal.
Demonstrating the performance of a POC test conducted by non-technical operators under field conditions is critical to inform decisions regarding broader programmatic adoption of these tests. The GeneXpert uses molecular techniques previously restricted to laboratory environments and presents new operational challenges to the clinic and non-laboratory trained operator. Yet our results are proof of concept that the GeneXpert CT/NG assay integrated into remote primary health services and performed at the POC by trained clinicians has excellent diagnostic performance, comparable with laboratory-based NAATs. While 90 min may be longer than desirable for a POC test result,3 this time frame remains a relatively extreme improvement compared with current circumstances and does not appear to be a major barrier to testing or treatment in these remote settings.16 Coupled with the logistic, clinical and public health advantages of offering appropriate management to patients and their partners at the time of consultation, this technology has the potential to transform management and control of STIs in many endemic settings. Expanded usage will nevertheless require evidence of cost-effectiveness and identification of sustainable funding. Furthermore, to maximise its public health impact, POC testing should ideally be combined with other strategies including greater use of opportunistic testing, partner notification, retesting and health promotion in communities with high rates of infection.
GeneXpert technology is already available for POC diagnosis, predominantly for tuberculosis in low/middle-income countries. The CT/NG assay, in combination with the recent advent and increasing use of the HIV viral load and human papilloma virus assays for cervical screening, provides a comprehensive suite of assays that can be performed on the one diagnostic device. Similar to remote Aboriginal communities in Australia, many low/middle-income countries have high burdens of STIs, limited access to timely diagnosis and highly mobile populations, so may benefit from the use of POC testing for STIs, potentially leveraging existing diagnostic GeneXpert devices and testing infrastructure.
Point-of-care (POC) testing for STIs has the potential to improve treatment completion rates by placing diagnostic capability in the hands of front-line clinicians.
A new compact molecular test for chlamydia and gonorrhoea is now available and potentially suitable for POC use.
In high prevalence settings we demonstrate for the first time high accuracy when routinely performed by clinicians in remote primary health services at POC.
Molecular testing at the POC could transform management and control of these infections in many endemic settings, including low/middle-income countries.
TTANGO is a collaboration between researchers named in the authorship list, the Queensland Aboriginal and Islander Health Council, Queensland Health, Aboriginal Health Council of Western Australia, West Australia Department of Health, West Australian Country Health Service, Aboriginal Health Council of South Australia, South Australia Health, National Reference Laboratory, and Medical Communication Associates. We are grateful to Cepheid for their generous in-kind support with supply of GeneXpert devices and reduced price assay cartridges.
The study would not have been possible without the commitment of all participating health services and their staff and the ongoing support and advice from our partners. The authors acknowledge the important contributions of the TTANGO Reference Group not otherwise named as coauthors: Donna Mak, Russell Waddell, Joseph Debattista, Fleur Francis and Tony Coburn. The authors are grateful to Western Diagnostic Pathology, WA; Pathwest Laboratory Medicine, WA; Clinipath Pathology, WA; Queensland Health Pathology and Scientific Services; Sullivan Nicolaides Pathology, QLD; and SA Pathology; and their staff who undertook testing for STIs on behalf of participating health services, and provided the anonymous line record data for the trial.
Handling editor Nicola Low
Contributors The study was designed by all authors. LMC, LJN, BH, AT and SGB were responsible for data collection. SNT, DMW and DJS were responsible for reference and further laboratory testing results. LMC, RJG and HW were responsible for data analysis. All authors contributed to the interpretation of results and findings. LMC drafted the first full manuscript with input to the final version from all other authors. All authors contributed to the interpretation and writing of the manuscript.
Funding This work was supported by the Australian National Health and Medical Council awards (NHMRC Project Grant Application No 1009902 and Program Grant Application No 1071269). Cepheid (Sunnyvale, CA) supplied the GenXpert devices and cartridges at reduced price.
Disclaimer The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests None declared.
Ethics approval West Australian Aboriginal Health Ethics Research Committee (HREC 396); Kimberley Aboriginal Health Planning Forum (HREC 2012-003); West Australian Community Health Board Research Ethics Committee (HREC 2012/16); Townsville (HREC/12/QTHS/133)/Cairns and Hinterland Hospital and Health Service (HREC 12/QCH/89-810); Aboriginal Health Council South Australia (HREC 04-13-500).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement As per the approved study protocol, access to these data is limited to select named investigators and remains the property of the participating health services. Access to these data may be considered by contacting the corresponding author of this manuscript.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.