Article Text
PDF
Abstract
Objectives To identify point-of-care (POC) and rapid nucleic acid amplification techniques (NAATs) for the diagnosis of chlamydia and gonorrhoea and assess their utility.
Methods Literature search for available POC and rapid NAATs. The performance from the best-performing assays were applied hypothetically to patients in the clinic in which 100 consecutive patients with chlamydia and 100 with gonorrhoea were diagnosed in 1737 and 4575 patients respectively, with 44/100 and 54/100 treated at first attendance, respectively.
Results 11 POC and 1 rapid NAAT were identified. Published performances for the best POC for chlamydia (CRT) were: sensitivity 41%–87%, specificity 89%–99.6%. Our data suggest that if this assay was used instead of our current NAAT, for every 100 patients diagnosed currently, 23–46 extra patients would be treated at first attendance; 10–35 would go undiagnosed with 7–191 false-positives. Best chlamydia rapid NAAT (GeneXpert): sensitivity 97.5%–98.7%, specificity 99.4%–99.9%. Anticipated performance for every 100 patients diagnosed currently: 0 extra patients treated at first attendance, 1–3 undiagnosed, 0–2 false-positives. Best POC for gonorrhoea (GC Check): sensitivity 54%–70%, specificity 97%–98%. Anticipated performance for every 100 patients diagnosed currently: 14–18 extra patients treated at first attendance, 28–32 undiagnosed, 92–137 false-positives. Best rapid NAAT for gonorrhoea (GeneXpert): sensitivity 96%–100%, specificity 99.9%–100%. Anticipated performance for every 100 patients diagnosed currently: 0 extra patients treated at first attendance, 0–4 undiagnosed, 0–5 false-positives. Rapid NAAT would reduce time to treatment by 4 days for initially untreated patients.
Conclusions POC assays would need to be used in conjunction with a NAAT, increasing early treatment rates expense and false-positive results. The rapid NAAT could be used alone, with a reduction in average time-to-treat and a small reduction in sensitivity and specificity.
- CHLAMYDIA INFECTION
- GONORRHOEA
- MODELING
- DIAGNOSIS
Statistics from Altmetric.com
Introduction
The ideal test for any sexually transmitted infection (STI) would be one that correctly identifies all infected patients at the time of first attendance.1–7 This would allow the infected patient to be treated immediately, reducing the risks of onward transmission, potentially preventing complications and avoiding patients going untreated if they subsequently fail to attend. A perfect rapid test would also allow partner notification to begin immediately, again optimising partner treatment and recall. In the last 15 years, there has been a steady rise in available point-of-care (POC) tests for chlamydia and gonorrhoea which produce results in less than an hour, based on an antibody/antigen reaction.1–4 ,7 More recently, test systems based on nucleic acid amplification techniques (NAATs) have been developed that produce results within a few hours.5 ,6 Although there have been some publications on the utility and accuracy of these tests, it is not entirely clear how they would perform against each other and when used in a range of settings. The literature looking at the published outcomes of available tests was therefore reviewed and their potential impact was modelled to our clinic cohort.
Methods
Setting
The Patrick Clements Clinic is an open-access sexual health service based in North West London.
The clinic cohort
In 2014, data were gathered on 100 patients diagnosed with chlamydia and 100 patients diagnosed with gonorrhoea, and also on how many patients tested negative over the period taken to diagnose these 100 patients and the time taken to treat each infected patient.
Literature review
Definitions
Rapid and POC testing was defined as any system that provided a chlamydia or gonorrhoea result within 4 h.
Search strategy
PubMed and Medline bibliographic databases were searched from January 2004 to the end of November 2014. Only English language papers were included. The following terms (and variations/alternative spelling of these terms) were used in the search: ‘point of care’, ‘POC’, ‘POCT’ ‘rapid test’,’ Chlamydia’, ‘Chlamydia trachomatis’, ‘Gonorh(o)ea’,‘Neisseria gonorrhoeae’ and ‘evaluation’. Further publications were looked for within reviews, editorials and systematic review articles. Once the names of commercially available tests had been identified, a further Medline search was performed using the specific name of each test.
Inclusion criteria
Papers were included in the analysis if they included the primary source of results of the performance of a rapid/POC test within a clinical cohort as against an identified standard NAAT and published data on any of sensitivity, specificity and/or time to produce the result. Papers were excluded if the rapid POC test was conducted on clinical isolates in a laboratory setting rather than on clinical samples or if the paper did not report on the performance characteristics of the test.
For each paper that met the inclusion criteria, information was extracted on the setting, participants (age, sex), evaluation design (sample size, specimen type), rapid/POC test (type, manufacturer, brand name), gold standard test (NAAT, culture), rapid/POC test performance (sensitivity, specificity) and operational characteristics (time to result).
Published test performances related to the clinic population
The results of the performance of each test from each published study that met the entry criteria were tabulated. The predicted best and worst performance of the best-performing non-NAAT POC test (results available within 1 h) and best-performing rapid NAAT tests (results available 2–4 h) were used to predict how they would work within our clinic for the diagnosis and treatment of chlamydia and gonorrhoea if the same levels of sensitivity and specificity applied. The standard against which these tests were compared was the performance of our current diagnostic test platform, the Becton-Dickinson ProbeTec CT/ NG (Chlamydia trachomatis/Neisseria gonorrhoeae) NAAT assay.
As data are very limited, there is no meta-analysis as such and the approach used is ad hoc. No assessment is possible as to whether the findings from different studies are compatible or not.
Assumptions made after the literature review
Calculations of the predicted utility of the best-performing assays were made based on the following assumptions. The publications using the non-NAAT POC assays indicate that a result is available within an hour or less and therefore available to most patients before they leave the clinic. Outcomes were estimated for the use of these assays in our clinic patients when used in conjunction with the clinics standard NAAT, but predicted outcomes if a NAAT was not used are also given. The results of the rapid NAAT assay are available after 2–4 h and in the only paper looking at the time to give patients their results, the result was available functionally after one working day. Therefore it was assumed that no extra patients would be treated on their initial day of attendance based on the rapid NAAT assay result. For the non-NAAT POC tests, the estimated reduction in time to treat for the 100 patients was calculated by multiplying the reduction in time to get the result, as compared with our current wait, by the number of extra patients who would be treated on the first day of attendance. For the rapid NAAT test, the estimated reduction in time to treat for the 100 patient cohort was calculated by multiplying the reduction in time to get the result, as compared with our current wait, by the number of patients who remained untreated after the initial visit (excluding never-treated patients). It was assumed that for the rapid NAAT test, the never-treated patients would remain untreated.
Results
The literature search identified 79 potential papers. Sixteen published papers were used to estimate the utility of the assays.8–23 Papers were excluded for any of the following reasons: editorial or commentary with no original data; organisms other than C. trachomatis or N. gonorrhoeae; test used only for trachoma or lymphogranuloma venereum (LGV) infection; data was from a highly selected population; the gold standard was not clear.
The current performance of our clinic in the diagnosis and treatment of 100 people with chlamydia and gonorrhoea infection is given in table 1. For the three patients with lone extra-genital CT infection and 20 patients with lone extra-genital gonococcal infection (GC), there was substantial supportive evidence that these were true infections including one or more of: multisite infection, concurrent GC/CT infection, subsequent GC culture +ve, contact subsequently GC +ve, high risk men who have sex with men (MSM) with frequent past/future GC +ve, high risk MSM with known GC exposure, high risk MSM with multiple unprotected sexual intercourse (UPSI) and high risk men who have sex with women with multiple UPSI.
Current performance in the diagnosis and treatment of 100 patients with chlamydia and 100 patients with gonorrhoea infections
Rapid and point-of-care test performance.
After the literature review, 16 papers met the inclusion criteria: 10 in relation to chlamydia, 3 for gonorrhoea and 3 for both (see web only appendix).8–23 For chlamydia, only one rapid NAAT assay was included in these papers but eight different non-NAAT POC assays were included. For gonorrhoea, only one rapid NAAT assay was included in these papers and three different non-NAAT POC assays were included. The assays with the best published performance in relation to sensitivity and specificity were identified, with rapid NAAT and POC tests considered separately for chlamydia and gonorrhoea. These assays were GeneXpert,14 ,15 ,19 the Chlamydia Rapid Test (CRT)9 ,11 ,16 ,18 and the GC Check.22 Their published performances are given in table 2. The non-NAAT POC tests can only be used on genital samples (urine and/or swabs) and will not detect extra-genital infection. The rapid NAAT can be used to detect both genital and extra-genital infection. For comparison, the performance of the BD (Becton Dickinson) ProbeTec assay against other standard NAATs is also given in table 2.23–27
Performance of the best-performing assays and their published worst performance
Other chlamydia assays reviewed but not used in the modelling were ACON,11 Handilab-C,12 ,13 Clearview,9 ,10 Biovei,21 Chlamydia Test Card,20 Quickview13 and Biorapid13 which had sensitivities and specificities ranging from 12%–80% and 25%–100% respectively. Other gonorrhoea assays reviewed and not used in the modelling were Biostar (OIA)8 and NOW.23 The NOW data was excluded as the patients in the published paper were highly selected. The sensitivity and specificity of the Biostar gonorrhoea assay were 60% and 89.9% respectively.
Table 3 shows what the outcomes would be in our clinic if the outcomes of the rapid NAAT and POC assays in table 2 were applied to the patients in table 1. In table 3, the clinic outcomes and also the estimated rates of true-positives, false-positives, true-negatives and false-negatives are shown if the best-performing assays had been used and had performed at their best and worst published outcomes.
Positive and negative test results predicted for our patients if the best-performing rapid NAAT and POC tests had been used
It is seen in table 1 that 44% of our patients with chlamydia and 54% of patients with gonorrhoea were treated on the initial day of attendance. Also, 3 patients with chlamydia and 20 patients with gonorrhoea had extra-genital infection only and 2 patients with chlamydia and 1 with gonorrhoea failed to return for treatment within 3 months. Based on our actual results, the utility of these best-performing rapid/POC assays was estimated in terms of how many extra patients would be treated on the day of initial attendance, the reduction in time spent untreated through earlier treatment and the number of patients who would remain untreated after three or more months (table 4). These data are also based on the published outcomes that show that the POC test results are available within 30–60 min and for the rapid NAAT the result takes 2–4 h (functionally, 1 day). For the predicted outcomes using non-NAAT POC, patients with extra-genital infection would not be detected by the POC tests and therefore not treated at the first visit.
Predicted utility of the different assays if used for our patients
Extra resources required to introduce these assays into the service
There is no published data on this subject. For the POC tests, extra resource would be required in terms of additional staff to test the samples and the cost of the extra assays. POC tests alone do not function well enough to replace the standard NAAT which would have to be used in addition. For the rapid NAAT, the performance in terms of sensitivity and specificity may be viewed as being sufficiently effective for it to be used as the only assay. The extra resources required would therefore be any additional costs that are incurred if this assay is more expensive than conventional NAATs. Laboratory costs would double if both a rapid NAAT and standard NAAT were used.
Discussion
Statement of principal findings
A literature search identified the performance of a number of commercially available non-NAAT POC and rapid NAAT assays for the early diagnosis of chlamydia and gonorrhoea.8–23 Applying their expected performance and potential utility to our patient cohort, it was found that the potential utility in terms of diagnostic outcomes of the assays varied greatly. None of the assays would achieve the ideal combination of being both highly sensitive and specific and allowing most infected patients to be treated before they leave the clinic.
The non-NAAT POC assays produce results that should be routinely available before the patient leaves, but the sensitivity of these assays is highly variable (12%–87%) meaning that some infected patients will remain untreated on the day of first attendance.8–13 ,16–8 ,20–22 Similarly, the specificity of these assays (25%–99.6%) would result in many false-positive results. By applying the expected outcomes of these assays to our clinic, even with the best-performing chlamydia non-NAAT POC test (the CRT), our data suggest that 10–35 patients out of every 100 would remain untreated on the day of initial attendance. The expected number of false-positive tests (depending on assay performance) would be anywhere between 7 and 191 for every 100 true- positive results. The published results for the best-performing gonorrhoea non-NAAT POC test (GC Check) suggest that it would leave 28–32 out of 100 patients untreated at the initial visit and 92–137 extra false-positives would occur for every 100 true-positives. The ‘missed’ patients would therefore be treated only if an NAAT was also used for such patients. The POC assays are not used to diagnose extra-genital infection and so all cases of extra-genital infection (20 gonorrhoea and 3 chlamydia infections for every 100 diagnosed infections in our clinic cohort) would be missed unless an additional NAAT was used for this purpose.
The published evidence suggests that most patients tested with the rapid NAAT (GeneXpert) are not treated before they leave the clinic and functionally the result is equivalent to being produced the following day, taking into account the time taken to contact patients and enabling them to return for treatment.15 Therefore using this assay is not likely to improve on the 56 patients with chlamydia and 46 patients with gonorrhoea out of every 100 infected who are currently not treated on the initial day of attendance in our clinic. The rapid NAAT assay for both chlamydia and gonorrhoea has a good sensitivity and specificity14 ,19 but the best and worst case scenarios would leave an extra 1–2 patients with chlamydia and 3–4 patients with gonorrhoea permanently untreated per 100 patients who were true-positive. Similarly, the numbers of patients with chlamydia and gonorrhoea who were false-positive would be 0–2 and 0–5 per 100 true-positives, respectively. However, the gold-standard used was our current assay, the BD ProbeTec. Therefore, it could be argued that some of the ‘false-positive’ results on the GeneXpert were truepositives that would be missed by the BD ProbeTec although the performance of BD ProbeTec has been found to be very good for both genital and extra-genital samples.28–30
Both non-NAAT POC and rapid NAAT assays would reduce the estimated time to treatment compared with our current performance. The total time to treatment per100 infected patients is estimated to fall by 204–208 days for chlamydia and 164–172 days for gonorrhoea using the rapid NAAT, based on the estimated reduction in time to obtain a result to 1 day as compared with our current median of 5 calendar days (3–4 working days). The reduction in total time to treatment for the non-NAAT POC assays is estimated to be 105–230 days per 100 patients for chlamydia and 70–90 days- per 100 patients for gonorrhoea.
The extra resources required to introduce these assays into a service such as ours are unknown, but could be significant, especially for the non-NAAT POC tests as they could never replace an NAAT and so both assay types would need to be used together. Therefore, testing costs would increase significantly and extra clinic staff would be required. It is unclear if the rapid NAAT would require extra resources when used in a service such as ours. Even if it did not, a balance would have to be made between the small number of false-positives and false-negatives on one hand and the average reduction in time to treat of 4 days for initially untreated patients (1.5–2 days per infected patient across 100 patients) on the other.
Currently, we observe a variation in laboratory turnaround of our standard NAAT from 3 to 9 days. Addressing the reasons for this variation could achieve a comparable reduction in time to treat.
Strengths and weaknesses of the study
The strength of this study is that it uses real-life clinic outcomes on which to base the assessment. The weakness is that the assessment uses published outcome data from elsewhere and does not have any data based on actual use of any of these non-NAAT POC or rapid NAAT assays in our clinic. Several assumptions about the utility of each assay have been made which may not be correct, such as assuming that never-treated patients will not be prevented by using the rapid NAAT assay. Also, there is not enough data to be able to estimate any specific financial outcomes.
An additional limitation is that given that the published studies are heterogeneous, the observed performance in the worst case study is probably worse than it really is in that setting and, likewise, the performance in the best study is probably better than the reality.
The findings for some of the POC tests are inconsistent which suggests that, for various reasons, the performance of the tests seems to fluctuate strongly across studies/settings. It is therefore difficult to predict precisely the performance of the tests if they were adopted in the clinic. Why this variability occurs can only be surmised. Perhaps, the POC assay results are difficult to interpret, or are prone to variations according to test conditions or vary according to populations of patients who are tested.
Meaning of the study
Although this paper makes several assumptions, it does give some insight into the potential utility of available assays. It is clear that in a service such as ours, where we expect to detect and treat the majority of infections in attending patients, the non-NAAT POC assays could only be used in conjunction with a standard NAAT assay. Therefore, while patients will be generally treated more quickly and never-treated patients could be avoided, this would be at the expense of a significant increase in the financial resources required. They would also lead to a significant increase in false-positive tests with the attendant distress caused to patients and their partners. It is possible that the rapid NAAT could be used without the additional use of a conventional standard NAAT but at the risk of a small number of false-positive and false-negative results. This might be offset by the significant fall in the time taken to treat patients with positive tests. The financial implications remain unclear. A modelling paper suggests that if rapid NAAT assay results were available before the patient left the clinic, and this was used in genitourinary medicine clinics in England, there would be a saving of £8million and that over 95 000 inappropriate treatments, 189 cases of pelvic inflammatory disease and 17 561 onward transmissions annually could be avoided.6
Unanswered questions and future research
It is clear from the published data that several questions remain unanswered. Very few of the published studies, despite there being research on assays that potentially produce rapid or POC results, have actually given any data on the time taken to give the result or the time taken to treat the patients. Instead, the majority of studies merely discuss the sensitivity and specificity of the assays. None of the published studies have included any financial data. Therefore, there is a need for more studies which look at actual outcomes in clinical cohorts in terms of time to results, time to treatment, positive and negative predictive values and financial and logistic implications.
If any of these assays were to be considered for use in a clinic, a period of testing by both current methodology and the new test would need to be conducted to establish how the tests might then perform subsequently in the clinic setting.
Conclusions
POC assays increase the early diagnosis rate for chlamydia and gonorrhoea but at the expense of a high false-positive and false-negative rate. The rapid NAAT assay reduces time to treat, with a small rate of false-positive and false-negative results and no improvement in patients treated at first visit compared with current standard of care. In a clinic such as ours, the performance of the non-NAAT POC assays for chlamydia and gonorrhoea is such that they would need to be used only in conjunction with an NAAT, greatly increasing expense and false-positive results, but increasing early treatment rates as well. The rapid NAAT could potentially be used alone in our clinic, with a small reduction in sensitivity and specificity, but also leading to a reduction in time-to-treat for many patients. The financial costs for each strategy are unknown.
Key messages
Non-nucleic acid amplification techniques (NAAT) point-of-care (POC) tests for chlamydia and gonorrhoea have variable sensitivities and specificities and would only work in practice in conjunction with an additional NAAT.
Rapid NAATs for chlamydia and gonorrhoea do not produce results quickly enough to be ready before most patients leave the clinic.
The sensitivity and specificity of rapid NAATs for chlamydia and gonorrhoea appear to be lower than standard NAATs leading to a few false-positive and false-negative results.
POC tests used in conjunction with a NAAT, and rapid NAATs used alone, would reduce the time taken to treat patients with chlamydia and gonorrhoea.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
Handling editor Jackie A Cassell
Acknowledgements The author would like to acknowledge Dr S Murphy and Dr J McSorley for critically commenting on the paper and to Paul Bassett (Statsconsultancy) for statistical advice. The efforts of the clinic staff were also key to the success of clinic outcomes.
Competing interests GB is local lead for a study of the CRT.
Provenance and peer review Not commissioned; externally peer reviewed.