Article Text
PDF
Abstract
Objective To systematically review and estimate pooled sensitivity and specificity of the polymerase chain reaction (PCR) technique compared to recommended reference tests in the diagnosis of suspected syphilis at various stages and in various biological materials.
Design Systematic review and meta-analysis.
Data sources Search of three electronic bibliographic databases from January 1990 to January 2012 and the abstract books of five congresses specialized in the infectious diseases' field (1999–2011). Search key terms included syphilis, Treponema pallidum or neurosyphilis and molecular amplification, polymerase chain reaction or PCR.
Review methods We included studies that used both reference tests to diagnose syphilis plus PCR and we presented pooled estimates of PCR sensitivity, specificity, and positive and negative likelihood ratios (LR) per syphilis stages and biological materials.
Results Of 1160 identified abstracts, 69 were selected and 46 studies used adequate reference tests to diagnose syphilis. Sensitivity was highest in the swabs from primary genital or anal chancres (78.4%; 95% CI: 68.2–86.0) and in blood from neonates with congenital syphilis (83.0%; 55.0–95.2). Most pooled specificities were ∼95%, except those in blood. A positive PCR is highly informative with a positive LR around 20 in ulcers or skin lesions. In the blood, the positive LR was <10.
Conclusions The pooled values of LR showed that T. pallidum PCR was more efficient to confirm than to exclude syphilis diagnosis in lesions. PCR is a useful diagnostic tool in ulcers, especially when serology is still negative and in medical settings with a high prevalence of syphilis.
- Syphilis
- Pcr
- Meta-Analysis
- Diagnosis
- Serology
Statistics from Altmetric.com
Introduction
Syphilis is a sexually-transmitted infection of concern worldwide. WHO estimated that 12 million new cases occurred in 1999,1 of which 92% affect individuals in low-income countries. In most Western countries, syphilis is a cause of epidemics, particularly among men having sex with men.2 When syphilitic chancres are present, there is also a higher risk of HIV transmission and acquisition.3
While the treatment of syphilis is simple, low cost and effective, its diagnosis remains challenging, particularly among asymptomatic cases (mainly latent forms), pregnant women and newborns, or in the early phase of the disease. The case definition of the US Centres for Disease Control and Prevention (CDC)4 combines the presence of clinical signs or history with direct detection of Treponema pallidum in clinical specimens (primary, secondary and late syphilis) and/or reactive non-treponemal or treponemal tests. For chancres, a gold standard is the rabbit infectivity test, which is not routinely performed because it is time-consuming and expensive.5 ,6 When lesions are present, dark-field microscopy (DFM) or direct fluorescent-antibody is the most specific method to detect directly T. pallidum, but both require skilled laboratory staff and are also not widely available.5 When direct detection techniques are unavailable, serology testing combining a non-treponemal and a treponemal test is recommended. These provide a sensitivity ranging from 76% to 100% and specificity from 97% to 99%, depending on the syphilis stage, compared with direct detection of T. pallidum in clinical specimens.5
Today, there is an urgent need of a reliable, easy and cheap diagnostic test that would benefit patients in countries with limited resources. In the 1980s, the amplification of T. pallidum DNA by PCR has been proposed as an interesting, but costly, alternative.7−11 To be more powered to assess the diagnostic performance of PCR and to conclude on the best media to use, we reviewed all published studies using T. pallidum PCR in the diagnosis of syphilis and provided pooled estimates for sensitivity and specificity at various syphilis stages and in different biological specimens. We aimed also to identify factors that would explain any observed variability in the PCR diagnostic accuracy.
Materials and methods
Search strategy and selection criteria
Two independent reviewers (AGA and SL) searched three databases (PubMed, Embase and Web of Science) and the abstract books from specialised conferences to identify published studies between January 1990 and January 2012. Details of the literature search and selection criteria are described in online supplementary appendix 1.
Quality assessment
We assessed the methodological quality of each selected study by using the quality assessment tool for diagnostic accuracy studies (QUADAS).12 ,13 Two independent assessments were made for studies published in English or French (by AGA and SL), and one for reports in Czech14 and Russian.15 Details on the use of the QUADAS items are described in online supplementary appendix 1.
Data extraction
Three reviewers (AGA, SL, BN) screened abstracts of potentially relevant studies and applied the inclusion criteria. Reasons for exclusion were recorded and discrepancies were solved by consensus. Data extraction from the selected studies was done independently by two reviewers (AGA, SL) and two-by-two tables (PCR by reference test) were constructed for each study by syphilis stage and biological specimen tested. Data were verified for consistency and accuracy.
We collected the following data: author; journal; year of publication; study country; design; population characteristics; syphilis stage; PCR target; type of PCR; reference tests used to diagnose syphilis; purification of T. pallidum DNA during extraction; use of inhibitory controls; number of positive PCR among cases and number of negative PCR among controls (if available); number of cases and controls tested; and definitions of the reference test used for syphilis diagnosis. For blood specimens, we considered whole blood, serum or plasma as ‘blood’ and we collected separate data for each blood compartment. Early syphilis was defined as the stages of syphilis (primary, secondary and early latent syphilis) that occur within the first year after acquisition of infection. If data were missing or unclear, the corresponding author was contacted by email.
Data analysis
Due to the pathophysiology of the disease, we performed the analysis by syphilis stage and biological specimen. We calculated pooled estimates of the PCR sensitivity and specificity for all studies and, in a second step, only for studies using adequate reference tests by using the inverse variance method with random effects16 on the logit-transformed proportions. For studies providing both sensitivity and specificity, we estimated the pooled positive and negative likelihood ratios (LR+ and LR−, respectively) to assess the usefulness of PCR in the syphilis diagnosis. Heterogeneity was measured with the indicator I2 which reflects the proportion of variance between studies that is due to factors other than chance.17 If I2 was greater than 50% and the number of study was ≥10, we explored potential heterogeneity factors defined a priori for the LR+ and LR−. We used meta-regression or Cochran tests to explore the between-strata heterogeneity only when at least two studies were available in each stratum. The impact of a potential publication bias was explored by using the Trim and Fill method when the number of studies was greater than 10. This method detects missing studies for the funnel plot to be symmetric and assesses the pooled result including the missing studies.18
All analyses were performed using Comprehensive Meta Analysis V.2 (Biostat, Englewood, New Jersey, USA) with the significance level set at 0.05.
Results
Sixty-nine studies were retrieved by the systematic review. Two studies were not used in the meta-analysis: one that used aqueous humour from secondary syphilis with panuveitis to test for PCR, and one on syphilitic stillbirth that assessed the mothers’ blood. The study flow chart is presented in online supplementary figure 1.
Studies’ description
A summary of the 69 study characteristics by syphilis stage is presented in table 1 (see also online supplementary table 1). In less than half of all studies, the primary objective was the evaluation of the diagnostic performance of PCR. In the others, the primary objective was to describe the aetiology of genital ulcer diseases by using PCR (56.5%). In all studies, amplification was performed on native DNA and only two studies performed amplification on an initial RNA sample.15 ,19 The controls used to assess PCR specificity were mostly patients with genital ulcers caused by Chlamydia trachomatis, Herpes virus or Haemophilus ducreyi, but also aphthous ulceration and caustic or traumatic ulcerations. In the study population, 48.6% of patients were HIV-positive (29 studies). The use of antibiotics within the weeks prior to sampling was reported in 52% of studies (12/23). Overall, 17 studies mentioned controlling for inhibitory substances before PCR (25.8%).
Summary of the 69 principal study characteristics by syphilis stage and in all stages
Quality assessment and impact on pooled estimates
Most studies fulfilled the quality criteria related to bias (66.7%–95.6%, depending on the item). The study population was judged as representative of those who will receive the test in practice in 85.3% of studies. Finally, most studies fulfilled the items related to quality reporting (61.8%–86.8%, depending on the item). Syphilis diagnosis was established following the CDC recommendations in 66.7% of studies (table 1). In genital ulcer diseases (n=42 studies), DFM was used to diagnose syphilis in half of them. Pooled sensitivities and specificities were significantly lower in studies comparing PCR to inaccurate reference tests than those using adequate reference tests (table 2).
Pooled sensitivities, specificities and LRs (positive and negative) of Treponema pallidum PCR (random effects) with their heterogeneity (I2), by syphilis stage and biological specimen tested
Pooled estimates of sensitivity and specificity
To correctly assess the diagnostic performance of PCR, we performed the meta-analysis on the 46 studies that used the recommended reference tests to diagnose syphilis. Pooled results are shown in table 2. Pooled sensitivities were very low for blood samples, except in studies of neonates with congenital syphilis (n=4). By blood compartment, depending on the stage, both serum and plasma provided higher sensitivities than whole blood, but not significantly (table 2). In contrast with blood, sensitivities were systematically higher than 70% for ulcer or skin lesions. In neurosyphilis, the sensitivity in cerebrospinal fluid (CSF) was close to 50%. We compared regular PCR with nested PCR, real-time PCR and multiplex PCR (data not shown). Sensitivities of nested PCR or regular PCR, depending on the stage and specimen, were higher than with real-time and multiplex PCR in all biological specimens and syphilis stages, without reaching statistical significance. Specificity was less often assessed than sensitivity. Globally, the pooled specificities were high (table 2). However, the pooled specificities for blood and CSF specimens were as low as 83% in some stages, while they were systematically higher than 95% for ulcer and skin lesions. Specificities remained comparable between the three blood compartments (table 2) and specificities were similar by the PCR method used.
In studies using adequate reference tests, a positive PCR in ulcer or skin lesions was highly suggestive of syphilis and 17.3–21.1 times as likely to occur in patients with early syphilis, as opposed to patients without. In contrast, a negative PCR was less useful and was 0.27–0.36 times as likely to occur in patients with early syphilis, as opposed to those without (table 2). In studies using adequate reference tests in the ulcers or skin lesions from early syphilis, we found significantly higher LR+ and lower LR− compared with those using inadequate reference tests (table 2). Regarding the usefulness of PCR in congenital syphilis diagnosis, the LR+ was higher than the threshold of 10 in the CSF of newborns while LR+ was lower in the blood.
Publication bias
A potential publication bias was detected for some pooled LR (data not shown), but it did not change the conclusions of the meta-analysis.
Factors explaining heterogeneity
Sources of heterogeneity were explored for LR− in ulcers from primary syphilis and for LR+ plus LR− in lesion swabs from early syphilis. We did not find any association between LR− and LR+ depending on the stage, the year of publication or the gross national income (data not shown). The only factor that explained heterogeneity in the LR+ of lesion swabs from early syphilis was the PCR target: polymerase A gene (polA gene) provided a significantly higher LR+ than the 47KD lipoprotein gene (tpp47), but both provided excellent LR+: 41.0 (25.3–66.6) versus 14.8 (8.9–24.5), respectively (p=0.004). The heterogeneity in the LR− of lesion swabs from early syphilis was explained by the type of PCR with a lower LR− when standard PCR was used compared with multiplex PCR: 0.31 (0.22–0.45) and 0.51 (0.44–0.59), respectively (p=0.02).
Discussion
This meta-analysis indicates that, when the reference tests are adequate, the sensitivity of T. pallidum PCR is moderate, while its specificity is very good. However, the diagnostic performance varied with the biological specimen tested, especially for sensitivity. The best performance was found in ulcers from primary syphilis and, more generally, symptomatic lesions provided higher sensitivities than blood or even CSF. In the blood, PCR sensitivity was higher in plasma or serum than in whole blood. Regarding the PCR method, nested PCR or regular PCR was more sensitive than multiplex or real-time PCR in all syphilis stages and specimens. The inadequacy of reference tests recommended by the CDC was also associated with lower sensitivity. In addition, the PCR target and the type of PCR also explained heterogeneity in the subgroup of studies that used adequate reference tests. PCR targeting the polA gene provided higher LR+ than those targeting the 47KD lipoprotein gene (tpp47). Regular PCR provided lower LR− than multiplex PCR.
Some studies have shown that PCR sensitivities and specificities were higher in blood compared with ulcer and chancre14 ,20–27 and this result was confirmed by our meta-analysis. The potential explanations are that DNA extraction from blood provided a lower yield than skin lesions,24 ,27 thus postulating a higher T. pallidum load in skin lesions than in blood,28 but also that substances may be present in whole blood and inhibit PCR.29 Nonetheless, clinical lesions are only present in the early symptomatic phase of the disease. In the case of asymptomatic disease, even if the ear lobe was proven to provide the highest DNA yield, these results were described in only one study30 and blood remains the easiest media to use. When stratified by blood compartment, sensitivities were higher in plasma or serum than in whole blood and this confirms the results from previous studies on a larger number of cases.20 ,30
Regarding the PCR technique itself, some authors suggested that its lack of sensitivity was related to non-specific inhibitory substances in whole blood that could be controlled when extracting DNA.8 Apart from DNA extraction, T. pallidum genes targeted and the PCR method used might also explain the great heterogeneity of PCR sensitivity. Castro et al30 compared the 47KD lipoprotein gene (tpp47) with polA gene and a combination of both on the same biological samples. They showed that PCR targeting 47KD lipoprotein gene (tpp47) was the most sensitive but we did not find any difference. They also worked on paired data and used the same material to compare PCR targets and this might have increased the power of the study. Regarding the PCR method used, we did not find significant differences in their diagnostic performance. Multiplex PCR has the advantage of identifying multiple infectious agents, but already showed a lower sensitivity to detect T. pallidum.31 By contrast, nested PCR increases sensitivity by amplifying a secondary target within the first run product.24 Finally, real-time PCR presents the advantages to provide rapid and quantitative results and also to reduce the likelihood of contamination. To our knowledge, no previous study has compared the performance of different PCR methods in syphilis diagnosis and meta-analysis allows making such comparisons on a larger number of cases.
The LR+ was above the threshold value of 10 that is considered as the lower limit of clinical usefulness for some authors, but the LR was above the threshold of 0.10 considered as the higher limit to rule out the diagnosis.32 However, we should not interpret the LR in isolation. The pretest probability of the disease should also be considered when estimating the post-PCR probability of syphilis. Figure 1 illustrates the post-test probability as a function of the pretest probability using the LR obtained in ulcers from primary syphilis. When the pretest probability of syphilis is 16%, the risk of having syphilis reaches 80% if the PCR is positive in ulcers, and 5% if the PCR is negative. For syphilis prevalence above 16%, a positive PCR increases the post-test probability above 80%. In contrast, below 16%, a negative PCR test is informative, as it pushes the probability of disease to very low values.
Post-test probabilities (with 95% CIs) of Treponema pallidum PCR for different values of syphilis prevalence taking a likelihood ratio (LR)+ of 21.0 (15.5–28.4) and a LR− of 0.27 (0.18–0.40). Horizontal dashed lines represent desired levels of the post-test probabilities to rule-out (level 0.05) and to rule-in (level 0.80) syphilis diagnosis.
Our study has several limitations. First, we were not able to test the heterogeneity factors in all subgroups due to the small number of studies available. Although HIV status may influence reference tests,33 disaggregated data to explore HIV as a potential heterogeneity factor were insufficient. Second, regarding the PCR technique itself, some hypotheses were not tested due to lack of information. For instance, DNA extraction and purification can be done either manually or automatically and this could also modify PCR sensitivity.9 It is also important to test systematically for the presence of substances that inhibit PCR34 and this was done in a small proportion of studies, but we were unable to obtain the information for the remaining reports. Furthermore, the amplification procedure needs to be done at least in triplicate in order to achieve acceptable reproducibility,35 but none of the selected studies reported using this method. Finally, the gold standard to diagnose syphilis remains animal infectivity testing or rabbit infectivity test, but this is not routinely performed.5 As an alternative, the CDC recommends the use of DFM or direct fluorescent-antibody for the direct detection of T. pallidum in clinical specimens from early syphilis. DFM is a reference test in the diagnosis of syphilis, but not the gold standard. As a consequence, the assessment of the diagnostic performance of PCR may be biased.36
In conclusion, this meta-analysis confirmed that PCR could be a useful tool in the diagnosis of ulcer or chancres following risky sexual behaviour, especially when serology is not yet reactive or in a large number of regions when DFM is not available. We showed also a large heterogeneity of diagnostic values between studies and the diagnostic performance of PCR varied largely according to the biological specimen. The summary of the published diagnostic values by specimen and clinical stage of syphilis may help to guide future clinical practice.
Key messages
-
In this meta-analysis, we summarised the published diagnostic values of Treponema pallidum PCR (TpPCR) by clinical stage and biological specimen.
-
Sensitivities were 31.2% in blood from latent syphilis, 75.9% in ulcers from early syphilis, and specificities were 83.5% and 96.5%, respectively.
-
The pooled values of likelihood ratios showed that TpPCR was more efficient to confirm than to exclude syphilis diagnosis in lesions.
Acknowledgments
We thank Ms Rosemary Sudan for editorial assistance and Dr Youri Popowski for his help in the translation of the Russian publication. We thank also all the following authors for their kind collaboration (R. Castro, J. Gatrix, IF. Hoffman and the University North Carolina Group, K. Nagy, MA. Kingston, H. Liu, IM. Mackay, I. Martin, GT. Noordhoek, N. O'Farrell, G. Riedner, AE. Singh, W. Sturm, M. Sutton, V. Woznicovà).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online appendix
Footnotes
-
Contributors AG-A coordinated the work, searched relevant abstracts, assessed published work, extracted data, and analysed and interpreted the data. SL searched relevant abstracts, assessed published work and extracted data. BN searched relevant abstracts. CC analysed and interpreted the data. TVP takes responsibility for the integrity of the data and the accuracy of the data analysis. AGA wrote the paper with important contributions from SL, BN, CC and TVP.
-
Funding None.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.