Objectives: To determine the sensitivity and specificity of the Architect Syphilis Chemiluminescence Assay (CLIA): a new highly automated screening test for syphilis.
Methods: To establish the sensitivity of the Architect Syphilis assay we tested 129 stored sera from serologically characterised cases of untreated syphilis. The sera were selected to contain a disproportionately high number of primary infections. There were 79 primary infections, 29 secondary infections, 9 early latent infections and 12 latent syphilis of unknown duration. To establish the specificity of the assay we tested 1107 sera that had been submitted for routine syphilis serology.
Results: The Architect CLIA and the Treponema pallidum particle agglutination test (TPPA) were in total agreement for all untreated infection with sensitivity of 98.4%. This was significantly higher than the sensitivity of the Murex immune capture enzyme (ICE) immunoassay (86%, p<0.001), the IgM enzyme immunoassay (EIA) (86.8%, p<0.001) and the Venereal Disease Research Laboratory test (VDRL) (83.7%, p<0.001). The difference in the sensitivity of the Architect and ICE assays was entirely due to primary stage syphilis (97.5% vs 77.2%, p<0.001). Although the specificity of Architect CLIA was very high (99.1%, 1049/1059) it was significantly lower (p = 0.016) than that of the Murex ICE assay (99.9%).
Conclusions: The Architect CLIA is significantly more sensitive than the Murex ICE screening assay in detecting primary syphilis but it is significantly less specific. Given the relatively high levels of early syphilis, we consider a small increase in the number of confirmatory tests required to exclude false-positive results is worthwhile to increase the detection of primary syphilis by 20%.
Statistics from Altmetric.com
Infectious syphilis re-emerged as a significant public health problem in the UK in the late 1990s, rising from 301 cases in 1997 to 3702 in 2006.1
Current guidelines for the management of syphilis recommend screening with an enzyme immunoassay (EIA) or the Treponema pallidum particle agglutination (TPPA) test.2 In a recent UK National Audit of early syphilis management an EIA was used to diagnose 89% of cases (regional range 18–100%).3 Primary syphilis is the most challenging of the stages of early syphilis to detect, with current serological screening tests giving lower sensitivity than other early stages.2 In one study, 17 (34%) cases of primary syphilis were initially negative by EIA, which had a sensitivity of 57% when compared with dark ground microscopy.4 Another recent evaluation reported a sensitivity of 84% for the Murex immune capture enzyme (ICE) immunoassay in detecting primary syphilis.5 Commercially available EIAs vary in their ability to detect primary syphilis,6 7 Although dark ground microscopy has high sensitivity in the diagnosis of primary syphilis, it is not widely used and is only performed in 29% of cases nationally.3
New highly automated syphilis assays based on chemiluminescence are now available. The Architect Syphilis Chemiluminescence Assay (CLIA) is a highly automated random access assay capable of processing 200 specimens per hour. It is based on paramagnetic microparticle chemiluminescent technology; microparticles are coated with the three recombinant antigens (TpN15, TpN17, TpN47), which are important in the immune response to syphilis. Instead of using EIA in the final detection, acridinium-labelled anti-human IgG and IgM monoclonal antibodies are used as conjugates. The end result of chemiluminescence is measured as relative light units (RLUs). Specimens yielding RLUs less than the cut-off are considered negative; specimens with RLUs greater than the cut-off are considered positive. The aim of this study was to evaluate the new Abbott Architect syphilis assay for the detection of antibodies to T pallidum.
To determine the sensitivity of the test we used a panel of sera from untreated cases, predominantly primary infections. The specificity of the test was determined by testing consecutive sera submitted for routine serological testing.
MATERIALS AND METHODS
Testing of known positive treponemal sera
To establish the sensitivity of the Architect Syphilis assay (Abbott, Wienbaden, Germany) we tested sera from cases of untreated syphilis diagnosed at the Genitourinary Medicine (GUM) Department Edinburgh between 2003 and 2007. All patients were screened for syphilis using the Murex ICE EIA (Murex, Dartford, UK), which detects both IgG and IgM antibodies. A full confirmatory test profile was performed on sera that gave a positive EIA (or an antibody index within 10% of the cut-off) on screening. A confirmatory test profile was also performed on sera from sexual contacts of patients with syphilis and those with clinical signs of syphilis when these were requested clinically—for example, if primary syphilis was suspected. Heat-inactivated serum was used for confirmatory testing, which comprised a quantitative Venereal Disease Research Laboratory (VDRL) carbon antigen test (Abbott, Wienbaden, Germany), a quantitative TPPA test (Fujirebio Mast Diagnostics, Bootle, UK) and the Mercia Syphilis M EIA (Microjen Bioproduct, Cambridge, UK) specific for anti-treponemal IgM.
The stage of syphilis of the cases diagnosed at GUM was obtained from the Health Protection Scotland National Extended Surveillance for Infectious Syphilis in Scotland (NESISS) database and the serological results were obtained from the Syphilis Specialist Laboratory database. We selected 129 sera, containing a disproportionately high number of primary diagnoses (79 from primary infections, 29 from secondary infections, 9 early latent infections and 12 latent syphilis of unknown duration) from the departmental bank of sera that had been stored at −70°C since the initial diagnosis.
Parallel testing of sera submitted for routine serological tests for syphilis
To establish the specificity of the assay, we tested 1107 anonymised sera that had been submitted for routine syphilis serology during November and December 2007: 558 were from GUM, 443 from an antenatal clinic, 46 from primary care, 38 from various hospital wards and departments, and 22 specimens were referred from other laboratories. Both the Murex ICE test and the Architect CLIA were performed according to the manufacturer’s instructions. Any serum with a Murex ICE antibody index of ⩾1 and/or an Architect Syphilis assay (RLU of specimen/cut-off) ratio ⩾1 was scored positive and was tested further using the confirmatory profile described above. Sera that gave a Murex ICE antibody index within 10% of the cut-off on screening, sera from sexual contacts of patients with syphilis and those with clinical signs of syphilis (as notified by the requesting clinician) also received a full confirmatory profile.
Sera were scored as treponemal or non-treponemal on the basis of the confirmatory serological tests. A positive result in only the Murex ICE EIA or the Architect CLIA was considered a false-positive; clinical information was also available for the majority of GUM patients who were positive on screening.
Table 1 shows the reactivity of the Architect syphilis assay with stored sera from known positive cases.
Overall the Architect and the TPPA were in total agreement with sensitivity of 98.4%. This was significantly higher than the 86% sensitivity of the ICE assay (χ2 12.2; p<0.001), the 86.8% sensitivity of the IgM (χ2 11.1; p<0.001) and the 83.7% sensitivity of the VDRL (χ2 15.5; p<0.001). The difference in the sensitivity of the Architect and ICE assays was entirely due to primary stage syphilis (97.5% vs 77.2%; χ2 12.9; p<0.001).
The stage of syphilis and treatment status where known and the serological reactivity for the 48 treponemal sera detected on routine screening are given in table 2.
As shown in table 2, all three treponemal tests give very high sensitivity compared with the low reactivity of the VDRL and IgM tests. This pattern suggests that the majority of the sera are likely to have come from patients who were previously treated. Therefore, although the numbers are small, the Architect CLIA is also likely to be highly sensitive in detecting treated infections.
The correlation of the Architect and ICE assays on testing 1107 routine sera in parallel is shown in table 3.
The overall correlation between the two assays was 98.9% (1095/1107). A positive TPPA test confirmed the treponemal nature of all 47 sera positive in both tests. The serum that was positive by ICE but negative by Architect was considered a false-positive as it was also negative by the TPPA, the IgM-EIA and the VDRL tests. Ten of the eleven sera that were Architect positive but ICE negative were considered false-positives as they were also negative by the TPPA, the IgM-EIA and the VDRL tests. This was supported by clinical data for all four false-positives originating from GUM. The remaining serum that was Architect positive but ICE negative was from a patient with a clinical diagnosis of primary syphilis. Other serological tests supported the diagnosis: TPPA positive (titre 160), VDRL positive (titre 4), IgM-EIA positive (index 1.77).
Based on the data in table 3, the sensitivity of the Architect CLIA was 100% (48/48) compared with 97.9% (47/48) for ICE. The specificity of Architect CLIA was 99.1% (1049/1059) compared with 99.9% (1058/1059) for ICE; this is a statistically significant difference (χ2 5.8; p = 0.016).
Table 4 shows the distribution of the sample/cut-off ratios for non-treponemal and treponemal sera.
As shown, 10 of the non-treponemal sera have ratios above the cut off value of 1. This ranged from >1.2 to 4.99. The two primary infections that were missed had cut off values in the 0.4–0.9 range and three of the primary infections had values of between 1 and 1.2. Raising the sample/cut-off ratio to a value that would remove only two of the ten false-positives would mean that an additional four cases of primary syphilis would be missed.
There was no difference in the sensitivity of the Architect CLIA (100%; 47/48) and the Murex ICE assay (97.9%; 47/48) when 1059 routine sera requesting syphilis serology were screened in parallel. However, only 48 sera were found to contain treponemal antibodies and only two of these were known to be from cases of untreated primary syphilis. One of these cases was negative in the Murex ICE assay.
The evaluation of sera from known cases of untreated syphilis confirmed that the Architect CLIA had significantly higher sensitivity (97.5%) than the Murex ICE assay (77.2%) in untreated primary infection. The sensitivity of both tests was 100% for all other stages of untreated infection.
Wheeler et al4 noted the failure of Murex ICE to detect a significant number of cases of primary syphilis where the test gave an overall sensitivity of 64% using unselected sera. Studies from our centre have reported higher sensitivities (84%5 and 75%8). Nevertheless, a significant number of primary infections were missed, which were detected by the TPPA with a sensitivity of 96%.5 8 The limitations of Murex ICE in detecting primary syphilis were not apparent from our earlier study where all seven cases of untreated primary infection were detected.9 The detection of primary syphilis by different EIAs is known to vary widely. In a comparative evaluation of nine different screening EIAs using 52 highly selected sera from patients with primary syphilis (all negative in the Treponema pallidum haemagglutination test (TPHA)), the sensitivity ranged from 48.5–76.9%.6
We found that although the specificity of the Architect CLIA was high (99.1%) it was significantly less than that of the Murex ICE (99.9%). A very high specificity for the ICE syphilis test has been reported previously (99.5%,6 99.8%9). Our findings of high sensitivity and specificity for the Architect CLIA are supported by a recently published evaluation from Japan,10 which reported a specificity of 100% (based on 500 hospital patient serum specimens that were non-reactive with both rapid plasma reagin (RPR) and TPPA) and a sensitivity of 100% (based on 121 hospital patient serum specimens that were reactive with both RPR and TPPA). The assay was also shown to have good reproducibility (intra-assay coefficient of variation less than 4%).10
Other evaluations of chemiluminescence support the potential of this methodology to give high sensitivity and specificity.11 12 The LIAISON chemiluminescent assay, in which TpN17 is used as antigen, gave an overall sensitivity of 99.2% with a panel of 131 clinically and serologically characterised syphilitic sera, including seven primary infections.11 CLIA detected all seven primary infections whereas the sensitivity of EIA and TPHA were 57.1%. The specificity was 99.9% when tested against 2494 blood donor sera. In the same study, a prospective evaluation of 1800 unselected routine screening samples for syphilis gave 100% specificity and 98.8% (83/84) sensitivity against the Western Blot. The serum that was missed by CLIA gave a faint reaction with TpN17 in the Western Blot assay, although there was a strong reaction with three other diagnostic bands.
In another study, the LIAISON chemiluminescent assay for the diagnosis of syphilis evaluated against the Captia Syphilis-G EIA and the TPPA on a total of 2645 samples, including 51 from patients with primary or secondary syphilis, gave an overall sensitivity of 95.8% and specificity of 99.1%.12
However, we consider the use of multiple recombinant antigens in the Architect CLIA has potential benefit. The use of three recombinant antigens (TpN15, TpN17 and TpN47), which are detectable throughout the course of syphilis,13–15 improves diagnostic sensitivity.16 It also maximises the opportunity to detect antibodies when the immune response may be abnormal as may happen in patients with HIV.17 Detection of antibodies to TpN47 was found to be particularly useful in the diagnosis of primary syphilis.18
Sensitivity and specificity of serological tests are normally negatively correlated. Our findings illustrate this: the Architect CLIA has the higher sensitivity in detecting primary syphilis but the Murex ICE assay has the higher specificity. Because the serological diagnosis of syphilis depends upon the principle of dual testing,19 where a positive screening test is confirmed with a second treponemal test of a different type,2 the use of a slightly less specific screening test will not compromise the accuracy of diagnosing syphilis. It will, however, lead to an increase in the number of confirmatory tests performed. Based on our findings, we consider that, given the current relatively high levels of infectious syphilis, an additional eight confirmatory tests per 1000 screening tests is worthwhile to potentially increase the detection of primary syphilis by 20%. We consider that a serum that is positive on Architect CLIA but negative in a specific anti-treponemal IgM-EIA and TPPA has a very high probability of being a false-positive. This view is based on our current findings that the sensitivities of the TPPA and Architect CLIA are identical and our previous findings that all sera from cases of primary syphilis that were missed by screening with Murex ICE were positive in either a specific anti-treponemal IgM-EIA or the TPPA.5 8
The results of our study suggest that the Architect CLIA has very high sensitivity in primary syphilis and would make an ideal screening test; however, because this was largely established with previously positive sera, these findings should be confirmed in a prospective study.
Primary syphilis is the most challenging stage of early syphilis to detect.
Serological screening by conventional enzyme immunoassays may fail to detect 15–40% of primary infections.
The Architect Syphilis Chemiluminescence Assay, a new highly automated screening test for syphilis, was significantly more sensitive than a conventional immunoassay in detecting primary infection (98% vs 77%).
The chemiluminescence assay was less specific than conventional enzyme immunoassay (99.1 vs 99.9%) but, given the relatively high levels of early syphilis, we consider this acceptable to increase the detection of primary syphilis by 20%.
We thank Dr Gordon Scott, Consultant in Genitourinary Medicine at Edinburgh Royal Infirmary, for providing clinical information in Lothian patients and Dr Sylvia Rafters, Consultant in Genitourinary Medicine in Lanarkshire, for providing clinical information on referred sera. We also thank the staff of the syphilis specialist laboratory for performing confirmatory serology.
Funding: Abbott kindly provided the Architect syphilis kits for this study and 1000 Euros towards the cost of the study.
Competing interests: None.
Contributors: HY and JD were responsible for study design, data analysis and writing the manuscript. JP reviewed the methodology and supervised all of the serological testing. LD performed the serological testing.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.